Bind an EMR Serverless Ray compute resource
Before you can run Ray jobs in DataWorks, bind a Ray cluster from an EMR Serverless Spark workspace as a Serverless Ray compute resource. After binding, select this resource on a Serverless Ray node in Data Studio to run tasks.
Prerequisites
-
You have created an EMR Serverless Spark workspace in E-MapReduce and an available Ray cluster in the workspace.
-
You have created a DataWorks workspace, and you have added your RAM account to the workspace and assigned it the workspace administrator role.
ImportantOnly workspaces that Use Data Studio (New Version) are supported.
-
You have used a serverless resource group and bound it to the target DataWorks workspace.
Limitations
-
Region restrictions: The supported regions are the same as those for binding an EMR Serverless Spark compute resource. Supported regions include China (Hangzhou), China (Shanghai), China (Beijing), China (Shenzhen), China (Chengdu), China (Hong Kong), Japan (Tokyo), Singapore, Indonesia (Jakarta), Germany (Frankfurt), US (Silicon Valley), and US (Virginia). The actual supported regions are those displayed in the console.
-
Permission requirements:
Operator
Permissions
Alibaba Cloud account
No additional permissions are required.
RAM account/RAM role
-
DataWorks administrative permissions: Only workspace members with the O&M and workspace administrator roles, or the
AliyunDataWorksFullAccesspermission, can create compute resources. For details, see Grant a user the workspace administrator permission. -
EMR Serverless Spark service permissions: You must have the
AliyunEMRServerlessSparkFullAccesspermission policy and theOwnerpermission for the target Spark workspace. For details, see Manage users and roles.
-
Bind a Serverless Ray compute resource
On the Compute Resources page, bind a Ray cluster as a Serverless Ray compute resource.
-
Select the type of compute resource to bind.
-
Click Associate Computing Resources to go to the Associate Computing Resources page.
-
On the Associate Computing Resources page, select Serverless Ray as the compute resource type to open the Bind Serverless RAY Compute Resource wizard.
-
-
Fill in the binding information in the wizard.
In the second step on the Enter Information page, configure the parameters described in the following table.
Parameter
Description
Spark Workspace
Select the EMR Serverless Spark workspace that contains the Ray cluster. You can also create a Spark workspace from the drop-down list.
Billing Method
Determined by the selected Spark workspace, such as Pay-as-you-go. Cannot be modified separately.
RAY Cluster
Select the Ray cluster to bind. The list shows all Ray clusters in the selected Spark workspace.
Engine Version
Automatically populated based on the selected Ray cluster, including the engine version and built-in Ray and Python versions. The actual versions are subject to the console display.
Computing Resource Instance Name
Identifies this compute resource in DataWorks tasks. When running a task, select the corresponding instance name on the node to use this binding.
Description
Optional. Notes the business purpose of this compute resource for easier management.
ImportantTo properly retrieve cluster information in DataWorks, do not remove the administrator role of the DataWorks service-linked roles
AliyunServiceRoleForDataWorksOnEmrandAliyunServiceRoleForDataWorksEnginefrom the E-MapReduce Serverless Spark workspace. -
Click Confirm to complete the Serverless Ray compute resource binding.
Next step
After binding, you can create a Serverless Ray node in Data Studio and select this compute resource to run Ray jobs.