Bind an EMR Serverless Ray compute resource

Updated at:
Copy as MD

Before you can run Ray jobs in DataWorks, bind a Ray cluster from an EMR Serverless Spark workspace as a Serverless Ray compute resource. After binding, select this resource on a Serverless Ray node in Data Studio to run tasks.

Prerequisites

Limitations

  • Region restrictions: The supported regions are the same as those for binding an EMR Serverless Spark compute resource. Supported regions include China (Hangzhou), China (Shanghai), China (Beijing), China (Shenzhen), China (Chengdu), China (Hong Kong), Japan (Tokyo), Singapore, Indonesia (Jakarta), Germany (Frankfurt), US (Silicon Valley), and US (Virginia). The actual supported regions are those displayed in the console.

  • Permission requirements:

    Operator

    Permissions

    Alibaba Cloud account

    No additional permissions are required.

    RAM account/RAM role

    • DataWorks administrative permissions: Only workspace members with the O&M and workspace administrator roles, or the AliyunDataWorksFullAccess permission, can create compute resources. For details, see Grant a user the workspace administrator permission.

    • EMR Serverless Spark service permissions: You must have the AliyunEMRServerlessSparkFullAccess permission policy and the Owner permission for the target Spark workspace. For details, see Manage users and roles.

Bind a Serverless Ray compute resource

On the Compute Resources page, bind a Ray cluster as a Serverless Ray compute resource.

  1. Select the type of compute resource to bind.

    1. Click Associate Computing Resources to go to the Associate Computing Resources page.

    2. On the Associate Computing Resources page, select Serverless Ray as the compute resource type to open the Bind Serverless RAY Compute Resource wizard.

  2. Fill in the binding information in the wizard.

    In the second step on the Enter Information page, configure the parameters described in the following table.

    Parameter

    Description

    Spark Workspace

    Select the EMR Serverless Spark workspace that contains the Ray cluster. You can also create a Spark workspace from the drop-down list.

    Billing Method

    Determined by the selected Spark workspace, such as Pay-as-you-go. Cannot be modified separately.

    RAY Cluster

    Select the Ray cluster to bind. The list shows all Ray clusters in the selected Spark workspace.

    Engine Version

    Automatically populated based on the selected Ray cluster, including the engine version and built-in Ray and Python versions. The actual versions are subject to the console display.

    Computing Resource Instance Name

    Identifies this compute resource in DataWorks tasks. When running a task, select the corresponding instance name on the node to use this binding.

    Description

    Optional. Notes the business purpose of this compute resource for easier management.

    Important

    To properly retrieve cluster information in DataWorks, do not remove the administrator role of the DataWorks service-linked roles AliyunServiceRoleForDataWorksOnEmr and AliyunServiceRoleForDataWorksEngine from the E-MapReduce Serverless Spark workspace.

  3. Click Confirm to complete the Serverless Ray compute resource binding.

Next step

After binding, you can create a Serverless Ray node in Data Studio and select this compute resource to run Ray jobs.