Associate an AnalyticDB for Spark computing resource
To develop and manage AnalyticDB for Spark tasks in DataWorks, you must first associate your AnalyticDB for MySQL cluster with DataWorks as an AnalyticDB for Spark computing resource. Once the resource is associated, you can use it for data development in DataWorks.
Prerequisites
An AnalyticDB for MySQL cluster is created.
After you create the cluster, you must create an interactive resource group with the engine type set to Spark for the cluster. Otherwise, you cannot associate the cluster as an AnalyticDB for Spark computing resource.
NoteWe recommend that you create the AnalyticDB for MySQL cluster in the same Region as the DataWorks workspace. You cannot associate a cluster and a workspace that are in different regions.
A DataWorks workspace is created. The RAM user who performs the operation has been added to the workspace and granted the Workspace Administrator role.
ImportantOnly workspaces that use the Use Data Studio (New Version) are supported.
A resource group is associated with the workspace.
If you use a Serverless resource group, ensure that the Serverless resource group can connect to the AnalyticDB for MySQL cluster.
If you use a deprecated exclusive resource group, ensure that the exclusive resource group for scheduling can connect to the AnalyticDB for MySQL cluster.
The resource group must be in the same VPC as the AnalyticDB for MySQL cluster, and the IP address of the resource group must be added to the whitelist of the AnalyticDB for MySQL cluster.
Limitations
Supported regions: China (Hangzhou), China (Shanghai), China (Beijing), China (Shenzhen), China (Hong Kong), Japan (Tokyo), Singapore, Malaysia (Kuala Lumpur), and Indonesia (Jakarta).
Permission requirements:
User
Permission description
Alibaba Cloud account
No additional permissions are required.
RAM user/RAM role
DataWorks permissions: Only workspace members who have the O&M or Workspace Administrator role, or members who have the
AliyunDataWorksFullAccesspolicy can create computing resources. For more information about how to grant permissions, see Grant a user the Workspace Administrator role.AnalyticDB for MySQL permissions: When you associate an AnalyticDB for Spark computing resource, if you need to create a database for the AnalyticDB for MySQL cluster, you must grant the
AliyunADBFullAccesspolicy to the RAM user. This policy grants full permissions to manage the AnalyticDB for MySQL cluster.
Go to the computing resource list page
-
Log on to the DataWorks console. In the navigation pane on the left, switch to the target region and click . Select your workspace from the drop-down list and click Go to Management Center.
-
In the navigation pane on the left, click Computing Resources to open the computing resource list page.
Associate an AnalyticDB for Spark computing resource
On the Computing Resources page, configure and associate the AnalyticDB for Spark computing resource.
Select the type of computing resource to associate.
Click Associate Computing Resources to open the Associate Computing Resources page.
On the Associate Computing Resources page, select AnalyticDB for Spark as the computing resource type to go to the Bind AnalyticDB for Spark Computing Resource configuration page.
Configure the AnalyticDB for Spark computing resource.
On the Bind AnalyticDB for Spark Computing Resource configuration page, configure the settings as described in the following table.
Parameter
Description
Configuration mode
Only the Alibaba Cloud Instance Mode is supported.
Alibaba Cloud account
Only the current Alibaba Cloud account is supported.
Instance
Select the AnalyticDB for MySQL cluster to bind. You can also click Create in the drop-down menu to create an AnalyticDB for MySQL cluster.
NoteWhen you create the cluster, you must create an interactive resource group with the engine type set to Spark. Otherwise, you cannot associate the cluster as an AnalyticDB for Spark computing resource.
Database name
Enter the name of an existing database in the AnalyticDB for MySQL cluster.
Computing resource instance name
Enter a name for the computing resource. You will use this name to select the resource for tasks at runtime.
Test the connectivity.
In the Connection Configuration section, select the resource group that DataWorks uses to run AnalyticDB for Spark tasks, and click Test Connectivity to ensure that the resource group can access your AnalyticDB for MySQL cluster. For more information, see Network connectivity solutions overview.
Click Confirm to complete the AnalyticDB for Spark computing resource configuration.
NoteWhen you bind an AnalyticDB for Spark computing resource, the system automatically creates an AnalyticDB for Spark data source with the same name in the Data Sources of the current workspace.
Next steps
After you configure the AnalyticDB for Spark computing resource, you can use it to develop ADB Spark node and ADB Spark SQL node tasks in Data Studio.