Compute resource management

Relationship between compute resources and data sources

DataWorks supports various compute resources. After you associate a compute resource, you can develop complex data processing and periodic scheduling tasks in Data Studio. When you associate most compute resources with DataWorks, a data source with the same name is automatically created. You can then use that data source in the Data Integration module for tasks such as data synchronization. The differences between a compute engine and a data source are as follows:

• A compute resource is a compute engine instance that executes data processing and analysis tasks.

• A data source connects to data storage services to store and manage data.

When you perform ETL data transformations using SQL nodes, such as converting a JSON column in an AnalyticDB for MySQL table into a new table, you must also associate the corresponding compute engine. SQL nodes in Data Studio rely on a compute engine to execute SQL statements, even if you are only using data transformation features.

Supported compute resources

In DataWorks, you can associate the following compute resources for data development.

Permission description

• Only workspace members who have the operator or workspace administrator role, or members who have the AliyunDataWorksFullAccess or AdministratorAccess permission policy can create compute resources. For more information, see Manage permissions on workspace-level services and Grant permissions to a RAM user.

• In addition to the preceding permissions, some compute resources may require additional permissions during creation. Follow the on-screen instructions to grant access.

Associate a compute resource

You can associate a compute resource from the corresponding entry based on whether the workspace has Use Data Studio (New Version) enabled.

Disassociate a compute resource

You can disassociate a compute resource as needed. On the compute resource page, find the associated compute resource and click Disassociate on the right side to disassociate the compute resource from the current workspace.

Appendix: Task execution environment description

In a standard mode workspace, a compute resource instance has two sets of configurations: one for the development environment and one for the production environment. Different environments can be configured to use different databases or instances. The system automatically maps access to the corresponding compute resource based on the runtime environment, thereby isolating development testing from production scheduling. For example, when a batch synchronization task is executed, the development environment automatically accesses the pre-configured development database, while production scheduling accesses the production database.