A MaxCompute compute source attaches a Dataphin project to a MaxCompute project, providing computing capabilities for processing offline computing tasks in the Dataphin project. If the compute engine of the Dataphin system is set to MaxCompute, the project must have a MaxCompute compute source to support features such as standardized modeling, ad hoc queries, MaxCompute nodes, and general scripts. This topic describes how to create a MaxCompute compute source.
MaxCompute data lakehouse solution
MaxCompute supports the data lakehouse solution, which is implemented through the MaxCompute data warehouse and data lake. For more information, see MaxCompute data lakehouse.
Prerequisites
You have set the Dataphin compute engine to MaxCompute. For more information, see Set the compute engine for a Dataphin instance to MaxCompute.
The system supports accounts with the super administrator and project administrator roles to create compute sources. For more information, see Data warehouse planning permission list.
Procedure
In the top navigation bar of Dataphin, choose Planning > Compute Source.
On the Compute Source page, click Create Compute Source and select MaxCompute Compute Source.
On the Create MaxCompute Compute Source page, configure the required parameters.
Parameter
Description
Basic Information Of Compute Source
Compute Source Type
Select MaxCompute.
Endpoint
The default is the Endpoint of the compute engine. This parameter cannot be modified.
AccessKey ID,AccessKey Secret
Enter the AccessKey ID and AccessKey Secret of the account that can access the data in the MaxCompute project.
You can obtain the AccessKey ID and AccessKey Secret of the account on the User Management page.
ImportantTo ensure that the Dataphin project is properly connected to the MaxCompute project, we recommend that you enter the AccessKey of the MaxCompute project administrator.
To ensure normal metadata collection, avoid modifying the AccessKey of the MaxCompute project.
Compute Source Configuration
MaxCompute Project
Select or enter a MaxCompute project. The dropdown list displays only the projects that are visible to the user corresponding to the entered AccessKey in the console. You can manually enter another project name or go to the MaxCompute console to create one.
NoteThis is the MaxCompute project name, not the DataWorks workspace name.
After you switch to a region in the upper-left corner of the MaxCompute console, you can view the specific MaxCompute project name on the Project Management tab.
External Project
Specify whether the project is an external project. The default value is No. If you select Yes, the project is an external project.
NoteIntegration tasks do not support reading or writing data in external MaxCompute projects.
The Hologres engine does not support accelerating queries on external MaxCompute projects.
The system cannot obtain the storage usage of tables in external MaxCompute projects from MaxCompute metadata.
Resources do not support storage health scores and storage usage.
Compute Project
Select or enter a compute project. The dropdown list displays only the projects that are visible to the user corresponding to the entered AccessKey in the console. You can manually enter another project name or go to the MaxCompute console to create one.
When the MaxCompute project is an external project, you need to set a MaxCompute project as the computing resource.
Compute Source Name
Enter a name for the compute source. The name must meet the following requirements:
It can contain Chinese characters, digits, letters, underscores (_), and hyphens (-).
It cannot exceed 64 characters in length.
Description
Enter a brief description of the compute source.
NoteIf you need to create multiple compute sources under different AccessKey IDs in batches, you can click Create Compute Source to create them. You can create a maximum of 10 compute sources at a time.
Click Test Connection to test the connectivity between Dataphin and the account of the AccessKey ID.
After the test is successful, click Verify And Submit. The system verifies the connectivity of the compute source and creates it.
What to do next
After you create a MaxCompute compute source, you can attach it to a project when you create the project. For more information, see Create a general project.