Create a Data Lake Analytics node in DataWorks to build an online extract, transform, and load (ETL) process.
Background information
Data Lake Analytics nodes connect to Alibaba Cloud Data Lake Analytics (DLA). For more information about DLA, see What is DLA?
Tasks on Data Lake Analytics nodes can run on serverless resource groups or old-version exclusive resource groups. We recommend serverless resource groups. For more information about how to purchase and use a serverless resource group, see Use serverless resource groups.
Limits
Data Lake Analytics nodes are supported in the following regions: China (Hangzhou), China (Shanghai), China (Beijing), China (Shenzhen), China (Hong Kong), Japan (Tokyo), Singapore, Germany (Frankfurt), UK (London), US (Silicon Valley), and US (Virginia).
Procedure
-
Go to the DataStudio page.
Log on to the DataWorks console. In the target region, click in the left-side navigation pane. Select a workspace from the drop-down list and click Go to Data Development.
-
Hover over the
icon and choose .Alternatively, open the target workflow, right-click Custom Rule, and choose .
- In the Create Node dialog box, configure the Name and Path parameters. Note The node name must be 1 to 128 characters in length and can contain letters, digits, underscores (_), and periods (.).
- Click Confirm.
-
Configure the Data Lake Analytics node.
-
Select a data source.
Select a data source. If the data source you need is not in the drop-down list, click New data source on the right to go to the Data source management page and create one. For more information, see Configure a Data Lake Analytics (DLA) data source.
-
Write SQL statements for the node.
After you select a data source, write SQL statements based on the syntax supported by DLA. You can write data manipulation language (DML) or data definition language (DDL) statements.
-
Click the
icon in the top toolbar. -
Click the
icon in the top toolbar to execute SQL statements.
To change the resources for a test run on the Data Studio page, click the
icon in the toolbar and select the target serverless resource group. NoteA serverless resource group is required to access a data source deployed in a virtual private cloud (VPC). You must select a serverless resource group connected to the data source.
-
-
In the right-side pane, click Scheduling to configure the node's scheduling properties. For more information, see Configure basic properties.
You must select a serverless resource group connected to the Data Lake Analytics node to periodically schedule tasks.
-
Click the
icon in the top toolbar to save the node. -
Click the
icon in the top toolbar. -
In the Commit Node dialog box, enter a Change Description.
-
Click OK.
- Perform O&M operations on the node. For more information, see Basic O&M for auto-triggered nodes.
Save and commit the node.
You must configure the Rerun attribute and Parent Nodes properties for the node before you can commit it.
If you use a workspace in standard mode, click Deploy in the upper-right corner after you commit the node. For more information, see Deploy tasks.