DataWorks:Serverless StarRocks SQL node

Prerequisites

• Computing resource constraints: Only EMR Serverless StarRocks computing resources are supported. Ensure network connectivity between the resource group and the computing resource.

• Resource group constraints: Only Serverless resource groups can run this type of task.

• (Optional, required for RAM users) The RAM user for task development has been added to the workspace with the Developer or Workspace Administrator role (which has extensive permissions — assign with caution). For more information about adding members, see Add members to a workspace.

  You can skip this step if you are using an Alibaba Cloud account.

Create a node

For instructions, see Create a node.

Develop the node

Write SQL statements in the editor. To define variables, use the ${variable_name} format and assign values in the Scheduling Parameters section of the Scheduling Settings panel on the right. This enables dynamic parameter passing in scheduled jobs. For more information, see Sources and expressions of scheduling parameters. Example:

SHOW TABLES; 
-- Defines a variable named var. If you assign the value ${yyyymmdd} to this variable,
-- you can create a table with a suffix that indicates the business date.
CREATE TABLE IF NOT EXISTS userinfo_new_${var} (
  ip STRING COMMENT 'IP address',
  uid STRING COMMENT 'User ID'
)PARTITIONED BY(
    dt STRING
); --This can be used with scheduling parameters.

Test the node

1. Configure run properties.

   In the Run Configuration panel on the right, configure the following parameters.

   Parameter	Description
   Computing resource	Select the EMR Serverless StarRocks computing resource to use. If no computing resources are available in the drop-down list, select Create Computing Resource. Important Ensure that the computing resource and the resource group are connected over the network. For more information, see network connectivity solution.
   Resource group	Select the resource group that passed the connectivity test when you attached the computing resource.
   Script Parameters	If you defined variables in the code by using the ${parameter_name} format, specify the corresponding Parameter name and Parameter Value in the Script Parameters section. At runtime, the system replaces the variables with the specified values. For more information, see Sources and expressions of scheduling parameters.

2. Run the node.

   To run the node, click Save and then Run.

Next steps

• Configure node scheduling: If you need to run a node periodically, configure its Scheduling Policy in the Scheduling Settings panel on the right.

• Publish a node: To run a task in the production environment, click the icon to publish the node. A node runs on schedule only after it is published to the production environment.

• Task O&M: After a task is published, monitor its periodic runs in the Operation Center. For more information, see Get started with Operation Center.