DataWorks:Parameter node

Overview

A parameter node is a virtual node that does not run data computation tasks or produce data. It is primarily used for cross-node parameter passing and centralized parameter management.

• Pass parameters between nodes

  In a workflow, if a downstream node requires output parameters from multiple upstream nodes across various levels, you can use a parameter node to consolidate them. Add all required parameters to the parameter node. Then, set the downstream node to depend directly on the parameter node to access all the parameters it needs.

  

  In the figure above, the Sq_MySQL_G node requires the output parameters from the Sq_MySQL_B and Sq_MySQL_E nodes. You can use a parameter node as a downstream node for both Sq_MySQL_B and Sq_MySQL_E. Add all parameters that Sq_MySQL_G needs to the parameter node. Then, set Sq_MySQL_G as a downstream node of the parameter node to receive its output parameters. This way, Sq_MySQL_G can obtain the parameters from Sq_MySQL_B and Sq_MySQL_E via the parameter node.

• Centralized parameter management

  When downstream task nodes in a workflow need to use certain constant or variable parameters, you can add all these parameters to a single parameter node. The downstream nodes that require these parameters can then depend directly on the parameter node. This approach centralizes parameter management for the workflow.

  

  In the figure above, the Sq2_MySQL_A, Sq2_MySQL_C, Sq2_MySQL_D, Sq2_MySQL_E, and Sq2_MySQL_F nodes all require parameters. You can use a parameter node to obtain output parameters from upstream nodes and configure necessary variable or constant parameters. This parameter node then serves as the upstream dependency for the nodes that require these parameters.

Prerequisites

• The RAM user that you want to use is added to your workspace.

  If you want to use a RAM user to develop tasks, you must add the RAM user to your workspace as a member and assign the Develop or Workspace Administrator role to the RAM user. The Workspace Administrator role has more permissions than necessary. Exercise caution when you assign the Workspace Administrator role. For more information about how to add a member and assign roles to the member, see Add members to a workspace.

• A serverless resource group is associated with your workspace. For more information, see the topics in the Use serverless resource groups directory.

• Before you can configure a parameter node, you must create one. For more information, see Create nodes for a scheduling workflow.

Note

To reference parameters from a parameter node, a task node must be its direct downstream node and set the parameter node as an upstream dependency.

Step 1: Configure the parameter node

You can create constant parameters, variable parameters, and pass-through variables in a parameter node. The following sections describe how to configure these three types of parameters.

After you finish configuring the parameters for the parameter node, configure its schedule settings. For more information, see Configure schedule settings.

Step 2: Use parameters in downstream nodes

After you configure the parameter node, you can directly use its parameters in downstream nodes for centralized parameter management and improved development efficiency.

1. Associate the upstream parameter node parameters.

   1. Open the parameter node editor and click schedule settings on the right side. In Scheduling Settings > Scheduling Dependency > Node dependency configuration, click Add Dependency and configure the parameter node as an upstream dependency.

   2. After you configure the dependency, switch to Scheduling Parameters and click Add parameters. Configure the following settings:

      1. Enter a custom name for the new parameter.

      2. Click the button to search for and associate an output parameter from the upstream parameter node.

2. After you configure the parameters, you can reference them in the node script by using ${parameter name} to obtain the parameters passed from the parameter node.

Step 2: Deploy and manage the node

1. After you complete the schedule settings, you can submit and deploy the node along with its upstream and downstream nodes to the production environment. For more information, see Deploy nodes.

2. After deployment, the task runs periodically based on the schedule you configured. You can view the deployed scheduled tasks and perform O&M operations in Operation and Maintenance Center > Node O&M > Auto Triggered Task O&M > Auto Triggered Task. For more information, see Manage scheduled tasks.