DataWorks provides for-each nodes. You can use a for-each node to traverse the output
of an assignment node in loops. You can also customize the workflow in a for-each
node. This topic provides an example on how to configure and use a for-each node.
In this example, the for-each node is used to traverse the output of an assignment
node in two loops, and the system displays the current number of loops in the Operation
Center for each loop.
Prerequisites
DataWorks of the Standard Edition or a more advanced edition is activated.
Background information
In DataWorks, a for-each node is used to traverse the output of an assignment node
in loops. Before you use a for-each node, you must configure the for-each node as
a descendant node of an assignment node. After the assignment node passes its output
to the for-each node, the for-each node traverses the output in loops. After you create
a for-each node, the following three inner nodes are created: start, sql, and end.
You can customize the workflow in a for-each node and configure built-in variables
to obtain the output of an assignment node. For more information, see Composition and application logic.
Procedure
Before you use a for-each node, you must configure the for-each node as a descendant
node of an assignment node. The following figure shows the configuration procedure.

- Configure node dependencies.
Configure an assignment node as an ancestor node of a for-each node. For more information,
see Create and configure a workflow.
- Configure inputs for the for-each node.
In the Parameters section of the Properties tab for the for-each node, add the outputs parameter of the assignment node to Input Parameters. For more information, see Configure an assignment node.
- Configure the inner nodes of the for-each node.
Customize the workflow in the for-each node based on your business requirements. Then,
configure built-in variables for the inner nodes to enable the inner nodes to obtain
and traverse the output of the assignment node in loops. For more information about
the built-in variables, see Built-in variables. For more information about how to configure a for-each node, see Configure a for-each node.
Create and configure a workflow
To create a workflow that contains an assignment node as the ancestor node and a for-each
node as the descendant node, perform the following steps:
- Go to the DataStudio page.
- Log on to the DataWorks console.
- In the left-side navigation pane, click Workspaces.
- In the top navigation bar, select the region where your workspace resides, find the
workspace, and then click Data Analytics in the Actions column.
- Create a workflow.
- Move the pointer over the
icon and select Workflow.
- In the Create Workflow dialog box, specify Workflow Name and Description.
Notice The workflow name must be 1 to 128 characters in length and can contain letters, digits,
underscores (_), and periods (.).
- Click Create.
- Create a for-each node.
- Move the pointer over the
icon and choose . Alternatively, find the required workflow in the Scheduled Workflow pane, click the
workflow name, right-click General, and then choose .
- In the Create Node dialog box, specify Node Name and Location.
Notice The node name must be 1 to 128 characters in length and can contain letters, digits,
underscores (_), and periods (.).
- Click Commit.
- Create an assignment node. For more information, see Configure an assignment node.
- On the configuration tab of the created workflow, drag Assignment Node in the General section to the canvas on the right side.
- In the Create Node dialog box, specify Node Name and Location. The default value of the Location parameter is the path of the current workflow.
Notice The node name must be 1 to 128 characters in length and can contain letters, digits,
underscores (_), and periods (.).
- Click Commit.
- Drag a directed line to configure the assignment node as an ancestor node of the for-each
node.
Configure an assignment node
- On the configuration tab of the created workflow, double-click the name of the assignment
node that you created. The configuration tab of the assignment node appears.
- Select SHELL from the Language drop-down list.
- Enter the following statement in the code editor:
- In the right-side navigation pane, click the Properties tab. In the Parameters section, view the information about the outputs parameter below Output Parameters. The outputs parameter is the default output parameter of the assignment node.
- Click the
icon in the top toolbar to save the assignment node.
- Commit the assignment node.
Notice You must specify Rerun and Parent Nodes on the Properties tab before you commit the assignment node.
- Click the
icon in the top toolbar.
- In the Commit Node dialog box, specify Change description.
- Click OK.
If the workspace that you use is in standard mode, you must click
Deploy in the upper-right corner to deploy the node after you commit the node. For more
information, see
Deploy nodes.
Configure a for-each node
- Double-click the for-each node that you created. By default, the start, sql, and end nodes are displayed on the configuration tab of the for-each node.
- Delete the sql node.
You can use a node other than an SQL node in the workflow of the for-each node.
- If you want to use an ODPS SQL node, skip this step.
- If you want to use a node other than an SQL node, delete the sql node first. In this example, a Shell node is used.
- Right-click the sql node and select Delete Node.
- In the Delete message, click OK.
- Create and configure a Shell node.
You can create other types of nodes by using the same method. If you want to use the
default sql node, skip this step.
- Drag Shell in the General section to the configuration tab of the for-each node.
- In the Create Node dialog box, specify Node Name.
Notice The node name must be 1 to 128 characters in length and can contain letters, digits,
underscores (_), and periods (.).
- Click Create.
- Drag directed lines to configure the start node as an ancestor node of the Shell node and the end node as a descendant node of the Shell node.
- Double-click the Shell node. The configuration tab of this node appears.
- Enter the following code:
echo ${dag.loopTimes} ----Display the current number of loops.
Note
- The start and end nodes in the workflow of the for-each node have fixed logic and
cannot be modified.
- After you modify the code of the Shell node, save the modification. No message that
reminds you to save the modification will appear when you commit the node. If you
do not save the modification, the code cannot be immediately updated to the latest
version.
A for-each node supports the following environment variables:
- ${dag.foreach.current}: the current data entry.
- ${dag.loopDataArray}: the output of an assignment node.
- ${dag.offset}: the offset of the current number of loops to 1.
- ${dag.loopTimes}: the current number of loops. The value is equivalent to the value of ${dag.offset} plus 1.
For more information about the variables, see
Built-in variables and
Examples of variable values.
- Configure the scheduling properties of the for-each node.
- On the configuration tab of the for-each node, click the Properties tab in the right-side navigation pane.
- Find the loopDataArray parameter below Input Parameters in the Parameters section and click Change in the Actions column. The loopDataArray parameter is the default input parameter
of the for-each node.
- Select the outputs parameter of the assignment node from the drop-down list in the Value Source column.

Note After you configure the assignment node as an ancestor node of the for-each node,
you must specify the input parameter for the for-each node on the Properties tab.
If you do not specify the input parameter, an error occurs when you commit the for-each
node.
- Click Save.
- Click the
icon in the top toolbar to save the for-each node.
- Commit the for-each node.
Notice You must specify Rerun and Parent Nodes on the Properties tab before you commit the node.
- Click the
icon in the top toolbar.
- In the Commit dialog box, select the inner nodes that you want to commit and enter your comments
in the Description field.
- Click Commit.
If the workspace that you use is in standard mode, you must click
Deploy in the upper-right corner to deploy the for-each node after you commit the for-each
node. For more information, see
Deploy nodes.
- Test the for-each node and view the result.
- On the configuration tab of the for-each node, click Operation Center in the upper-right corner.
- In the left-side navigation pane of the Operation Center page, choose .
- On the page that appears, click the name of the for-each node in the node list. In
the directed acyclic graph (DAG) on the right side, right-click the assignment node
and choose . In the Patch Data dialog box, select the assignment node and for-each node and click
OK.
- On the Patch Data page, wait until the data backfill node instance is run. Then, click DAG in the Actions column of the instance.
- In the DAG that appears, right-click the assignment node and select View Runtime Log to view the operational logs of the node.
- On the Patch Data page, right-click the for-each node in the DAG and select View Internal Nodes.
- On the page that appears, click Loop 1 in the middle pane, right-click the Shell node in the DAG, and then select View Runtime Log.

On the page that appears, view the operational logs of the Shell node in the first
loop.

- Use the same method to view the operational logs of the Shell node in the second loop.