Auto triggered nodes are automatically run as scheduled after they are committed to the scheduling system. You can view the details of an auto triggered node from the DAG of the node or the auto triggered node list.

Background information

By default, the Cycle Task page displays the nodes in all the workflows that are created by using the current Alibaba Cloud account.
Notice Do not perform operations on the projectname_root node, which is the root node of the workspace. All the instances of auto triggered nodes depend on this node. If this node is frozen, the instances of auto triggered nodes cannot be run.
Instances can be generated in the following ways:
  • Next Day:
    • If a node is committed and deployed before 23:30, instances are generated from the next day.
    • If a node is committed and deployed after 23:30, instances are generated from the day after the next day.
  • Immediately After Deployment: Instances are generated immediately after a node is committed and deployed. However, if the node is committed and deployed after 23:30, instances cannot be immediately generated. For more information, see Immediate instance generation.
If you cannot find the instances generated by your node, perform operations by following the instructions in What do I do if I cannot find the instance of a node?.
The Cycle Task page displays the auto triggered nodes that are committed to the scheduling system in a list or in directed acyclic graphs (DAGs).
  • Manage auto triggered nodes

    You can specify filter conditions to search for your desired node. Then, you can view the basic information of the node and perform operations such as testing the node and generating retroactive data for the node. For more information, see Manage auto triggered nodes.

  • DAG

    A DAG presents the ancestor and descendant nodes of a node. You can view the details and lineage of a node and test it from the DAG of the node. A DAG provides the node aggregation, upstream analysis, and downstream analysis features. The node aggregation feature allows you to aggregate nodes in a DAG from the dimensions such as workspace, owner, and priority, and view the total number of nodes from your required dimension. The remaining two features allow you to analyze the ancestor and descendant nodes of a specific node. This way, you can quickly locate the ancestor node that blocks the running of the node and view the number of the descendant nodes of the node based on the analysis results. These features help you better understand the running status of all nodes. For more information, see Manage auto triggered nodes in a DAG.

Limits

DAGs have the following limits:
  • Only users of the DataWorks Standard Edition or a more advanced edition can use the node aggregation, upstream analysis, and downstream analysis features provided by DAGs.
  • Users of the DataWorks Basic Edition or Standard Edition can use the node aggregation, upstream analysis, and downstream analysis features provided by DAGs on a trial basis free of charge as of May 31, 2021. From June 1, 2021, users must update their DataWorks to the Professional Edition before they can use these features. For more information about DataWorks editions, see DataWorks advanced editions.
  • You can use the node aggregation, upstream analysis, and downstream analysis features of the DAGs of auto triggered nodes that are deployed only in the China (Shenzhen) region.

Go to the Cycle Task page

  1. Go to the DataStudio page.
    1. Log on to the DataWorks console.
    2. In the left-side navigation pane, click Workspaces.
    3. In the top navigation bar, select the region where your workspace resides, find the workspace, and then click Data Analytics in the Actions column.
  2. On the DataStudio page, click the Cycle Task icon in the upper-left corner and choose All Products > Operation Center.
  3. In the left-side navigation pane, choose Cycle Task Maintenance > Cycle Task. On the Cycle Task page, view the auto triggered nodes or the DAGs of the auto-triggered nodes.

Manage auto triggered nodes

In the middle of the page, click the Rightward arrow icon to show the node list. Auto triggered nodes
Operation Description
Filter Specify filter conditions to search for your desired node in the section marked with 1 in the preceding figure.
You can search for nodes by node name or node ID. You can also specify the following conditions to search for your desired node: Solution, Workflow, Node Type, Owner, Region, Scheduling Resource Group, Engine Type, Engine Instance, Baseline, My Nodes, Modified Today, Frozen Nodes, Isolated Nodes, and Expired Nodes.
Note If you search for nodes by node name, the search results are affected by other filter conditions you specified. Only nodes that meet both the specified search condition and other filter conditions are returned in the search result.
DAG Click DAG in the Actions column that corresponds to a node to view the DAG of the node. You can view the node information, such as properties, operation logs, and code, in the DAG.
Test Click Test in the Actions column that corresponds to a node to test the node. For more information, see Test instances.
Patch Data Click Patch Data in the Actions column that corresponds to a node to generate retroactive data for the node. For more information, see Retroactive instances.
More Click More in the Actions column that corresponds to a node to perform the following operations on the node:
  • Select Freeze to freeze the node. After the node is frozen, the system generates instances for the node but does not run the instances of the node and its dependent descendant instances.
  • Select Unfreeze to unfreeze the node. After the node is unfrozen, the system normally runs the instances of the node and its descendant instances.
  • Select View Instances to view the instances of the node.
  • Select Configure Alert Rule to configure alert rules for the node.
  • Select Change Owner to change the owner of the node.
  • Select Add to Baseline to add the node to a baseline.
  • Select Modify Scheduling Resource Group to change the resource group that is used to run the node. You can perform this operation if multiple resource groups exist in the workspace. In the Modify Scheduling Resource Group dialog box, select a resource group from the New Resource Group drop-down list and click OK.
  • Select Configure Data Quality Rules to configure rules for monitoring the data quality of the node.
  • Select View Lineage to view the lineage of the node.
  • Select View Node Details to go to the Node Information page. On this page, you can view the node information on the Ancestor Nodes and Descendent Nodes tabs.
Batch operations In the section marked with 3 in the preceding figure, you can perform the following operations on the nodes: Change Owner, Modify Scheduling Resource Group, Modify Data Integration Resource Group, Freeze, Unfreeze, Configure Alert Rule, Add to Baseline, and Undeploy.

Manage auto triggered nodes in a DAG

Click DAG in the Actions column that corresponds to a node to view the DAG of the node. You can perform the following operations in a DAG:DAG features
  • Aggregate nodes
    If an auto triggered node has multiple ancestor and descendant nodes or the ancestor and descendant nodes are distributed at multiple levels, you can aggregate the nodes. The nodes can be aggregated from dimensions such as node status, workspace, owner, and priority. Then, you can view the number of nodes from your required dimension. This allows you to understand the number of nodes from different dimensions and helps the system run the nodes. The following figures show the node distribution when the ancestor and descendant nodes of an auto triggered node are not aggregated or are aggregated by priority.
    • The following figure shows the node distribution when the ancestor and descendant nodes of an auto triggered node are not aggregated. Ancestor and descendant nodes of an auto triggered node not aggregated
    • The following figure shows the node distribution when the ancestor and descendant nodes of an auto triggered node are aggregated by priority. From the figure, you can quickly understand that the current auto triggered node has six descendant nodes whose priorities are 1. Ancestor and descendant nodes of an auto triggered node aggregated by priority
  • Analyze ancestor nodes
    In most cases, an auto triggered node has both upstream and downstream relationships. If an auto triggered node is not run for a long time, you can analyze the ancestor nodes of the node. You can view the ancestor node that blocks the running of the node in the DAG of the node, and quickly locate and troubleshoot the issue. This improves the running efficiency of the node.
    Note You can analyze the ancestor nodes of only the auto triggered nodes that are not run.
    The following figure shows how to analyze the ancestor nodes of an auto triggered node. For example, the 2_ node is not run for a long time. In this case, you can select the node and click Upstream Analysis in the upper-left corner to analyze the ancestor nodes of the node.An auto triggered node is not runThe analysis results show that the ancestor nodes that block the running of the 2_ node are the table data synchronization and metric statistics nodes, as shown in the following figure. Then, you can quickly troubleshoot the issue based on the analysis result.
  • Analyze descendant nodes
    If an auto triggered node has multiple descendant nodes or the descendant nodes of an auto triggered node are distributed at multiple levels, you can analyze the descendant nodes of the auto triggered node. You can aggregate the descendant nodes by workspace, owner, or priority. Then, you can view the number of nodes at different levels or the total number of nodes at all levels from your required dimension.
    Note
    • By default, the descendant nodes of an auto triggered node are aggregated by owner. The system calculates the total number of nodes at all levels from the owner dimension.
    • If you analyze the descendant nodes of an auto triggered node, the analysis results are displayed by level, and a maximum of six levels of nodes can be displayed. If you want to view more levels of nodes, click Continue Analysis in the upper-left corner.
    In the following example, the descendant nodes of the tag node are analyzed. The following figures show the analysis results that are displayed by using different methods.
    • The descendant nodes of the tag node are aggregated based on the workspaces to which the descendant nodes belong, and the analysis results are presented by level. This way, the number of the descendant nodes in different workspaces is displayed at different levels. Analysis results displayed by level
    • The descendant nodes of the tag node are aggregated based on the workspaces to which the descendant nodes belong, and the analysis results are presented by using the merging method. This way, all the descendant nodes are placed at the same level, and the number of the descendant nodes that belong to different workspaces is displayed. Analysis results displayed by using the merging method
  • Select a display pattern for a DAG

    You can click the icons in the upper-right corner of a DAG panel to adjust the display pattern of the DAG based on your business requirements. For example, you can click Toggle Full Screen View or Fit Screen to perform the operation.

    In the following examples, the DAG of the 0_2 node is displayed after the descendant nodes of the 0_2 node are ungrouped or grouped:
    • The following figure shows the DAG of the 0_2 node when the descendant nodes of the 0_2 node are ungrouped. In this pattern, you can clearly view the upstream and downstream relationships of all the nodes. DAG of the 0_2 node when the descendant nodes of the 0_2 node are ungrouped
    • The following figure shows the DAG of the 0_2 node when the descendant nodes of the 0_2 node are grouped. In this pattern, every five descendant nodes of the 0_2 node are placed at the same level. This way, the descendant nodes are displayed in an orderly manner, and you can quickly obtain the total number of the descendant nodes. DAG of the 0_2 node when the descendant nodes of the 0_2 node are grouped
  • Right-click your desired node in a DAG and perform operations on the node. DAG
    Operation Description
    Show Ancestor Nodes If a workflow contains three or more nodes, specific nodes are automatically hidden in the DAG in Operation Center. You can select the number of levels to view all nodes at one or more levels. Select the largest number to view the most node dependencies.
    Show Descendant Nodes
    View Node Details Allows you to go to the Node Information page to view the node information, including the input table, output table, ancestor nodes, and descendant nodes.
    View Code Allows you to view the code of the node.
    Edit Node Allows you to go to the DataStudio page and modify the current node.
    View Instances Allows you to view the instances of the node.
    View Lineage Allows you to view the lineage of the node.
    Test Allows you to test the node. After you select Test, the Test dialog box appears. You must specify Test Name and Data Timestamp and click OK. Then, the Test Instance page appears.
    Run Allows you to generate retroactive data for the node. The following modes for generating retroactive data are supported: Current Node Retroactively, Current and Descendant Nodes Retroactively, and Mass Nodes Retroactively.
    Freeze Allows you to freeze the current node and pause the scheduling of the node.
    Unfreeze Allows you to resume the scheduling of the frozen node.
    Configure Data Quality Rules Allows you to configure rules for monitoring the data quality of the node.