This topic describes how to view the directed acyclic graph (DAG) of an instance and use the features provided by a DAG to manage instances. You can view the DAG of an auto triggered node instance, a data backfill instance, or a test instance.

Limits

DAGs have the following limits:
  • Only users of DataWorks Professional Edition or a more advanced edition can use the node aggregation, upstream analysis, and downstream analysis features provided by DAGs.
  • Users of DataWorks Basic Edition or Standard Edition can use the node aggregation, upstream analysis, and downstream analysis features provided by DAGs on a trial basis free of charge as of May 31, 2021. From June 1, 2021, users must update their DataWorks to Professional Edition before they can use these features. For more information about DataWorks editions, see DataWorks advanced editions.
  • You can use the node aggregation, upstream analysis, and downstream analysis features of the DAGs of auto triggered nodes that are deployed only in the China (Shenzhen) region.

Manage instances in a DAG

Click DAG in the Actions column that corresponds to a node to view the DAG of the node. You can perform the following operations in a DAG:DAG
  • Aggregate nodes
    If an auto triggered node has multiple ancestor and descendant nodes or the ancestor and descendant nodes are distributed at multiple levels, you can aggregate the nodes. The nodes can be aggregated from dimensions such as node status, workspace, owner, and priority. Then, you can view the number of nodes from your required dimension. This allows you to have a command of the number of nodes from different dimensions and helps the system run the nodes. The following figures show the node distribution when the ancestor and descendant nodes of an auto triggered node are not aggregated or are aggregated by priority.
    • The following figure shows the node distribution when the ancestor and descendant nodes of an auto triggered node are not aggregated. Ancestor and descendant nodes of an auto triggered node not aggregated
    • The following figure shows the node distribution when the ancestor and descendant nodes of an auto triggered node are aggregated by priority. The figure shows that the current auto triggered node has six descendant nodes whose priorities are 1. Ancestor and descendant nodes of an auto triggered node aggregated by priority
  • Analyze ancestor nodes
    In most cases, an auto triggered node has both upstream and downstream relationships. If an auto triggered node is not run for a long time, you can analyze the ancestor nodes of the node. You can view the ancestor node that blocks the running of the node in the DAG of the node, and quickly locate and troubleshoot the issue. This improves the running efficiency of the node.
    Note You can analyze the ancestor nodes only of the auto triggered nodes that are not run.
    The following figure shows how to analyze the ancestor nodes of an auto triggered node. For example, the 2_ node is not run for a long time. In this case, you can click the node and click Upstream Analysis in the upper-left corner of the DAG to analyze the ancestor nodes of the node. An auto triggered node is not runThe analysis results show that the ancestor nodes that block the running of the 2_ node are the table data synchronization and metric statistics nodes. You can quickly troubleshoot the issue based on the analysis results.
  • Analyze descendant nodes
    If an auto triggered node has multiple descendant nodes or the descendant nodes of an auto triggered node are distributed at multiple levels, you can analyze the descendant nodes of the auto triggered node. You can aggregate the descendant nodes by node status, workspace, owner, or priority. Then, you can view the number of nodes at different levels from your required dimension or the total number of nodes at all levels from your required dimension.
    Note
    • By default, the descendant nodes of an auto triggered node are aggregated by owner. The system calculates the total number of nodes at all levels from the owner dimension.
    • If you analyze the descendant nodes of an auto triggered node and display the analysis results by level, a maximum of six levels of nodes can be displayed. If you want to view more levels of nodes, click Continue Analysis in the upper-left corner.
    In the following example, the descendant nodes of the tag node are analyzed. The following figures show the analysis results that are displayed by using different methods.
    • The descendant nodes of the tag node are aggregated based on the workspaces to which the descendant nodes belong, and the analysis results are presented by level. This way, the number of the descendant nodes in different workspaces is displayed at different levels. Analysis results displayed by level
    • The descendant nodes of the tag node are aggregated based on the workspaces to which the descendant nodes belong, and the analysis results are presented by using the merging method. This way, all the descendant nodes are placed at the same level, and the number of the descendant nodes that belong to different workspaces is displayed. Analysis results displayed by using the merging method
  • Select a display pattern for a DAG

    You can click the icons in the upper-right corner of a DAG panel to adjust the display pattern of the DAG based on your business requirements. For example, you can click Toggle Full Screen View or Fit Screen to perform the operation.

    In the following examples, the DAG of the 0_2 node is displayed after the descendant nodes of the 0_2 node are ungrouped or grouped:
    • The following figure shows the DAG of the 0_2 node when the descendant nodes of the 0_2 node are ungrouped. In this pattern, you can clearly view the upstream and downstream relationships of all the nodes. DAG of the 0_2 node when the descendant nodes of the 0_2 node are ungrouped
    • The following figure shows the DAG of the 0_2 node when the descendant nodes of the 0_2 node are grouped. In this pattern, every five descendant nodes of the 0_2 node are placed at the same level. This way, the descendant nodes are displayed in an orderly manner, and you can quickly obtain the total number of the descendant nodes. DAG of the 0_2 node when the descendant nodes of the 0_2 node are grouped
  • Right-click your desired node in a DAG and perform operations on the node. DAG
    Note After you click the Refresh icon in the upper-right corner, only the DAG of the node is refreshed.
    Operation Description
    Show Ancestor Nodes or Show Descendant Nodes If a workflow contains three or more nodes, specific nodes are automatically hidden in the DAG in Operation Center. You can select the number of levels to view all nodes at one or more levels.
    View Runtime Log Allows you to view the operational logs of the instance in a state such as running, successful, or failed.
    View Code Allows you to view the code of the instance.
    Edit Node Allows you to modify the node on the DataStudio page.
    View Lineage Allows you to view the lineage of the instance.
    Stop Allows you to stop the instance. You can stop an instance only in the pending or running state. After you perform this operation, the instance enters the failed state.
    Rerun Allows you to rerun the instance if it is in an abnormal state such as failed.
    Rerun Descendent Nodes Allows you to rerun all the instances generated for the descendant nodes of the current node. If multiple descendant instances exist, all the instances are rerun.
    Set Status to Successful Allows you to set the status of the current instance to successful and run its pending descendant instances. Perform this operation if a node fails to be run.
    Note You can perform this operation only on failed nodes.
    Emergency Operations Allows you to perform emergency operations in emergency scenarios. Emergency operations take effect only once on the current instance.

    Select Delete Dependencies to delete the dependencies of the current instance. You can perform this operation to start the current instance if the ancestor instances of the current instance fail and the current instance does not depend on the data of the ancestor instances.

    Freeze Allows you to freeze the current instance and pause the scheduling of the instance.
    Unfreeze Allows you to resume the scheduling of the frozen instance.