All Products
Search

This Product

    DataWorks:View and manage auto triggered nodes

    Document Center

    DataWorks:View and manage auto triggered nodes

    Last Updated:Aug 15, 2023

    Auto triggered nodes are automatically run as scheduled after they are committed to the scheduling system. You can view the list of auto triggered nodes in the current workspace, view the configurations of auto triggered nodes that are deployed in the production environment, and manage auto triggered nodes on the Operation Center page.

    Background information

    Limits

    Only users of DataWorks Professional Edition or a more advanced edition can use the node aggregation, ancestor node analysis, and descendant node analysis features provided on the DAG page. For more information about how to update DataWorks, see Billing of DataWorks advanced editions.

    Precautions

    • Do not freeze the projectname_root node, which is the root node of a workspace. Instances of auto triggered nodes depend on this node. If this node is frozen, the instances of auto triggered nodes cannot be run.

    • After you modify an auto triggered node, you must deploy the node. After the node is successfully deployed, the modification takes effect in the production environment. You may encounter the following unexpected situations when you deploy the node: The node fails to be deployed, the deployment process is blocked, or the version of the deployed node does not meet your expectations. In these cases, we recommend that you go to the Cycle Task page in Operation Center to troubleshoot issues.

      • You can use the DAG of a node to check whether dependencies of the node are correct.

      • You can view the information on the details page of a node to check whether parameter configurations for the node in the production environment meet your business requirements.

    Go to the Cycle Task page

    1. Log on to the DataWorks console. In the left-side navigation pane, choose Data Modeling and Development > Operation Center. On the page that appears, select the desired workspace from the drop-down list and click Go to Operation Center.

    2. In the left-side navigation pane of the page that appears, choose Cycle Task Maintenance > Cycle Task. On the Cycle Task page, view the auto triggered node list or the DAG of an auto triggered node.

    View the auto triggered node list

    You can view the auto triggered nodes that you commit to the scheduling system in the auto triggered node list and perform O&M operations on the nodes. Auto triggered node list

    Operation item

    Description

    Filter

    Allows you to specify conditions to search for your desired node in the rectangle marked with 1 in the preceding figure.

    You can search for your desired node by node name or node ID. You can also specify the following conditions to search for your desired node: Solution, Workflow, Node Type, Owner, Region, Scheduling Resource Group, Engine Type, Engine Instance, Baseline, My Nodes, Modified Today, Frozen Nodes, Isolated Nodes, and Expired Nodes.

    Note

    • Frozen node: A frozen node is not automatically scheduled, and its descendant nodes are blocked from running. If a node is frozen, instances generated for the node are also frozen. Frozen instances are not automatically scheduled, and the descendant nodes of the node are blocked from running.

    • Isolated node: An isolated node does not depend on another node. This type of node is not automatically scheduled. If an isolated node has a large number of descendant nodes, the running of the descendant nodes is affected. For more information, see Scenario: Isolated node.

    • Expired node: An expired node does not generate instances and is not run.

    Auto triggered node list

    Allows you to view the auto triggered nodes that meet the specified conditions in the rectangle marked with 2 in the preceding figure.

    Operations on a single node

    Allows you to perform operations on an auto triggered node in the rectangle marked with 3 in the preceding figure.

    • DAG: You can perform this operation to view dependencies of an auto triggered node. On the DAG page, you can right-click the node in the DAG to perform related operations. For more information, see View the DAG of an auto triggered node.

    • Test: You can perform this operation to test whether the running of an auto triggered node meets expectations. After you click Test in the Actions column of the node, a test instance is generated for the node. You can go to the Test Instance page to view the status of the test instance. For more information, see Test an auto triggered node and view test instances generated for the node.

      Note

      A test instance generates data. Perform this operation only when it is necessary.

    • Backfill Data: You can perform this operation to backfill data for an auto triggered node for a period of time in the past or in the future. After you click Backfill Data in the Actions column of the node, a data backfill instance is generated for the node. You can go to the Patch Data page to view the status of the data backfill instance. For more information, see Backfill data for an auto triggered node and view data backfill instances generated for the node.

    • More:

      • Freeze: You can perform this operation if an auto triggered node and its descendant nodes do not need to be run for a period of time.

        Note

        Do not perform this operation on the projectname_root node, which is the root node of a workspace. All instances of auto triggered nodes depend on this node. If this node is frozen, the instances of auto triggered nodes cannot be run.

      • Unfreeze: You can perform this operation to unfreeze an auto triggered node that is frozen.

      • View Instances: You can perform this operation to view instances generated for an auto triggered node on the Cycle Instance page.

      • Configure Alert Rule: You can perform this operation to configure a custom alert rule to monitor the status of an auto triggered node every day. After the configuration is complete, you can choose Alarm > Rule Management to view the configured custom alert rule. For more information, see Create a custom alert rule.

      • Change Owner: You can perform this operation to change the owner of an auto triggered node.

      • Add to Baseline: You can perform this operation to add an auto triggered node to a baseline. For more information about how to use a baseline, see Overview.

      • Modify Scheduling Resource Group: You can perform this operation to change the resource group for scheduling that is used to run an auto triggered node to another resource group for scheduling.

      • Configure Data Quality Monitoring Rule: You can perform this operation to configure a data quality monitoring rule to check data for an auto triggered node during the running of the node. For more information, see Overview.

      • View Lineage: You can perform this operation to view data lineage of an auto triggered node.

      • View Node Details: You can perform this operation to view basic properties of an auto triggered node. For more information about related parameters, see View the details of an auto triggered node.

    Batch operations

    Allows you to perform an operation on multiple auto triggered nodes at the same time. You can click the following buttons in the rectangle marked with 4 in the preceding figure to perform operations on the selected nodes at the same time: Change Owner, Modify Scheduling Resource Group, Modify Data Integration Resource Group, Freeze, and Unfreeze. You can also select multiple auto triggered nodes, click More, and then select Add rule monitoring or Add baseline monitoring from the shortcut menu to perform the corresponding operation on the auto triggered nodes at the same time.

    Items per page

    Allows you to select the number of items that you want to view on a single page in the rectangle marked with 5 in the preceding figure. The maximum number that you can select is 200.

    View the DAG of an auto triggered node

    In the auto triggered node list, click the name of an auto triggered node or click DAG in the Actions column of the node. On the DAG page, the system automatically displays the level-1 ancestor nodes and level-1 descendant nodes of the node. You can view the other ancestor and descendant nodes of the current node by clicking the icons and buttons for the node aggregation and descendant node analysis features in the upper-left corner of the DAG page. You can also right-click the node to perform related operations. Cycle Task

    Functionality

    Description

    Operations that you can perform on the DAG page

    Allows you to use the node aggregation and descendant node analysis features and adjust the display style of the DAG of a node by clicking icons and buttons in the rectangles marked with 1 in the preceding figure. For more information, see Appendix: Use the features provided in a DAG.

    • Node aggregation: You can use this feature to aggregate nodes from different dimensions such as node status, workspace, owner, or priority.

    • Descendant node analysis: If an auto triggered node has multiple descendant nodes or the descendant nodes of an auto triggered node are distributed at multiple levels, you can use this feature to calculate the number of descendant nodes that are affected by the current node.

    • Adjust the display style of the DAG: You can adjust the display style of the DAG of a node by clicking the icons in the upper-right corner of the DAG page.

    Operations on a single node

    Allows you to perform related operations on an auto triggered node in the DAG of the node in the rectangle marked with 2 in the preceding figure.

    • Show Ancestor Nodes: allows you to view the ancestor nodes of an auto triggered node at different levels. A maximum of six levels of ancestor nodes can be displayed at the same time. You can perform this operation to view the ancestor nodes that affect data output of the current node.

    • Show Descendant Nodes: allows you to view descendant nodes of an auto triggered node at different levels. A maximum of six levels of descendant nodes can be displayed at the same time. You can perform this operation to view the descendant nodes whose data output is affected by the current node.

    • View Code: allows you to view the code of an auto triggered node in the production environment. If the code of the node does not meet expectations, you need to confirm whether the latest code of the node is successfully deployed to the production environment. For more information about how to deploy a node to the production environment, see Deploy nodes.

    • Edit Node: allows you to edit an auto triggered node on the configuration tab of the node on the DataStudio page.

    Note

    For information about other operations, see View the auto triggered node list.

    Details about a single node

    Allows you to view properties of an auto triggered node in the rectangle marked with 3 in the preceding figure. You can also click Show Details to view the property details, context parameters, operation logs, and code of the node. For more information, see View the details of an auto triggered node.

    View the details of an auto triggered node

    On the tabs that appear after you click Show Details, you can view properties such as scheduling parameters of and resource groups that are used by an auto triggered node in the production environment and check whether the code of the node meets your expectations.

    2

    Tab

    Description

    General

    On this tab, you can view the scheduling properties of an auto triggered node in the production environment. For more information about the basic parameters, see Configure basic properties.

    Note

    If the parameter configurations for an auto triggered node on the DataStudio page are inconsistent with those for the node on the Cycle Task page in Operation Center, we recommend that you check whether you successfully deploy the node after you modify the node.

    • Rule Monitoring: allows you to view the information about the alert rule that is used to monitor the status of the current auto triggered node. You can also click Create on the right side to quickly create a custom alert rule for the auto triggered node. For more information, see Create a custom alert rule.

      Note

      You can view only the details of the alert rule that is used to monitor the status of an auto triggered node. You cannot view the data quality monitoring rule that is used to check data for an auto triggered node.

    • Baseline monitoring: allows you to view the information about the baseline to which the current auto triggered node belongs. You can also click Create on the right side to quickly create a baseline for the auto triggered node. For more information, see Manage baselines.

      Note

      After you create and configure a baseline, the baseline is available to choose as an option on the Cycle Task page on the next day. To check whether the baseline alert rule is successfully created on the day when you configure the baseline, choose Smart Baseline > Baselines.

    Context

    On this tab, you can view the input and output parameters of an auto triggered node. For more information about the context parameters, see Configure input and output parameters.

    Operation Log

    On this tab, you can view operation records of an auto triggered node, including the operation time, operator, and specific operations.

    Code

    On this tab, you can view the latest code of an auto triggered node in the production environment. If the code of the node does not meet expectations, you need to confirm whether the latest code of the node is successfully deployed to the production environment. For more information, see Deploy nodes.