After an auto triggered node is committed and deployed to the production environment, DataWorks runs the node as scheduled. You can view the details of the auto triggered node and perform O&M operations on the node in the list of auto triggered nodes. For example, you can manually run, suspend, or undeploy the auto triggered node. This topic describes the O&M operations that you can perform on an auto triggered node.

Overview

Auto triggered nodes are nodes that are automatically run as scheduled after they are committed to the scheduling system. You can view the auto triggered nodes in a specified workspace on the Cycle Task page in Operation Center. This topic describes the following O&M operations that you can perform on an auto triggered node:

Create and deploy an auto triggered node

Before you can view an auto triggered node on the Cycle Task page in Operation Center, you must deploy the node to the scheduling system in the production environment. For more information, see Deploy nodes.
Note
  • After you perform operations such as addition, modification, and undeployment on nodes, resources, or functions in the production environment, you must commit and deploy the nodes, resources, or functions to make the configurations take effect.
  • After you modify an auto triggered node, you must deploy the node. After the node is deployed, the modification takes effect in the production environment. You may encounter the following unexpected situations when you deploy the node: The node fails to be deployed, the deployment process is blocked, or the version of the deployed node does not meet your expectations. In these cases, we recommend that you go to the Cycle Task page in Operation Center to troubleshoot issues.
    • You can use the directed acyclic graph (DAG) of the node to check whether the dependencies of the node are correct.
    • You can view the information on the details page of the node to check whether parameter configurations for the node in the production environment meet your business requirements.

View auto triggered nodes

In the list of auto triggered nodes, you can view the auto triggered nodes that are committed and deployed to the scheduling system in the production environment. You can check whether the code, scheduling parameter configurations, scheduling dependencies, and lineage of the auto triggered nodes meet your business requirements. You can view the details of and perform the related operations on an auto triggered node in the list of auto triggered nodes or in the DAG of the current auto triggered node. For more information, see View and manage auto triggered nodes.
Note
  • Only the auto triggered nodes that are deployed to the production environment are displayed in the list of auto triggered nodes on the Cycle Task page in Operation Center. If the configurations of an auto triggered node do not meet your expectations, perform a check by following the instructions that are described in Create and deploy an auto triggered node.
  • The following two types of nodes are not automatically scheduled: nodes that do not depend on other nodes and nodes that depend on their descendant nodes.

Run auto triggered nodes

DataWorks generates auto triggered node instances that are scheduled to run on the next day for an auto triggered node every night. You can click Backfill Data or Test in the Actions column of the auto triggered node to generate a data backfill instance or a test instance for the auto triggered node.
Instance type Scenario Relationship with an auto triggered node

(How an instance is generated)

Instance trigger method

(How an instance is triggered to run)

Auto triggered node instance Periodic extract, transform, and load (ETL) operations are required. Every night, DataWorks generates auto triggered node instances that are scheduled to run on the next day based on the snapshot information of an auto triggered node at a specific point in time.
Note Nodes cannot be automatically scheduled to generate auto triggered node instances in Operation Center in the development environment.
DataWorks triggers an auto triggered node instance to run.
Data backfill instance You want to backfill data of a period of time in the past or in the future for the current auto triggered node and its descendant nodes. This indicates that you must perform ETL operations on the data of that period of time. You must manually backfill data for the current auto triggered node to generate data backfill instances for the node. After you backfill data, the data backfill instances are generated and triggered to run.
Execute the test and view the test instance You want to test the current auto triggered node to check whether the node can be run as expected.
Note The code of the auto triggered node is run during the test.
You must manually test the current auto triggered node to generate test instances for the node. After you perform the test, the test instances are generated and triggered to run.
Note
  • Impacts of different instance generation modes on auto triggered node instances:
    When you create an auto triggered node, you can configure the Instance Generation Mode parameter to determine the time when instances are generated after the auto triggered node is deployed to the production environment. You can view the instances that are generated for the current auto triggered node on the Cycle Instance page in Operation Center in the production environment based on the setting of the Instance Generation Mode parameter. For more information, see Configure immediate instance generation for a node.
    • Instance Generation Mode set to Next Day:
      • If you commit and deploy an auto triggered node before 23:30, DataWorks generates instances for the auto triggered node on the next day and runs the instances as scheduled.
      • If you commit and deploy an auto triggered node after 23:30, DataWorks generates instances for the auto triggered node on the third day and runs the instances as scheduled.
    • Instance Generation Mode set to Immediately After Deployment:
      • If you commit and deploy an auto triggered node before 23:30, DataWorks generates instances for the auto triggered node on the current day and runs the instances as scheduled.
        • If the scheduling time for running your auto triggered node is 10 minutes or more than 10 minutes later than the time you commit and deploy the auto triggered node, DataWorks generates an instance for the node and runs the instance.
        • If the scheduling time for running your auto triggered node is less than 10 minutes from the time you commit and deploy the auto triggered node, DataWorks generates an instance whose running is complete for the node. The instance is an expired instance that is generated in real time. The expired instance is not actually run.
      • If you commit and deploy an auto triggered node after 23:30, the immediate instance generation feature does not take effect. You can find the instance generated for the node on the third day.
  • Impacts on instance generation for an auto triggered node on the current day if you change the instance generation mode of the auto triggered node from Next Day to Immediately After Deployment: Instances that have been run are retained. Instances that are not run are replaced with the instances that are immediately generated.

Conditions for an auto triggered node to run and troubleshooting

An auto triggered node can be scheduled to run only when the following conditions are met: Ancestor nodes of the auto triggered node are successfully run, the scheduling time of the auto triggered node has arrived, scheduling resources are sufficient, and the auto triggered node is not frozen. For more information, see Prerequisites for an auto triggered node to successfully run. FlowchartIf an auto triggered node fails to run, you can use the upstream analysis feature provided in the DAG of the auto triggered node to quickly identify ancestor nodes that block the running of the auto triggered node. Then, you can use the intelligent diagnosis feature to diagnose failure causes or related issues of the ancestor nodes. The intelligent diagnosis feature can also be used to quickly troubleshoot issues when dependencies between the auto triggered node and its ancestor nodes are complex. This improves O&M efficiency.

Suspend scheduling

Important Do not perform operations on the projectname_root node, which is the root node of your workspace. All the instances of auto triggered nodes depend on this node. If this node is frozen, the instances of auto triggered nodes cannot be run.
Operation Scenario Description
Freeze an auto triggered node If an auto triggered node and its descendant node do not need to be run for a certain period of time in the future, you can freeze the auto triggered node.
  • Freeze operation: In automatic scheduling scenarios, an auto triggered node can start to run only after all ancestor nodes of the node are successfully run. If one of the ancestor nodes is frozen, the running of the auto triggered node is blocked. If an auto triggered node and its descendant node do not need to be run for a certain period of time in the future, you can freeze the auto triggered node.
  • Impact: After you freeze an auto triggered node, auto triggered node instances generated for the auto triggered node are also frozen, and descendant instances of the auto triggered node instances cannot be run.
Freeze an instance If an instance generated for an auto triggered node does not need to be run, you can freeze the instance. The freeze operation takes effect only on the current instance. Other instances that are generated on the same day as the current instance and the instances that are generated later than the current day are not affected.
Set Recurrence to Dry Run for an auto triggered node If an auto triggered node does not need to be run for a certain period of time, but you do not want the node to block the running of its descendant nodes, you can set the Recurrence parameter to Dry Run for the auto triggered node. The auto triggered node in the dry-run state generates dry-run instances. The dry-run instances have no run logs, do not consume time, and do not generate data.
Note The operation of setting the Recurrence parameter to Skip Execution for an auto triggered node on the DataStudio page achieves the same effect as the operation of freezing an auto triggered node in Operation Center. The modifications that you made on an auto triggered node on the DataStudio page take effect only in the development environment. If you want the modifications to take effect in the production environment, you must deploy the auto triggered node. For more information, see Create and deploy an auto triggered node.
Note For more information about the impacts that are imposed after you freeze or unfreeze an auto triggered node or auto triggered node instance, see Node freezing and unfreezing.

Undeploy auto triggered nodes

If an auto triggered node does not need to be run, you can undeploy the auto triggered node. If auto triggered nodes in a specified workflow do not need to be run, you can undeploy the auto triggered nodes in the workflow at the same time. To undeploy an auto triggered node, perform the following steps:
  1. Delete the auto triggered node on the DataStudio page. The operation takes effect only in the development environment. The node is undeployed from the development environment. A node undeployment record is generated on the Create Deploy Task page.
  2. Go to the Create Deploy Task page, find the node undeployment record, and then click Deploy in the Actions column to deploy the undeployment operation. If the operation is successful, the auto triggered node is undeployed from the production environment.
  3. Go to the Cycle Task page in Operation Center to check whether the auto triggered node that you deleted is no longer displayed in the list of auto triggered nodes.
Note
  • When you undeploy an auto triggered node, DataWorks checks whether other nodes depend on the auto triggered node in the development and production environments. An auto triggered node can be undeployed only if no other nodes depend on the node. For more information about FAQ related to the undeploy operation, see Node undeployment.
  • After an auto triggered node is undeployed, it is no longer displayed on the Cycle Task page. DataWorks no longer generates instances for the auto triggered node on the next day. However, the instances that have been generated for the auto triggered node are not directly deleted and can be retained for 30 days. After 30 days elapse, the system automatically delete the instances. You cannot manually delete the instances.
  • When you undeploy an auto triggered node, the following situations occur on the instances that are generated for the auto triggered node:
    • The instances become dry-run instances. The status of the instances is directly set to successful, but the code of the instances is not run. If one of the instances does not need to be run, you can freeze the instance. If you use the shared resource group for scheduling whose billing method is pay-as-you-go to schedule the auto triggered node, you are not charged for scheduling dry-run instances.
    • The instances that are running can continue to run normally.
  • For more information about how to undeploy multiple auto triggered nodes in a specified workflow at the same time, see Perform operations on multiple DataWorks objects at a time.

Recover undeployed auto triggered nodes

After you undeploy an auto triggered node, the auto triggered node is moved to the recycle bin. You can recover the auto triggered node from the recycle bin. For more information about how to recover an auto triggered node that is undeployed, see Create and deploy an auto triggered node.

Note After an auto triggered node is recovered from the recycle bin, a new node ID is generated but the other configurations remain unchanged.

Manage auto triggered nodes

In the list of auto triggered nodes on the Cycle Task page, you can perform the following operations:
  • Modify properties that are related to the running of nodes. For example, you can select multiple auto triggered nodes and click Modify Scheduling Resource Group, Modify Data Integration Resource Group, Freeze, or Unfreeze to perform the related operation on the selected nodes at the same time. For more information, see View and manage auto triggered nodes.
  • Configure custom alert rules to monitor the status of auto triggered node instances. For more information, see Manage custom alert rules.
  • Configure monitoring rules for table data that is generated when auto triggered node instances, data backfill instances, or test instances generated for auto triggered nodes are run. For more information, see Overview.
Note If you want to redefine scheduling properties of an auto triggered node, you can go to the DataStudio page, find the desired auto triggered node, and modify the configurations of the scheduling properties on the configuration tab of the auto triggered node. For more information, see Configure basic properties. You can also go to the batch operation tab to modify multiple nodes at the same time. For more information, see Perform operations on multiple DataWorks objects at a time.

View operation records of auto triggered nodes

Entry point Description
Operation Log tab of the DAG page of an auto triggered node or auto triggered node instance You can view the operation records of an auto triggered node or auto triggered node instance on the Operation Log tab.
Versions tab on the DataStudio page If no details about an operation that is performed on and deployed for an auto triggered node are recorded, you can go to the configuration tab of the auto triggered node and compare an existing version of the node with the version of the node in the production environment to obtain details about version changes. For more information, see Deploy nodes.
Operation History page in Operation Center You can go to the Operation History page in Operation Center to view the operation records of an auto triggered node, an auto triggered node instance, or a baseline. For more information, see View operation records.