Test an auto triggered task and view test instances generated for the task - DataWorks

A test instance is generated each time you test an auto triggered task. You can view the details of the test instance in the list of test instances or in the directed acyclic graph (DAG) of the test instance.

Background information

You can test an auto triggered task to check whether the auto triggered task can be run as expected in the production environment. A test instance is generated each time you test an auto triggered task. You can view the details of the test instance on the Test Instance page.

For more information about how to test an auto triggered task, see View and manage auto triggered nodes.
For more information about how to view the details of a test instance in the list of test instances, see Manage test instances in the instance list.
For more information about how to view the details of a test instance in the DAG of the test instance, see Manage test instances in the DAGs of the instances.

Note

The code of the auto triggered task is run during the test. Make sure that the test does not affect your business before you test the auto triggered task.

Limits

Limits on feature usage:
- Only users of DataWorks Professional Edition or a more advanced edition can use the intelligent diagnosis feature. If you use another edition, you can use the feature free of charge on a trial basis. However, we recommend that you upgrade the DataWorks service to DataWorks Professional Edition to use more features. For more information, see Intelligent diagnosis.
- Only users of DataWorks Professional Edition or a more advanced edition can use the aggregation, upstream analysis, and downstream analysis features provided by DAGs. For more information, see Billing of DataWorks advanced editions.
- Some features can be used only by users who are granted O&M permissions. If the entry point of a feature is dimmed or is not displayed, you can check whether you are granted the required O&M permissions on the Workspace Members tab of the Workspace page in SettingCenter. For more information, see Manage permissions on workspace-level services.
Limits on logs and instances:
- You cannot manually delete a test instance. DataWorks deletes a test instance about 30 days after the test instance expires. If you no longer need to run a test instance, you can freeze the instance.
- Test instances that are run on the shared resource group for scheduling are retained for one month (30 days), and the operation logs for the test instances are retained for one week (7 days).
- Test instances that are run on an exclusive resource group for scheduling are retained for one month (30 days), and the operation logs for the test instances are also retained for one month (30 days).
- The system regularly clears excess operation logs every day when the size of operation logs generated for the test instances that finish running exceeds 3 MB.

Test an auto triggered task

Go to the Operation Center page.
Log on to the DataWorks console. In the top navigation bar, select the desired region. In the left-side navigation pane, choose Data Development and Governance > Operation Center. On the page that appears, select the desired workspace from the drop-down list and click Go to Operation Center.
Test an auto triggered task.
In the left-side navigation pane of the Operation Center page, choose Auto Triggered Node Operation > Auto Triggered Nodes. On the Auto Triggered Nodes page, use one of the following methods to test an auto triggered task:
- Method 1: In the list of auto triggered tasks, find your desired auto triggered task and click Test in the Actions column. If the current page is not displayed in list mode, you can click the icon in the middle of the page. The current page is displayed in list mode.
- Method 2: In the DAG of your desired task, right-click the task and select Test. If the current page is not displayed in DAG mode, you can click DAG in the Actions column of your desired task to open the DAG of the task.

Configure parameters in the Test dialog box. 冒烟测试

Parameter	Description
Test Name	The name of the test instance.
Data Timestamp	The data timestamp of the test instance. Note If you select a date earlier than the previous day, the test instance is immediately run. If you select the previous day, the test instance is run when the scheduling time of the instance arrives.

Manage test instances in the instance list

You can view the status of and perform related operations on the test instances that are generated for an auto triggered task in the instance list.

Operation	Description
Filter	Allows you to specify conditions to search for test instances in the area marked with 1 in the preceding figure. You can search for test instances by node name or node ID. You can also specify the filter conditions to search for test instances: such as Node Type, Run At, Data Timestamp, Scheduling Resource Group, My Nodes, Tested by Me Today, and Frozen Nodes. Note By default, the data timestamp is set to the day before the current day.
List of test instances	Allows you to view the test instances that meet the specified conditions in the area marked with 2 in the preceding figure.
Actions	Allows you to perform operations on a test instance in the area marked with 3 in the preceding figure. DAG: You can perform this operation to view the dependencies of a test instance. In the DAG of the test instance, you can right-click the instance to perform related operations. For more information, see Appendix: Use the features provided in a DAG. To diagnose: You can perform this operation to carry out end-to-end analysis on a test instance. If the test instance is not run as expected, you can click To diagnose to troubleshoot issues. For more information, see Intelligent diagnosis. Rerun: You can perform this operation to rerun a test instance that is in the Succeeded or Failed state. After the test instance is successfully run, its descendant instances that are in the Pending (Ancestor) state can be scheduled to run. This operation is used to process a test instance that fails to run or a test instance that is not run as scheduled. Note Only the test instances that are in the Succeeded or Failed state can be rerun. More: Rerun Descendant Nodes: You can perform this operation to rerun the descendant instances of a test instance that is in the Succeeded or Failed state. You can select the descendant instance that you want to rerun. After the selected instance is successfully run, its descendant instances that are in the Pending (Ancestor) state can be scheduled to run. This operation is used to recover data. Note Only descendant instances in the Successful or Failed state can be selected. The value No appears in the Meet Rerun Condition column of instances in other states, and you cannot select the instances. Set Status to Successful: You can perform this operation to set the status of a test instance that fails to run to Succeeded. You can perform this operation if you do not want a test instance that fails to run to block the running of its descendant instances. This operation is used to process a test instance that fails to run. Stop: You can perform this operation to stop a test instance that does not need to be run. After you perform this operation on a test instance, the test instance fails to run and exits. Note Only test instances in the Pending (Schedule), Pending (Resources), or Running state can be stopped. Freeze: You can perform this operation if a test instance and its descendant instances do not need to be run for a certain period of time. For more information, see Manage auto triggered nodes. Note Do not perform this operation on the projectname_root node, which is the root node of your workspace. All instances of auto triggered tasks depend on this node. If this node is frozen, the instances of auto triggered tasks cannot be run. Unfreeze: You can perform this operation to unfreeze a test instance that is frozen. If the test instance is not run, it is automatically run after its ancestor instances are successfully run. If all the ancestor instances of the test instance are successfully run, the state of the test instance is directly set to Failed. You must manually rerun the test instance. Note The unfreeze operation takes effect only on the current test instance. If the auto triggered task for which the test instance is generated is frozen, instances that are scheduled to run on the next day are also frozen. View Lineage: You can perform this operation to view the lineage of a test instance. View Node Details: You can perform this operation to view the basic information of a test instance. View Runtime Log: After the auto triggered task for which the test instance is generated is started, you can perform this operation to view the execution details of the auto triggered task based on the operation logs. For more information about the core parameters in the operation logs, see Appendix: Parameters in operation logs.
Batch operations	Allows you to perform operations on multiple test instances at the same time. You can click the following buttons in the area marked with 4 in the preceding figure to perform operations on multiple test instances at the same time: Stop, Rerun, Set Status to Successful, Freeze, and Unfreeze.
Items per page	Allows you to select the number of items that you want to view on a single page in the area marked with 5 in the preceding figure. The maximum number that you can select is 200.

Manage test instances in the DAGs of the instances

In the list of test instances, you can click the name of a test instance or DAG in the Actions column of a test instance to open the DAG of the test instance. In the DAG, you can right-click the test instance to perform the related operations. DAG图

Functionality	Description
Operations that you can perform in the DAG of a test instance	You can click DAG in the Actions column of a test instance to open the DAG of the test instance. You can perform the following operations in the DAG of the test instance: Search: In the area marked with 1 in the preceding figure, you can search for a desired test instance by node name, node ID, instance ID, or instance name. DAG display style: In the area marked with 2 in the preceding figure, you can adjust the display style of a DAG based on your business requirements by clicking the icons. The icons from left to right are Update Node Status, Zoom In, Zoom Out, 1:1, and Fit Screen.
Operations on a single instance	You can right-click your desired test instance in the DAG and perform operations on the instance. View Code: You can perform this operation to view the code of the task for which the current test instance is generated in the production environment. If the code of the task does not meet your expectations, you must check whether the latest code of the task is successfully deployed to the production environment. Edit Node: You can perform this operation to go to the configuration tab of the task for which the current test instance is generated on the DataStudio page. Note For more information about other operations, see Manage test instances in the instance list.
Details about a single instance	In the area marked with 4 in the preceding figure, you can perform the following operations: View Logs: After the auto triggered task for which the test instance is generated is started, you can perform this operation to view the execution details of the auto triggered task based on the operation logs. For more information about the core parameters in the operation logs, see Appendix: Parameters in operation logs. Show Details: You can perform this operation to view detailed information about a test instance on the following tabs: General, Context, Runtime Log, Operation Log, and Code. For more information, see View the details about a test instance.

View the details about a test instance

In the DAG of a test instance, you can click the test instance, and then click Show Details in the pane that appears in the lower-right corner of the canvas. Then, you can view the details about the test instance on the following tabs: General, Context, Runtime Log, Operation Log, and Code.

查看实例详情

Tab	Description
General	On this tab, you can view the scheduling properties of a test instance in the production environment. For more information about the basic parameters, see Configure the resource property. Instance status interpretation: If the instance is in the Pending (Ancestor), Pending (Schedule), Pending (Resources), or Freeze state, you can follow the instructions that are described in What are the conditions that are required for a node to successfully run? to troubleshoot issues. Time spent for waiting for resources: If a test instance is in the Pending (Resources) state for a long period of time, you can use the intelligent diagnosis feature to identify the instances that occupy resources at the time when the current test instance is waiting for resources. Then, you can quickly identify the instances on which exceptions occur and troubleshoot issues. Long running duration: If the running duration of a test instance is much longer than the average running duration over a historical period of time, you can troubleshoot the issue based on task types: Non-synchronization task: You can consult the owner of the compute engine instance on which the test instance generated for a non-synchronization task is run. Batch synchronization task: The running speed of a test instance that is generated for a batch synchronization task may be slow in a specific phase or the instance is in the Pending (Resources) state for a long period of time. For more information, see What do I do if a batch synchronization node runs for an extended period of time? Rule-based monitoring: You can view information about the alert rule that is associated with the task for which the current instance is generated. If no alert rule is associated with the task, you can click Create on the right of the Rule Monitoring parameter to quickly create an alert rule to monitor the status of the task. For more information, see Create a custom alert rule. Note You can view only information about the alert rule associated with the task for which a test instance is generated on this tab. Information about monitoring rules used to monitor the data quality of the task for which a test instance is generated is not displayed on this tab. Baseline monitoring: You can view information about the baseline with which the task that generates the current instance is associated. If the task is not associated with baselines, you can click Create on the right of the Baseline monitoring parameter to quickly create a baseline. For more information, see Manage baselines.
Context	On this tab, you can view all input and output parameters of the task for which a test instance is generated. For more information, see Configure input and output parameters.
Runtime Log	On this tab, after the auto triggered task for which the test instance is generated is started, you can view the execution details of the auto triggered task based on the operation logs. For more information about the core parameters in the logs, see Appendix: Parameters in operation logs.
Operation Log	On this tab, you can view the operation records of a task or an instance, including the operation time, operator, and specific operations.
Code	On this tab, you can view the latest code of the task for which the current test instance is generated in the production environment. If the code of the task does not meet your expectations, you must check whether the latest code of the task is successfully deployed to the production environment. For more information, see Deploy nodes.

Troubleshoot instance running issues

If a test instance is not run as expected, you can use the intelligent diagnosis feature to identify the reason why the instance is not run as expected or identify the issue that occurs on the instance.