Auto triggered tasks are automatically run by the scheduling system at scheduled times. On the Auto Triggered Task page in the Operation Center, you can view and manage the scheduled tasks in your workspace. You can also view the configuration details of tasks in the production environment.
Background information
In DataWorks, auto triggered tasks are run automatically based on a schedule. You can view all scheduled tasks on the auto triggered task management page. You can also run tasks manually using the data backfill and test features to generate instances for specific data processing or debugging. Tasks configured for automatic scheduling create and run new recurring instances when specified conditions are met. For example, recurring instances for the next day are generated at 23:30 every night.
Changes to an auto triggered task in the production environment affect instances that are generated after the change. Most modifications take effect at the next scheduled time. For example, today's recurring instances were generated at 23:30 last night. Data backfill instances, however, are generated immediately based on the current task configuration and are run when manually triggered. For more information about Operations and Maintenance (O&M) operations such as creating, publishing, running, pausing, and unpublishing tasks, see Basic O&M operations for auto triggered tasks.
Limits
Only users of DataWorks Professional Edition or higher can use the node aggregation, upstream analysis, and downstream analysis features in the Directed Acyclic Graph (DAG). For more information about how to upgrade your edition, see Select and pay for a software version.
Notes
Do not perform operations on the
WorkspaceName_rootnode. This is the root node of the workspace. All instances of auto triggered tasks depend on this node. If you freeze this node, the instances will not run.Changes to a task must be published to take effect in the production environment. A publish operation may fail, the publishing flow may be blocked, or the published version may not be what you expect. After you publish a task, go to the page and check the following items:
In the DAG, check whether the upstream and downstream dependencies of the task are correct.
In the node details, check whether the parameter settings of the task in the production environment are as expected.
Go to the Auto Triggered Task page
Log on to the DataWorks console. In the top navigation bar, select the desired region. In the left-side navigation pane, choose . On the page that appears, select the desired workspace from the drop-down list and click Go to Operation Center.
In the Operation Center, click in the navigation pane on the left. The Auto Triggered Task page appears, where you can view the task list or the DAG.
View auto triggered tasks
The auto triggered task list displays the tasks that have been submitted to the scheduling system. You can use the intelligent search and filter by condition features to find specific tasks.
Intelligent search mode
The intelligent search feature for auto triggered tasks lets you enter search queries. The system automatically parses your query and quickly filters the task list.
Perform an intelligent search.
Click the Intelligent Search button in the filter box. In the dialog box that appears, enter your search query, such as
sort by task type, and press Enter. The system automatically finds and displays matching tasks.Save a new view.
To use the search conditions again, click in the search bar. In the Save View dialog box, enter a custom View Name and click Save. You can then find and use this new view in the view search bar to search for tasks.
NoteIf you no longer need the new view, find its name in the view search bar. Hover over the view name, click the ... button on the right, and select Edit or Delete.
Close intelligent search.
If you want to perform an exact search by filtering conditions, press the Esc key or click the Close Intelligent Search button in the task operations bar to exit intelligent search mode.
Filter by condition mode
You can use the filter by condition feature to precisely filter the task list based on specific criteria.
Simple filtering.
In the toolbar, you can select multiple filter conditions, such as Task Name/ID, Task Type, Expires in 180 Days, and Referenced, to filter the task list.
Complex filtering.
Click the Filter button in the filter box. You can then combine multiple conditions, such as Task Name/ID, Engine Type, Baseline, and Priority, to precisely find the tasks you need.
Paused (frozen) node: A frozen task is not automatically scheduled and blocks its downstream tasks. A paused (frozen) auto triggered task generates frozen instances. These instances are not scheduled to run and will block their downstream tasks.
Orphaned node: A task that does not depend on any parent nodes. This type of task does not run automatically. If an orphaned node has many downstream dependencies, the execution of those downstream tasks is affected. For more information, see Scenario: Orphaned node.
Expired node: A node that is outside its validity period does not generate instances and will not run.
Manage auto triggered tasks
Manage a single task
To manage a single auto triggered task, find the task in the list and use the options in the Actions column.
Feature | Description | |
DAG | The DAG shows the upstream and downstream dependencies of the auto triggered task. You can perform operations on the task in the DAG. For more information, see View the DAG of an auto triggered task. | |
Test | Tests whether the auto triggered task runs as expected. After you perform a test, a test instance is generated. You can view the running status of the test task on the Test Instance page. For more information, see Run a test and view the test instance. Note A test run also generates business data. Proceed with caution. | |
Data Backfill | Backfills historical data or data for a future period for an auto triggered task. After you backfill data, a data backfill instance is generated. You can view the running status of the data backfill task on the Data Backfill Instance page. For more information, see Data backfill instance O&M. | |
More | Pause (Freeze) | If a task and its downstream tasks do not need to run for a period of time, you can pause (freeze) the task. An auto triggered task that is frozen generates frozen instances. Frozen instances do not run and block their downstream tasks. Note Do not perform operations on the |
Resume (Unfreeze) | Resumes a frozen auto triggered task. | |
Change Owner | Changes the owner of the current task. | |
Change Scheduling Resource Group | Changes the scheduling resource group used to run the scheduled task. | |
Modify Tags | Modifies, adds, or detaches tags for the scheduled task asset. To manage tags in a centralized manner, go to Tag Management. | |
Configure Data Quality Monitoring | Configures data quality monitoring rules for the current task to verify data when the task runs. For more information, see Data Quality overview. | |
View Instances | Click to go to the Recurring Instance page for the current task. | |
View Data Lineage | Views the data lineage information of the current task. | |
View Auto Triggered Task Details | Views the basic properties of the task. For more information about the parameters, see View node details. | |
Add Monitoring Rule | Customizes alert rules to help you monitor the daily running status of auto triggered tasks. After the configuration is complete, you can view the added monitoring rule on the page. For more information, see Rule management. | |
Add to Baseline Monitoring | Adds the current task to a baseline. For more information about how to use baselines, see Intelligent baseline overview. | |
Manage tasks in batches
To manage multiple auto triggered tasks at once, select the tasks in the list. You can then perform batch operations, such as Pause (Freeze) and Resume (Unfreeze), using the options below the list.
View the DAG of an auto triggered task
Click DAG in the Actions column of an auto triggered task to open the DAG details page.
DAG panel features
On the DAG details page, you can use the following features in the DAG panel to aggregate nodes, analyze downstream dependencies, and adjust the display style of the DAG.
Feature | Description | |
| Click the icons in the upper-left corner to aggregate node information by the following dimensions:
| |
| If there are too many auto triggered tasks or levels, use the Downstream Analysis feature to count the number of downstream tasks affected by the current task. | |
| Click the icons in the upper-right corner to adjust the display style of the DAG. | |
DAG operations
On the DAG details page, you can right-click a node in the flow to view its upstream and downstream relationships, code details, and other information. The available operations are as follows:
Expand Parent Nodes: Expands the upstream tasks of the current task by level, up to six levels. This feature helps you understand which tasks affect the data output of the current task.
Expand Child Nodes: Expands the downstream tasks of the current task by level, up to six levels. This feature helps you understand which tasks are affected by the data output of the current task.
View Code: View the code details of the current task in the production environment. If the code is not as expected, confirm that the latest version of the task has been successfully published. For more information about how to publish a production task, see Publish a task.
Edit Node: Opens the current task on the Data Studio page.
For more information about other operations, see Manage auto triggered tasks.
View node details
You can view task properties and expand the details to see detailed properties, context parameters, operation logs, and code. This lets you verify that the scheduling parameters, resource group, and code in the production environment are as expected. The procedure is as follows:
On the DAG details page, click a node in the flow. In the node information panel that appears, click Expand Details to view related information in the tabs below.
Feature | Description |
Properties | View the scheduling properties of the task in the production environment. For more information about the UI parameters, see Scheduling Configuration. Note If the parameter settings on the Data Studio page are inconsistent with those on the Auto Triggered Task page in Operation Center, first confirm whether the task changes were successfully published.
|
Operation Logs | Views the operation records for the task, including the operation time, operator, and specific operation. |
Code | Views the latest code of the current task in the production environment. If the code is not as expected, confirm that the latest version of the task has been successfully published. For more information, see Publish a task. |