Other frequently asked questions about Operation Center tasks, instances, resource groups, and scheduling behavior.
Tasks and instances
What is the relationship between auto triggered tasks, recurring instances, data backfill instances, and test instances?
Every night, DataWorks automatically generates and schedules the next day's recurring instances based on auto triggered tasks.
Beyond recurring instances, you can manually spawn two other instance types from an auto triggered task:
-
Data backfill instances — run a task over a historical time range to fill in missing or corrected data.
-
Test instances — run a task on demand to validate logic without affecting production scheduling.
When you create a node, you can choose to generate an instance immediately after publishing. After publishing, the instance appears in the recurring instances list in Operation Center.
Resource groups
How long are logs and instances retained?
Retention periods differ between shared and exclusive scheduling resource groups:
|
Resource group type |
Instance retention |
Log retention |
|
Shared |
30 days |
7 days |
|
Exclusive |
30 days |
30 days |
Two additional limits apply regardless of resource group type:
-
Logs for completed instances larger than 3 MB are cleared daily.
-
Instances cannot be manually deleted after they are generated. To stop a task from running, freeze the auto triggered task or the specific instance. Freezing a task and freezing an instance have different effects — see Task freezing and unfreezing for details.
Why can't I run big data computations on scheduling resource groups?
Scheduling resource groups are designed primarily for scheduling tasks. Because they have limited resources, they are not suitable for computation tasks. Use MaxCompute for big data workloads — MaxCompute is built for massive-scale data processing and scales independently from your scheduling infrastructure.
Other
How do I set task priority?
Task priority in DataWorks is set at the baseline level. Adjust the baseline priority to change the priority of daily instances for an auto triggered task. Valid values are 1, 3, 5, 7, and 8 — higher values receive more scheduling resources when DataWorks allocates scheduling resources.
To change the priority:
-
Go to Operation Center.
-
In the left navigation pane, choose Intelligent Monitoring > Intelligent Baseline.
-
Click Edit next to the baseline.
-
In the Modify Baseline dialog box, set Priority.
-
Click Finish.
How do I view task priority?
In Operation Center, open the Instance Properties page for the instance. The priority value is displayed there.