A scheduling cycle defines how often a node automatically runs in the production environment. Based on this cycle, the DataWorks scheduling system generates recurring instances and runs them according to node dependencies and the scheduled time of each instance.
Key concepts
Recurring instance: For each data timestamp, the scheduling system generates a runtime entity for an auto triggered task based on its scheduling configuration, such as running at 00:00 every day. This entity is a recurring instance. The node's runtime, status, and logs are associated with this instance.
Cross-cycle dependency: In DataWorks, nodes with different scheduling cycles can depend on each other. For example, a daily downstream node can depend on an hourly upstream node. The dependency between nodes is a dependency between their recurring instances. For more information, see Cross-cycle dependencies.
Dry-run: For nodes that do not run daily, such as weekly, monthly, or yearly nodes, the scheduling system generates a dry-run instance on non-scheduled days. When this instance reaches its scheduled time, its status immediately changes to "Success", but it does not run the code logic within the node. The main purpose of a dry-run is to resolve dependencies and ensure that downstream daily nodes can be triggered correctly.
The instance status is "Success", the running time is 0 seconds, and no logs are generated.
It does not use scheduling or computing resources.
It does not block the execution of downstream nodes. Even if an upstream node is a dry-run instance, downstream nodes run as scheduled after their running conditions are met.
Basic principles and scenarios
Conditions for scheduled execution
A recurring instance runs only after both of the following conditions are met. The conditions can be met in any order.
All its upstream instances have run successfully. This includes instances that completed as a dry-run.
The instance has reached its scheduled time.
Therefore, the configured scheduled time is only the expected scheduled time. The actual running time of a node is affected by factors such as the completion time of its upstream nodes, available computing resources, and the node's specific running conditions.
Workflow scheduling scenarios
In a workflow that consists of nodes A, B, and C (A→B→C), the scheduled time configuration affects the execution of the entire workflow:
Scenario 1: Unified start time | |
 | The entire workflow must start after 03:00. To achieve this, set the scheduled time of the start node A to |
Scenario 2: Different start times for each node | |
 | Node A is scheduled to run at 03:00, node B must not run until after 05:00, and node C must not run until after 06:00. Set the scheduled times of A, B, and C to |
Scenario 3: Only some nodes have specific start times | |
 | Node A is scheduled to run at 03:00, node B must not run until after 05:00, and node C has no specific time requirement. Set the scheduled time of A to |
Impact of updating scheduled times
After you modify the scheduled time of a node in Scheduling Settings and redeploy the node, the impact depends on the selected instance generation mode:
T+1 next day generation: After you select this option and deploy the node, the scheduled time of instances that have already been generated for the past two days (T and T-1), including completed and pending instances, is updated to the new time. Future instances that have not been generated will be generated based on the new time.
Immediate instance generation: If you select this option, new instances are generated immediately based on the new configuration. The scheduled time of historical instances remains unchanged.
Schedule type configuration
DataWorks supports six scheduling cycles: minute, hour, day, week, month, and year. On the node editing page in Data Studio, click Scheduling Settings on the right side, and configure the settings in the Scheduling time section.
The configuration method for the scheduled time depends on how the node is organized:
Workflow nodes: If a node is organized within a workflow, the scheduled time is configured at the workflow level. Individual nodes within the workflow cannot have their own scheduled time configurations. To modify the scheduled time, go to the schedule settings of the workflow.
Standalone nodes: If a node does not belong to any workflow, its scheduled time is configured independently.
Minute schedule
The minimum interval for a minute schedule is 1 minute.
Configuration example
Goal: The node runs every 30 minutes between 00:00 and 23:59 every day.
The cron expression is automatically generated based on the selected time and cannot be manually modified.
Scheduling details
The preceding configuration generates 48 recurring instances per day, with scheduled times of 00:00, 00:30, 01:00, ..., 23:30. The business date (bizdate) of each instance is the current day.
For more dependency scenarios related to minute schedules, see Minute schedule dependency scenarios.
Hour schedule
Notes
The time range follows the inclusive-inclusive principle. For example, if you configure a node to run every 1 hour between
00:00and03:00, the scheduling system generates 4 instances with scheduled times of00:00,01:00,02:00, and03:00.You can set a time range and interval, or directly specify multiple run time points.
Configuration example
Goal: The node runs every 6 hours between 00:00 and 23:59 every day.
The cron expression is automatically generated based on the selected time and cannot be manually modified.
Scheduling details
The scheduling system generates 4 instances per day, with scheduled times of 00:00, 06:00, 12:00, and 18:00.
For more dependency scenarios related to hour schedules, see Hour schedule dependency scenarios.
Day schedule
A daily node runs once at the specified scheduled time every day. When you create a scheduled task, the default scheduled time is randomly generated within the 00:00 to 00:30 range. You can modify it as needed.
Configuration example
Goal: The node runs once at 13:00 every day.
The cron expression is automatically generated based on the selected time and cannot be manually modified.
Scheduling details
The scheduling system generates one instance per day for this task, with a scheduled time of 13:00.
For more dependency scenarios related to day schedules, see Day schedule dependency scenarios.
Week schedule
On non-scheduled days, a weekly node triggers a dry-run to ensure that downstream dependencies can be executed properly. For more information, see Dry-run.
Configuration example
Goal: The task runs at the specified time every Monday and Friday. Instances generated on Monday and Friday are executed normally, while instances on other days are dry-run instances.
The cron expression is automatically generated based on the selected time and cannot be manually modified.
Scheduling details
The scheduling system automatically generates instances for the task and runs them.
When you use the backfill data feature, pay attention to the selected business date. In DataWorks, business date = scheduled run date - 1. For example, to backfill a weekly task that runs on Monday, the business date should be the previous Sunday. If you select a different date, the backfill instance will be a dry-run.
Month schedule
On non-scheduled days, a monthly node triggers a dry-run to ensure that downstream dependencies can be executed properly. For more information, see Dry-run.
The month schedule supports setting Specified time to Last day of every month.
Configuration example
Goal: The task runs at the specified time on the last day of every month. Instances generated on the last day of each month are executed normally, while instances on other days are dry-run instances.
The cron expression is automatically generated based on the selected time and cannot be manually modified.
Scheduling details
The scheduling system automatically generates instances for the task and runs them.
When you use the backfill data feature, pay attention to the selected business date. In DataWorks, business date = scheduled run date - 1. For example, to backfill a month-end task that runs on January 31, the business date should be January 30. If you select a different date, the backfill instance will be a dry-run.
Year schedule
On non-scheduled days, a yearly node triggers a dry-run to ensure that downstream dependencies can be executed properly. For more information, see Dry-run.
Configuration example
Goal: The task runs on the 1st and last day of January, April, July, and October every year. Instances generated on the specified dates are executed normally, while instances on other days are dry-run instances.
Scheduling details
The scheduling system automatically generates instances for the task and runs them.
For more dependency scenarios, see Cross-cycle dependencies.