A scheduling cycle defines the frequency at which a node automatically runs in the production environment. The DataWorks scheduling system generates scheduled instances based on the configured scheduling cycle. These instances are triggered to run based on their scheduled times and node dependencies.
Key concepts
Scheduled instance: The scheduling system generates a runtime entity, called a scheduled instance, for each business date based on the scheduling configuration of a scheduled task, such as a task that runs at 00:00 every day. The task's execution, status, and logs are all associated with this instance.
Cross-cycle dependency: DataWorks supports dependencies between nodes that have different scheduling cycles. For example, a daily downstream node can depend on an hourly upstream node. The dependency between nodes is essentially a dependency between their scheduled instances. For more information, see Dependencies between nodes with different scheduling cycles.
Dry run: For tasks that are not scheduled daily, such as weekly, monthly, or yearly tasks, the scheduling system generates a dry-run instance on non-scheduled dates. When this instance reaches its scheduled time, its status immediately changes to "Success" but it does not execute the code logic within the node. The main purpose of a dry run is to resolve dependencies and ensure that downstream daily nodes can be triggered correctly.
The instance status is "Success", the execution duration is 0 seconds, and no execution logs are generated.
It does not consume scheduling or computing resources.
It does not block the execution of downstream nodes. Even if an upstream node is a dry run, downstream nodes will run as scheduled after their running conditions are met.
Basic principles and scenarios
Scheduling execution conditions
A scheduled instance runs only when both of the following conditions are met. The order does not matter:
All its upstream instances have run successfully, including instances that successfully completed a dry run.
The instance has reached its own scheduled time.
Therefore, the configured scheduled time is only an expected start time. The actual execution time of a node is affected by factors such as the completion time of its upstream nodes, the availability of runtime resources, and the node's specific execution conditions.
Workflow scheduling scenarios
In a workflow that consists of nodes A, B, and C (A→B→C), the configuration of scheduled times affects the execution of the entire flow:
Scenario 1: Unified start time | |
 | If the entire business flow must start after 03:00, set the scheduled time of the start node A to |
Scenario 2: Different start times for each node | |
 | Node A is scheduled to run at 03:00, node B must run after 05:00, and node C must run after 06:00. Set the scheduled times for A, B, and C to |
Scenario 3: Specific start times for some nodes | |
 | Node A is scheduled to run at 03:00, node B must run after 05:00, and node C has no specific time requirement. Set the scheduled time of A to |
Impact of updating the scheduled time
After you modify the scheduled time of a node in Scheduling Settings and redeploy the node, the impact depends on the selected instance generation method:
T+1 next day generation: If you select this option and deploy the node, the scheduled times of instances that have already been generated for the last two days (T and T-1), including completed and pending instances, are updated to the new time. Future instances that have not yet been generated will be generated based on the new time.
Immediate instance generation: If you select this option, new instances are immediately generated based on the new configuration. The scheduled times of historical instances remain unchanged.
Schedule type configuration
DataWorks supports six scheduling cycles: minute, hour, day, week, month, and year. On the node editing page in Data Studio, click Scheduling Settings on the right side and configure the settings in the Scheduling time section.
How you configure the scheduled time depends on how the node is organized:
Workflow nodes: If a node is organized within a workflow, the scheduled time is configured at the workflow level. Individual nodes within the workflow cannot have their own scheduled times. To modify the scheduled time, go to the schedule settings of the workflow.
Standalone nodes: If a node does not belong to any workflow, the scheduled time is configured independently on the node itself.
Minute-level schedule
The minimum interval for a minute-level schedule is 1 minute.
Configuration example
Goal: The node is scheduled to run every 30 minutes during the time period from 00:00 to 23:59 every day.
The cron expression is automatically generated based on the selected time and cannot be manually modified.
Scheduling details
The preceding configuration generates 48 scheduled instances per day, with scheduled times of 00:00, 00:30, 01:00, ..., 23:30. The business date (bizdate) of each instance is the current day.
For more dependency scenarios related to minute-level schedules, see Minute-level schedule dependency scenarios.
Hourly schedule
Notes
The time range follows the inclusive-inclusive principle. For example, if a node is configured to run every 1 hour during the
00:00to03:00time period, the scheduling system generates 4 instances with scheduled times of00:00,01:00,02:00, and03:00.You can set a time range and interval, or directly specify multiple run times.
Configuration example
Goal: The node is scheduled to run every 6 hours during the time period from 00:00 to 23:59 every day.
The cron expression is automatically generated based on the selected time and cannot be manually modified.
Scheduling details
The scheduling system generates 4 instances per day, with scheduled times of 00:00, 06:00, 12:00, and 18:00.
For more dependency scenarios related to hourly schedules, see Hourly schedule dependency scenarios.
Daily schedule
A daily-scheduled node runs once per day at the specified scheduled time. When you create a scheduled task, the default scheduled time is randomly generated within the 00:00 to 00:30 time range. You can modify it as needed.
Configuration example
Goal: The node runs once every day at 13:00.
The cron expression is automatically generated based on the selected time and cannot be manually modified.
Scheduling details
The scheduling system generates one instance per day for this task, with the scheduled time set to 13:00 on the current day.
For more dependency scenarios related to daily schedules, see Daily schedule dependency scenarios.
Weekly schedule
On non-scheduled dates, a weekly-scheduled node triggers a dry run to ensure that downstream dependencies can be executed properly. For more information, see Dry run.
Configuration example
Goal: The task runs at the specified time every Monday and Friday. Instances generated on Mondays and Fridays are executed as scheduled, while instances on other days are dry runs.
The cron expression is automatically generated based on the selected time and cannot be manually modified.
Scheduling details
The scheduling system automatically generates and runs instances for the task.
When you use the backfill data feature, pay attention to the selected business date. In DataWorks, business date = scheduled run date - 1. For example, to backfill data for a weekly-scheduled task that runs on Monday, set the business date to the previous Sunday. If you select a different date, the backfill instance will be a dry run.
Monthly schedule
On non-scheduled dates, a monthly-scheduled node triggers a dry run to ensure that downstream dependencies can be executed properly. For more information, see Dry run.
Monthly schedules support setting Specified time to Last day of every month.
Configuration example
Goal: The task runs at the specified time on the last day of every month. The instance generated on the last day of each month is executed as scheduled, while instances on other days are dry runs.
The cron expression is automatically generated based on the selected time and cannot be manually modified.
Scheduling details
The scheduling system automatically generates and runs instances for the task.
When you use the backfill data feature, pay attention to the selected business date. In DataWorks, business date = scheduled run date - 1. For example, to backfill data for a month-end task that runs on January 31, set the business date to January 30. If you select a different date, the backfill instance will be a dry run.
Yearly schedule
On non-scheduled dates, a yearly-scheduled node triggers a dry run to ensure that downstream dependencies can be executed properly. For more information, see Dry run.
Configuration example
Goal: The task runs on the 1st and the last day of January, April, July, and October every year. Instances generated on the specified dates are executed as scheduled, while instances on other days are dry runs.
Scheduling details
The scheduling system automatically generates and runs instances for the task.
For more dependency scenarios, see Dependencies between nodes with different scheduling cycles.