Scheduling principles and examples for complex dependency scenarios in DataStudio - DataWorks

DataWorks allows you to create dependencies between auto triggered tasks that have different scheduling frequencies, such as minute, hour, day, week, month, and year. Each task execution is an instance, and the scheduling frequency determines how many instances are generated. This topic explains how DataWorks establishes dependencies between the instances of ancestor and descendant tasks when their scheduling frequencies differ.

Background

In DataWorks, an auto triggered task generates instances based on its scheduling frequency. For example, an hourly task generates a corresponding number of instances each day. Task dependencies are established between their instances. When ancestor and descendant tasks have different scheduling frequencies, the number of instances and how they depend on each other will vary.
DataWorks supports various scheduling dependency scenarios. You can choose to configure a same-cycle scheduling dependency or a cross-cycle scheduling dependency. For more information, see Configure a same-cycle scheduling dependency and Configure a cross-cycle scheduling dependency.
For cross-time-zone scenarios where a daily task depends on hourly or minute-based tasks, in addition to indirectly achieving this through self-dependency and the proximity principle, you can enable Advanced Scheduling Dependency Configuration to directly use the Specified Range dependency to configure the required dependency range (such as [-5, 19]), or use the Specified Set dependency to precisely configure dependencies on instances at fixed times, without needing to set self-dependency for the upstream hourly tasks. For more information, see Configure a same-cycle scheduling dependency.

Understand these concepts before configuring a scheduling dependency.

No.	Description	References
1	DataWorks supports multiple scheduling frequencies, including minute, hour, day, week, month, and year. When an ancestor task and a descendant task have different scheduling frequencies, DataWorks establishes the scheduling dependency using the principle of scheduling time proximity. Note If an hourly task depends on another hourly task and they generate the same number of instances per day, this principle does not apply. The dependency is not related to the scheduling time. By default, a daily task depends on all instances generated by its hourly or minute-based ancestor tasks on the same day. The daily task starts only after all data for that day from the ancestor tasks is processed.	Mount dependencies: proximity principle
2	A scheduling dependency establishes a data dependency between tasks. The descendant task will not run until its ancestor task completes successfully, regardless of the descendant's own scheduling time.	Impact of dependencies on task execution
3	You can further understand the proximity principle through the following scenario examples.	Appendix 1: Summary of complex dependency scenarios The scenarios include: Minute tasks depending on other tasks Hour tasks depending on other tasks Day tasks depending on other tasks Other tasks depending on weekly, monthly, or yearly tasks

Dependency resolution: The principle of scheduling time proximity

When an auto triggered task runs in DataWorks, multiple instances are generated. Descendant instances depend on ancestor instances. Unless a specific ancestor instance is designated, the descendant instance follows the proximity principle when establishing dependencies. This means the descendant instance depends on the ancestor instance whose scheduling time is the closest to but not later than the descendant's scheduling time and has not been claimed by another descendant instance. The dependency principles for different scenarios are as follows:

Note

If the descendant task's scheduling time is earlier than the ancestor task's scheduling time, the descendant task will not be scheduled even when its scheduled time arrives. It must wait until the ancestor task completes before it can be scheduled.
When dependencies are established based on the proximity principle, if no ancestor instance is scheduled earlier than the first instance of the descendant task on a given day, the descendant instance depends on the first ancestor instance of that day by default.

Scenario

Description

Diagram

Dependencies between hourly and minute-based tasks

Dependencies are related to the scheduling time of instances
- Default behavior: Descendant instances are assigned dependencies based on the proximity principle. DataWorks assigns each descendant instance to the ancestor instance whose scheduling time is earlier than or the same as the descendant's scheduling time and has not been claimed by another descendant instance.
  Note
  When the upstream node runs at a higher frequency than the downstream node (that is, the ancestor generates more instances than the descendant), the downstream instance actually depends on all ancestor instances produced within its scheduling interval (start, end].
- Special case: If both the ancestor and descendant nodes have self-dependency configured (that is, cross-cycle dependency on the node itself), the current descendant instance depends on only one ancestor instance whose scheduling time is earlier than or the same as the descendant's scheduling time.
  Note
  You can refer to the diagrams for detailed scenarios where hourly tasks and minute-based tasks are configured with self-dependencies.
Dependencies are not related to the scheduling time of instances.
When an hourly task depends on another hourly task, or a minute-based task depends on another minute-based task, and the ancestor and descendant nodes generate the same number of instances per day, the dependencies are established in a one-to-one mapping based on the instance sequence.

The following figure shows an overview of the detailed scenarios for dependencies between hourly and minute-based tasks:

Hourly task depends on hourly task

Hourly task depends on minute-based task

Minute task depends on an hourly task

Example:

An hourly task depends on another hourly task.

Scenario 1: Hourly task B depends on hourly task A.
Configuration:
- The time range is set to 00:00-23:59 each day.
- Both task A and task B are scheduled every 5 hours.
- Task A and task B generate the same number of instances per day.
Dependency: Task A and task B generate the same number of instances per day, so the instance dependencies are mapped one to one.
Scenario 2:
Configuration:
- Task A is scheduled every 4 hours during the 06:10-21:59 time range each day, generating 4 instances in total.
- Task B is scheduled every 4 hours during the 08:00-23:59 time range each day, generating 4 instances in total.
Dependency: Task A and task B generate the same number of instances per day, so the instance dependencies are mapped one to one.

Daily task depends on an hourly or minute task

Default behavior: By default, a daily task depends on all instances generated by its hourly or minute-based ancestor tasks on the same day. The daily task starts only after all data for that day from the ancestor tasks is processed.

Other cases: If the daily task only needs to depend on the hourly or minute instance closest to its own scheduling time, you can configure self-dependency for the hourly or minute task. After this configuration, the daily task starts as soon as that hourly or minute instance completes.

For details about the dependencies and execution behavior in various scheduling scenarios, see Appendix 1: Summary of complex dependency scenarios.

How dependencies affect task execution

Once a descendant task has a dependency configured, the task will not run at its scheduled time if the ancestor task has not completed successfully.

For example, hourly task B depends on daily task A, and hourly task B does not have self-dependency configured.

Daily task A: The scheduled time is set to 07:00.
Hourly task B: The scheduled times are set to 00:00, 08:00, and 16:00.

If daily task A has not completed, hourly task B will not run even when its scheduled time 00:00 arrives. The earliest actual run time of descendant task B is 07:00.

Appendix 1: Summary of complex dependency scenarios

If a descendant task's scheduled time is earlier than that of its ancestor task, the descendant task will not be dispatched even when its scheduled time arrives. It must wait until the ancestor task completes. For the first instance of the descendant task on a given day, if the ancestor task has no instance scheduled earlier, the descendant task depends on the ancestor task's first instance of that day by default.

Hourly task depends on other tasks

Scenario	Dependency and execution details	Diagram
Hourly task depends on an hourly task	Ancestor and descendant have the same number of instances The instances generated by the ancestor and descendant tasks on the same day are mapped one to one. The first descendant instance depends on the first ancestor instance, the second descendant instance depends on the second ancestor instance, and so on. Ancestor and descendant have different numbers of instances The instances generated by the ancestor and descendant tasks on the same day are mapped based on the principle of scheduling time proximity. Each descendant instance depends on the ancestor instance whose scheduled time is closest to (at or before) the descendant instance's own scheduled time. Special case: In a scheduling dependency configuration where the ancestor task runs more frequently than the descendant task (the "more-ancestor-fewer-descendant" scenario), the dependency logic for descendant instances follows the interval coverage principle rather than simple proximity. Dependency scope: A descendant instance depends on all ancestor instances produced within its cycle interval `(start, end]`. `start` refers to the scheduled time of the previous descendant instance. `end` refers to the scheduled time of the current descendant instance.
Hourly task depends on a daily task	Hourly task without self-dependency All instances generated by the descendant hourly task on the current day depend on the ancestor daily task. The descendant hourly instances start running only after the daily task completes. At that point, all hourly instances whose scheduled times have already arrived run concurrently. Note If you do not want concurrent execution, configure a self-dependency for the downstream hourly task. All periodic instances generated by the downstream hourly task run independently of each other. Hourly task with self-dependency Only the first hourly instance depends on the ancestor daily task. Each subsequent hourly instance depends on the hourly instance from the previous cycle. The current hourly instance starts running only after the daily task and the previous-cycle hourly instance both complete. Even if the scheduled time has arrived, hourly instances do not run concurrently. Note A self-dependency creates a cross-day dependency. If the last periodic instance from the previous day has not completed, today's task cannot be scheduled.
Hourly task depends on a minute task	Minute task without self-dependency The descendant hourly task depends on all minute-level instances generated by the ancestor minute task within that hour. Both the minute task and the hourly task have self-dependency The descendant hourly task depends on the ancestor minute-level instance whose scheduled time is closest to (at or before) the descendant's own scheduled time within that hour.

Daily task depends on other tasks

Scenario	Dependency and execution	Diagram
Daily task depends on a same-cycle daily task	Ancestor daily task without self-dependency By default, the descendant daily task instance depends on the ancestor daily task instance of the same cycle. Ancestor daily task with self-dependency When the ancestor daily task has a self-dependency configured, a cross-cycle dependency exists between the descendant daily task and the ancestor daily task. Note A self-dependency creates a cross-day dependency. If the last periodic instance from the previous day has not completed, today's task cannot be scheduled.
Daily task depends on hourly tasks of the current day	Hourly task without self-dependency The descendant daily task depends on all hourly instances generated by the ancestor hourly task on the current day. The daily task starts processing only after all hourly data for the day has been processed. Note If the daily task only needs to depend on a specific hourly instance, you can configure a self-dependency for the hourly task. After that specific hourly instance runs successfully, the daily task is automatically scheduled. Hourly task with self-dependency The descendant daily task depends on the ancestor hourly instance whose scheduled time is closest to (at or before) the descendant's own scheduled time.
Daily task depends on hourly or minute tasks of the previous day	Hourly or minute task without self-dependency The descendant daily task depends on all hourly or minute instances generated by the ancestor task on the previous day. Hourly or minute task with self-dependency The descendant daily task depends on the last hourly or minute instance generated by the ancestor task on the previous day.	The following example uses a daily task that depends on an hourly task of the previous day.

Minute task depends on other tasks

Scenario	Dependency description	Diagram
Minute task depends on an hour task	Minute task without self-dependency Each instance of the descendant minute task depends on the nearest hourly instance whose scheduling time is earlier than or equal to its own scheduling time, without duplicating dependencies. Note In this scenario, multiple downstream minute-based periodic instances depend on the same upstream hourly periodic instance. Both minute and hour tasks have self-dependency configured Each instance of the descendant minute task depends on its own previous instance and, based on the principle of scheduling time proximity, depends on the nearest ancestor hourly instance whose scheduling time is earlier than or equal to its own scheduling time.
Minute task depends on a daily task	Minute task without self-dependency All minute instances generated by the descendant minute task on a given day depend on the daily task. The minute task can run only after the daily task reaches its scheduling time and completes execution. Note After the daily task completes, minute instances whose scheduling times have already arrived will run concurrently. To prevent concurrent execution, configure self-dependency for the minute task. The execution of different instances of the minute task does not affect each other. Minute task with self-dependency configured Only the first instance generated by the descendant minute task depends on the daily task. All other instances depend directly on their own previous instance. A minute instance starts scheduling only after both the daily task and the previous minute instance complete execution. In this case, minute instances will not run concurrently even if their scheduling times have already arrived. Note A self-dependency creates a cross-day dependency. If the last periodic instance from the previous day has not completed, today's task cannot be scheduled.

Other tasks depending on weekly, monthly, or yearly tasks

When daily, hourly, or minute tasks depend on weekly, monthly, or yearly tasks, the weekly, monthly, or yearly tasks generate dry-run instances on non-scheduling days. These instances do not actually process data, do not consume resources, and do not block the execution of descendant tasks.

The following example uses the scenario of a daily task depending on a weekly task without self-dependency:

Ancestor weekly task: Runs normally on Monday and Friday. On Tuesday, Wednesday, Thursday, Saturday, and Sunday, dry-run instances are generated and automatically set to a success status at the scheduling time without executing code logic. Dry-run instances do not affect the normal execution of descendant instances.
Descendant daily task: Generates a scheduled instance every day and depends on the instance generated by the ancestor weekly task each day (including dry-run instances). After the ancestor weekly task's instance for the day completes successfully, the corresponding descendant instance is triggered to run.

Appendix 2: Workflow scheduling

DataWorks allows you to add ancestor and descendant nodes to a workflow as a whole. For details about workflow scheduling, see Workflow scheduling.

Appendix 3: Cross-timezone global data aggregation (specified range dependency)

When a data warehouse in China centrally processes data from multiple regions worldwide, hourly ancestor tasks in each region run based on local time, while the descendant daily task aggregates daily data based on the China timezone. Due to timezone differences, the descendant daily task must use a specified range dependency to depend on all instances of hourly ancestor tasks in each region within a specific time window.

Scenario example (data timestamp: 2026-01-22)

Region	Time difference from China	Dependency type	Offset configuration	Actual dependent instances
India	2.5 hours behind	Specified range	[-3, 21]	01-21 21:00 ~ 01-22 21:00 (24 instances)
Saudi Arabia	5 hours behind	Specified range	[-5, 19]	01-21 19:00 ~ 01-22 19:00 (24 instances)

Cross-day dependency behavior in backfill scenarios

The cross-day dependency behavior of instances generated by backfill tasks differs from that of regular scheduling instances.

1. Cross-day dependencies configured by specified range or specified set

When you run a backfill task, cross-day dependencies configured by a specified range or specified set do not take effect.

Specific behavior: When backfill instances are generated for multiple business dates, the system automatically ignores all dependencies that cross business date boundaries during dependency resolution. This means that a backfill instance only mounts upstream instances from the same business date, and does not mount upstream instances that point to the previous day or the next day as defined in the configuration.
Example: A daily task is configured with a range dependency on an upstream hourly task [-4h, 0] (depending on the 4 instances before 00:00 of the current day). In regular scheduling, the instance on May 11 depends on the instances from May 10, 20:00-23:00. However, when you backfill data for May 11, this cross-day dependency is ignored, and the instance will not mount any upstream instances.

2. Default cross-day dependency in backfill (serialization mechanism)

By default, when you run a non-grouped batch backfill, an implicit serial dependency exists between instances of different business dates.

Specific behavior: All backfill instances for business date D wait until all backfill instances for business date D-1 complete successfully before they start running. This means that if any instance on date D-1 fails, none of the instances on date D will be scheduled.
Exception: If you select grouped execution for the backfill, this serialization mechanism does not take effect. Task instances within each execution group run independently and are not affected by the status of instances in other groups.