DataWorks allows you to create dependencies between auto triggered tasks that have different scheduling frequencies, such as minute, hour, day, week, month, and year. Each task execution is an instance, and the scheduling frequency determines how many instances are generated. This topic explains how DataWorks establishes dependencies between the instances of ancestor and descendant tasks when their scheduling frequencies differ.
Background
In DataWorks, an auto triggered task generates instances based on its scheduling frequency. For example, an hourly task generates a corresponding number of instances each day. Task dependencies are established between their instances. When ancestor and descendant tasks have different scheduling frequencies, the number of instances and how they depend on each other will vary.
DataWorks supports various scheduling dependency scenarios. You can choose to configure a same-cycle scheduling dependency or a cross-cycle scheduling dependency. For more information, see Configure a same-cycle scheduling dependency and Configure a cross-cycle scheduling dependency.
Understand these concepts before configuring a scheduling dependency.
No. | Description | References |
1 | DataWorks supports multiple scheduling frequencies, including minute, hour, day, week, month, and year. When an ancestor task and a descendant task have different scheduling frequencies, DataWorks establishes the scheduling dependency using the principle of scheduling time proximity.
Note If an hourly task depends on another hourly task and they generate the same number of instances per day, this principle does not apply. The dependency is not related to the scheduling time. By default, a daily task depends on all instances generated by its hourly or minute-based ancestor tasks on the same day. The daily task starts only after all data for that day from the ancestor tasks is processed.
| Dependency resolution: The principle of scheduling time proximity |
2 | A scheduling dependency establishes a data dependency between tasks. The descendant task will not run until its ancestor task completes successfully, regardless of the descendant's own scheduling time. | How dependencies affect task execution |
3 | You can review the examples in the appendix to better understand the principle of scheduling time proximity. | Appendix 1: Complex dependency scenarios The scenarios include: |
Dependency resolution: The proximity principle
When an auto triggered task runs, it generates one or more instances. A descendant instance depends on an ancestor instance. By default, the system resolves dependencies using the principle of scheduling time proximity: a descendant instance depends on the nearest available ancestor instance that is not already a dependency for another instance. The following principles apply in different scenarios:
Note A descendant task scheduled to run before its ancestor task will not be triggered until the ancestor completes.
When applying the principle of scheduling time proximity, if the ancestor task has no instances scheduled before the descendant's first instance of the day, the descendant's first instance defaults to depending on the ancestor's first instance of the day.
Scenario | Description | Diagram |
Dependencies between hourly and minute-based tasks | Dependency is based on the scheduling time of instances Default behavior: A descendant instance depends on the nearest ancestor instance scheduled at or before its scheduling time, provided that the ancestor instance is not already a dependency for another instance.
Note When an ancestor node has a higher scheduling frequency than the descendant node (a many-to-one scenario), a descendant instance depends on all ancestor instances generated within its time window (start, end]. Special case: If a self-dependency is configured for both the ancestor and descendant tasks, the current descendant instance depends on a single ancestor instance that is scheduled at or before its own scheduling time.
Note For a visual explanation, see the diagrams for the scenario where both hourly and minute-based tasks have a self-dependency configured.
Dependency is not based on the scheduling time of instances. If two tasks of the same frequency (such as hourly to hourly) generate the same number of instances per day, a one-to-one dependency is created between their corresponding instances.
| The following diagrams show the sub-scenarios for dependencies between hourly and minute-based tasks. An hourly task depends on another hourly task An hourly task depends on a minute-based task A minute-based task depends on an hourly task Example: An hourly task depends on another hourly task.  Scenario 1: Hourly node B depends on hourly node A. Configuration: Daily scheduling is set from 00:00 to 23:59. Nodes A and B are both scheduled every 5 hours. Nodes A and B generate the same number of instances each day.
Dependency relationship: Because nodes A and B generate the same number of instances per day, a one-to-one dependency is established between their corresponding instances. Scenario 2: Configuration: Node A is scheduled every 4 hours from 06:10 to 21:59 daily, generating a total of four instances. Node B is scheduled every 4 hours from 08:00 to 23:59 daily, generating a total of four instances.
Dependency relationship: Because nodes A and B generate the same number of instances per day, a one-to-one dependency is established between their corresponding instances.
|
A daily task depends on an hourly or minute-based task | Default behavior: A daily task depends on all instances generated by its hourly or minute-based ancestor tasks for that day. The daily task starts only after all those ancestor instances complete. Other cases: If the daily task only needs to depend on the most recent instance from the hourly or minute-based task, you can configure a self-dependency for the ancestor task. Once that specific instance of the ancestor task completes, the daily task will start. |  |
For more details on dependencies and execution behavior in each scenario, see Appendix 1: Complex dependency scenarios.
How dependencies affect task execution
A descendant task with a configured dependency will not execute until its ancestor task runs successfully, regardless of its own scheduling time.
For example, an hourly task B depends on a daily task A. No self-dependency is configured for the hourly task.
Task A (daily): The scheduling time is set to 07:00.
Task B (hourly): The scheduling times are 00:00, 08:00, and 16:00.
If task A has not completed, task B will not run, even at its scheduled time of 00:00. The earliest that task B can actually run is 07:00.
Appendix 1: Complex dependency scenarios
A descendant task scheduled before its ancestor task will not trigger until the ancestor completes. For the first descendant instance of the day, if no ancestor instance is scheduled before it, it will default to depending on the first ancestor instance of that day.
An hourly task depends on other tasks
Sub-scenario | Dependency and execution | Diagram |
An hourly task depends on another hourly task | Ancestor and descendant tasks generate the same number of instances A one-to-one dependency is established between instances from the same day: the first descendant instance depends on the first ancestor instance, the second on the second, and so on. Ancestor and descendant tasks generate a different number of instances Instances generated on the same day depend on each other based on the principle of scheduling time proximity. A descendant instance depends on the nearest ancestor instance scheduled at or before its own scheduling time. Special case: In scheduling dependency configurations, if an ancestor node runs more frequently than a descendant node (a many-to-one scenario), the descendant's dependency logic follows the interval coverage principle, not just the proximity principle. Dependency scope: A descendant instance depends on all ancestor instances generated within its time window (start, end].
|  |
An hourly task depends on a daily task | Self-dependency is not configured for the hourly task All of the descendant's hourly instances for the day depend on the ancestor daily task and will only start after it completes. Any hourly instances with passed scheduling times will then run concurrently.
Note To prevent concurrent execution, you can configure a self-dependency for the descendant hourly task. The execution of all descendant hourly instances does not affect one another.
Self-dependency is configured for the hourly task Only the first hourly instance depends on the ancestor daily task. All other hourly instances depend on their preceding hourly instance. An hourly instance starts only after both the daily task and the previous hourly instance are complete. In this case, hourly instances will not run concurrently, even if their scheduled times have passed.
Note A self-dependency can create a cross-day dependency. If the last instance from the previous day has not completed, it can prevent today's tasks from being scheduled.
| |
An hourly task depends on a minute-based task | Self-dependency is not configured for the minute-based task The descendant hourly task depends on all minute-based instances generated within that hour. Self-dependency is configured for both the minute-based and hourly tasks The descendant hourly task depends on the nearest minute-based instance scheduled at or before its own scheduling time within that hour.
|  |
A daily task depends on other tasks
Sub-scenario | Dependency and execution | Diagram |
A daily task depends on a same-cycle daily task | Self-dependency is not configured for the ancestor daily task By default, the descendant daily instance depends on the same-cycle ancestor daily instance. Self-dependency is configured for the ancestor daily task If a self-dependency is configured for the ancestor daily task, the dependency from the descendant daily task becomes a cross-cycle dependency.
Note A self-dependency can create a cross-day dependency. If the last instance from the previous day has not completed, it can prevent today's tasks from being scheduled.
|  |
A daily task depends on an hourly task from the same day | Self-dependency is not configured for the hourly task The descendant daily task depends on all hourly instances generated by the ancestor task on that day. The daily task starts only after all hourly data for that day is processed.
Note If the daily task only needs to depend on a specific hourly instance, you can configure a self-dependency for the hourly task. The daily task will be scheduled automatically after that specific hourly instance runs successfully. Self-dependency is configured for the hourly task The descendant daily task depends on the nearest ancestor hourly instance that is scheduled at or before its own scheduling time.
| |
A daily task depends on an hourly or minute-based task from the previous day | Self-dependency is not configured for the hourly or minute-based task The descendant daily task will depend on all hourly or minute-based instances generated by the ancestor task on the previous day. Self-dependency is configured for the hourly or minute-based task The descendant daily task will depend on the last hourly or minute-based instance generated by the ancestor task on the previous day.
| The following example shows a daily task that depends on an hourly task from the previous day.  |
A minute-based task depends on other tasks
Sub-scenario | Dependency description | Diagram |
A minute-based task depends on an hourly task | Self-dependency is not configured for the minute-based task A descendant minute-based instance depends on the nearest ancestor hourly instance that is scheduled at or before its own scheduling time, without duplicate dependencies.
Note In this scenario, multiple descendant minute-based instances may depend on the same ancestor hourly instance. Self-dependency is configured for both the minute-based and hourly tasks Each descendant minute-based instance depends on its preceding instance. It also depends on the nearest ancestor hourly instance scheduled at or before its own scheduling time, based on the principle of scheduling time proximity.
|  |
A minute-based task depends on a daily task | |  |
Weekly, monthly, or yearly dependencies
When a daily, hourly, or minute-based task depends on a weekly, monthly, or yearly task, the system generates a dry-run instance for the ancestor task on its non-scheduled days. A dry-run instance does not process any data, consume resources, or block descendant tasks.
The following example shows a scenario where a daily task depends on a weekly task for which a self-dependency is not configured:
Ancestor weekly task: Runs normally every Monday and Friday. On Tuesday, Wednesday, Thursday, Saturday, and Sunday, it generates a dry-run instance. When the scheduled time arrives, the dry-run instance is immediately marked as successful without actually running any code. A dry-run instance does not affect the normal operation of descendant instances.
Descendant daily task: Generates a scheduled instance every day, which depends on the instance (including any dry-run instance) generated by the ancestor weekly task for that day. After the weekly task's instance for the day completes successfully, the corresponding descendant instance is triggered.
Appendix 2: Workflow scheduling
In DataWorks, you can treat an entire workflow as a single unit and configure ancestor and descendant nodes for it. For more information, see Periodic workflow orchestration.