A dependency on the previous cycle means that a node's current instance depends on the instance of a specific node from the previous cycle. The current instance runs only after the previous cycle's instance completes successfully. Configure a cross-cycle dependency when a node needs data produced by an upstream node on the previous day, or when an hourly or minutely task must wait for its own instance from the previous hour or minute.
Usage notes
| Category | Description | Related information |
|---|---|---|
| Display of cross-cycle dependencies | Cross-cycle dependencies appear as dashed lines in the Directed Acyclic Graph (DAG) in DataWorks. | Appendix: DAG features |
| Check whether same-cycle dependencies are still needed | After you configure a cross-cycle dependency, confirm whether the same-cycle dependency is also needed. The automatic parsing feature for same-cycle dependencies is enabled by default, and a downstream node runs only after all its ancestor nodes complete successfully. If the same-cycle dependency is not needed, delete it to prevent unexpected behavior. | Delete a dependency |
| Using cross-cycle dependencies in complex scenarios | When same-cycle dependencies do not meet your requirements, cross-cycle dependencies can fill the gap. For example, if a daily node depends on an hourly node, the daily node depends on all hourly instances for that day by default. Configure a self-dependency for the hourly node to instead attach the daily node to the hourly instance from a specific cycle. | Principles and examples of scheduling configurations for complex dependency scenarios |
| Preview dependencies before publishing | Preview the node's dependencies before you publish the task to confirm that the instance relationships for each cycle are correct. This prevents unexpected scheduling delays in the production environment. | Preview dependencies |
| Publish both upstream and downstream nodes | Publish both the upstream and downstream nodes that share a cross-cycle dependency to the production environment. After publishing, view the dependency in Operation Center. | Publish a task |
Entry point
In Data Studio, open the node editing page. In the right-side navigation pane, click Scheduling Configuration. Under Scheduling Dependency > Previous Cycle, configure the node's dependencies.
Dependency types
Three types of cross-cycle dependencies are supported. Choose the type based on what the current node's instance needs to wait for:
| Dependency type | What the current instance waits for |
|---|---|
| Current node (self-dependency) | Its own instance from the previous cycle |
| First-level child nodes | Its immediate downstream nodes' instances from the previous cycle |
| Other nodes | A specific other node's instance from the previous cycle |
Dependency on the previous cycle: Current node (self-dependency)
Use self-dependency when a node's current instance needs the business data its own previous-cycle instance produced.
The current instance runs only after its own previous-cycle instance completes successfully.
Effect on daily tasks that depend on hourly or minutely tasks
When an hourly or minutely task has a self-dependency, it changes how a daily task attaches to it:
| Hourly/minutely task setting | How the daily task attaches |
|---|---|
| No self-dependency (default) | Depends on all instances of the hourly or minutely task for that day; aggregates all table data those instances produced. |
| Self-dependency enabled | Attaches only to the instance with the closest scheduled time, based on the scheduling time proximity principle. Does not depend on all instances for that day. |
For details on specific dependency combinations, see Appendix: Summary of complex dependency scenarios.
When you backfill data for a task with a self-dependency, instances may run sequentially in chronological order. For details, see Data backfill for tasks with a self-dependency.
Dependency on the previous cycle: First-level child nodes
Use this type when the current node's instance needs confirmation that its downstream nodes successfully processed the data in its sink table (output table) from the previous cycle.
The current instance runs only after all its immediate downstream nodes' previous-cycle instances complete successfully.
Example: Node C has two downstream nodes, Node A and Node B. With a first-level child nodes dependency, Node C's current instance (cycle T) waits for Node A's and Node B's instances from the previous cycle (T-1) to complete successfully before it runs.
Dependency on the previous cycle: Other nodes
Use this type when the current node logically depends on data from a separate business process, but does not directly read or write that data itself.
The current instance runs only after the specified other node's previous-cycle instance completes successfully.
Example: Node A and Node B are both downstream nodes of Node C. With an "other nodes" dependency, Node B depends on Node D's instance from the previous cycle (T-1). Node B's current instance (cycle T) runs only after Node D's T-1 instance completes successfully.
Inherit the dry-run property from upstream nodes
This configuration is typically used with branch nodes.
Problem: A node has multiple downstream nodes. Some downstream nodes are set to dry-run during execution. If a dry-run downstream node also has a self-dependency, its dry-run property propagates to its child nodes, causing them to remain in dry-run state indefinitely.
Fix: In the Scheduling Dependency section, set Skip Dry-Run Property of Upstream Node to Yes. This prevents the node's dry-run state from being inherited by its child nodes in the next cycle.
Example scenario:
-
Assign_Node is an assignment node. Branch_Node is a branch node. Shell_Node1 and Shell_Node2 are downstream nodes of Branch_Node. All four nodes run on a daily schedule.
-
During execution, Shell_Node1 is set to dry-run; Shell_Node2 runs normally.
-
Shell_Node1 has a self-dependency on its previous-cycle instance.
-
In cycle T, the recurring instance is
Shell_Node1'. In cycle T-1, it wasShell_Node1.
Because Shell_Node1 is a descendant of Branch_Node and inherits its dry-run property, it always performs a dry-run. The self-dependency causes each cycle's instance to wait for the previous cycle's dry-run instance, perpetuating the dry-run state. Setting Skip Dry-Run Property of Upstream Node to Yes breaks this chain.
What's next
After you configure a cross-cycle dependency, verify that it behaves as expected:
-
Preview the node's dependencies to confirm the instance relationships before publishing. See Confirm that the dependencies meet your expectations.
-
Publish both the upstream and downstream nodes to the production environment.