edit-icon download-icon

Dependency

Last Updated: Mar 26, 2018

In scheduling configuration, you must configure two task-level dependencies: dependency attribute and cross-cycle dependency.

Dependency attribute

  • Automatic recommendation: The system automatically scans the node code to parse the source and target tables and recommend the task from which the source table is generated. Only tasks of SQL nodes and workflow tasks have this feature, and only the tables generated by SQL tasks can be parsed.

  • Owner project: It shows all the projects in the current organization, and the expected project can be selected from its drop-down list.

  • Upstream task: It corresponds to a task in the owner’s project and is used to set the upstream task of the current task. It is not a required item. Fuzzy search by workflow name is supported.

Description

One workflow can depend on multiple upstream workflows. Likewise, multiple upstream workflows can depend on one workflow. The dependency attribute is not a required item. Dependency must be configured when a downstream workflow needs to depend on data generated by upstream workflows.

Scenarios

For example, assume that the number of nodes configured in Workflow A has reached the upper limit, but the service demand has not yet been met. In that case, you can create Workflow B and continue to configure node tasks. You can set Workflow A as the upstream task on which Workflow B depends. The system runs Workflow A first and then Workflow B.

Cross-cycle dependency

Configures the cross-cycle dependencies of node/workflow tasks. For example, a daily scheduling task depends on the performed data yesterday to get the data for execution today, so you can configure it to be dependent on yesterday’s cycle of the task. As a result, yesterday’s instance must perform the task successfully first, for the instance to be scheduled successfully today. Such dependencies are mainly embodied in the dependencies of task scheduling instances.

Cross-cycle dependency descriptions:

  • Not dependent on the previous scheduling cycle: This option is selected by default for all tasks. It means that no dependencies exist on the previous scheduling instance of any task.

  • Self-dependency, continuing operation only after the previous scheduling cycle of the task ends:Applicable scenario - The data source of Task A in the current cycle is dependent on the execution result of the previous cycle of Task A; or Task A of hourly/minutely scheduling does not allow concurrent instances.

  • Continuing operation is only after the previous cycle of the downstream task ends: The operation can continue only after the previous cycle of first-tier sub-tasks has concluded. This applicable scenario is not common. Once this option is checked, all the instances depend on the previous cycle instance of all the first-tier sub-tasks, after the task has been directly depended on by the other tasks.

  • Continuing operation is only after the previous cycle of a user-specified task ends: Applicable scenario - Daily Task A is dependent on the data produced by daily Task B yesterday.

Notes

  • The scheduling dependency configured in Dependency Attribute is within the same cycle, and it does not conflict with the cross-cycle dependency. You can configure Task A to be dependent on Task B in the Dependency Attribute, and dependent on Task B across cycles. As a result, Task A is dependent on both the current cycle of Task B and the previous cycle of Task B.

  • If Task A is a hourly/minutely scheduling task, Task B is a daily scheduling task, and Task B is configured to be dependent on the previous cycle of Task A, then the instances of Task B of the day are dependent on all the instances of Task A yesterday.

  • If both Task A and Task B are hourly/minutely scheduled with the same scheduling cycle, and Task B is configured to be dependent on the previous cycle of Task A, then each Task B instance is dependent on all the instances in the previous cycle of Task A.

  • If the daily Task A is configured to be dependent on the previous cycle of Task B across cycles, the data supplementing instance is dependent on the instances of the previous automatic scheduling cycle of Task B during data supplementation of Task A. But if no instance exists in the previous automatic scheduling cycle, no dependency is created.

Thank you! We've received your feedback.