Dependency check node - DataWorks - Alibaba Cloud Documentation Center

Standard upstream and downstream dependencies might not be sufficient for complex business scenarios. For example, a daily node may need to wait for all hourly nodes from the previous day to complete. You might also need to combine conditions from multiple nodes using AND/OR logic. In these cases, you need a more flexible tool. This topic describes how to use the dependency check node to build complex dependency checks across cycles, workflows, and workspaces. This lets you precisely control when a node triggers to ensure accurate and timely data processing.

Function overview

The dependency check node is a powerful and flexible tool for handling complex, cross-cycle dependency scenarios. For example, a daily node might need to wait for all hourly nodes from the previous day to succeed before it runs. A node might also need to meet conditions from multiple nodes in different workspaces.

Core features:

Cross-entity dependency: Set dependencies across workspaces and workflows.
Cross-cycle dependency: Specify the exact time range for ancestor instances in a cross-cycle dependency.
Complex logic combination: Combine multiple dependency check conditions using AND and OR relationships.
Custom check policy: Set a custom check frequency and maximum wait time. The node fails if the check times out.

Configuration interface overview

The configuration for a dependency check node has two main parts: Check dependencies and Check settings.

Configure a dependency check node

The core of configuring a dependency check node is building its dependency check logic. You can define multiple check items and check groups and combine them using AND/OR relationships. Each check item points to instances of one or more ancestor nodes within a specific time range. When the dependency check node runs, it evaluates the entire logical expression. It then returns a Boolean value (True/False) that determines whether to execute or block the descendant node.

Configure dependency check items

You can set multiple check groups in a dependency check node. Each group can contain multiple Check item to build a complex dependency network.

Add check groups and check items:
- Click + Add Item to create a new dependency check condition.
- Organize multiple check items into a check group.

Configure the check item content:

Task: Enter the Output Name, Name, or ID of the ancestor node to search for and select it.
- You can select nodes from the same workflow, a different workflow in this workspace, or another workspace for which you have permissions.
  Important
  For simple cross-workspace or cross-workflow scheduling dependencies, see Configure cross-workflow and cross-workspace scheduling dependencies.
- If you select a node that has not been published to the production environment, the system displays the following prompt: The selected node is not published to the production environment. The check may fail at runtime.

Period: The period specifies which historical instances of the ancestor node are checked at runtime. Select a period type based on your requirements. The following table provides details.

Note

The conditions within a single check item are combined using an AND relationship. For example, if you select all hourly nodes for the current day, all of them must run successfully for the check to return True.

Period type	Options	Description
Minute	`Previous X minutes` (X is 0-59)	Checks the instance from `X` minutes before the current time. `Current minute` corresponds to `Previous 0 minutes`.
Hour	`Previous X hours` (X is 0-23)	Checks the instance from `X` hours before the current time. `Current hour` corresponds to `Previous 0 hours`.
Day	`Previous X days` (X is 0-7)	Checks the instance from `X` days ago. `Today` corresponds to `Previous 0 days`.
Week	`Monday to Sunday of this week`, `Monday to Sunday of last week`, `Last Monday`, `Last Tuesday` ... `Last Sunday`.	Checks instances within the specified week range.
Month	`First day to last day of this month`, `First day of this month`, `Last day of last month`, etc.	Checks instances within the specified month range.

Set the logical relationship (AND/OR):
- On the left side of the interface, you can set the logical relationship between check groups and between check items within a group.
  - AND: All related check items or check groups must succeed.
  - OR: Only one of the related check items or check groups needs to succeed.

Configure check settings

At the bottom of the page, define the node's check behavior:

Check interval: Specifies the frequency for checking the status of all dependencies. The minimum value is 1 minute/check.
Max duration: Specifies the maximum wait time for the node. If this duration is exceeded and the dependencies are still not met, the dependency check node fails. The maximum value is 1440 minutes (24 hours).

Runtime logic and statuses

Checking: Within the Max duration, if any dependency condition is not met, the node remains in the Running state. It retries the check periodically based on the Check interval setting.
Succeeded: Within the Max duration, if all dependency conditions are met according to the specified AND/OR logic, the node status changes to Running Succeeded.
Failed:
1. The dependency conditions are still not met after the Max duration is exceeded.
2. During the check, if a required dependency item (for example, in an AND relationship) points to a node that does not exist, the check terminates early and the node status is set to Failed.

Use case: Making a daily node wait for all hourly nodes from the previous day

Consider the classic scenario from the introduction. You have a daily summary report node (rpt_daily_summary). It must wait for all 24 instances of an hourly node named ods_hourly_log from the previous day (00:00 to 23:59) to run successfully before it can start.

Drag a dependency check node
In your workflow, drag a dependency check node upstream of your daily node, rpt_daily_summary, and connect them.
Configure the dependency check item
Double-click the dependency check node to configure its dependency check item.
- Add dependent node: Click + Add Item. In the Task input box, search for and select the hourly node, ods_hourly_log.
- Define the time range: You need to check all hourly instances from the "previous day". Set Period to Day, and then select Previous 1 day.
  The system will check if all instances (24) of this hourly node from the previous calendar day have succeeded.
- Set the logical relationship: Because there is only one check item, you can use the default AND logical relationship for the check group and its items.
Configure the check policy
- Check interval: For example, set it to 5 minutes/check. This means if the dependency is not yet met, the system waits 5 minutes before checking again.
- Max duration: For example, set it to 180 minutes. This means the node will wait a maximum of 3 hours from the daily node's scheduled time. If the hourly nodes from the previous day have not all finished after 3 hours, the dependency check node will fail. This prevents the downstream daily node from starting. This also prevents indefinite waiting and exposes issues promptly.

After you complete the configuration, save and publish it. Now, your daily report node will wait for all hourly data to be ready before it runs.

Appendix: Differences from scheduling dependencies

The checked nodes in a dependency check node's configuration are different from the scheduling dependencies in a scheduling configuration. The differences are as follows:

Attribute	Checked node in a dependency check node	Scheduling dependency node
Trigger method	The dependency check node actively polls the status of the check items. The check item nodes do not trigger or modify the status of the dependency check node.	When the target node reaches its scheduled runtime, it checks if the status of the scheduling dependency node meets the requirements.
DAG visualization	The dependency is contained within the node configuration and is not displayed on the DAG.	Displayed as upstream and downstream dependency lines in the DAG.
Data backfill/rerun interaction	Rerunning or backfilling data for a check item node does not automatically trigger the descendant node.	Backfilling data for a scheduling dependency node can optionally trigger the target node.