Advanced time property configuration

Mode one: Dependency-driven execution

Use this mode when the business goal is to complete the entire business flow as quickly as possible. In this mode, all nodes in the flow run as soon as their input data is ready. This approach completes the full-path computation in the shortest possible time.

• Configuration method: Set a specific scheduled time, such as 02:00, for only the first node (or multiple nodes with no upstream dependencies). Set the scheduled time for all descendant nodes to 00:00.

• Execution logic: Descendant nodes remain in a waiting state because their upstream dependencies are not met. As soon as an ancestor node runs successfully, its descendant node is immediately triggered to run.

• Configuration and execution example:

  Node	Scheduled time configuration	Actual runtime (estimated)	Trigger logic
  A (First node)	02:00	02:00	Triggered to run at the specified time.
  B (Downstream)	00:00	~02:10 (after A is complete)	Node A, a dependency, runs successfully and immediately triggers Node B.
  C (Downstream)	00:00	~02:18 (after B is complete)	Node B, a dependency, runs successfully and immediately triggers Node C.

Mode two: Time-constrained execution

Use this mode when a node in a flow must run after a specific time due to external dependencies or business rules. For example, a node in the flow, such as Node B, may need to start its computation after 5:00 AM due to external business rules or system window constraints.

• Configuration method: Set a specific scheduled time, such as 05:00, for the time-constrained node.

• Execution logic: The node must meet two conditions to run: upstream dependencies are met and its own scheduled time has been reached. Even if the ancestor Node A finishes at 02:00, Node B will wait until its scheduled time of 05:00 to start running.

• Configuration and execution example:

  Node	Scheduling Configuration	Instance runtime	Trigger logic
  A (First node)	02:00	02:00	Triggered to run at the specified time.
  B (Time-constrained)	05:00	05:00	The upstream dependency A is met. The node waits for its own scheduled time to be reached.
  C (Time-constrained)	08:00	08:00	The upstream dependency B is met. The node waits for its own scheduled time to be reached.

Ensuring on-time completion of critical tasks

When data output has a strict delivery time requirement (for example, Node E must be completed before 09:00 daily), you need to plan the scheduled times for the entire business flow by working backward from the endpoint.

Using baseline priority for peak-load shifting

The dependency-driven method is easy to configure. However, it can cause many tasks to start at the same time, such as at 00:00. This leads to computing resource competition and task queuing. You can use the priority configuration in baseline management to give core tasks a higher priority to run.

• Identify task priorities : Distinguish between core tasks, such as data extraction for the Operational Data Store (ODS) layer, and non-core tasks, such as some internal reports.

• Set scheduling priorities for tasks: Use baselines to increase the priority of core tasks in acquiring resources.

• Optimization comparison diagram:

By combining scheduled times with baselines, you can achieve a reasonable allocation of scheduling resources. This enables priority-based intelligent scheduling and reduces the Operations and Maintenance (O&M) costs and human errors associated with setting individual scheduled times for each task.

Configuring cross-cycle dependencies

When a task's execution depends on the previous cycle's instance of its ancestor task, you need to configure a cross-cycle dependency. For example, a summary task for day T+1 needs to wait for all hourly tasks on day T to complete.

• Scenario: A daily summary task B needs to run at 02:00 every morning. Its data source is an hourly task A. It must run only after all hourly instances from 00:00 to 23:00 of the previous day (T) have run successfully.

• Configuration method: When configuring the scheduling dependency for Node B, set its dependency on the ancestor Node A as a cross-cycle dependency . For more information, see Configure dependency on the previous cycle (cross-cycle dependency).

  Set Dependency Mode to Other Nodes. Set Skip Dry-run Property of Upstream Task to No. Then, add the ancestor Node A to the list below.

• Execution result: After configuration, the Node B instance with a data timestamp of 2025-12-02 will wait for all Node A instances with a data timestamp of 2025-12-01 to run successfully before it is triggered.

Implementing complex recurring schedules

For tasks with special periodic patterns, such as quarterly or semi-annual tasks for account closing, you can use a combination of scheduling cycles and scheduling parameters.

• Scenario: A financial closing task needs to run on the last closing day of each quarter and depends on data from the entire past quarter.

  When setting a closing day, a buffer period is usually reserved to handle special month-end items, such as cross-month supplementary orders, refund reversals, and manual audits.

• Configuration method:

  1. Set the scheduling cycle: In the task's time properties, select "Year" for scheduling. Specify the months as 1, 4, 7, 10 and select Last day of the month for the date. DataWorks automatically handles different month lengths (30/31 days) and leap years.

  2. Use scheduling parameters: In your code, use scheduling parameters or user-defined functions to dynamically calculate the required data date range. For example, you can obtain the start and end dates of the quarter for the current data timestamp. For more information, see Supported formats for scheduling parameters.

• Execution logic: DataWorks automatically identifies whether the 30th or 31st (or even February 29th in a leap year) is the "last day". During this period, instances on non-month-end days automatically perform a "dry-run". This ensures both the continuity of dependency logic and the precise triggering of financial calculations.

Using a scheduling calendar for trading day schedules

The scheduled time (cron expression) defines when a task runs within its scheduling cycle. The scheduling calendar acts as a filter for execution dates. Combining these features allows for precise control to run tasks only on specific business dates, such as trading days or promotion days.

• Scenario: A securities company's trade settlement task must run at 22:00 on every trading day (non-holidays) . If it's a weekend or a public holiday, the task must automatically stop running to avoid creating invalid instances or wasting resources on dry-runs.

• Solution: Scheduling calendar + Scheduled time

  • Create a custom calendar: In the DataWorks Resource Center, maintain a "Trading Calendar" by manually or automatically synchronizing all trading dates for the year. For more information, see Configure a scheduling calendar.

  • Configure scheduling properties: Set the task to trigger daily at 22:00 and select the custom "Trading Calendar".

• Execution logic:

  • Trading day: The system detects that the current date is in the calendar. The task starts running on time at 22:00.

  • Non-trading day (such as Spring Festival) : The system automatically skips generating an instance for this task, or the generated instance is in a "dry-run" state and does not consume actual computing resources.

1. Static scheduling planning and configuration

Goal: Decouple scheduling logic from business logic using a layered strategy.

Core strategies:

1. Linear business flows

   Configure the scheduled time for only the first node. For example, set the first node's scheduled time to 07:00. Downstream tasks are automatically triggered through dependencies to maximize execution efficiency.

2. Time-dependent tasks

   1. Set a precise scheduled time for specific tasks independently. When setting the time, avoid setting an ancestor node's scheduled time later than its descendant node's. This could cause the descendant node to fail to run on time.

   2. Use a scheduling calendar and effective date range to control the task's active period. For example, control a task to run only on weekdays between January 1, 2026, and December 31, 2026.

3. Dynamic scheduling parameters

   Use scheduling parameters such as ${yyyymmdd} for dynamic replacement of time parameters.

2. Dynamic control with intelligent baselines

Goal: Guarantee the delivery time of core tasks and reduce manual intervention costs.

Prerequisite: You have created a baseline and completed task priority classification.

Core mechanisms:

1. Commitment time and priority definition:

   Define a committed completion time for core tasks, such as 09:00, and attach them to a high-priority baseline. The system will automatically identify the critical path based on priority, ensuring that high-priority tasks, such as ODS layer extraction, receive priority access to computing resources.

2. Automatic resource "peak-load shifting":

   You do not need to manually set staggered start times for each non-core task. The scheduling engine will automatically queue non-core report tasks to avoid resource contention during peak times, prioritizing resource supply for the critical path.

3. Dynamic prediction and real-time alerting:

   Based on historical runtimes, the system can dynamically predict in the early morning whether the day's pipeline will miss its delivery time. If at 07:00 it predicts a delay to 09:15, the system will immediately trigger an alert and highlight the bottleneck node on the critical path. This changes the approach from "post-mortem recovery" to "proactive intervention".