Set Offline Pipeline Dependencies and Recurrence Rules - Dataphin

Scheduling configurations define the rules for periodically scheduling offline pipeline nodes. This topic describes how to configure scheduling for recurring offline pipelines.

Background information

Dependencies represent semantic relationships between two or more nodes. The status of an ancestor node affects whether its descendant nodes can run.
For nodes with dependencies, the scheduling rule is as follows. First, a descendant node can be scheduled only after its ancestor node completes running. Second, the system determines whether to execute the schedule based on the node’s scheduled time.
Scheduling configurations submitted before the scheduled time take effect at the scheduled time. Dependencies configured after the scheduled time generate instances only after a one-day delay.

Procedure

On the Dataphin homepage, in the top menu bar, select Development > Data Integration.
On the Integration page, in the top menu bar, select Project.
In the navigation pane on the left, select Integration>Offline Integration. In the Offline Integration list, click the target task name.
On the node tab, click Property on the right to open the Property panel.

On the Scheduling Configuration tab, configure the parameters in the Scheduling Configuration section.

Parameter	Description
Configuration Method	You can select Custom Configuration or Schedule Template. After you select Scheduling Template, you can select from the Basic Scheduling Templates configured in Planning > Common Business Logic > Offline Scheduling Template, or click Create Scheduling Template to create a new one if no suitable template is available. Once a template is selected, the Schedule Type, Effective Date, and Recurrence parameters cannot be modified, but you can configure the Priority, Rerun Property, and Conditional Scheduling.
Schedule Type	Supports three scheduling types: Normal, Dry-Run, and Skip Execution. Normal: Schedules according to the scheduling cycle and executes normally (processes actual data). This is usually the default option for nodes. Dry-Run: Schedules according to the scheduling cycle but performs a dry run (does not process actual data). When the system schedules this node, it returns success directly without actual execution. Use this type when a node does not need to execute for a certain period and should not block its descendant nodes. Skip Execution: If selected, the system starts scheduling according to the time configured in the scheduling cycle, but the node status is set to skipped (does not process actual data). When the system schedules this node, it returns failure directly and blocks the execution of descendant nodes that depend on the current node. Use this type when a node is temporarily not needed but will be used again later.
Priority	The priority of the current offline pipeline task can be set to Medium, Low, or Lowest. When you create an offline pipeline node, the default priority here is inherited from the default priority in Management Hub > Development Platform Settings > Node Task-related Settings. Note After you publish this node to the production environment or submit it in the Basic environment, you cannot modify its priority when editing the node. Modify it in O&M in the production environment. The priority value at this time is the latest value from the production environment.
Scheduling Time Zone	The scheduling time zone set for the current tenant in Management Hub > System Settings > Tenant Settings > Basic Settings. Instance generation time, scheduling cycle, and effective date are all calculated based on this time zone.
Recurrence	Recurrence defines how often the code in this node actually executes in the production environment's scheduling system. After a node is successfully submitted, its scheduling configuration generates automatically scheduled recurring instances starting from the day after submission, based on the time properties of the node. These instances run according to the execution results and timestamps of upstream dependent instances. For Recurrence, select Daily, Weekly, Monthly, Hourly, or Minutely: Daily: The node runs automatically once a day. When you create a recurring node, the system defaults to running once a day at 00:00. As needed, click the icon to specify the running time. Weekly: The node runs automatically once at a specific time on specific days of the week. As needed, click the icon to specify the running time. Note Outside the specified scheduled time, to ensure the normal operation of descendant instances, the system generates instances and directly sets them to run successfully. It does not execute any logic or consume resources. Monthly: The node runs automatically once at a specific time on specific days of the month. As needed, click the icon to specify the running time. Note Outside the specified scheduled time, to ensure the normal operation of descendant instances, the system generates instances daily and directly sets them to run successfully. It does not execute any logic or consume resources. Hourly: Supports scheduling by Time Range, Time Frame, Custom Time Range, and Custom Time Point. Time Range: The node automatically starts scheduling daily within a specified time range at set intervals. Click the icon to set the Start Time and End Time for the time range. Click the icon to select Interval of n hours. For example, if you select a time range of 00:00 to 23:00 and an interval of 1 hour, the current node automatically schedules once every hour within the 00:00 to 23:00 time range each day. On-the-hour scheduling: Click the drop-down list box to select an On-the-hour time, and the scheduling system automatically generates and runs an instance for the node at the selected time. Custom Time Point: The scheduling system automatically generates and runs instances for the node at the selected custom time points. Click the drop-down list box to select custom time points. Click +Add Custom Time Point to add multiple time points. Custom Time Range: The node automatically starts scheduling daily within certain time ranges at set intervals. Click the icon to set the Start Time and End Time for the time range. After the interval, enter Interval of n minutes (where n is an integer between 5 and 360). Click +Add Custom Time Range to add up to 10 time ranges. Time ranges cannot overlap. For example, in Time Range 1, if the time range is 00:00 to 03:00 and the interval is set to 30 minutes, the current node automatically schedules once every 30 minutes within the 00:00 to 03:00 time range each day. Minute-level scheduling: supports Daily or Hourly. Daily: The node runs once at the specified interval within the designated time range daily. Hourly: The node runs once at the specified interval within the time range from the specified start time of each hour to the end of that hour.
Scheduling Run Plan	Click Preview. The scheduling run plan displays all scheduling instances and their scheduling types for the current node for each day of a selected month, based on the configured scheduling cycle and conditional scheduling. For the preview date type, select Data Timestamp or Run Date (Scheduling Date). If all instances within a day have multiple scheduling type statuses, the system displays all included scheduling type statuses by color, along with the name of each scheduling type status and its corresponding number of instances. For example, the following figure shows that on the 4th of a certain month, the current node has 44 normal scheduling instances, 2 skipped instances, and 12 dry-run instances. Hover over a day's scheduling type module to view a detailed list of scheduling instances for the current node on that day. This includes the run time (the time of the node's recurring schedule under the Scheduling Time Zone), scheduling type, and condition name.
Conditional Scheduling	After you enable conditional scheduling, you must select a Configuration Method. You can choose either Custom Settings or Scheduling Template. Important For multiple sets of scheduling conditions (up to 10), the system evaluates conditions in top-to-bottom order. If a condition is met, the corresponding scheduling action executes, and evaluation of all subsequent conditions stops. If no condition is met, the default scheduling configuration executes. Conditional scheduling takes effect only when the scheduling type is Normal. Scheduling conditions and start times are calculated using the configured Scheduling Time Zone. When an instance is generated, the system determines whether conditional scheduling is met based on the instance's information, thereby deciding the scheduling type. Custom Settings Click +Add Scheduling Condition. In the Edit Conditional Scheduling dialog box, configure the conditional scheduling information. Condition Name: Supports any character, up to 32 characters in length. Status: Enabled by default. If disabled, the conditional schedule is ignored during scheduling. Meet the following conditions: The rule for condition evaluation. If the condition evaluates to true, the system schedules according to the Execute Schedule configuration. For configuration details, see Conditional Scheduling Rules. Execute Schedule: Supports custom and follow scheduling properties: Custom: If the condition evaluates to true, the system runs the schedule based on the configured Scheduling Type. Follow Scheduling Property: Consistent with the scheduling policy in the scheduling properties, equivalent to the scheduling settings when conditional scheduling is disabled. Scheduling Type: For configuration details, see Scheduling Type above. Click OK. After completing the conditional scheduling settings, click Preview Scheduling Run Plan to view the dates when conditional scheduling is met in the calendar. Important After modifying the conditional scheduling settings, submitting, and publishing them to the production environment, the changes take effect immediately for instances that are in the Not Running state at the time of publishing. The changes do not apply to instances that have already entered the Waiting for Run Time state. If cross-node parameter judgment types are used in conditional scheduling, provide possible parameter values for preview. Scheduling Template After you select Scheduling Template, you can select any of the Conditional Scheduling Templates configured in Planning > Common Business Logic > Offline Scheduling Templates. If no template meets your requirements, you can click Create Scheduling Template to create a new one. After you select a template, you cannot add new scheduling conditions. You can click the View Details icon next to each scheduling condition to view the conditional schedule details. Note If the current node's scheduling cycle is Daily, Weekly, or Monthly, the Start Time parameter in the referenced conditional scheduling template takes effect. If the current node's scheduling cycle is Hourly or Minutely, the Start Time parameter becomes invalid.
Cross-Tenant Dependency	Specifies whether to allow dependencies from nodes that belong to other tenants. If you enable this option, you must select the Allowed Tenants. You can select one or more tenants that are in the same time zone as the current tenant. Note This option is configurable only if cross-tenant dependency is enabled for the current tenant.

Click OK to complete the offline pipeline scheduling configuration.