All Products
Search
Document Center

Dataphin:Offline task scheduling attribute configuration

Last Updated:Jan 21, 2025

Scheduling attributes define the recurring schedule of nodes within the production environment. By configuring these attributes, you can set task scheduling cycles and effective dates. This topic explains how to configure the scheduling attributes for a task.

Procedure

  1. On the Dataphin home page, navigate to the top menu bar and select Development > Data Development.

  2. At the top menu bar of the Development page, choose Project.

  3. In the left-side navigation pane, click Data Processing > Script Task. Then, in the Script Task list, select the desired task name.

  4. Under the task tab, click Attribute on the right to display the Attribute panel.

  5. Within the Attribute panel for the offline integration task, set the parameters in the Scheduling Attribute section.

    Parameter

    Description

    Schedule Type

    Supports three scheduling types: normal scheduling, dry-run scheduling, and skip execution.

    • Normal scheduling: Schedule according to the time configuration of the scheduling cycle and execute normally (i.e., real data will be run). This option is usually selected by default for tasks.

    • Dry-run scheduling: Schedule according to the time configuration of the scheduling cycle, but all executions are dry-run (i.e., real data will not be run). When the task is scheduled, it directly returns success without actually executing the task. This type of scheduling is usually selected when a node does not need to be executed within a certain period and does not block the execution of its descendant nodes.

    • Skip execution: When skip execution is selected, scheduling will start according to the time configured in the scheduling cycle, but the node status is set to pause (i.e., real data will not be run). When the task is scheduled, the system directly returns failure, and the descendant nodes that depend on the current node cannot be run. This is usually used in scenarios where a task does not need to be executed temporarily but will continue to be used later.

    Priority

    The priority of the current offline task can be set to Medium, Low, or Lowest.

    When a new offline task is created, the default value of the priority here is taken from the default priority in Management Center > Development Platform Settings > Node Task Related Settings.

    Note

    When the task is published to the production environment or submitted in the Basic environment, the priority cannot be modified during task editing. It must be modified in production operations. At this time, the priority value is the latest value in the production environment.

    Effective Time

    After configuring the effective date, the scheduling task will take effect and be automatically scheduled within the effective date. Tasks that exceed the validity period will not be automatically scheduled.

    Recurrence

    The scheduling cycle refers to how often the code in the task will be executed in the production environment scheduling system. When a task is successfully submitted, the task scheduling will submit the time attribute configuration of the task on the next day after submission to generate recurring instances for automatic scheduling, and run based on the results of upstream dependency instances and time points.

    Scheduling Cycle can be set to Daily, Weekly, Monthly, Hourly, and Minute:

    • Daily scheduling: The scheduling task runs automatically once a day. When a recurring task is created, the system's default period is to run once at 00:00 every day. You can click the image icon to specify the running time point as needed.

    • Weekly scheduling: The scheduling task runs automatically once on specific days of the week at specific time points. You can click the image icon to specify the running time point as needed.

      Note

      To ensure the normal operation of downstream instances during non-designated scheduling times, the system will generate instances and set them to run successfully without executing any logic or occupying resources.

    • Monthly scheduling: The scheduling task runs automatically once on specific days of the month at specific time points. You can click the image icon to specify the running time point as needed.

      Note

      To ensure the normal operation of downstream instances during non-designated scheduling times, the system will generate instances every day and set them to run successfully without executing any logic or occupying resources.

    • Hourly scheduling: Supports Time Period, On the Hour, Custom Time Period, and Custom Time Point scheduling.

      • Time Period scheduling: The scheduling task automatically starts scheduling every day within the specified time period according to the set time interval. Click the image icon to set the Start Time and End Time of the time period, and click the image icon to select Interval n Hours. For example, if the time period is set to 00:00~23:00 and the interval is set to 1 hour, the current task will be automatically scheduled once every hour within the 00:00~23:00 time period every day.

      • On the Hour scheduling: Click the drop-down list box to select the On The Hour time. The scheduling system will automatically generate instances for the task and run them at the selected on-the-hour time.

      • Custom Time Point scheduling: The scheduling system automatically generates instances for the task and runs them according to the selected custom time points. Click the drop-down list box to select custom time points. You can click +add Custom Time Point to add multiple time points.

      • Custom Time Period scheduling: The scheduling task automatically starts scheduling every day within certain time periods according to the set time interval. Click the image icon to set the Start Time and End Time of the time period, and enter Interval n Minutes (n is an integer between 5 and 360) after the interval. You can click +add Custom Time Period to add up to 10 time periods. The time between multiple time periods cannot overlap. For example, in Time Period 1, if the time period is set to 00:00~03:00 and the interval is set to 30 minutes, the current task will be automatically scheduled once every 30 minutes within the 00:00~03:00 time period every day.

    • Minute scheduling: Supports Daily or Hourly.

      • Daily: The scheduling task runs once at the interval time within the specified time period every day.

      • Hourly: The scheduling task runs once at the interval time within the time period from the specified start time every hour to the end of that hour.

    Scheduling Run Plan

    Click Preview. The scheduling run plan displays all scheduling instances and their scheduling types for each day of a certain month for the current scheduling task according to the configured scheduling cycle and conditional scheduling. The preview date type can be selected as Data Timestamp or Runtime (Scheduling Date).

    If there are multiple scheduling type statuses for all instances in one day, all scheduling type statuses contained will be displayed by color, and the name of each scheduling type status and the corresponding number of instances will be displayed. For example, the following figure shows that on the 4th of a certain month, the current scheduling task has 44 normal scheduling instances, 2 paused instances, and 12 dry-run instances. image

    Hover the mouse over the scheduling type module of a certain day to view the detailed scheduling instance list of the current scheduling task on that day, including scheduling type, scheduling conditions, and condition names.

    Conditional Scheduling

    After enabling conditional scheduling, you can set multiple groups of scheduling conditions. The system will perform condition judgment in the order of from top to bottom. Once a condition is hit, the corresponding scheduling action is executed, and all subsequent condition judgments are terminated. If no condition is hit, the default scheduling configuration is executed.

    Important

    Conditional scheduling is only effective when the scheduling type is Normal Scheduling.

    1. Click +add Scheduling Condition.

    2. In the Edit Conditional Scheduling dialog box, configure the relevant information for conditional scheduling.

      • Condition Name: Supports any character, with a length of up to 32 characters.

      • Effective Status: Enabled by default. When disabled, this conditional scheduling will be ignored during scheduling.

      • Meet the Following Conditions: The rule for condition judgment. When the condition judgment is true, scheduling is performed according to the configuration information of Execute Scheduling. For configuration instructions, see Conditional Scheduling Rule Description.

      • Execute Scheduling: Supports custom and follow scheduling attributes:

        • Custom: When the condition judgment is true, scheduling is run according to the configured Scheduling Type.

        • Follow Scheduling Attributes: Consistent with the scheduling policy in scheduling attributes, equivalent to the scheduling settings when conditional scheduling is disabled.

      • Scheduling Type: For configuration instructions, see the above Scheduling Type.

    3. Click OK.

      After completing the conditional scheduling settings, click Preview Scheduling Run Plan to view the dates hit by conditional scheduling in the calendar.

      Important
      • After modifying the conditional scheduling settings, submitting and publishing to the production environment will take effect in real-time for instances that are in the Not Running status at the time of publication. It will not take effect for instances that have entered the Waiting for Runtime status.

      • When cross-node parameter judgment types are used in conditional scheduling, possible parameter values must be provided for preview.

  6. To finalize the configuration, click OK.