, - Dataphin - Alibaba Cloud Documentation Center

Generation and running rules

Dataphin generates recurring instances for the next day at 23:00 every night. By default, these instances are in the Not Started state.

After all upstream instances that a recurring instance depends on complete successfully, the status of the recurring instance changes from Not Started to Waiting for Scheduling Time.
The status of the recurring instance changes from Waiting for Scheduling Time to Running when its scheduled time is reached, sufficient scheduling resources are available, and all throttling rules are met.

Note
Pausing an instance only suspends its current run and does not affect other instances generated by the same task. To pause multiple recurring instances, change the scheduling property of the task to Pause Scheduling. For more information, see Manage integration and computing tasks.

Access the recurring instance page

In the top navigation bar of the Dataphin homepage, choose Develop and O&M > Task O&M.
In the top navigation bar, select the production or development environment.
In the left-side navigation pane, choose Instance O&M > Recurring Instances.

Recurring instance page overview

The recurring instance page mainly displays lists of Integration and Computing Tasks and Modeling Tasks. On this page, you can perform O&M operations on these recurring instances.

Section	Description
① Search and filter area	Filter: You can quickly filter instances by selecting My Instances, Failed Instances, or Unfinished Instances, or by selecting a business date and project. The following filter options are also supported: My Instances: All instances for which the current logged-in user is responsible. Failed Instances: All failed instances in projects for which the current logged-in user has the O&M - Access Directory permission. Unfinished Instances: All unfinished instances in projects for which the current logged-in user has the O&M - Access Directory permission. Project: Filters by the project to which the task belongs. You can select multiple projects for which the current logged-in user has the O&M - Access Directory permission. Business Date: You can select Today, Yesterday, All, or a specific date or a date range. Instance type: You can select Integration and Computing Tasks or Modeling Tasks. Sub-type: The supported sub-types vary based on the selected instance type: Integration and Computing Tasks: The supported sub-types vary based on the bound compute source type. The available options are displayed on the page. Modeling Tasks: Supports dimension logic tables, fact logic tables, and summary logic tables. O&M owner: Filters by the O&M owner of the instance. Running status: Filters by one of the seven running statuses: Successful, Failed, Running, Waiting for Scheduling Time, Throttling, Waiting for Scheduling Resources, and Not Started. For more information about each status and its icon, see Recurring instance running status. Priority: Filters by one of the five instance running priorities: Highest, High, Medium, Low, and Lowest. Note If the baseline feature is enabled, only Medium, Low, and Lowest priorities are supported for tasks. Scheduling cycle: You can select one or more of the six scheduling cycles to filter by: Minute, Hour, Day, Week, Month, and Year. Running mode: Filters by the instance running mode. Running modes include Normal Run, Dry Run, and Paused Run. Start time: Filters by the start time of the recurring instance. You can select Last 1 Hour, Today, Yesterday, or a specific date or date/time range (accurate to the second). Instances on non-scheduling days: Refers to dry run instances generated by the system on non-scheduling days. This does not include instances configured for dry run or instances that are part of a conditional schedule. You can choose to Show or Hide them. Note A logical table instance is hidden only when all its fields are dry-run on a non-scheduling day. End time: Filters by the end time of the instance. You can select Last 1 Hour, Today, Yesterday, or a specific date or date/time range (accurate to the second). Scheduled time: Filters by the scheduled run time of the recurring instance. You can select 00:00-06:00, 06:00-12:00, 12:00-18:00, or a specific time range (accurate to the second). Running duration: Filters by the total time or a time range from when the instance started to when it ended. Scheduling resource group: Filters by the scheduling resource group specified for the instance run. Retries: You can enter a range for the number of retries in the format `a~b`. The default value of a is 0. Leaving b empty includes all values greater than a. You can enter 0 or a positive integer for both a and b. Note This filter is not supported for Modeling Tasks. Auto Retries: You can enter a range for the number of automatic retries in the format `a~b`. The default value of a is 0. Leaving b empty includes all values greater than a. You can enter 0 or a positive integer for both a and b. Note This filter is not supported for Modeling Tasks. Tag: Select one or more task tags to filter by. Directory: Filters by the directory where integration tasks, compute tasks, data collection, or workflow instances are located within the selected project. Note Only Integration and Computing Tasks support filtering by a specific directory. Click Reset to clear all filter conditions and restore the default settings. Search: You can search for recurring instances by entering a node ID or node name. You can also click the icon in the search box to open the Batch Search dialog box. In the Batch Search dialog box, you can search by manually entering multiple task IDs or by uploading a document. Batch search can be combined with other filter conditions. Batch Search-Manual Entry: In the Batch Search dialog box, select Manual Entry, enter up to 5,000 task IDs separated by line breaks, and then click OK to search. Batch Search-Upload Document: In the Batch Search dialog box, select Upload Document and click Upload to upload an Excel file. The system reads the first column (the task ID column) of the first sheet from rows 2 to 5,001. You can click Download Task ID List Template to download a template for editing.
② List operations	Customize list columns: Click the icon to select the columns to display and adjust their order. Note The first column and the Actions column cannot be hidden or reordered. If you clear your browser cache or switch to a different browser, the column settings are reset to default. Refresh list: Click the icon to refresh the current list. Batch Operation Records: Click the Batch Operation Records button to view the execution status of batch operations on recurring instances in the Batch Operation Records panel. This includes the instance name, instance ID, operation type, operation time, and status. You can filter by operation type and status, or search by instance name or ID. For records with a Failed status, you can hover over the prompt icon next to the status to view the reason for the failure.
③ Recurring instance list	The recurring instance page lists all recurring instances in projects for which the current user has O&M - Access Directory permission. The list is divided into Integration and Computing Tasks and Modeling Tasks. The Actions column shows the available O&M operations for each recurring instance. You can adjust the width of each column. Your settings are saved in your local browser and are lost if you clear your browser's cache. The supported O&M operations for each recurring instance type are as follows: For O&M operations supported by Integration and Computing Task instances, see Supported operations for Integration and Computing Task instances. For O&M operations supported by Modeling Task instances, see Supported operations for recurring Modeling Task instances. Note The recurring instance list displays only instances from the current tenant.
④ Batch operations	The batch operations area lets you manage multiple recurring instances at once. These operations include Rerun, Terminate, Set to Succeeded and Continue Scheduling, Pause, Resume, Modify Scheduling Resource Group, and Modify Priority. Select Select All to select all instances across all pages that match the current filter conditions. For batch O&M operations supported by Integration and Computing Task instances, see Batch operations for Integration and Computing Task instances For batch O&M operations supported by Modeling Task instances, see Logical table fields and batch operations.
⑤ Time zone conversion	When the tenant's scheduling time zone is different from the system time zone (the time zone displayed in the User Center), you can switch between Scheduling Time Zone and System Time Zone to view the Scheduled Time, Start Time, and End Time in the respective time zones. Note The Start Time, End Time, and Scheduled Time filters only support filtering by the System Time Zone.

Recurring instance indicators

Indicator	Tooltip	Description
	Delayed	For an event fact logic table with data latency detection enabled, this indicator appears if Dataphin detects data latency after the instance completes successfully.
	Dry Run	The scheduling property of the task for this recurring instance is set to dry run.
	Paused	The scheduling property of the task for this recurring instance is set to Pause Scheduling.

Recurring instance running status

Status icon	Running status	Description
	Not Started	An instance that has not started running.
	Waiting for Scheduling Resources	All upstream node dependencies have completed successfully. The instance is waiting for scheduling resources.
	Throttling	The instance is being throttled.
	Waiting for Scheduling Time	All upstream node dependencies have completed successfully. The instance is waiting for its scheduled time to run.
	Running	The instance is running.
	Failed	The instance failed to run. If you terminate an instance that is in a waiting or running state, its status changes to Failed.
	Successful	The instance ran successfully. Instances of tasks scheduled for a dry run are directly marked as Successful.

Recurring instance DAG

Click an instance name in the recurring instance list. A Directed Acyclic Graph (DAG) centered on the current node is displayed on the right. The DAG provides a dynamic visualization of the node's upstream and downstream dependencies. By default, it displays the Main node (the selected node) and its first-level upstream and downstream nodes. If an instance has no upstream or downstream nodes, the DAG shows only the current Main node. You can also perform O&M operations on upstream and downstream nodes.

You can adjust the DAG view pane:

Click the icon to hide the instance list and enlarge the DAG.
Click the icon to hide the DAG.
Hover over the icon and drag to expand or shrink the DAG view pane.

The DAG for Modeling Task instances is the same as the DAG for Integration and Computing Task instances. The following figure uses an Integration and Computing Task instance as an example:

Section	Description
① Node information	Displays a summary of the currently selected node. Click View Node Details to see more information, including Instance Overview, Run Diagnosis, Run Logs, Operation Logs, and Node Code. Instance Overview: Displays basic information (such as the node name, ID, instance ID, tenant, and project), run information (such as running status and runtime), and resource usage (memory/CPU allocation and actual memory/CPU usage). Note Except for SQL tasks (Compute Engine SQL tasks and Database SQL tasks), resource usage information is displayed in the node details of recurring and backfill instances only when the Scheduling Resource Market is enabled under System Settings > Resource Settings for the metastore tenant. Gantt charts and run diagnosis are not available for external tenant nodes (nodes that do not belong to the current tenant). Run Diagnosis: View the instance's upstream dependencies, scheduled time, throttling rules, scheduling resources, and execution details. For more information, see Instance run diagnosis. Run Logs: View the run logs for the successful or failed instance run. Operation Logs: View the operation logs for the current instance, including the operation time, operator, and specific action. Node Code/Materialized Code: View the code for the current instance. Integration and Computing Task instances: Includes Instance Overview, Run Diagnosis, Run Logs, Operation Logs, and Node Code. Modeling Task instances: Includes Instance Overview, Run Diagnosis, Run Logs, Operation Logs, and Materialized Code. External tenant nodes (nodes not in the current tenant): Only Instance Overview is available.
② Node filter area	Provides shortcuts to set the number of upstream and downstream levels to expand from the Main node. If too many nodes are displayed, you can search for a node name to quickly locate a specific node within the current DAG view.
③ Scheduling dependency graph	Displays the scheduling dependency graph of the instance. You can expand more upstream and downstream nodes and perform O&M operations on them. Hover over a node in the DAG to view its name, type, scheduling cycle, O&M owner, and description. If a node has upstream or downstream nodes that are not in the current tenant, external tenant nodes are marked with a cross-tenant tag, and you can only expand to view their immediate parent and child nodes. For O&M operations supported by recurring Integration and Computing Task DAG nodes, see Supported operations for Integration and Computing Task DAG nodes. For O&M operations supported by recurring Modeling Task DAG nodes, see Supported operations for Modeling Task DAG nodes.
④ Canvas adjustment area	Provides shortcuts to adjust the DAG display scale. You can set a specific scale (default is 100%), zoom in (up to 200%), zoom out (down to 20%), fit to canvas, and enter full screen. This area also displays the node ID and name of the Main node in the current DAG.
⑤ Instance list	When viewing the DAG, the instance list displays the name and business date of the instance. When you hover over an instance, its node name, node ID, scheduling cycle, business date, and scheduled runtime are displayed, as shown in the following figure: