Dataphin scheduled instance operations maintenance - Dataphin

A scheduled instance is generated each time a recurring task runs according to its scheduling configuration. You can manage these instances by viewing run status, rerunning them, and inspecting node code.

Scheduled instance generation and run rules

At 23:00 every day, Dataphin generates the scheduled instances for the following day. By default, these instances are in the Not running state.

After all of an instance's upstream dependencies succeed, its status changes from Not running to Waiting for schedule time.
When the scheduled time is reached, Dataphin has sufficient scheduling resources, and the instance passes all throttling rules, its status changes from Waiting for schedule time to Running.
Note
Pausing an instance stops only the current run and does not affect other instances generated by the same task. To pause multiple scheduled instances, change the task's scheduling property to Pause Scheduling. Manage integration and compute tasks.

Access the Scheduled Instances page

In the top navigation bar of the Dataphin console, choose Development > Task O&M.
In the top navigation bar, select the production environment or development environment.
In the navigation pane on the left, choose Instance O&M > Scheduled Instances.

Scheduled instances page

The Scheduled Instances page lists Integration and Compute Tasks and Modeling Tasks, where you manage submitted scheduled instances.

Section	Description
① Search and filter area	Filter: Quickly filter by selecting My Instances, Failed Instances, or Unfinished Instances, or by choosing a business date and project. Additional filter options: My Instances: All instances where the current user is the operator. Failed Instances: All failed instances in projects where the current user has O&M - Access Directory permissions. Unfinished Instances: All unfinished instances in projects where the current user has O&M - Access Directory permissions. Project: Filters instances by project. You can select multiple projects where you have O&M - Access Directory permissions. Business Date: You can select Today, Yesterday, All, or a specific date or date range. Instance Type: You can select Integration and Compute Tasks or Modeling Tasks. Subtype: The available subtypes depend on the selected business type: Integration and Compute Tasks: The available subtypes vary based on the bound compute source type. See the UI for the complete list of options. Modeling Tasks: Supports dimension logical tables, fact logical tables, and summary logical tables. Operator: Filters instances by their operator. Run status: Filters instances by status. The available options are Succeeded, Failed, Running, Waiting for schedule time, Throttling, Waiting for scheduling resources, and Not running. Scheduled instance run status. Priority: Filters instances by their run priority. The available options are Highest, High, Medium, Low, and Lowest. Note If the baseline feature is enabled, task priorities only support Medium, Low, and Lowest. Scheduling cycle: Filters instances by one or more scheduling cycles. The available options are Minute, Hour, Day, Week, Month, and Year. Run mode: Filters instances by their run mode. The available options are Normal Run, dry run, and Paused Run. Start time: Filters instances by their start time. You can select Last Hour, Today, Yesterday, or a specific date or time range (accurate to the second). Non-scheduling day instances: These are dry run instances generated by the system on non-scheduling days. This does not include instances configured as dry runs or conditional scheduling dry runs. You can choose to Show or Hide these instances. Note Logical table instances are hidden only if all their fields are executed as a non-scheduling day dry run. End time: Filters instances by their end time. You can select Last Hour, Today, Yesterday, or a specific date or time range (accurate to the second). Scheduled time: Filters instances by their scheduled run time. You can select 00:00–06:00, 06:00–12:00, 12:00–18:00, or a custom time range (accurate to the second). Duration: Filters instances by their total run time, or by a specified duration range. Scheduling resource group: Filters instances by the scheduling resource group used for the run. Number of retries: Enter a range for the number of retries in the format `a~b`. The default value for 'a' is 0. 'b' is optional; if left empty, it includes all values greater than 'a'. Both 'a' and 'b' can be 0 or any positive integer. Note This filter is not supported for modeling tasks. Number of automatic retries: Enter a range for the number of automatic retries in the format `a~b`. The default value for 'a' is 0. 'b' is optional; if left empty, it includes all values greater than 'a'. Both 'a' and 'b' can be 0 or any positive integer. Note This filter is not supported for modeling tasks. Tag: Select one or more task tag values to filter instances. Click Reset to clear all filter conditions and restore the default settings. Search: Search for a scheduled instance by node ID or node name. Click the icon to open the Batch Search dialog box. In Batch Search, enter multiple task IDs or upload a file. Batch search can be combined with other filters. Batch Search - Manual Input: In Batch Search, select Manual Input, enter up to 5,000 task IDs separated by line breaks, and click Confirm. Batch Search - Upload File: In the Batch Search dialog box, select Upload File and click Upload to upload an Excel file. The system reads the first sheet, first column (Task ID), rows 2 to 5,001. Click Download Task ID List Template for a template.
② List actions	Customize list header: Click the icon to select columns to display and reorder them. Note The first column and the Actions column cannot be hidden or reordered. The header settings revert to default if you clear your browser cache. Refresh list: Click the icon to refresh the current list.
③ Scheduled instance list	The Scheduled Instances page lists all recurring tasks in projects where you have O&M - Access Directory permissions, divided into Integration and Compute Tasks and Modeling Tasks. The Actions column provides management options. You can adjust column widths; these settings are saved in your browser and cleared if you clear your cache. Supported operations in the integration and compute task instance list. Supported operations for scheduled modeling task instances. Note Only instances for the current tenant are displayed.
④ Batch actions	You can use batch actions to manage multiple scheduled instances at once. Available actions: Rerun, Terminate, Mark as Succeeded and Continue, Pause, Resume, Modify scheduling resource group, and Modify priority. Batch operations for integration and compute task instances. Logical table fields and batch operations.
⑤ Time zone conversion	If the tenant's scheduling time zone differs from the system time zone (shown in user center), switch between Scheduling time zone and System time zone to view the Scheduled time, Start time, and End time accordingly. Note The Start Time, End Time, and Scheduled Time filters only support filtering by the System Time Zone.

Scheduled instance markers

Marker	Tooltip	Description
	Delayed	For an event fact logical table with data delay detection enabled, this marker appears if Dataphin detects a data delay after the scheduled instance has succeeded.
	dry run	This instance's corresponding task is configured for a dry run.
	Paused	The scheduling property of this scheduled instance's task is set to paused.

Scheduled instance run status

Icon	Status	Description
	Not running	The instance has not started running.
	Waiting for scheduling resources	All upstream dependencies have succeeded, and the instance is waiting for scheduling resources.
	Throttling	The instance is being throttled.
	Waiting for schedule time	All upstream dependencies have succeeded, and the instance is waiting for its scheduled run time.
	Running	The instance is currently running.
	Failed	The instance run failed. If you terminate an instance that is waiting or running, its status changes to Failed.
	Succeeded	The instance run succeeded. The system immediately marks instances configured for a dry run as succeeded.

Scheduled instance DAG

Click an instance name to display a DAG on the right, centered on the current node. The DAG visualizes the node's upstream and downstream dependencies. By default, it shows the main node and its immediate upstream and downstream nodes. If an instance has no dependencies, only the main node is shown.

Adjust the DAG view:

Click the icon to hide the instance list and enlarge the DAG.
Click the icon to hide the DAG.
Hover over and drag the icon to expand or shrink the DAG view.

The DAG for modeling task instances shares the same interface as integration and compute task instances. The following example uses an integration and compute task instance:

Section	Description
① Node information	Displays a summary of the selected node. Click View Node Details for more information, including instance overview, run diagnostics, run log, operation log, node code, and task parameters. Instance overview: Displays basic information (name, ID, instance ID, tenant, and project), run information (status and run time), and resource consumption (memory/CPU allocation and actual usage). Note Except for SQL tasks (compute engine SQL tasks and database SQL tasks), resource consumption is displayed in node details of scheduled instances and backfill instances only when System Settings > Resource settings > Scheduling resource dashboard is enabled for the metawarehouse tenant. The Gantt chart and run diagnostics cannot be viewed for cross-tenant nodes (nodes not belonging to the current tenant). Run diagnostics: View upstream dependencies, scheduled time, throttling rules, scheduling resources, and execution details. Instance run diagnostics. Run log: View the run log for the current instance's execution. Operation log: View operations performed on the current instance, including time, operator, and action. Node code/Materialized code: View the code for the current instance. Integration and compute task instances: Instance overview, Run diagnostics, Run log, Operation log, and Node code. Modeling task instances: Instance overview, Run diagnostics, Run log, Operation log, and Materialized code. Cross-tenant nodes (nodes that belong to other tenants): displays only the Instance overview.
② Node filter area	Set the number of upstream and downstream levels to display. Search by node name to locate a specific node.
③ Scheduling dependency graph	Displays the scheduling dependency graph. You can expand upstream and downstream nodes and manage them. Hover over a node to view its name, type, scheduling cycle, operator, and description. If a node has dependencies belonging to another tenant, a cross-tenant tag is displayed, and you can only expand its parent and child nodes. Supported operations for DAG nodes of integration and compute task instances. Supported operations for DAG nodes of modeling task instances.
④ Canvas controls	Adjust the DAG display scale. Set a specific zoom level (default 100%), zoom in (up to 200%), zoom out (down to 20%), fit to canvas, or view in full screen. This area also displays the node ID and name of the main node.
⑤ Instance list	When viewing the DAG, this list displays each instance's name and business date. Hover over an instance to view its node name, node ID, scheduling cycle, business date, and scheduled run time, as shown below: