Dataphin:Manage real-time tasks

Section

Description

① Search and filter

Search for real-time tasks by task object name or ID. This is useful for quick searches when you know the name or ID.

• My Tasks: Real-time tasks for which the current logon account is the owner.

• My Started Instance Tasks: Real-time tasks that are started and for which the current logon account is the owner.

• Filter: Click Filter to expand all filter options. The filter options include the following:

  • Subtype: The available subtypes vary based on the real-time DPI engine. For more information, see the UI.

  • Last Operator: Filter by the last operator of the task.

  • Owner: Filter by the task owner. If "Tasks I operate" is selected, this filter option includes the current logged-in account by default.

  • Engine Type: Filter by DPI engine type. You can select Ververica Flink, Blink, Open Source Flink, or FusionInsight Flink.

  • Resource Queue: Filter by the resource queue where the task runs. This option is available when the real-time DPI engine is Alibaba Blink or open source Flink.

  • Cluster: Filter by the cluster where the task runs. This option is available when the real-time DPI engine is Ververica Flink.

  • Engine Version: The database engine version that the task uses.

  • Production Instance Started: Filter by whether the production instance is running. This option is available only for Prod and Basic projects.

  • Test Instance Started: Filter by whether the test instance is running. This option is available only for Dev projects.

  • Project: Filter by the project to which the task belongs. You can select multiple projects for which the current logon user has O&M - Access Folder permissions.

• Reset: After you expand all filter options, click Reset to clear the selected filter conditions.

• Refresh: Click the Refresh icon to refresh the real-time task list.

② Real-time task list

The real-time task page lists the real-time tasks that are generated by the system. The list displays information such as task object, last operator, owner, production instance status, last submission time, DPI engine type, and database engine version.

• Task Object: The real-time task submitted to the Operation Center. It includes the script name and script ID.

• Last Operator: The account name of the person who last operated on the task.

• Owner: The account name of the task owner.

• Production Instance Started: The status of the production instance for the real-time task.

• Project: The project to which the task belongs. The format is Project English Name (Project Chinese Name).

• Last Submission Time: The time when the real-time task was last submitted to the Operation Center.

• Engine Type: The DPI engine type that the current real-time task uses.

• Cluster: The production environment cluster for the task. This field is displayed only when the real-time DPI engine is Ververica Flink. For other DPI engine types, this field is empty.

• Resource Queue: The resource queue that the task uses at runtime. This field is displayed only when the real-time DPI engine is Alibaba Blink or open source Flink. For other DPI engine types, this field is empty.

• Engine Version: The database engine version that the task uses.

The Actions column lists the O&M operations for real-time tasks. For more information about individual operations, see Individual operations.

③ Batch operations

In the batch operations area, you can change owners and start production instances for multiple real-time tasks at a time. This improves O&M efficiency.

For more information about batch operations for real-time tasks, see Batch operations.

Operation

Description

DAG

In the Actions column of the target real-time task, click the icon to view the DAG of the task. For more information, see Operations supported in the DAG of a real-time task.

View Instance

In the Actions column of the target real-time task, click the icon to view the instances generated by the task.

Start Instance

In the Actions column of the target real-time task, click the icon to start an instance. For more information, see Start a real-time instance.

View Task Code

In the Actions column of the target real-time task, click the icon and select View Task Code to view the code of the task.

View Task Parameters

In the Actions column of the target real-time task, click the icon and select View Task Parameters to view the parameters defined for the task node.

Change Owner

In the production environment, click the icon in the Actions column of the target real-time task, select Change Owner, and select a new owner in the Change Owner dialog box.

Modify Resource Configuration

In the Actions column of the target real-time task, click the icon and select Modify Resource Configuration. Modify the resource parameters for the task, such as Core and Memory.

Operation

Description

Start Production Instance

Start multiple real-time integration instances or real-time computing instances at the same time.

Start Real-time Computing Instance:

1. Select multiple real-time computing tasks that are not running, or select All Tasks On The Current Page. Then, click Start Production Instance and select Start Real-time Computing Instance.

2. In the Batch Start Real-time Computing Instances dialog box, configure the parameters.

   • Start n instances: The number of instances to start and their names. Multiple instance names are separated by commas (,).

   • Start Mode: Supports stateless start and start from a valid state.

     • Stateless Start: Starts the task without an initial state.

     • Start From Valid State: Starts the task from an existing and valid state.

       Note

       If the selected instances include stateless ones, you can only select Stateless Start for the batch operation.

   • Data Read Point: Specifies a start time. All source tables that support startTime (such as Log Service, Kafka, MySQL, MongoDB, and OceanBase) start to read data from this time.

     Note

     The parameter settings in the SQL job take precedence over this start time.

   • Data Timestamp: Assigns a value to a time variable. Time-based variables are calculated based on the data timestamp. The default value is the current day.

     Note

     The parameter settings in the SQL job take precedence over this start time.

3. Click OK to start the instances.

Change Owner

Change the owners of multiple real-time tasks at the same time.

1. Select multiple real-time tasks, or select All Tasks On The Current Page. Then, click Change Owner.

2. In the Change Owner dialog box, select a new owner.

3. Click OK.

Operation

Description

Expand Parent Nodes

Expand the upstream nodes of the main node at different levels in the DAG.

Expand Child Nodes

Start Real-time Instance

To run a real-time instance, click the icon to manually start the instance and configure its parameters. For more information, see Start a real-time instance.

View Real-time Instance

Go to the real-time instance page to view the instances generated by the current real-time task.

View Node Code

View the code of the real-time task node.

View Task Parameters

View the parameters of the real-time task node.

View Operation Logs

View the operation logs of the real-time task node.

Edit Development Node

Go to the edit page of the current task in the Dev project to edit the task. This operation is available only in the Dev-Prod development mode.

Edit Node

Go to the edit page of the current task to edit it. This operation is available only in Basic mode.

View Production Node

View the configuration of the current task in the Prod project.

View Task Parameters

View the parameters of the task.

View Topology

View the resource group nodes and component nodes of the real-time task.

Modify Resource Configuration

Modify the resource parameters for the task, such as Core and Memory.