Dataphin:Computing tasks

Area

Description

① Computing Task Directory

The computing task directory organizes your tasks and supports the following operations:

• Directory Toolbar: Supports filter, refresh, new folder, and locate.

• Create Computing Task: Click the icon to create a computing task. The types of tasks that you can create vary by computing engine. The available task types are shown on the page. In multi-engine mode, the available computing task types are determined by the engine type of the project compute source.

• Computing Task: Supports operations such as acquire lock, edit, clone, rename, move, delete, take offline, take offline and delete, and change its project.

  Important

  • For a Basic-mode project, deleting a task immediately removes it from the production environment. Proceed with caution.

  • For a Dev-Prod mode project, after you delete a task, you must release this change from the pending release list to the production environment.

• Computing Task List: Click the computing task list to manage your created computing tasks in a list view.

② Toolbar

The toolbar provides the following functions:

• Switch version: For a Python computing task, you can switch its execution version. Python 2.7, Python 3.7, and Python 3.11 are supported.

• Switch data source: For a database SQL task, you can switch the task's data source.

• Switch Database/Schema: For a database SQL task where the database type is AnalyticDB for PostgreSQL, Oracle, Presto, MySQL, StarRocks, Doris, SelectDB, ClickHouse, or OceanBase (MySQL mode), you can switch the data source's Database/Schema.

• Switch Catalog: For a database SQL task where the database type is Presto or Trino, you can switch the data source's Catalog.

• Save: Saves the current computing task.

• Pre-compile: Pre-compiles the current computing task.

• Run: Runs the current computing task. If you have already set the parameters, you do not need to enter them again for subsequent runs.

  If you do not have the required row-level permission for a table, a permission check failure dialog box appears with details about the missing permissions. Click the apply for permissions icon to navigate to the Management Center > Permission Management > My Permissions > row-level permission page. After your permission request is granted, you can release the task again.

• Run with parameters: Runs the computing task using newly entered parameters, ignoring any existing ones.

• Submit: Submits the computing task to the Release Center or O&M Center.

• Go to release: For a submitted task, takes you to the Release Center to perform release operations.

  If the production account for the current project does not have the required row-level permission for relevant tables, the system blocks the task release and displays an error about the missing permissions. Contact a security administrator to grant the permissions, and then release the task again.

• Go to O&M: For a submitted task, takes you to the O&M Center to perform operations. For more information, see O&M Center.

• Format: Formats the code for SQL-type tasks.

• Refresh: Refreshes the current computing task.

• Acquire lock: Acquires the edit lock for the computing task.

• Locate: Locates the current computing task in the computing task directory.

• Delete, take offline and delete: Deletes or takes offline and deletes the current computing task.

  Important

  • For a Basic-mode project, deleting a task immediately removes it from the production environment. Proceed with caution.

  • For a Dev-Prod mode project, after you delete a task, you must release this change from the pending release list to the production environment.

③ Editor

The code editor supports keyboard shortcuts and efficiency features for developing computing tasks. For more information, see:

• Editor shortcut keys.

• Editor code efficiency.

④ Configuration and Management

Configure and manage the computing task's attributes, dependencies, History, and data lineage.

• Attribute Configuration: Configure the attributes of the computing task. The attribute settings vary by task type.

  • For information about configuring attributes for an offline computing task, see Configure offline pipeline task attributes.

  • For information about configuring attributes for a real-time computing task, see:

    • Real-time mode configuration overview.

    • Offline mode configuration overview.

• Dependencies: For an offline computing task, you can view its dependencies, which include tables it reads from, tables that read from it, and tasks that depend on it. For more information, see View and manage task dependencies.

• History: View and manage historical versions of the task. For more information, see View and manage historical versions of real-time tasks.

• Data lineage: For Python, Shell, and Virtual computing tasks, you can configure custom data lineage. For more information, see custom data lineage configuration.

⑤ Console

When you run or pre-compile a task, the log and execution result are displayed in the console.

For SQL query tasks, the console displays the execution result, log, and code. For more information, see Console for SQL query task results.

Section

Description

① Console operations

• View Execution Result: Click the icon on the left side of the console to view the execution result of the current task.

• View Log: Click the icon on the left side of the console to view the log for the current task run.

• View Code: Click the icon on the left side of the console to view the code for the current task.

• View and manage multiple execution results: Click the icon in the upper-right corner of the console to view the results from multiple task runs.

• Fullscreen Console: Click the icon to display the console in full screen.

• Pull Up Console/Pull Down Console: Click the or icon to pull the console up or down. You can also drag the top edge of the console to resize it.

• Collapse Console/View Console: Click Collapse console at the bottom of the console to hide it. After it is collapsed, you can click View console to show it again.

② Execution result operations

• Column search and filter: Click Column search and filter to view all column names in a drop-down list. Click the icon next to a column name to locate and highlight that column in the table. You can use the drop-down list to filter columns. Selected columns are displayed in the results table, while unselected columns are hidden.

• Single-row details: Click Single-row details to display each row as a list of fields and values. You can quickly navigate to the previous, next, first, or last row. Click list view to return to the table format.

• Auto-fit column width: You can manually adjust the width of each column in the results table. Click Auto-fit column width to automatically adjust the column widths based on the content length.

• Content search: Click the icon to search for keywords across the entire table.

• Download data: Click the icon and select Download sample data or Download all data. For more information, see Download result data.

• Transpose rows and columns: Click the icon to transpose the rows and columns in the current table.

• Single-column filtering and sorting: Click the icon next to a column name to filter and sort the data in that column. Filtering and sorting are supported for every column in the results table.

• Select entire table: Click the cell in the top-left corner of the table to select the entire table, including the header row. You can then right-click to copy the entire table.