Dataphin:Compute Tasks

Area

Description

① Compute Tasks Directory

Your compute tasks appear here for unified management. The directory supports these actions:

• Directory toolbar: Filter, refresh, create folders, and locate tasks.

• Create a compute task: Click the button to create a new compute task. Supported task types depend on your compute engine. Check the page for available options.

• Compute tasks: Lock, edit, clone, rename, move, delete, unpublish, unpublish and delete, or change the project ownership.

  Important

  • In Basic mode projects, deleting a task updates it directly in the production environment. Proceed with caution.

  • In Dev-Prod mode projects, after deleting a task, go to the Pending Publishing list to publish it to the production environment.

• Compute Tasks List: Click this list to manage your compute tasks in table view.

② Toolbar

The toolbar includes Save, Precompile, Run, Run with Parameters, Submit, Unpublish, Format, Refresh, Acquire Lock, Locate, Unpublish, and Unpublish and Delete.

• Switch version: For Python compute tasks, switch between Python versions. Supported versions are Python 2.7, Python 3.7, and Python 3.11.

• Switch data source: For database SQL compute tasks, switch the data source.

• Switch Database/Schema: For database SQL compute tasks using AnalyticDB for PostgreSQL, Oracle, Presto, MySQL, StarRocks, Doris, SelectDB, ClickHouse, or OceanBase (MySQL mode), switch the Database or Schema.

• Switch Catalog: For database SQL compute tasks using Presto or Trino, switch the Catalog.

• Save: Save the current compute task.

• Precompile: Precompile the current compute task.

• Run: Run the current compute task. After you set parameters for the first time, you do not need to re-enter them for subsequent runs.

  If you lack row-level permissions on required tables, the access denied dialog box shows details. Click the request permission icon to go to Management Hub > Permission Management > My Permissions > Row-Level Permissions to request access, then republish.

• Run with Parameters: Run the compute task without using saved parameters. You must enter parameters each time.

• Submit: Submit the compute task to the Publishing Center or Operation Center.

• Go to Publish: For compute tasks already submitted to the Publishing Center, go to the Publishing Center to publish them.

  If your tenant account lacks row-level permissions on required tables, publishing is blocked. The system shows the missing permissions. Contact your security administrator to grant access, then republish.

• Go to O&M: For compute tasks already submitted to the Operation Center, go to the Operation Center to manage them. For more information, see the Operation Center.

• Format: For SQL compute tasks, format the code.

• Refresh: Refresh the current compute task.

• Acquire Lock: Acquire an edit lock for the compute task.

• Locate: Find the current compute task in the directory.

• Delete and Unpublish and Delete: Delete or unpublish and delete the current compute task.

  Important

  • In Basic mode projects, deleting a task updates it directly in the production environment. Proceed with caution.

  • In Dev-Prod mode projects, after deleting a task, go to the Pending Publishing list to publish it to the production environment.

③ Editor

Develop compute tasks in the code editor. The editor supports many keyboard shortcuts and productivity features. For more information, see:

• Editor Keyboard Shortcuts.

• Editor Code Productivity Features.

④ Configuration and Management

Configure and manage task properties, related objects, history, lineage, and other metadata.

• Property Configuration: Set task properties. Options vary by compute type.

  • For offline compute tasks, see Configure Offline Pipeline Task Properties.

  • For real-time compute tasks, see:

    • Real-Time Mode Configuration Overview.

    • Offline Mode Configuration Overview.

• Related Objects: For offline compute tasks, view related objects — tables read by the current task, tables reading from the current task, and tasks that depend on the current task as an upstream dependency. For more information, see View and Manage Related Objects.

• History: View and manage task history versions. For more information, see View and Manage Real-Time Task History Versions.

• Lineage: For Python, Shell, and Virtual compute tasks, define custom lineage. For more information, see Custom Lineage Configuration.

⑤ Console

The console displays operational logs and results when you run or precompile a task.

For SQL query compute tasks, the console shows three sections: results, logs, and code. For details, see Console for SQL Query Compute Task Results.

Section

Description

① Console controls

• View results: Click the icon on the left side of the console to view the current compute task’s results.

• View log: Click the icon on the left side of the console to view the log for the current run.

• View code: Click the icon on the left side of the console to view the current compute task’s code.

• View and manage multiple results: Click the icon in the top-right corner of the console to view all results from multiple runs.

• Full-screen console: Click the icon in the top-right corner to expand the console to full screen.

• Pull up console / Pull down console: Click the or icon in the top-right corner to pull up or pull down the console. Or hover your mouse over the top edge of the console to drag it freely.

• Collapse console / Expand console: Click Collapse console at the bottom of the console to hide it. Click Expand console to show it again.

② Result table controls

• Column search and filter: Click Column search and filter to open a dropdown list showing all column names. Click the icon next to a column name to jump to and highlight that column. You can filter columns in the dropdown list. Selected columns appear in the result table. Clear a selection to hide that column.

• Row details: Click Row details to view each row as field-value pairs. Navigate quickly to the previous row, next row, first row, or last row. Click List view to return to table view.

• Auto-fit column width: Adjust column widths freely in the result table. Click Auto-fit column width to let the system set optimal widths based on content length.

• Search content: Click the icon to search the entire table for text matching your keyword.

• Download data: Click the icon and choose Download sample data or Download all data. For details, see Download Result Data.

• Transpose rows and columns: Click the icon to swap rows and columns in the current table.

• Filter and sort single-column data: Click the icon next to a column name to filter or sort data in that column. Every column supports filtering and sorting.

• Select entire table: Click the top-left cell to select the whole table, including the table header (column names). Right-click to copy the full table.