Dataphin:Computing tasks

Area

Description

①Computing tasks directory

Created computing tasks are managed in the tasks directory. The tasks directory supports the following management operations:

• Directory toolbar: Supports Filter, Refresh, Create Folder, and Locate.

• Create computing task: Click the button to create computing tasks. Because different compute engines support different types of tasks, the supported computing tasks vary based on what is displayed on the page.

• Computing task: Supports Get Lock, Edit, Clone, Rename, Move, Delete, Unpublish, Unpublish and Delete.

  Important

  • For Basic mode projects, after a delete task is submitted successfully, it is immediately updated to the production environment. Exercise caution when performing this operation.

  • For Dev-Prod mode projects, after a delete task is submitted successfully, go to the To Be Published list to publish the object to the production environment.

• Computing Tasks List: Click the computing tasks list to manage created computing tasks in list view.

②Toolbar

The toolbar supports functions including save, precompile, run, run with parameters, submit, publish, format, refresh, get lock, locate, unpublish, and unpublish and delete.

• Switch version: When the computing task is Python, you can switch the execution version of the Python computing task. Python 2.7, Python 3.7, and Python 3.11 versions are supported.

• Switch data source: When the computing task is database SQL, you can switch the data source of the computing task.

• Schema: When the computing task is database SQL and the database type is AnalyticDB for PostgreSQL, Oracle, or Presto, you can switch the Schema of the data source.

• Catalog: When the computing task is database SQL and the database type is Presto, you can switch the Catalog of the data source.

• Save: Save the computing task that is currently being developed.

• Precompile: Precompile the current computing task.

• Run: Run the computing task that is currently being developed. After the parameters are set for the first time, you do not need to enter the parameters again for subsequent runs.

  If you do not have row-level permissions for the relevant tables, view the permission details in the Permission Verification Failed dialog box. Contact a security administrator to grant the permissions, and then run the node again.

• Run with parameters: Run without using the parameters that exist in the computing task. Each time you run the computing task with parameters, you need to enter the parameters again.

• Submit: Submit the computing task to the publishing center or Operation Center.

• Go to publish: For computing nodes that have been submitted to the publishing center, go to the publishing center to perform publishing operations.

  If the tenant account for the current project does not have row-level permissions for the relevant tables, the system blocks the node publication and displays a message about the missing permissions. Contact a security administrator to grant the required permissions and then republish.

• Go to O&M: For computing tasks that have been submitted to the Operation Center, you can go to the Operation Center to perform O&M operations. For more information, see Operation Center.

• Format: When the computing task is SQL type, you can format the computing task code.

• Refresh: Refresh the current computing task.

• Get lock: Obtain the edit lock for the computing task.

• Locate: Locate the current computing task in the tasks directory.

• Delete, Unpublish and Delete: Delete or unpublish and delete the current computing task.

  Important

  • For Basic mode projects, after a delete task is submitted successfully, it is immediately updated to the production environment. Exercise caution when performing this operation.

  • For Dev-Prod mode projects, after a delete task is submitted successfully, go to the To Be Published list to publish the object to the production environment.

③Editor

You can develop computing tasks in the code editor. The editor supports various shortcut keys and efficiency methods. For more information, see:

• Editor shortcut key list.

• Editor code efficiency.

④Configuration and management

Configure and manage the property configuration, related items, history, and lineage information of computing tasks.

• Property configuration: Configure the properties of computing tasks. Different computing types have different property configurations.

  • For offline computing task property configuration, see Configure offline pipeline task properties.

  • For real-time computing task property configuration, see:

    • Overview of real-time mode configuration.

    • Overview of offline mode configuration.

• Related: Offline computing tasks support viewing related objects of the computing task, which are tables that the current computing task references and reads, tables that reference and read the current task, and tasks that use the current task as an upstream dependency. For more information, see View and manage task-related objects.

• History: View and manage task history versions. For more information, see View and manage real-time task history versions.

• Lineage: Python, Shell, and VIRTUAL computing tasks support custom lineage. For more information, see Custom lineage configuration.

⑤Console

When you run or precompile a task, the run logs and results are displayed in the console.

For SQL query computing tasks, the console includes three parts: run results, logs, and code. For more information, see SQL query computing task results console.

Section

Description

①Console overall operations

• View run results: Click the icon on the left side of the console to view the run results of the current computing task.

• View logs: Click the icon on the left side of the console to view the log information of the current run of the computing task.

• View code: Click the icon on the left side of the console to view the code of the current computing task.

• View and manage multiple run results of the current computing task: Click the icon in the upper-right corner of the console to view the various run results after multiple runs of the current computing task.

• Full screen console: Click the icon in the upper-right corner of the console to display the console in full screen.

• Pull up console/Pull down console: Click the or icon in the upper-right corner of the console to pull up or pull down the console. You can also move your mouse to the upper edge of the console to freely pull up or pull down the console.

• Collapse console/View console: Click Collapse Console at the bottom of the console to collapse the console. After collapsing, you can click View Console to view the console again.

②Run results operations

• Column search and filter: Click Column Search And Filter to view all column names of the current table in the dropdown list. Click the icon after a column name, and the system will locate and highlight the corresponding column, making it easy for you to quickly view the content of that column. The dropdown list supports filtering target columns. Selected columns will be displayed in the run results table. If you deselect a column, the corresponding column will be hidden in the run results table.

• Single row details: Click Single Row Details to display the content of each row in the form of field and field value. You can quickly view the previous row, next row, first row, or last row. If you click List View, the display will return to the table format.

• Automatic column width: The run results table supports freely adjusting the width of each column. After clicking Automatic Column Width, the system will automatically adjust each column to an appropriate width based on the content length of each column.

• Content search: Click the icon to search for content related to the entered keyword throughout table.

• Download data: Click the icon and select Download Sample Data or Download All Data. For more information, see Download the result data.

• Row-column conversion: Click the icon to swap the rows and columns of the current table.

• Single column data filtering and sorting: Click the icon after a column name to filter and sort the data in the current column. Each column in the run results table supports data filtering and sorting functions.

• Select all table: Click the cell in the first row and first column of the table, and the system will automatically select the entire table (including the table header, i.e., column names). You can right-click to copy the entire table.