Offline computing node submission - Dataphin - Alibaba Cloud Documentation Center

When you complete the development and submission of an offline computing task, the Dataphin system automatically analyzes the data lineage of tables and fields within the task, verifies the task object's change type and content, and conducts a preliminary check to ensure the task meets submission requirements, reducing the likelihood of erroneous submissions.

Data lineage parsing instructions

Data lineage parsing instructions The system analyzes the data lineage in the development environment upon task submission and in the production environment upon task publication. The number of data lineage entries parsed is limited to 100,000 entries. Entries beyond this limit will not be recorded or appear in the asset catalog.

Submission details instructions

During the submission of an offline computing task, you can review the submission details, pre-check results, and enter remarks in the Submitting Log dialog box.

Submission content
The submission content includes the task object's name, type, change type, and details of the change. Information on the change encompasses basic information, computing code, runtime parameters, schedule configuration, schedule dependencies, runtime configuration, resource configuration.

Pre-check

The following pre-checks are performed during task submission. If any check item is incorrectly configured, the system will not support submission.

Check item	Description
Schedule dependencies	Dataphin optimizes business data output by systematically executing each process node according to the configured schedule dependencies. For more information, see Configure offline task schedule dependencies.
Runtime parameters	Runtime parameter configuration is essential for assigning values to variables within the code of a computing task, facilitating node scheduling. These parameter variables are designed to be automatically substituted with their respective values. For additional details, refer to Parameter configuration and node parameter usage.
Cross-node parameters	Pass-through parameters to direct descendant nodes of the current object node. For more information, see Parameter configuration and node parameter usage.

Submission remarks
You can include remarks with the task submission, limited to 128 characters.

Check item instructions

After all pre-checks are passed, click the OK And Submit button, and you can view the check items and their results in the Submitting Log dialog box.

Check item	Description
Configuration check	Checks include pre-check, code length, number of referenced resources, and offline code template version. Pre-check: Verifies completion of all required properties. Code length: Ensures the code does not exceed 100,000 characters. Otherwise, submission is not supported. Number of referenced resources: Checks if the number of resources referenced is within the limit of 100 resources. Offline code template version: Checks for changes in the offline code template version.
Parameter configuration	All variables in the code are parsed and listed, including parameter name, type, and current value. Checks if local variables have been assigned values.
Syntax check	For SQL tasks only, verifies the correctness of SQL syntax. Submissions with errors are not supported.
Object check	For SQL tasks only, checks if all referenced objects are submitted and published to the production environment. Lists all referenced objects in the code, including name, type, and check result. Object name: The format is `[catalog].[object name]`. Project space name variable: If `catalog` is a variable `${catalog}`, then `catalog` refers to the production space name. Hard coding: Directly written space names in the code are taken as is. If omitted, the system defaults to `${catalog}`. Omitting catalog: When catalog is omitted, the system defaults to the `${catalog}` project space name variable. Object type: Checked types include physical tables, fields, and logical tables. Check result: Indicates success or failure. Success: All objects are submitted and published to the production environment. Note If the catalog is hard-coded, the system will verify the existence of objects only within the specified environment. Failure: Objects not submitted or published to the production environment will result in failure. Note The check will fail if it encounters fields absent in the objects under scrutiny.
Permission check	The system parses all objects in the code and displays a list showing if you have operation permissions for them. The list includes the object name, object type, check result, row-level permission status, and the permission request operation. Object name: The format is `[catalog].[object name]`. Project space name variable: If `catalog` is a variable `${catalog}`, then `catalog` refers to the production space name. Hard coding: Directly written space names in the code are taken as is. If omitted, the system defaults to `${catalog}`. Omitting catalog: When catalog is omitted, the system defaults to the `${catalog}` project space name variable. Object type: Checked types include physical tables, global objects, logical tables, and functions. Note When cross-project reference authentication is enabled for the project where the referenced function is located, authentication is required for both personal accounts and production accounts during pre-editing, running, and submission. Check result: Checks if the current user and the tenant account have read and write permissions for the object type. The result can be Success or Failed. Note If the object is a table, you need read and write permissions for all referenced fields. If the object is a function, the tenant account needs read and write permissions for the corresponding offline function in Basic projects. In Dev-Prod projects, the personal account needs read and write permissions for the corresponding offline function. Row-level permission status: You must purchase the row-level permissions value-added service. If a table has row-level access control enabled and an account lacks row-level permissions, the system adds a Row-level Permission Status column. This column notifies you about the permission status for that account. The status can be Success, Failed, or Success with threats. Failed: The status is Failed if the personal account does not have row-level permissions for the object. Success with threats: In a Dev-Prod project, the status is Success with threats if the personal account has row-level permissions but the tenant account does not. Permission request: If an object check fails, click the icon at the bottom to request permissions. For more information, see My permissions.
Specification check	For SQL tasks only, the system performs a scan based on predefined specifications and presents the results for each check. For more information, see Coding specifications.
Dependency check	The system evaluates the task's dependency and output configurations. Dependency configuration: Verifies the existence of dependency node objects configured for the task in the development environment. Output configuration: Checks for duplicate task output names within the current tenant.
Code review	The system automatically checks for configurations that require a code review within the task. If such configurations are present, as in the case of referencing a global variable with code review requirements, a code review will be mandated. For more information, see View built-in approval templates. Important If code review is enabled for the project where the current node resides, a code review is triggered upon submission. This happens regardless of whether the offline node references a global variable that requires a review. If code review is enabled for the project and the node references a global variable that requires a review, all approval tasks in the code review check must pass before you can submit the node.
Operation execution	This check confirms the task submission has been executed. During this phase, the submission cannot be revoked.