Dataphin:Manage publishing tasks

Publish a job

Publishing a node moves it from the list of objects to be published to the production environment. If a node is changed in the development environment, the existing version of the node in the production environment is updated after you publish the change.

• Before you publish a single node or multiple nodes in a batch, the system runs a dry run to check whether the number of data processing units (DPUs) required for publishing exceeds the available quota. If the quota is exceeded, the publishing is blocked.

• If the dry run passes, the system checks each object to be published. After all checks pass, the system verifies the DPU quota again before publishing each object. If the quota is exceeded, the publishing of the current object fails.

Procedure

1. On the Dataphin homepage, choose Developer > Publish from the top menu bar.

2. In the navigation pane on the left, choose Publish Object Management > Objects To Publish.

3. (Optional) On the Objects To Publish page, click Filter to filter objects by project, recent submitter, object type, recent submission time, or change type. You can also select the My Submissions and Today's Submissions options to quickly filter the objects.

4. On the Objects To Publish page, you can publish a single object or multiple objects in a batch.

   • Publish a single object: Click the icon in the Actions column for the target object to open the Publish Configuration dialog box.

   • Publish multiple objects in a batch

     1. Select multiple objects to publish, or click Select All On This Page at the bottom of the page.

     2. Click Batch Publish at the bottom of the page to open the Publish Configuration dialog box.

5. In the Publish Configuration dialog box, enter a Publish Name and Remarks, and select the Dependent Objects and a Publish Scope.

   • Dependent Objects: This list includes all upstream objects that will be published because your selected objects depend on them. If you do not have permission to publish a dependent object, the object does not appear in the list, and the system automatically generates an approval request.

   • Publish Scope: You can select Must-follow And Optional Dependencies, Only Must-follow Dependencies, or Only Selected (Do Not Publish Upstream Dependencies). The default is Only Selected (Do Not Publish Upstream Dependencies). If you select Only Selected (Do Not Publish Upstream Dependencies), the system clears the dependent object list and publishes only the selected objects, not their upstream dependencies.

     Note

     Assume that object A depends on object B. For example, node A depends on node B, or an atomic metric A depends on its source logical table B. When you publish the objects, the following rules apply:

     • Mandatory dependency: If the change type of B is New and the change type of A is New or Changed (meaning A adds a new upstream dependency on B), you must publish B before you publish A. Otherwise, the publishing of A fails. In this case, B is a mandatory dependency of A. If the change type for both A and B is Delete, A is a mandatory dependency of B.

     • Optional dependency: If the change type of B is Changed and the change type of A is New or Changed, A can be published successfully even if B is not published first. In this case, B is an optional dependency of A. Note that in this case, if you publish only A without publishing B, A will run based on the old logic of B. This may lead to unexpected results.

     • If you do not have the permission to publish B, B is not included in the list of objects to be published. In this case, the publishing of downstream objects that depend on B may fail.

6. Click OK to publish.

   The system publishes objects based on their dependency order.

   For example, you are publishing object A. The system detects that the dependencies are A <- L1 <- L2 <-L3 (A depends on L1, L1 depends on L2, and L2 depends on L3), and each dependency is mandatory. If you have the permission to publish L1 and L3 but not L2, the deployment package contains the publishing records of L3, L1, and A. The publishing of L1 and A fails. The publishing record of L2 is not included.

After the node is published, you can view the check details in the publishing details. For more information, see View publishing details.

Republish

If a node fails to publish, you can modify it based on the failure message and republish it from the Publishing History list.

• For objects that require release approval, if the node has already been approved, you do not need to request approval again to republish it. The republishing is automatically approved.

• Each republication undergoes a sales validation.

Procedure

1. From the top menu bar on the Dataphin homepage, choose Developer > Publish.

2. In the left navigation pane, choose Publish Object Management > Publishing History.

3. On the Publishing History page, select Failed in the upper-right corner to quickly filter for failed objects.

4. (Optional) Click Filter in the upper-right corner to filter objects by project, publisher, object type, publish time, change type, or publish status.

5. In the list of publishing records, republish a single object or multiple objects.

   • Republish a single object: Click the icon in the Actions column for the object that you want to republish.

   • Republish failed and canceled objects: Click the icon next to the Publish Name of the target deployment package to republish all failed and canceled objects in the package.

6. In the dialog box that appears, click OK to republish.

Release approval

Dataphin lets you configure the release approval feature at the project level. If you enable this feature, an approval process is triggered when you publish an object. The object can enter the check phase only after the publish request is approved. This feature helps you better manage the release process. This section describes how to enable the release approval feature and view the approval process.

1. In the top menu bar on the Dataphin homepage, choose Planning > Project.

2. On the project management page, click Information Settings on the target project card to open the Information Settings dialog box.

3. In the Information Settings dialog box, under Publish Settings, click the switch to enable or disable release approval. If you enable this feature, configure the following parameters:

   • Approver: You can select Administrator or Custom. If you select Administrator, all administrators of the current production project are assigned as approvers. If you select Custom, you must select up to 10 default approvers.

   • Approval Policy: The default is Parallel Approval. With this policy, a request is approved if any approver approves it, and the process is terminated if any approver rejects it.

4. On the Publishing History page, click the icon next to a deployment package name to view its release approval process.

   In the release approval process dialog box, you can view information, such as the approval status and approvers for each approval request. You can click the icon to navigate to the Task Hub to view the details of the corresponding approval request.

5. Approvers can click the icon in the upper-right corner of the page to go to the Task Hub and approve publishing requests for nodes. For more information, see View and process tasks.

Edit task

1. On the Pending Objects List page, click the icon in the Actions column for the target object, and select Edit to open the developer page for the corresponding project.

2. Navigate to the module that corresponds to the node to edit it.

Remove objects

Removing an object deletes it from the list of objects to be published. This operation does not affect existing versions of the object in the production or development environments. To apply changes to the object in the production and development environments, you must submit and publish the object again.

Procedure

1. (Optional) On the Objects To Publish page, click Filter to narrow down the list of objects by project, recent submitter, object type, recent submission time, or change type. You can also select My Submissions and Today's Submissions to quickly filter the objects.

2. On the Objects To Publish page, you can remove one or more objects.

   • To remove a single object from being published: In the Actions column for the target object, click the icon and select Remove.

   • To remove objects in a batch: Select the objects to remove, or click Select All On This Page. Then, click Batch Remove at the bottom of the page.

3. In the dialog box, click OK to complete the removal.

View publishing history

On the Publishing History page, you can view published nodes, including the publishing records for all versions of each node. You can perform the following operations on the publishing records:

• View publishing details: For more information, see View publishing details.

• Republish: Republish objects that failed to publish. For more information, see Republish.

• View version details: Click the number in the Version Number column for an object to view the details of its current published version.

• Go to O&M: This feature is available for objects that are successfully published and do not have a publish type of Delete. Click the icon in the Actions column for an object to go to the corresponding module on the O&M page.

On the Objects To Publish page, you can click the icon in the Actions column for the target object to view its publish history details.

View publishing details

1. On the Publishing History page, click the icon in the Actions column of the target object to open the Publishing Details dialog box.

2. In the Publishing Details dialog box, you can view information for the current object, including Basic Information, Flow Check, and Exception Alert.

   Publishing feature	Description
   Release approval	This process is available if release approval is enabled for the object. When you republish an object that was approved but failed to be published, the release approval step is automatically passed.
   Control rule check	This process is available if the object hits a release control policy. If a control rule is hit, information such as Control Policy, Control Rule, Exception Description (if any), and Check Result is displayed. Click the icon in the Actions column to view the details of the rule check.
   Execution condition check	Different object types have different check conditions. You can view the check details in the publishing details. The type, name, exception description (if any), and check result of the execution condition check are displayed. If the check fails, you can modify the object based on the on-screen instructions and use the shortcut button to republish it. If the release approval has been granted, you do not need to apply for approval again to republish the object.
   Availability check	This process is available if the object consumes DPUs. Only auto triggered tasks are subject to a resource usage check. One-time tasks do not require this check. Integration sync includes offline pipeline nodes, sync tasks, and real-time integration nodes. Data processing includes offline computing nodes and real-time computing nodes. Data modeling includes logical dimension tables, logical fact tables, derived metrics, and derived metrics. Note After a node is published, a resource usage check is required for objects that consume DPUs. Only auto triggered tasks are subject to this check. For more information about statistics, see View resource usage and Statistical methods. Files (ad hoc query files), table management (metadata tables, mirrored tables, and offline physical tables), offline computing templates, real-time computing templates, and registered metrics do not require a resource usage check.

3. For successfully published objects, click Close to close the Publishing Details dialog box. For failed objects, click Republish to retry publishing the current object.