Gracefully undeploy tables and tasks - DataWorks - Alibaba Cloud Documentation Center

Data Asset Governance provides a comprehensive undeployment solution for invalid tasks or tables that are identified during data governance. The solution supports various features, such as undeployment risk evaluation, undeployment notification push, and undeployment process management. This solution eliminates the need to manually identify undeployment risks. This solution also resolves the issue that you cannot undeploy tasks whose descendant tasks are scheduled as normal or you cannot determine whether to undeploy tasks due to unpredictable risks. The solution allows you to undeploy multiple tasks or tables at the same time. This improves the undeployment efficiency.

Background information

Graceful undeployment supports undeployment impact analysis and undeployment process management, which are unavailable in the traditional undeployment. Task undeployment notifications can be sent to related contacts by using various methods at each phase of the task undeployment process.

Operation	Undeployment method	Comparison	Description
Select the objects that you want to undeploy	Direct undeployment	To directly undeploy a task, go to Operation Center. In the left-side navigation pane, choose Auto Triggered Node O&M > Auto Triggered Nodes. On the page that appears, find the task that you want to manage, click More, and then click Freeze in the Actions column.	You cannot view the impacts and risks of the undeployment and you may undeploy the wrong task.
Select the objects that you want to undeploy	Graceful undeployment	You can manually specify the objects that you want to undeploy or select the objects that are automatically recommended by the system. You can evaluate the impact scope and risks of the undeployment.	Select the objects that you want to undeploy based on node IDs, lineage, and tables. The impacts and risks of the undeployment are evaluated.
Configure undeployment settings	Direct undeployment	None.	Notifications cannot be pushed to the corresponding contacts.
Configure undeployment settings	Graceful undeployment	You can send notifications to the specified contacts in each phase of the graceful undeployment process by internal messages, emails, DingTalk chatbots, or webhooks.	The undeployment process can be handled through multiple phases and notifications are pushed to the corresponding contacts in each phase.
Specify the method to execute the undeployment plan	Direct undeployment	None.	After you click to undeploy a task, the task is immediately undeployed. You cannot cancel the undeployment.
Specify the method to execute the undeployment plan	Graceful undeployment	Manually Trigger Execution Automatic Execution	You can select a method to undeploy a task based on the scenario to save you sufficient time to cancel the undeployment.
View and manage the undeployment plan	Direct undeployment	None.	The selected task is directly undeployed without a silence period. You may easily select the wrong task and cause business interruptions.
View and manage the undeployment plan	Graceful undeployment	You can view the details of an undeployment plan and perform various operations on the tasks that are specified in the plan. For example, you can view the details of the tasks, cancel the task undeployment, or roll back the undeployment operation.	The undeployment can be canceled or rolled back when you identify errors within the silence period.

Limits

You can add only objects in the same workspace to an undeployment plan.
Up to 200 objects can be added to an undeployment plan.
The graceful undeployment feature allows you to roll back the operations performed on a task before the Undeploy Task phase of the process, and allows you to roll back the operations performed on a table before the Undeploy Completed phase of the process.

Permission management

To create an undeployment plan, you must meet one of the following requirements:
- You have an Alibaba Cloud account.
- You have created a Resource Access Management (RAM) user and the AliyunDataWorksFullAccess policy is attached to the RAM user.
- You are a workspace administrator.
- You are a tenant-level data governance administrator.
- You are a workspace-level data governance administrator.
- You are an O&M engineer.
Only Alibaba Cloud accounts, RAM users to which the AliyunDataWorksFullAccess policy is attached, and workspace administrators can perform all operations on an undeployment plan.
O&M engineers have full permissions only on the undeployment plans that they create. O&M engineers have only read permissions on the undeployment plans that are created by other users.
Tenant-level or workspace-level data governance administrators can create, configure, and delete undeployment plans but cannot start or execute tasks on which they do not have permissions.

Usage notes

You cannot select an object that is already added to an undeployment plan. Each object can be added only to one undeployment plan.
Specific types of objects cannot be added to an undeployment plan. For example, you cannot add the root node of a workspace to an undeployment plan or add a zero load node that you created for an undeployment plan to the undeployment plan.
If an object that you want to undeploy has descendant objects and the scheduling dependencies between the descendant objects and the object are not removed within the specified period of time, the system generates a zero load node and configures the zero load node as the ancestor node of the descendant objects.
The operations that you need to perform to create a node undeployment plan vary based on the type of node. You can follow the on-screen instructions to create a node undeployment plan for a type of node.
The graceful undeployment feature allows you to roll back an undeployment operation only before the object is undeployed.

Procedure

Step 1: Create an undeployment plan

Go to the Data Asset Governance page.
Log on to the DataWorks console. In the top navigation bar, select the desired region. In the left-side navigation pane, choose Data Governance > Data Asset Governance. On the page that appears, click Go to Data Asset Governance.
In the left-side navigation pane, choose Governance > Automation > Shutdown to go to the Shutdown page.
In the upper part of the Shutdown page, select the workspace to which the objects that you want to undeploy belong from the Workspace drop-down list. Only the workspaces to which you are added as a member are displayed in the drop-down list.
To join a workspace, you must contact the administrator of the workspace. For more information, see Add a RAM user to a workspace as a member and assign roles to the member.
On the Shutdown page, click Create Plan.
You can also go to the To-Do List page, find the table or task that you want to undeploy, and click Create Plan in the Actions column to go to the Create Plan page. For more information, see Handle governance issues.

Step 2: Select the objects that you want to undeploy

1. Specify the basic information about an undeployment plan

You can specify the workspace to which the objects that you want to undeploy belong from the Workspace drop-down list and enter a custom plan name in the Plan Name field.

2. Select the objects that you want to undeploy

In the To-Be-Undeployed Object section, you can select deployed tasks or tables based on your business requirements.

Undeploy tasks: Select Add to-be-undeployed Object or Import Based on Lineage to add tasks that you want to undeploy.
- Add to-be-undeployed Object: Add tasks that you want to undeploy by specifying the node IDs of the tasks.
- Import Based on Lineage: Select a task and select tasks based on lineage levels.
  Note
  - You can view the node ID of the task in the General section of the Properties panel on the task configuration page.
  - You can also go to Operation Center and choose Auto Triggered Node O&M > Auto Triggered Nodes in the left-side navigation pane to view the node ID of the task.
Undeploy tables: You can select Add to-be-undeployed Object to undeploy tables.

Note

You cannot select an object that is already added to an undeployment plan. Each object can be added only to one undeployment plan.
Specific types of objects cannot be added to an undeployment plan. For example, you cannot add the root node of a workspace to an undeployment plan or add a zero load node that you created for an undeployment plan to the undeployment plan.

Item	Description
Undeployment Impact Assessment	Low: The table or output table of the task within the previous month has not been consumed. High: The table or output table of the task within the previous month has been consumed.
Impact Item Description	The description of the undeployment impact assessment.
The output table is consumed.	Information about the consumption of the output table of the task.
Task That Generates Table	The output task of the table. You can click it to view the task information.
Number of Affected Baselines/Highest Baseline Level	The number of baselines affected by the output task of the table and the highest baseline level. The number of baselines affected by the task and the highest baseline level. Note For more information about task baseline monitoring, see Manage baselines.

Step 3: Configure undeployment settings

After you add objects that you want to undeploy, you can configure fine-grained undeployment settings, including the Start Undeployment, Delayed Scheduling, Suspend Scheduling, Undeploy Task, Rename Output Table, Reset Table Lifecycle, Delete Output Table, and Undeploy Completed phases.

Phase	Description	Scenario
Start Undeployment	Start the graceful undeployment. This phase is selected by default and cannot be unselected.	-
Delayed Scheduling	Delay the scheduling time of the objects that you want to undeploy. In this case, you delay the time when the output tables of the objects and their descendant objects are generated.	If no data is generated by the ancestor platform, you can select this phase to ensure that the tasks can run as normal.
Suspend Scheduling	Freeze the objects that you want to undeploy. To freeze the objects, go to Operation Center. In the left-side navigation pane, choose Auto Triggered Node O&M > Auto Triggered Nodes. On the page that appears, find the objects that you want to freeze and freeze the objects. The objects and their descendant objects no longer generate tables.	You can select this phase if you have suspended a business but want to keep the workflow.
Undeploy Task	Undeploy the desired objects. After the objects are undeplyed, no instance is generated on the next day. The scheduling mode of the existing instances is set to dry run and the system no longer runs these instances. This phase is selected by default and you can unselect it.	You can select this phase to undeploy a node or workflow that is no longer needed.
Rename Output Table	Rename the output table of a task or table.	You can select this phase if you want to identify the output table of an undeployed object.
Reset Table Lifecycle	Reset the lifecycle of the output table of a task or table.	You can select this phase if you want to retain the output table of an undeployed object for a period of time. Note Only MaxCompute tables are supported.
Delete Output Table	Delete the output table of a task or table.	You can select this phase you do not need the output table of an undeployed object.
Undeploy Completed	End the graceful undeployment. This phase is selected by default and cannot be unselected.	-

Note

The steps in the phases of the graceful undeployment plan are performed in sequence. You can specify whether to perform the step in each phase. If you unselect a phase, the phase is skipped by default and the system proceeds to the next phase.
The graceful undeployment plan allows you to send notifications to the specified contacts by internal messages, emails, and DingTalk chatbots. For more information about how to configure a DingTalk chatbot, see Scenario practices: Send alert notifications to a DingTalk group. For information about internal messages, go to Data Asset Governance and choose Overview > Reminder in the left-side navigation pane to view messages on the Governance Reminder page.

Step 4: Specify the method to execute the undeployment plan

You can specify the method to perform the step in each phase of the undeployment plan. You can select Manually Trigger Execution or Automatic Execution for the Execution Mode parameter.

Manually Trigger Execution
- Note: Each step of the undeployment process can be performed only after the previous step is confirmed by the creator of the undeployment plan.
- Recommended scenario: The task or table is used in important situations of the business workflow.
Automatic Execution
- Note: Each step of the undeployment process is automatically performed based on the silence period that you configured. A step is automatically performed when the silence period of the previous step ends.
- Recommended scenario: You need to undeploy a large number of business workflows and the importance of the workflows is low.

Step 5: View and manage the undeployment plan

On the Shutdown page, you can view the list of undeployment plans. You can execute, cancel, or delete an undeployment plan.

Note

If you cancel an undeployment plan, the system cancels undeployment for tasks that are in the Undeploying state in the undeployment plan. Undeployed tasks are not affected.
If you delete an undeployment plan, the system cancels undeployment for tasks that are in the Undeploying state in the undeployment plan. Undeployed tasks are not affected. After the undeployment is canceled, the system deletes the undeployment plan. The deletion operation is irreversible.

You can click Execution Details in the Actions column of an undeployment plan to view the configuration details of the plan. You can also perform the operations that are described in the following table on a single task.

Operation	Description
View undeployment details	You can view the undeployment details of a single task, such as the undeployment progress, task status, and execution details of each phase. You can also perform various operations on the task, such as canceling undeployment for the task and rolling back the undeployment operation for the task. Cancel undeployment for a single task: You can cancel the unfinished steps of the task undeployment process for the task. Roll back the undeployment operation for a single task: You can roll back all undeployment steps for the task to the initial states. You cannot roll back a single step for the task. Note If an undeployment plan is canceled, you cannot perform the rollback operation. If a substep of the step in the current phase is being performed, you cannot perform the rollback operation. You cannot roll back the operations performed on a task after the undeployment phase. You cannot roll back the operations performed on an object (task or table) after the undeployment completion phase.
Go to Operation Center	You can go to Operation Center in the production environment to view the O&M details of the task. For more information, see Overview.
Go to the configuration tab of the task	You can go to the configuration tab of the task in DataStudio to modify the task. For more information, see DataStudio (old version).