Resource group orchestration lets you reassign the scheduling resource group for multiple nodes in a workflow at once. Use this feature when you need to redistribute workloads across resource groups — for example, moving compute-intensive nodes to a higher-spec resource group or consolidating nodes onto a serverless resource group.
After you switch resource groups, commit the changes and redeploy the affected nodes to apply the new assignment in the production environment.
Limitations
Resource group orchestration applies to auto triggered nodes only. Manual nodes are not supported.
Zero load nodes cannot be reassigned — they do not consume scheduling resources.
Resource group orchestration requires DataWorks Standard Edition or above.
Prerequisites
Before you begin, ensure that you have:
A workflow with at least one auto triggered node. See Create an auto triggered workflow.
One or more resource groups for scheduling in your workspace. To purchase a serverless resource group, see Create and use a serverless resource group.
How it works
Each resource group for scheduling has a maximum number of nodes it can run in parallel, determined by its specifications. When a node's resource group is changed, the node runs on the new resource group after the next deployment to production.
For resource group specifications and parallel node limits, see Billing of serverless resource groups.
To change resource groups for nodes already in the production environment, see View the auto triggered node list.
Switch resource groups for nodes
Go to the DataStudio page. Log on to the DataWorks console. In the top navigation bar, select a region. In the left-side navigation pane, choose Data Development and Governance > Data Development. Select your workspace from the drop-down list and click Go to Data Development.
Open the Resource Group Orchestration tab. In the Business Flow section, hover over the target workflow and click the
icon to the right of the workflow name.
Select and reassign nodes.
(Optional) Filter nodes by node name, node type, or engine type.
Select the nodes you want to reassign.
Click Switch Resource Groups and select the target resource group from the list.
Click OK.
NoteTo assign the same resource group to multiple nodes, select them all and switch in one batch. To assign different resource groups to different sets of nodes, perform the operation in separate batches.
Click the
icon to commit the changes.Redeploy the affected nodes. Go to the Deploy page and redeploy each node whose resource group was changed. The nodes run on the new resource group in the production environment after redeployment. For instructions, see Deploy nodes.
More operations
| Operation | Steps |
|---|---|
| Change the resource group for a single node | On the Resource Group Orchestration tab, click Switch in the Actions column for the node. |
| Modify a resource group's configuration | Click Settings to the right of the resource group name. On the DataWorks Overview page, go to Resource Groups in the left-side navigation pane. On the Exclusive Resource Groups tab, find the resource group and click Details in the Actions column. Applies to serverless resource groups and old-version exclusive resource groups. We recommend that you run tasks on serverless resource groups. |
| Purchase a resource group | Click Purchase Resource Group in the upper-right corner of the Resource Group Orchestration tab. |