The resource group orchestration feature of DataWorks allows you to change the resource groups for scheduling for nodes in the specified workflow. If you have multiple resource groups for scheduling in your workspace, you can change the resource groups for scheduling for nodes based on your business requirements. This helps you promote resource efficiency. This topic describes how to use the resource group orchestration feature of DataWorks.

Prerequisites

  • A workflow is created. The resource group orchestration feature allows you to change the resource groups for scheduling for nodes in a specified workflow. For more information about how to create a workflow, see Create a workflow.
  • One or more resource groups for scheduling are created for your workspace. This way, you can use the resource group orchestration feature to allocate resource groups for scheduling in the current workspace.

    If you activate DataWorks, the system automatically creates the shared resource group for scheduling for you to run nodes. If the shared resource group for scheduling cannot meet your requirements, you can purchase exclusive resource groups for scheduling as needed. For more information about exclusive resource groups for scheduling, see Exclusive resource groups for scheduling.

Background information

The resource group orchestration feature is used to change the resource groups for scheduling used by multiple nodes in a specified workflow during data development. The maximum number of concurrent nodes that can be run by a resource group for scheduling varies based on the specifications of the resource group. You can specify the nodes to run on the resource group for scheduling whose specifications meet the needs of the nodes. For more information about the specifications of exclusive resource groups for scheduling and the maximum number of concurrent nodes supported by each specification, see Performance metrics and billing standards of exclusive resource groups for scheduling.
Note For information about how to change the resource groups for scheduling for multiple nodes in the production environment, see View auto triggered nodes.

Limits

  • DataWorks allows you to change the resource groups for scheduling only for auto triggered nodes.
  • Zero load nodes do not occupy scheduling resources. Therefore, you cannot change the resource groups for scheduling for zero load nodes.

Procedure

  1. Go to the DataStudio page.
    1. Log on to the DataWorks console.
    2. In the left-side navigation pane, click Workspaces.
    3. In the top navigation bar, select the region where the required workspace resides, find the workspace, and then click Data Analytics.
  2. Go to the Resource Group Orchestration tab.
    On the DataStudio page, move the pointer over the required workflow and click the Resource Group Orchestration tab icon next to the workflow name to go to the Resource Group Orchestration tab.
    Note Before you go to the Resource Group Orchestration tab, you must find the workflow in which you want to change the resource groups for scheduling for the nodes.
    Resource Group Orchestration tab
  3. Change the resource groups for scheduling for multiple nodes.
    Note
    • You can specify the same resource group for scheduling for the nodes that run on different resource groups for scheduling at a time.
    • If you want to specify different resource groups for scheduling for the nodes that run on multiple resource groups, you must specify a resource group for scheduling for part of the nodes at each time.
    Change the resource groups for scheduling for multiple nodes
    1. Optional:Find the required nodes by configuring the filter conditions, such as the Name, Node Type, and Engine Type parameters.
    2. Check the nodes in the search results and select the nodes for which you want to change the resource groups for scheduling.
    3. Click Switch Resource Groups and select a resource group for scheduling from the list.
    4. Click OK.
    You can perform the following operations for the resource groups for scheduling:
    • Scale out the resource groups for scheduling: You can scale out only the shared resource groups for scheduling. Click Expand Capacity next to the Common scheduler resource group. Then, the Public Resource Groups tab appears. You can click Buy Now in the Public Resource Group Usage section to purchase resource plans of shared resource groups for scheduling.
    • Modify specifications: You can modify specifications for only exclusive resource groups for scheduling. Click Settings next to the name of an exclusive resource group for scheduling. Then, the Exclusive Resource Groups tab appears. You can view the details of the exclusive resource group for scheduling and modify specifications as needed.
    • Purchase resource groups for scheduling: If no resource groups are available in the current workspace, you can click Purchase Resource Group in the upper-right corner to purchase resource groups for scheduling.
    • Change the resource group for scheduling configured for a single node: Click Switch in the Actions column that corresponds to the node to change the resource group for scheduling for the node.
  4. Click the Submit icon icon to commit the resource group changes.
  5. Deploy the nodes whose resource groups for scheduling are changed.
    After the resource groups for scheduling of nodes are modified, you must deploy the nodes to the production environment on the Deploy page. After that, the nodes are run on the newly allocated resource group for scheduling in the production environment.