Create custom resource groups for scheduling - Workbench| Alibaba Cloud Documentation Center

DataWorks provides you with custom resource groups for scheduling and custom resource groups for Data Integration to ensure the flexibility of node scheduling and the timeliness of data synchronization. This topic describes how to create a custom resource group for scheduling and change the resource group for a node to the created custom resource group for scheduling.

Prerequisites

Custom resource groups for scheduling are only available in DataWorks Enterprise Edition or more advanced editions.

Background information

Custom resource groups for scheduling provide you with the capability of increasing the number of nodes that can be concurrently scheduled. Custom resource groups for Data Integration provide you with the capability of flexibly improving the data synchronization speed. You must purchase Elastic Compute Service (ECS) instances to use custom resource groups for scheduling and custom resource groups for Data Integration.

Custom resource groups and exclusive resource groups have the following differences:

Custom resource groups are hosted on your ECS instances. You can log on to these ECS instances. In addition, you must manage these ECS instances on your own.
Exclusive resource groups are hosted by the system. You do not need to worry about availability or server issues.

Resource groups for scheduling are used to run nodes or distribute nodes that are delivered by the scheduling system. DataWorks provides default resource groups for scheduling and custom resource groups for scheduling.

Note By default, the nodes you commit are run on default resource groups for scheduling provided by DataWorks.

For a single user, the scheduling capabilities of default resource groups cannot bear the increasing number of concurrent scheduling nodes. If a large number of nodes need to be concurrently scheduled, they may wait for resources. This also delays the running of these nodes.

If you require higher scheduling performance, purchase ECS instances and configure custom resource groups for scheduling. For more information about commonly asked questions and answers about the use, configuration files, and commands of custom resource groups, see Custom resource groups.

Procedure

Go to the Custom Resource Groups tab.
1. Log on to the DataWorks console.
2. In the left-side navigation pane, click Resource Groups.
3. On the Resource Groups page, click the Custom Resource Groups tab.
Create a custom resource group for scheduling.
1. On the Custom Resource Groups tab, click Add scheduling resources in the upper-right corner.
2. In the Add scheduling resources dialog box, enter a resource name and select the workspace.
3. Click Confirm.
Manage servers.
1. On the Custom Resource Groups tab, find the created custom resource group and click Server Management in the Operation column.
2. In the Management Server dialog box, click Add server.
3. In the Add scheduling resources dialog box, select a network type and enter the universally unique identifier (UUID) and server IP address of an ECS instance.
  Note
  
  For the network type, you can select Classic Network only for ECS instances in the China (Shanghai) region. We recommend that you select VPC.
  
  Do not confuse ECS instances on the classic network and their UUIDs and server IP addresses with those in virtual private clouds (VPCs).
4. Click Confirm.
Initialize a server.
1. On the Custom Resource Groups tab, find the created custom resource group and click Server initialization in the Operation column.
2. Log on to the target ECS instance and follow the initialization procedure in the Server initialization dialog box.
Go to the DataStudio page.
1. Go back to the DataWorks console.
2. In the left-side navigation pane, click Workspaces.
3. In the top navigation bar, select the region where the target workspace resides. Find the workspace to which the created custom resource group belongs and click Data Analytics in the Actions column.
Change the resource group for a node to the created custom resource group for scheduling.
You can change the resource group for a node in Operation Center or on the Properties tab of the node.
- To change the resource group for a node in Operation Center, perform the following steps:
  1. On the DataStudio page, click in the upper-left corner and choose All Products > Task Operation > Operation Center.
  2. In the left-side navigation pane, choose Cycle Task Maintenance > Cycle Task.
  3. On the page that appears, click the rightwards arrow in the middle of the page to show the node list. Find the target node, click More in the Actions column, and then select Change Resource Group. The Change Resource Group dialog box appears.
    
    Notice You are not allowed to change the resource group for zero load nodes, workflow nodes, or Machine Learning experiment nodes.
    
    To change the resource group for multiple nodes of the same type, select the nodes on the Cycle Task page and click Change Resource Group in the lower part of the page.
  4. In the Change Resource Group dialog box, select the created custom resource group for scheduling and click OK.
- To change the resource group for a node on the Properties tab, perform the following steps:
  1. Click in the upper-left corner and choose All Products > Data Development > DataStudio. Switch to the workspace to which the created custom resource group belongs.
  2. On the Data Analytics tab of the DataStudio page, double-click the target workflow. On the workflow dashboard that appears, double-click the target node to go to the node configuration tab.
  3. In the right-side navigation pane, click the Properties tab. In the Resource Group section, select the created custom resource group for scheduling. For more information, see Configure the resource group.
  On the node configuration tab, you can also click . In the Arguments dialog box, select the created custom resource group for scheduling to run a test for the node on the DataStudio page and click OK.