DataWorks provides custom resource groups for scheduling and custom resource groups for Data Integration to ensure that nodes are flexibly scheduled and data is synchronized as early as possible. This topic describes how to create a custom resource group for scheduling and change the resource group for a node.

Prerequisites

DataWorks Enterprise Edition or a more advanced edition is activated so that you can use custom resource groups for scheduling.

Background information

Custom resource groups for scheduling allow you to increase the number of nodes that can be concurrently scheduled. Custom resource groups for Data Integration allow you to increase the data synchronization speed. You must purchase Elastic Compute Service (ECS) instances to use custom resource groups for scheduling and custom resource groups for Data Integration.
Custom resource groups and exclusive resource groups have the following differences:
  • Custom resource groups are hosted on your ECS instances that are completely under your control. You can log on to these ECS instances. However, you must manage these ECS instances on your own.
  • Exclusive resource groups are fully managed. You do not need to worry about availability or server issues.
Resource groups for scheduling are used to run or distribute nodes that are delivered by the scheduling system. DataWorks supports the shared resource groups for scheduling and custom resource groups for scheduling.
Note By default, the nodes you commit for scheduling are run on the shared resource group for scheduling provided by DataWorks.

For a single user, the shared resource group cannot unlimitedly bear the increasing number of concurrent nodes for scheduling. If a large number of nodes need to be concurrently scheduled, they may wait for resources. This delays the running of these nodes.

If you require higher scheduling performance, purchase ECS instances and configure custom resource groups for scheduling. For more information about commonly asked questions and answers about the use, configuration files, and commands of custom resource groups, see Custom resource groups.

Procedure

  1. Go to the Custom Resource Groups tab.
    1. Log on to the DataWorks console.
    2. In the left-side navigation pane, click Resource Groups.
    3. On the Resource Groups page, click the Custom Resource Groups tab.
      Custom Resource Groups
  2. Create a custom resource group for scheduling.
    1. On the Custom Resource Groups tab, click Add scheduling resources in the upper-right corner.
    2. In the Add scheduling resources dialog box, set the Resource Name and Belonging workspace parameters.
    3. Click Confirm.
  3. Manage servers.
    1. On the Custom Resource Groups tab, find the created custom resource group and click Server Management.
    2. In the Management Server dialog box, click Add server.
    3. In the Add scheduling resources dialog box, set the parameters as required.
      Add scheduling resources
      Parameter Description
      Network type Valid values: Classic Network and VPC.
      Note You can select Classic Network only in the China (Shanghai) region. We recommend that you set the Network type parameter to VPC.
      Server Name Enter the host name of the ECS instance, which cannot be customized. You can run the hostname command on an ECS instance to obtain the host name.
      Note This parameter is displayed only when the Network type parameter is set to Classic Network.
      ECS UUID You can run the dmidecode | grep UUID command on an ECS instance to obtain the universally unique identifier (UUID) of the ECS instance.
      Note This parameter is displayed only when the Network type parameter is set to VPC.
      Machine IP You can run the hostname -i command on an ECS instance to obtain the host IP address.
      Notice Do not confuse ECS instances on the classic network and their UUIDs and server IP addresses with those in virtual private clouds (VPCs).
    4. Click Confirm.
  4. Initialize a server.
    1. On the Custom Resource Groups tab, find the created custom resource group and click Server initialization.
    2. Log on to the required ECS instance and follow the initialization procedure in the Server initialization dialog box.
  5. Go to the DataStudio page.
    1. Go back to the DataWorks console.
    2. In the left-side navigation pane, click Workspaces.
    3. Select the region where the required workspace resides. Find the workspace to which the created custom resource group belongs and click Data Analytics.
  6. Change the resource group for scheduling for a node.
    You can change the resource group for scheduling for a node in Operation Center. You can also change the resource group on the Properties tab or the configuration tab of the node.
    • Change the resource group for scheduling for a node in Operation Center
      1. On the DataStudio page, click the DataWorks icon icon in the upper-left corner and choose All Products > Task Operation > Operation Center.
      2. In the left-side navigation pane, choose Cycle Task Maintenance > Cycle Task.
      3. Click the rightwards arrow in the middle of the page to show the node list. Find the required node and choose More > Modifying a scheduling Resource Group.Change the resource group
        Notice You are not allowed to change the resource group for zero load nodes, workflow nodes, or Machine Learning experiment nodes.
        To change the resource group for multiple nodes at a time, select the required nodes on the Cycle Task page and click Modifying a scheduling Resource Group in the lower part of the page.Change the resource group for multiple nodes at a time
      4. In the Modify scheduling resource groups in batches dialog box, select the created custom resource group for scheduling and click OK.
    • Change the resource group for scheduling for a node on the Properties tab
      1. Click the DataWorks icon icon in the upper-left corner and choose All Products > Data Development > DataStudio. Switch to the required workspace.
      2. On the Data Analytics tab, double-click the required node. The node configuration tab appears.
      3. In the right-side navigation pane, click the Properties tab. In the Resource Group section, select the required resource group on which the node is run. For more information, see Configure the resource group.Resource Group
    • Change the resource group for scheduling that is used for test run on the configuration tab of the node
      1. Click the Run icon in the toolbar.
      2. In the Arguments dialog box, set the Resource Group and Custom Arguments parameters for testing and running the node on the DataStudio page.
      3. Click OK.