DataWorks supports custom resource groups. If you have idle servers, you can add them in a custom resource group to run your nodes. This topic describes the details of custom resource groups.

Limits

  • You must activate DataWorks Enterprise Edition or a more advanced edition before you can submit a ticket to enable the whitelist feature and use custom resource groups for scheduling.
  • You must activate DataWorks Professional Edition or a more advanced edition before you can use custom resource groups for Data Integration.
  • You can use custom resource groups for Data Integration to run only data integration nodes. You can use custom resource groups for scheduling to run only Shell nodes.

Scenarios

  • Network: When you synchronize data, you must connect to your data center.
  • Migration: If a local node already exists, you can directly schedule the node on your own server to reduce the workload of script migration.

Other instructions

  • Create custom resource groups
    • For more information about how to create custom resource groups for Data Integration, see Create a custom resource group for Data Integration.
    • To create a custom resource group for scheduling, perform the following steps:
      1. Log on to the DataWorks console. In the left-side navigation pane, click Resource Groups. On the page that appears, click the Custom Resource Groups tab.
      2. Click Add scheduling resources in the upper-right corner.
      3. In the Add scheduling resources dialog box, enter a resource name, select the required workspace, and then click Confirm.
      4. Find the created custom resource group and click Server Management in the Actions column. In the Management Server dialog box, click Add server.
      5. In the Add scheduling resources dialog box, set the parameters as required and click Confirm.
        Parameter Description
        Network type The type of the network. Valid values: Classic Network and VPC. You can select the appropriate network type based on the actual network environment.
        Server Name or ECS UUID
        • The Server Name parameter is displayed only after you select Classic Network for the Network type parameter. Enter the hostname of an Elastic Compute Service (ECS) instance.
        • The ECS UUID parameter is displayed only after you select VPC for the Network type parameter. Enter the UUID of an ECS instance. You can run the dmidecode | grep UUID command to query the UUID.
        Machine IP The internal IP address of the ECS instance. You can log on to the ECS instance and run the hostname -i command to query the internal IP address.
      6. After the server is added, go back to the Custom Resource Groups tab and refresh the page. Find the created custom resource group and click Server initialization in the Actions column. Log on to the ECS instance and follow the initialization procedure as prompted.
  • The procedure for changing the resource group for a node to a custom resource group is the same as that to an exclusive resource group.
    • To change the resource group for a node to a custom resource group for scheduling, go to Operation Center.
    • To change the resource group for a node to a custom resource group for Data Integration, go to the DataStudio page. Change the resource group, and commit and deploy the node.