DataWorks allows you to bind an exclusive resource group for scheduling to a virtual private cloud (VPC) so that the resource group can access data stores in the VPC.

Prerequisites

An exclusive resource group is properly connected to a data store before you use the exclusive resource group for scheduling to access the data store.

Background information

Assume that you do not need to connect an exclusive resource group to a data store. If you need to fix issues where nodes are delayed because resources in the shared resource groups for scheduling are insufficient, you can ignore the information about network configuration in this topic. In this case, you can purchase an exclusive resource group for scheduling in any zone without the need to bind the resource group to a VPC.
Notice
  • We recommend that you purchase an exclusive resource group for scheduling in the same zone as the data store to be accessed.
  • The exclusive resource group for scheduling must be bound to the same VPC and vSwitch as the data store to be accessed.
  • If you have purchased an exclusive resource group for scheduling in a different zone from the data store to be accessed, bind the resource group to the same VPC as the data store. Then, configure a routing rule for the resource group to access the data store. For more information, see Add a route.
  • An exclusive resource group for scheduling may still fail to access a data store after you bind the resource group to the same VPC and vSwitch as the data store. This is because the resource group is not in the whitelist configured for the data store. In this case, add relevant information, such as the vSwitch classless inter-domain routing (CIDR) block of the VPC, to the whitelist. For more information, see Add the exclusive resource group for scheduling to the whitelist of the data store to be accessed.

The maximum number of concurrent instances that are allowed varies with the specifications of exclusive resource groups for scheduling. For more information, see Performance metrics and billing standards of exclusive resource groups for scheduling.

Create an exclusive resource group for scheduling

  1. Log on to the DataWorks console.
  2. In the left-side navigation pane, click Resource Groups.
  3. Purchase an exclusive resource group for scheduling.
    If you have not purchased any exclusive resources for scheduling in the required region, perform the following steps to purchase resources:
    1. On the Exclusive Resource Groups tab, click Create a dedicated resource group.
    2. In the Create a dedicated resource group panel, click Purchase next to Order Number. The buy page appears.
      Buy page
    3. Set the Region, Type, Exclusive scheduling resources, Units, and Duration parameters.
      Note
      • In this example, set the Type parameter to Exclusive scheduling resources.
      • Exclusive resource groups cannot be shared among regions. For example, the exclusive resource groups in the China (Shanghai) region can only be used by workspaces in the China (Shanghai) region.
    4. Click Buy Now.
  4. Add the exclusive resource group for scheduling.
    1. Return to the Create a dedicated resource group panel and set the parameters as required.
      Parameter Description
      Resource Group Type The type of the exclusive resource group. In this example, select Exclusive Resource Groups.

      Valid values: Exclusive Resource Groups and Exclusive Resource Groups for Data Integration. The former type is used to schedule general nodes, whereas the latter type is used to schedule sync nodes.

      Resource Group Name The name of the resource group. The name must be unique within all resource groups of a tenant.
      Note A tenant indicates an Alibaba Cloud account. Multiple RAM users may exist under a tenant.
      Resource Group Description The description of the resource group.
      Order Number The order number of the exclusive resource group.
    2. Click OK.
      Note The exclusive resource group is initialized within 20 minutes. Wait until its status changes to Running.

Add a VPC binding

Exclusive resource groups are deployed in a VPC that is managed by DataWorks. To allow exclusive resource groups to access data stores in your own VPC, you must bind the exclusive resource groups to your VPC.

  1. Find the required resource group and click Add VPC Binding.
    Before the binding, authorize DataWorks to access your cloud resources in the RAM console.
  2. After the authorization is complete, click Add Binding.
  3. In the Add VPC Binding panel, set the parameters as required.
    Note The exclusive resource group and the data stores to be accessed must be in the same zone. Select the vSwitch to which the data stores are bound.
    Parameter Description
    Resource Group Name Select an exclusive resource group to be bound to a VPC from the Resource Group Name drop-down list.
    VPC The VPC to which the exclusive resource group is to be bound.
    VSwitch The vSwitch to which the data store to be accessed is bound.
    Security Groups The security group to be bound to the required VPC.
  4. Click OK.
  5. Optional:If the exclusive resource group and the data store to be accessed are bound to the same VPC but reside in different zones, configure a routing rule.
    1. Find the required resource group and click Custom Route.
    2. In the Custom Route panel, click Add Route.
    3. In the Add Route dialog box, set the parameters as required.
      Add Route
      Parameter Description
      Destination Type Valid values: VPC and IDC.
      Destination VPC The region and name of the destination VPC.
      Note This parameter is displayed only when the Destination Type parameter is set to VPC.
      Connection Method Valid values: Switch, Fixed IP Address and CIDR Block.
      • If you set the Connection Method parameter to Switch, you must also set the Destination VSwitch parameter.
        Note This parameter is displayed only when the Destination Type parameter is set to VPC.
      • If you set the Connection Method parameter to Fixed IP Address, you must also set the Fixed IP Address parameter.
      • If you set the Connection Method parameter to CIDR Block, you must also set the Destination CIDR Block parameter.
      Routing By default, the parameter cannot be modified.
    4. Click Generate Route.

Add the exclusive resource group for scheduling to the whitelist of the data store to be accessed

The exclusive resource group for scheduling may still fail to access a data store when the resource group and data store reside in the same zone and are bound to the same VPC and vSwitch. This is because the resource group is not in the whitelist configured for the data store. In this case, add the following information to the whitelist:
  • vSwitch CIDR block of the VPC to which the exclusive resource group for scheduling is bound.

    After you bind the exclusive resource group to the VPC, log on to the DataWorks console and click Resource Groups. On the Exclusive Resource Groups tab, find the exclusive resource group and click Add VPC Binding to view the value in the VSwitch CIDR Block column.

  • Elastic IP address (EIP) and CIDR block of the exclusive resource group for scheduling.
    1. Log on to the DataWorks console.
    2. In the left-side navigation pane, click Resource Groups.
    3. On the Exclusive Resource Groups tab, find the required resource group and click View Information.
    4. In the dialog box that appears, copy the values of the EIPAddress and CIDR Blocks parameters to the whitelist of the data store.

Manage the exclusive resource group for scheduling

  • Change the workspace to which the exclusive resource group for scheduling is bound.
    You must bind an exclusive resource group for scheduling to a workspace before you can select the resource group in the workspace. You can bind an exclusive resource group for scheduling to multiple workspaces.
    1. On the Exclusive Resource Groups tab of the Resource Groups page, find the required resource group and click Change Workspace.
    2. In the Modify home workspace dialog box, select the workspaces to which the resource group is to be bound.
    3. Click OK.
  • Change the exclusive resource group that is used to test a node on the DataStudio page.
    1. On the DataStudio page, double-click the required node. The node configuration tab appears.
    2. Click the Run with Arguments icon in the top toolbar.
    3. In the Arguments dialog box, set the Resource Group parameter to the exclusive resource group for testing the node.
    4. Click OK.
  • Change the exclusive resource group that is used to schedule a node.
    You can change the exclusive resource group that is used to schedule a node on the Properties tab of the node or in Operation Center.
    • Change the resource group for scheduling for a node on the Properties tab
      1. Click the Icon icon in the upper-left corner and choose All Products > Data Development > DataStudio. Then, switch to the required workspace.
      2. On the Data Analytics tab, double-click the required node. The node configuration tab appears.
      3. In the right-side navigation pane, click the Properties tab. In the Resource Group section, select the resource group that is used for node scheduling. For more information, see Configure the resource group.Resource Groups
    • Change the resource group for scheduling for a node in Operation Center
      1. Click the Icon icon in the upper-left corner and choose All Products > Task Operation > Operation Center. In Operation Center, switch to the required workspace.
      2. In the left-side navigation pane, choose Cycle Task Maintenance > Cycle Task.
      3. Click the rightwards arrow in the middle of the page to show the node list. Find the required node, and choose More > Modifying a scheduling Resource Group.Change the resource group
        Notice You are not allowed to change the resource group for zero load nodes, workflow nodes, or Machine Learning experiment nodes.
        To change the exclusive resource group for multiple nodes at a time, select the nodes on the Cycle Task page and click Modifying a scheduling Resource Group in the lower part of the page.Change the resource group for multiple nodes at a time
      4. In the Modify scheduling resource groups in batches dialog box, select the required exclusive resource group for scheduling and click OK.