DataWorks allows you to bind an exclusive resource group for scheduling to a virtual private cloud (VPC) so that the resource group can access data stores in the VPC.

Prerequisites

An exclusive resource group is properly connected to a data store before you use the exclusive resource group for scheduling to access the data store.

Background information

Assume that you do not need to connect an exclusive resource group to a data store. If you need to fix issues where nodes are delayed because resources in the shared resource groups for scheduling are insufficient, you can ignore the network configuration in this topic. In this case, you can purchase an exclusive resource group for scheduling in any zone without the need to bind the resource group to a VPC.
Notice
  • To allow an exclusive resource group for scheduling to access data stores in a VPC, you must bind the exclusive resource group to the VPC where the data store resides and then to a vSwitch that resides in the VPC. Then, the system adds a route for the exclusive resource group. The destination of this route is the Classless Inter-Domain Routing (CIDR) block of the VPC. This ensures that the exclusive resource group can access all data stores in this VPC.
  • Assume that a whitelist is configured for the data store that you want to access. If the data store is accessed over the Internet, add the Elastic IP address (EIP) of the exclusive resource group for scheduling to the whitelist. If the data store is accessed over an internal network, add the CIDR block of the vSwitch to the whitelist and security group of the data store. The vSwitch is the one that is bound to the exclusive resource group for scheduling. For more information, see Add the exclusive resource group for scheduling to the whitelist of the data store to be accessed.

The maximum number of concurrent instances that are allowed varies with the specifications of the resources in an exclusive resource group for scheduling. For more information, see Performance metrics and billing standards of exclusive resource groups for scheduling.

Create an exclusive resource group for scheduling

  1. Log on to the DataWorks console.
  2. In the left-side navigation pane, click Resource Groups.
  3. Purchase an exclusive resource group for scheduling.
    If you have not purchased an exclusive resources for scheduling in the required region, perform the following steps to purchase resources:
    1. On the Exclusive Resource Groups tab, click Create a dedicated resource group.
    2. In the Create a dedicated resource group panel, click Purchase next to Order Number. The buy page appears.
      Purchase
    3. Set the Region, Type, Exclusive Resources for Scheduling, Units, and Duration parameters.
      Note
      • In this example, set the Type parameter to Exclusive Resources for Scheduling.
      • Exclusive resource groups cannot be shared among regions. For example, the exclusive resource groups in the China (Shanghai) region can only be used by workspaces in the China (Shanghai) region.
    4. Click Buy Now.
  4. Add the exclusive resource group for scheduling.
    1. Return to the Create a dedicated resource group panel and set the parameters as required.
      Parameter Description
      Resource Group Type The type of the exclusive resource group. In this example, select Exclusive Resource Groups.

      Valid values: Exclusive Resource Groups and Exclusive Resource Groups for Data Integration. The former type is used to schedule general nodes, whereas the latter type is used to schedule sync nodes.

      Resource Group Name The name of the resource group. The name must be unique within all resource groups of a tenant.
      Note A tenant indicates an Alibaba Cloud account. Each tenant may have multiple RAM users.
      Resource Group Description The description of the resource group.
      Order Number The order number of the exclusive resource group.
    2. Click OK.
      Note The exclusive resource group is initialized within 20 minutes. Wait until its status changes to Running.

Add a VPC binding

Exclusive resource groups are deployed in a VPC that is managed by DataWorks. To allow exclusive resource groups to access data stores in your own VPC, you must bind the exclusive resource groups to your VPC.

  1. Find the required resource group and click Network Settings in the Actions column. The VPC Binding tab appears.
    Before the binding, authorize DataWorks to access your cloud resources in the Resource Access Management (RAM) console.
  2. After the authorization is complete, click Add Binding.
  3. In the Add VPC Binding panel, set the parameters as required.
    Add Binding
    Note You must bind the exclusive resource group for scheduling to the VPC where your data store resides.
    Parameter Description
    Resource Group Name Select an exclusive resource group to be bound to a VPC from the Resource Group Name drop-down list.
    VPC The VPC to which the exclusive resource group is to be bound.
    VSwitch If you set the VPC parameter to another VPC or no vSwitch can be used, you can click Create VSwitch to create a vSwitch for the exclusive resource group. After the vSwitch is created, set this parameter to the created vSwitch.
    Note
    Security Groups The security group to be bound to the required VPC.
  4. Click OK.
  5. Optional: If the exclusive resource group and the data store to be accessed are bound to the same VPC but reside in different zones, configure a routing rule.
    1. Find the required resource group and click Custom Route.
    2. In the Custom Route panel, click Add Route.
    3. In the Add Route dialog box, set the parameters as required.
      Add Route
      Parameter Description
      Destination Type Valid values: VPC and IDC.
      Destination VPC The region and name of the destination VPC.
      Note This parameter is displayed only when the Destination Type parameter is set to VPC.
      Connection Method Valid values: Switch, Fixed IP Address, and CIDR Block.
      • If you set the Connection Method parameter to Switch, you must also set the Destination VSwitch parameter.
        Note This parameter is displayed only when the Destination Type parameter is set to VPC.
      • If you set the Connection Method parameter to Fixed IP Address, you must also set the Fixed IP Address parameter.
      • If you set the Connection Method parameter to CIDR Block, you must also set the Destination CIDR Block parameter.
      Routing By default, the value cannot be changed.
    4. Click Generate Route.

Add the exclusive resource group for scheduling to the whitelist of the data store to be accessed

Assume that the resource group and data store reside in the same zone and are bound to the same VPC and vSwitch. The exclusive resource group for scheduling may still fail to access a data store. This is because the resource group is not in the whitelist that is configured for the data store. In this case, add the following information to the whitelist:
  • If the data store is accessed over an internal network, add the CIDR block of the vSwitch that resides in the VPC to the whitelist. The vSwitch is the one that is bound to the exclusive resource group for scheduling.
    After you bind the exclusive resource group to the VPC, log on to the DataWorks console and click Resource Groups in the left-side navigation pane. On the Exclusive Resource Groups tab, find the exclusive resource group and click Network Settings in the Actions column. On the VPC Binding tab, view the value in the VSwitch CIDR Block column.VSwitch CIDR block
  • If the data store is accessed over the Internet, add the EIP and CIDR block of the exclusive resource group for scheduling to the whitelist.
    1. Log on to the DataWorks console.
    2. In the left-side navigation pane, click Resource Groups.
    3. On the Exclusive Resource Groups tab, find the required resource group and click View Information in the Actions column.
    4. In the dialog box that appears, copy the values of the EIPAddress and CIDR Blocks parameters. Then, add the EIP and CIDR block to the whitelist of the data store.CIDR block

Bind the exclusive resource group to a workspace

You must bind an exclusive resource group for scheduling to a workspace before you can select the resource group in the workspace. You can bind an exclusive resource group for scheduling to multiple workspaces.
  1. On the Exclusive Resource Groups tab of the Resource Groups page, find the required resource group and click Change Workspace.
  2. In the Modify home workspace dialog box, select the workspaces to which the resource group is to be bound.
  3. Click OK.
  • Bind the exclusive resource group to a workspace.
    You must bind an exclusive resource group for scheduling to a workspace before you can select the resource group in the workspace. You can bind an exclusive resource group for scheduling to multiple workspaces.
    1. On the Exclusive Resource Groups tab of the Resource Groups page, find the required resource group and click Change Workspace in the Actions column.
    2. In the Modify home workspace dialog box, select the workspaces to which the resource group is to be bound.
    3. Click OK.
  • Change the exclusive resource group that is used to test a node on the DataStudio page.
    1. On the DataStudio page, double-click the required node. The node configuration tab appears.
    2. Click the Run with Arguments icon in the top toolbar.
    3. In the Arguments dialog box, set the Resource Group parameter to the exclusive resource group that is used to test the node.
    4. Click OK.
  • Change the exclusive resource group that is used to schedule a node.
    You can change the exclusive resource group that is used to schedule a node on the Properties tab of the node or in Operation Center.
    • Change the resource group for scheduling for a node on the Properties tab
      1. Click the Icon icon in the upper-left corner and choose All Products > Data Development > DataStudio. Switch to the required workspace.
      2. In the Data Analytics pane, double-click the required node to go to the node configuration tab.
      3. In the right-side navigation pane, click the Properties tab. The Properties panel appears. In the Resource Group section, select the resource group that is used for node scheduling. For more information, see Configure the resource group.Resource Group
    • Change the resource group for scheduling for a node in Operation Center
      1. Click the Icon icon in the upper-left corner and choose All Products > Task Operation > Operation Center. In Operation Center, switch to the required workspace.
      2. In the left-side navigation pane, choose Cycle Task Maintenance > Cycle Task.
      3. Click the rightwards arrow in the middle of the page to show the node list. Find the required node, and choose More > Modifying a scheduling Resource Group.Change the exclusive resource group
        Notice You are not allowed to change the resource group for zero load nodes, workflow nodes, or Machine Learning experiment nodes.
        To change the exclusive resource group for multiple nodes at a time, select the nodes on the Cycle Task page and click Modifying a scheduling Resource Group in the lower part of the page.Change the resource group for multiple nodes at a time
      4. In the Modify scheduling resource groups in batches dialog box, select the required exclusive resource group for scheduling and click OK.