DataWorks allows you to associate an exclusive resource group for scheduling with a virtual private cloud (VPC) so that the resource group can access data sources in the VPC.

Prerequisites

An exclusive resource group is properly connected to a data source before you use the resource group to access the data source.

Background information

If you do not need to connect an exclusive resource group to a data source and you want to fix issues where nodes are delayed because resources in the shared resource groups for scheduling are insufficient, you can ignore the network configuration in this topic. In this case, you can purchase an exclusive resource group for scheduling in any zone without the need to associate the resource group with a VPC.
Notice
  • To allow an exclusive resource group for scheduling to access the data sources in a VPC, you must associate the exclusive resource group with the VPC and then with a vSwitch that resides in the VPC. Then, the system adds a route for the exclusive resource group. The destination of this route is the CIDR block of the VPC. This ensures that the exclusive resource group can access all data sources in this VPC.
  • If a whitelist is configured for a data source that you want to access and the data source is accessed over the Internet, add the Elastic IP address (EIP) of the exclusive resource group for scheduling to the whitelist. If the data source is accessed over an internal network, add the CIDR block of the vSwitch to the whitelist and security group of the data source. The vSwitch is the one that is associated with the exclusive resource group for scheduling. For more information, see Add the exclusive resource group for scheduling to the whitelist of the data source to be accessed.

The maximum number of concurrent instances that are allowed by an exclusive resource group for scheduling is determined by the specifications of resources in the resource group. For more information, see Performance metrics and pricing of exclusive resource groups for scheduling.

Create an exclusive resource group for scheduling

  1. Log on to the DataWorks console.
  2. In the left-side navigation pane, click Resource Groups.
  3. Purchase an exclusive resource group for scheduling.
    If you have not purchased an exclusive resource group for scheduling in the required region, perform the following steps to purchase one:
    1. On the Exclusive Resource Groups tab, click Create Resource Group for Scheduling.
    2. In the Create a dedicated resource group panel, click Purchase next to Order Number. The buy page appears.
      Purchase button
    3. Set the Region, Type, Exclusive Resources for Scheduling, Units, and Duration parameters.
      Note
      • In this example, set the Type parameter to Exclusive Resources for Scheduling.
      • Exclusive resource groups cannot be shared among regions. For example, the exclusive resource groups in the China (Shanghai) region can only be used by workspaces in the China (Shanghai) region.
    4. Click Buy Now.
  4. Add the exclusive resource group for scheduling.
    1. Return to the Create a dedicated resource group panel and set the parameters as required.
      Parameter Description
      Resource Group Type The type of the exclusive resource group. In this example, select Exclusive Resource Groups.

      Valid values: Exclusive Resource Groups and Exclusive Resource Groups for Data Integration. The former type is used to schedule general nodes, whereas the latter type is used to schedule sync nodes.

      Resource Group Name The name of the resource group. The name must be unique within the current tenant.
      Note A tenant refers to an Alibaba Cloud account. Each tenant may have multiple RAM users.
      Resource Group Description The description of the resource group.
      Order Number The order number of the exclusive resource group.
    2. Click OK.
      Note The exclusive resource group is initialized within 20 minutes. Wait until its status changes to Running.

Associate the exclusive resource group with a VPC

Exclusive resource groups are deployed in a VPC that is managed by DataWorks. To allow exclusive resource groups to access data sources in your own VPC, you must associate the exclusive resource groups with your VPC.

  1. Find the created resource group and click Network Settings in the Actions column. The VPC Binding tab appears.
    Before the association, authorize DataWorks to access your cloud resources in the RAM console.
  2. After the authorization is complete, click Add Binding.
  3. In the Add VPC Binding panel, set the parameters as required.
    Add VPC Binding panel
    Note You must associate the exclusive resource group for scheduling with the VPC in which your data source resides.
    Parameter Description
    Resource Group Name Select the created exclusive resource group from the Resource Group Name drop-down list.
    VPC The VPC with which the exclusive resource group is to be associated.
    VSwitch The vSwitch with which the exclusive resource group is to be associated. If you set the VPC parameter to another VPC or no vSwitch is available, you can click Create VSwitch to create a vSwitch. After a vSwitch is created, set this parameter to the vSwitch.
    Note After you associate the exclusive resource group with the VPC in which the data source resides and a vSwitch that resides in the VPC, a route is automatically added. The destination of this route is the CIDR block of the VPC. This ensures that the exclusive resource group can access all the data sources in this VPC.
    Security Groups The security group to be associated with the required VPC.
  4. Click OK.
  5. Optional: If the exclusive resource group and the data source to be accessed are associated with the same VPC but reside in different zones, configure a routing rule.
    1. Find the resource group and click Custom Route.
    2. In the Custom Route panel, click Add Route.
    3. In the Add Route dialog box, set the parameters as required.
      Add Route dialog box
      Parameter Description
      Destination Type The type of destination. Valid values: VPC and IDC.
      Destination VPC The region and name of the destination VPC.
      Note This parameter is displayed only if the Destination Type parameter is set to VPC.
      Connection Method The connection method. Valid values: Switch, Fixed IP Address, and CIDR Block.
      • If you set the Connection Method parameter to Switch, you must also set the Destination VSwitch parameter.
        Note This parameter is displayed only if the Destination Type parameter is set to VPC.
      • If you set the Connection Method parameter to Fixed IP Address, you must also set the Fixed IP Address parameter.
      • If you set the Connection Method parameter to CIDR Block, you must also set the Destination CIDR Block parameter.
      Routing By default, the information cannot be changed.
    4. Click Generate Route.

Add the exclusive resource group for scheduling to the whitelist of the data source to be accessed

An exclusive resource group for scheduling may still fail to access a data source even if the resource group and the data source reside in the same zone and are associated with the same VPC and vSwitch. This is because the resource group is not in the whitelist that is configured for the data source. In this case, add the required information to the whitelist based on the following description:
  • If the data source is accessed over an internal network, add the CIDR block of the vSwitch that resides in the VPC to the whitelist. The vSwitch is the one that is associated with the exclusive resource group for scheduling.
    To view the CIDR block after you associate the exclusive resource group with the VPC, log on to the DataWorks console and click Resource Groups in the left-side navigation pane. On the Exclusive Resource Groups tab, find the exclusive resource group and click Network Settings in the Actions column. On the VPC Binding tab, view the value in the VSwitch CIDR Block column. VSwitch CIDR Block
  • If the data source is accessed over the Internet, add the EIP and CIDR block of the exclusive resource group for scheduling to the whitelist.
    1. Log on to the DataWorks console.
    2. In the left-side navigation pane, click Resource Groups.
    3. On the Exclusive Resource Groups tab, find the required resource group and click View Information in the Actions column.
    4. In the Exclusive Resource Groups dialog box, copy the values of the EIPAddress and CIDR Blocks parameters. Then, add the EIP and CIDR block to the whitelist of the data source. CIDR Blocks

Associate the exclusive resource group with a workspace

You must associate an exclusive resource group for scheduling with a workspace before you can select the resource group in the workspace. You can associate an exclusive resource group for scheduling with multiple workspaces.
  1. On the Exclusive Resource Groups tab of the Resource Groups page, find the required resource group and click Change Workspace in the Actions column.
  2. In the Modify home workspace dialog box, click Bind in the Actions column of the workspace with which the resource group is to be associated.
  • Associate the exclusive resource group with a workspace.
    You must associate an exclusive resource group for scheduling with a workspace before you can select the resource group in the workspace. You can associate an exclusive resource group for scheduling with multiple workspaces.
    1. On the Exclusive Resource Groups tab of the Resource Groups page, find the required resource group and click Change Workspace in the Actions column.
    2. In the Modify home workspace dialog box, click Bind in the Actions column of the workspace with which the resource group is to be associated.
  • Change the exclusive resource group that is used to test a node on the DataStudio page.
    1. On the DataStudio page, double-click the required node. The node configuration tab appears.
    2. Click the Run with Parameters icon icon in the top toolbar.
    3. In the Parameters dialog box, set the Resource Group parameter to the exclusive resource group that is used to test the node.
    4. Click OK.
  • Change the exclusive resource group that is used to schedule a node.
    You can change the exclusive resource group that is used to schedule a node in the Properties panel or in Operation Center.
    • Change the resource group for scheduling for a node in the Properties panel
      1. Click the More icon icon in the upper-left corner and choose All Products > Data Development > DataStudio. Switch to the required workspace.
      2. In the Scheduled Workflow pane, double-click the required node in the Business Flow directory to go to the configuration tab of the node.
      3. In the right-side navigation pane, click the Properties tab. The Properties panel appears. In the Resource Group section, select the required exclusive resource group for scheduling from the drop-down list. For more information, see Configure a resource group. Resource Group
    • Change the resource group for scheduling for a node in Operation Center
      1. Click the More icon icon in the upper-left corner and choose All Products > Task Operation > Operation Center. In Operation Center, switch to the required workspace.
      2. In the left-side navigation pane, choose Cycle Task Maintenance > Cycle Task.
      3. Click the rightwards arrow in the middle of the page to show the node list. Find the required node, and choose More > Modify Scheduling Resource Group in the Actions column. Change the resource group for a node
        Notice You are not allowed to change the resource group for zero load nodes, workflow nodes, or Machine Learning experiment nodes.
        To change the exclusive resource group for multiple nodes at a time, select the nodes on the Cycle Task page and click Modify Scheduling Resource Group in the lower part of the page. Change the resource group for multiple nodes at a time
      4. In the Modify Scheduling Resource Group dialog box, select the required exclusive resource group for scheduling from the drop-down list and click OK.