In scenarios where a large number of data integration nodes must be concurrently run, enterprises can use exclusive resource groups for Data Integration to ensure that data is fast and reliably transmitted.

Limits

  • When you bind an exclusive resource group for Data Integration to a virtual private cloud (VPC), select the vSwitch that is associated with the data store to which you want to connect. If the exclusive resource group for Data Integration and the data store reside in different zones, bind the exclusive resource group to the VPC where the data store resides and select a vSwitch. Then, configure a route for the exclusive resource group for Data Integration. For more information, see Add a route.
  • An exclusive resource group for Data Integration cannot access the classic network of Alibaba Cloud. If your data store is on the classic network, we recommend that you use shared resource groups to run sync nodes.
  • An exclusive resource group for Data Integration may still fail to connect to a data store even though the exclusive resource group is in the same zone and is bound to the same VPC and vSwitch as the data store. In this case, you must add the elastic IP address (EIP) of the exclusive resource group and the Classless Inter-Domain Routing (CIDR) block of the vSwitch to which the exclusive resource group is bound to the whitelist and security group of the data store. For more information, see Add the information about an exclusive resource group for Data Integration to the whitelist of a data store.

Features of exclusive resource groups for Data Integration

  • Your nodes can be run as scheduled if your exclusive resource group for Data Integration is not fully occupied.
  • An exclusive resource group for Data Integration can be connected to your VPC or data center.
  • Compared with shared resource groups, exclusive resource groups for Data Integration support more types of data stores.
  • Exclusive resource groups for Data Integration can be shared by multiple workspaces in DataWorks. This way, resources can be reused.

Pricing of exclusive resource groups for Data Integration

For more information about the specifications of exclusive resource groups for Data Integration, see Performance metrics and billing standards of exclusive resource groups for Data Integration.

Purchase an exclusive resource group for Data Integration

  1. Log on to the DataWorks console.
  2. Select the region and click Resource Groups in the left-side navigation pane.
  3. On the Exclusive Resource Groups tab, click Create a dedicated resource group.
  4. In the Create a dedicated resource group panel, click Purchase next to Order Number.
  5. On the buy page, set the region, exclusive resource type, quantity of resources, and billing cycle as required. Then, click Buy Now.
    Note You must select exclusive resources for Data Integration as the exclusive resource type.
  6. After you confirm that the order information is correct, select the check box for I have read and agree to DataWorks Exclusive Resources Agreement of Service and click Pay.

Create an exclusive resource group for Data Integration

  1. On the Exclusive Resource Groups tab of the Resource Groups page, click Create a dedicated resource group.
  2. In the Create a dedicated resource group panel, set the parameters.
    Parameter Description
    Resource Group Type The type of the resources. A value of Exclusive Resource Groups indicates an exclusive resource group that can be used to schedule all types of nodes. A value of Exclusive Resource Groups for Data Integration indicates an exclusive resource group that can be used to schedule only data synchronization nodes.
    Resource Group Name The name of the resource group. The name must be unique for a tenant.
    Note A tenant indicates an Alibaba Cloud account. Multiple RAM users may exist under one tenant.
    Resource Group Description The description of the resource group.
    Order Number The order number of the exclusive resources that you purchase. If you have not purchased exclusive resources, click Purchase to go to the buy page and purchase exclusive resources.
  3. After you complete the configuration, click OK.
    Note The exclusive resource group is initialized within 20 minutes. Wait until its status changes to running.

Configure network settings

Exclusive resource groups are deployed in a VPC that is managed by DataWorks and are disconnected from other network environments. To use an exclusive resource group, you must configure network settings for the exclusive resource group. Bind the exclusive resource group to a VPC that can connect to data stores. This way, the exclusive resource group can connect to the data stores by using the VPC.

  1. Find the resource group and click Network settings.
    Note Before you bind the exclusive resource group to a VPC, authorize DataWorks to access your cloud resources in the Resource Access Management (RAM) console.
  2. Bind the exclusive resource group to a VPC.
    1. Click Add Binding in the upper-left corner of the VPC binding page. In the Add VPC Binding panel, set the parameters based on the network environment.
      The following table describes the parameters.
      Parameter Configuration when the data store and exclusive resource group reside in the same VPC Configuration when the data store and exclusive resource group reside in different VPCs
      VPC If your data store is deployed in a VPC of Alibaba Cloud, we recommend that you set this parameter to the VPC where the data store resides. If your data store is not deployed in a VPC of Alibaba Cloud, or the data store and exclusive resource group need to be deployed in different VPCs, you can click Create VPC to create a VPC for the exclusive resource group. After the VPC is created, set this parameter to the created VPC.
      VSwitch If you set the VPC parameter to the VPC where the data store resides, we recommend that you select the vSwitch that is associated with the data store. If you set the VPC parameter to another VPC or no vSwitch can be used, you can click Create VSwitch to create a vSwitch for the exclusive resource group. After the vSwitch is created, set this parameter to the created vSwitch.
      Note In this case, you must configure a vSwitch route to ensure that the exclusive resource group for Data Integration can connect to the data store.
      Security Groups Security groups allow or deny access to your exclusive resource instances from the Internet or internal network. You can select an existing security group based on your business needs, or click Create Security Group to create a security group for the exclusive resource instances. For more information about how to add a security group rule, see Add security group rules.
    2. Click OK to bind the exclusive resource group to the VPC.
  3. Optional:Add host configurations.
    Your data store may not be able to be accessed by using IP addresses. For example, the data store must be accessed by using hostnames. In this case, you must add host configurations by performing the following steps. Otherwise, the connectivity test fails when you create a connection to the data store by using the hostnames of the data store.
    1. Click Host configuration. In the upper-left corner of the Host configuration page, click added. In the Add domain name Host configuration dialog box, set the parameters. The following table describes the parameters.
      Parameter Description
      IP address Enter the actual IP address of the data store.
      Host domain name Enter the hostname that is used to access the data store. If multiple hostnames exist, add them in different lines.
      Note The domain name can contain digits, letters, hyphens (-), and periods (.). It must start with a letter and end with a letter or digit.
    2. If the data store has multiple IP addresses, click added to add more host configurations.
      Note
      • The IP address or hostnames that are added in a host configuration must be different from those in existing host configurations.
      • You can map one IP address to multiple hostnames in a host configuration. However, one hostname can point to only one IP address.
  4. Optional:Add DNS configurations.
    Your data store may not be able to be accessed by using IP addresses. For example, the data store is accessed by using the domain name of a Server Load Balancer (SLB) instance, and an internal Domain Name System (DNS) server resolves the domain name to IP addresses of the data store. In this case, you must add DNS configurations by performing the following steps. Otherwise, the connectivity test fails when you create a connection to the data store by using the domain names of the data store.
    Note If a hostname that is added in a host configuration is also configured in a DNS configuration, the system preferentially uses the host configuration to look for the data store.
    1. Click DNS configuration. In the lower-left corner of the DNS configuration page, click Add. After you set the parameters for a DNS configuration, click Save. The following table describes the parameters.
      Parameter Description
      Domain Optional. If your data stores are accessed by using the same top-level domain, set this parameter to the top-level domain.

      For example, the domain name that is used to access Data store 1 is domain1.example.com, and the access domain name that is used to access Data store 2 is domain2.example.com. In this case, we recommend that you set this parameter to example.com.

      Note The domain name can contain digits, letters, hyphens (-), and periods (.). It must start with a letter and end with a letter or digit.
      NameServer Enter the IP address of the DNS server that resolves the domain name of the data store. If multiple DNS servers exist, add the IP addresses of the DNS servers in different lines.
    2. To modify an existing DNS configuration, click Modify in the lower-right corner.
After you complete the network configuration for the exclusive resource group for Data Integration, you must add the EIP of the exclusive resource group and the elastic network interface (ENI) IP address of the VPC to the whitelist of the data store.

Change the workspace

You must bind an exclusive resource group to a workspace so that the exclusive resource group can be used by nodes. You can bind an exclusive resource group to multiple workspaces.

  1. Go back to the DataWorks console. In the left-side navigation pane, click Resource Groups.
  2. Find the exclusive resource group and click Change Workspace.
  3. In the Modify home workspace dialog box, select the required workspace and click OK.
    After you bind an exclusive resource group for Data Integration to a workspace, you can select the exclusive resource group on the Resource Group configuration tab for sync nodes to use the exclusive resource group.
    1. Click the Icon icon in the upper-left corner and choose All Products > DataStudio. Then, switch to the required workspace.
    2. On the DataStudio page, double-click the required node to go to the configuration tab of the node.
    3. In the right-side navigation pane, click the Resource Group configuration tab.
    4. On the Resource Group configuration tab, set the Programme and Exclusive Resource Groups parameters.
    5. On the configuration tab of the node, click the Save icon in the toolbar.