If a large number of Data Integration nodes must be run in parallel, you can use exclusive resource groups for Data Integration to ensure fast and reliable data transmission. If you want to access data sources in network environments different from your nodes over an internal network, you can also use exclusive resource groups for Data Integration to run your nodes.

Pricing of exclusive resource groups for Data Integration

For more information about the specifications and pricing of exclusive resource groups for Data Integration, see Performance metrics and pricing of exclusive resource groups for Data Integration.

Limits

  • To allow an exclusive resource group for Data Integration to access a data source in a virtual private cloud (VPC), you must associate the exclusive resource group with the VPC and a vSwitch that resides in the VPC. This way, the system can add a route for the exclusive resource group. The destination of this route is the CIDR block of the VPC. This ensures that the exclusive resource group can access all data sources in this VPC.
  • If a whitelist is configured for the data source that you want to access over the Internet, you must add the elastic IP address (EIP) of your exclusive resource group for Data Integration to the whitelist. If a whitelist is configured for the data source that you want to access over an internal network, you must add the CIDR block of the vSwitch with which your exclusive resource group for Data Integration is associated to the whitelist and security group of the data source. For more information, see Add the EIP or CIDR block of an exclusive resource group for Data Integration to the whitelist of a data source.
  • If you use an exclusive resource group for Data Integration to access a data source that is deployed in a data center or access a data source that does not belong to your account, you must connect the exclusive resource group to the data source based on the instructions provided in Select a network connectivity solution. Then, add a route whose destination is the IP address of the data source.
  • An exclusive resource group for Data Integration cannot access the data sources that are deployed on the classic network of Alibaba Cloud. If your data source is deployed on the classic network, we recommend that you deploy it in a VPC.

Features of exclusive resource groups for Data Integration

  • Your nodes can be run as scheduled if your exclusive resource group for Data Integration is not fully occupied.
  • An exclusive resource group for Data Integration can be connected to your VPC or data center.
  • Exclusive resource groups for Data Integration support more types of data sources than shared resource groups.
  • An exclusive resource group for Data Integration can be shared by multiple DataWorks workspaces. This way, resources in the exclusive resource groups can be reused.

Purchase exclusive resources for Data Integration

  1. Log on to the DataWorks console.
  2. In the top navigation bar, select a region. In the left-side navigation pane, click Resource Groups.
  3. On the Exclusive Resource Groups tab, click Create Resource Group for Data Integration.
  4. In the Create a dedicated resource group panel, click Purchase next to Order Number. The buy page for DataWorks exclusive resources appears.
  5. On the buy page, configure the parameters based on your business requirements. Then, click Buy Now.
    Note You must set the Type parameter to Exclusive Resource Groups for Data Integration.
  6. On the Confirm Order page, confirm that the order information is correct, read the terms of service, select DataWorks Exclusive Resources Agreement of Service, and then click Pay.

Create an exclusive resource group for Data Integration

  1. On the Exclusive Resource Groups tab of the Resource Groups page, click Create Resource Group for Data Integration.
  2. In the Create a dedicated resource group panel, configure the parameters.
    Parameter Description
    Resource Group Type The type of the resource group. Valid values: Exclusive Resource Groups and Exclusive Resource Groups for Data Integration. A resource group of the Exclusive Resource Groups type can be used to run all types of nodes, and a resource group of the Exclusive Resource Groups for Data Integration type can be used to run only data synchronization nodes.
    Resource Group Name The name of the resource group. The name must be unique within all the resource groups of a tenant.
    Note A tenant refers to an Alibaba Cloud account. Each tenant may have multiple RAM users.
    Resource Group Description The description of the resource group.
    Order Number The order number of the exclusive resources that you purchased. If no exclusive resources are purchased, click Purchase next to Order Number to go to the buy page and purchase exclusive resources.
  3. After the parameters are configured, click OK.
    Note The exclusive resource group is initialized within 20 minutes. Wait until the state of the exclusive resource group changes to Running.

Configure network settings

Exclusive resource groups are deployed in the VPC in which DataWorks is hosted. Exclusive resource groups are disconnected from other network environments. To use an exclusive resource group, you must configure network settings for the exclusive resource group to associate it with a VPC that can connect to data sources. This way, the exclusive resource group can access the data sources over the VPC.

  1. Find the resource group for which you want to configure network settings, and click Network Settings in the Actions column. The VPC Binding tab appears.
    Note If you associate an exclusive resource group with a VPC for the first time, you must use your Alibaba Cloud account to assign the AliyunDataWorksAccessingENIRole role to DataWorks before you perform the association. This way, DataWorks can access the cloud resources within your Alibaba Cloud account. You can assign the role on the Cloud Resource Access Authorization page or in the message that appears after you use your Alibaba Cloud account to log on to the DataWorks console.
  2. Associate the exclusive resource group with a VPC.
    1. On the VPC Binding tab, click Add Binding in the upper-left corner. In the Add VPC Binding panel, configure the parameters based on the network environment.
      The following table describes the parameters.
      Parameter Configuration if your data source and exclusive resource group reside in the same VPC Configuration if your data source and exclusive resource group reside in different VPCs
      VPC If your data source is deployed in a VPC, we recommend that you set this parameter to the VPC in which your data source resides. If your data source is not deployed in a VPC, or your data source and exclusive resource group need to be deployed in different VPCs, you can click Create VPC on the right side of this parameter to create a VPC for the exclusive resource group. After the VPC is created, set this parameter to the new VPC.
      VSwitch If you set the VPC parameter to the VPC in which your data source resides, we recommend that you select the vSwitch to which the data source is connected. If you set the VPC parameter to another VPC or no vSwitch is available, you can click Create VSwitch on the right side of this parameter to create a vSwitch for the exclusive resource group. After the vSwitch is created, set this parameter to the new vSwitch.
      Note After you associate the exclusive resource group with the VPC in which the data source resides and a vSwitch that resides in the VPC, a route is automatically added. The destination of this route is the CIDR block of the VPC. This ensures that the exclusive resource group can access all data sources in this VPC.
      Security Groups Security groups allow or deny access to the resources in your exclusive resource group over the Internet or an internal network. You can select an existing security group based on your business requirements, or click Create Security Group on the right side of this parameter to create a security group for the resources in the exclusive resource group. For more information about how to create a security group, see Add security group rules.
    2. Click OK.
  3. Optional:Add host configurations.
    You may fail to access your data source by using IP addresses. For example, you can access your data source only by using hostnames. In this case, you must perform the following steps to add host configurations. Otherwise, the connectivity test fails when you add the data source by using its hostnames.
    1. Click the Hostname-to-IP Mapping tab. Then, click Add in the upper-left corner of the tab. In the Create Hostname-to-IP Mapping dialog box, configure the parameters. The following table describes the parameters.
      Parameter Description
      IP Address The actual IP address of the data source.
      The hostname The hostname that is used to access the data source. If you want to specify multiple hostnames, place each hostname on a separate line.
      Note The domain name can contain digits, letters, hyphens (-), and periods (.). It must start with a letter and end with a letter or digit.
    2. If the data source has multiple IP addresses, click Add to add more host configurations.
      Note
      • The IP address or hostnames that are added in a host configuration must be different from the IP addresses or hostnames in existing host configurations.
      • You can map one IP address to multiple hostnames in a host configuration. However, one hostname can point to only one IP address.
  4. Optional:Add Domain Name System (DNS) configurations.
    You may fail to access your data source by using IP addresses. For example, you can access your data source only by using the domain name of a Server Load Balancer (SLB) instance, and an internal DNS server resolves the domain name to IP addresses of your data source. In this case, you must perform the following steps to add DNS configurations. Otherwise, the connectivity test fails when you add the data source by using its DNS configuration.
    Note If a domain name that is added in a host configuration is also configured in a DNS configuration, the system preferentially uses the host configuration to access the data source.
    1. Click the DNS Configuration tab. Then, click Add in the lower-left corner of the tab. After you configure the parameters for a DNS configuration, click Save. The following table describes the parameters.
      Parameter Description
      Domain Optional. If you can use the same second-level domain to access your data sources, set this parameter to the second-level domain.

      For example, the domain name that is used to access Data source 1 is domain1.example.com, and the domain name that is used to access Data source 2 is domain2.example.com. In this example, we recommend that you set this parameter to example.com.

      Note The domain name can contain digits, letters, hyphens (-), and periods (.). It must start with a letter and end with a letter or digit.
      NameServer Enter the IP address of the DNS server that resolves the domain name of the data source. If you want to specify multiple DNS servers, place the IP address of each DNS server on a separate line.
    2. To modify an existing DNS configuration, click Modify in the lower-left corner.

Configure a whitelist

Add the EIP or CIDR block of your exclusive resource group to a whitelist of your data source. For more information, see Configure a whitelist.

Associate an exclusive resource group with a workspace

You must associate an exclusive resource group with a workspace to allow nodes in the workspace to use the exclusive resource group. You can associate an exclusive resource group with multiple workspaces.

  1. In the left-side navigation pane of the DataWorks console, click Resource Groups.
  2. Find the exclusive resource group that you want to associate with a workspace, and click Change Workspace in the Actions column.
  3. In the Modify home workspace dialog box, find the workspace with which you want to associate the exclusive resource group and click Bind in the Actions column.
    After you associate an exclusive resource group for Data Integration with a workspace, you can select the exclusive resource group on the Resource Group configuration tab of a synchronization node.
    1. Click the Icon icon in the upper-left corner and choose All Products > DataStudio. Then, switch to the workspace.
    2. In the Data Analytics pane, double-click the desired node to go to the configuration tab of the node.
    3. In the right-side navigation pane, click the Resource Group configuration tab.
    4. On the Resource Group configuration tab, select the exclusive resource group for Data Integration.
      Exclusive resource group for Data Integration
    5. On the configuration tab of the node, click the Save icon in the toolbar.