When data integration nodes run in high concurrency and load shifting is not an option, enterprises need exclusive computing resources to guarantee that data is transmitted quickly and reliably. To address such problems, DataWorks provides exclusive resource groups for data integration.

Background information

An exclusive resource group for data integration can only access the data stores in the same region and zone as the resource group.
  • You must bind a purchased exclusive resource group for data integration to a zone of your Virtual Private Cloud (VPC). The exclusive resource group for data integration must be in the same zone as the data stores that you want to access.
  • When you bind an exclusive resource group for data integration or an exclusive resource group for scheduling to a VPC, select the VSwitch associated with the data stores that you want to access.
  • After an exclusive resource group for data integration is bound to a VPC, the exclusive resource group for data integration can only access the data stores in the specified zone of the VPC. We recommend that you make sure that the same zone as that the VPC to be bound is selected when you add an exclusive resource group for data integration.
  • An exclusive resource group for data integration cannot access data stores deployed on a classic network of Alibaba Cloud. If your data stores are on a classic network, we recommend that you run sync nodes on the default resource group.

Purchase an exclusive resource group for data integration

  1. Log on to the DataWorks console.
  2. In the left-side navigation pane, click Resource Groups. The Exclusive Resource Groups tab appears by default.
  3. Click Add Exclusive Resource Group.
  4. In the Add Exclusive Resource Group dialog box that appears, click Purchase next to Order Number.
  5. On the purchase page, set Region, Type, Exclusive Resource Groups for Data Integration, Units, and Duration, and click Buy Now.
    Note Here, Type must be set to Exclusive Resource Groups for Data Integration.

    Exclusive resources cannot be used across regions. For example, the exclusive resources in the China (Shanghai) region can only be used by workspaces in the China (Shanghai) region.

  6. After confirming that the order information is correct, select the check box for DataWorks Exclusive Resources Agreement of Service and click Pay.

Add an exclusive resource group for data integration

  1. On the Exclusive Resource Groups tab, click Add Exclusive Resource Group.
  2. In the Add Exclusive Resource Group dialog box that appears, set the parameters.
    Parameter Description
    Resource Group Type The type of the exclusive resource group. The valid values are Exclusive Resource Groups and Exclusive scheduling resources. The two types of resource groups are applicable to general node scheduling and data synchronization, respectively.
    Resource Group Name The name of the exclusive resource group, which must be unique within all resource groups of a tenant.
    Note A tenant account indicates an Alibaba Cloud account. Multiple Resource Access Management (RAM) users may exist under a tenant account.
    Resource Group Description The description of the exclusive resource group.
    Order Number The order number of the exclusive resource group. If you have not purchased any exclusive resource groups, click Purchase next to Order Number to go to the purchase page and purchase an exclusive resource group.
    Zone The zone of servers available in the region. Select a zone based on your requirements.
  3. After the configuration is completed, click Create.
    Note The exclusive resource group is initialized within 20 minutes. Wait until its status changes to Running.

Bind an exclusive resource group for data integration to a VPC

Exclusive resource groups are deployed in a VPC managed by DataWorks. To allow exclusive resource groups to access data stores in your own VPC, you must bind the exclusive resource groups to your VPC.

  1. Click Add VPC Binding in the Actions column of the exclusive resource group for data integration.
    Note Before binding the exclusive resource group for data integration to your VPC, you must authorize DataWorks to access your cloud resources in the RAM console.
  2. Click Add Binding in the upper-right corner. In the Add VPC Binding dialog box that appears, set the parameters.
    • If no VPC is available, click Create VPC to create a VPC on the VPCs page of the VPC console.

      Click Create VPC. In the Create VPC dialog box that appears, set the parameters and click OK.

      After a VPC is created, you can view it on the VPCs page.

    • If no VSwitch is available, click Create VSwitch to create a VSwitch on the VSwitches page of the VPC console.

      Click Create VSwitch. In the Create VSwitch dialog box that appears, set the parameters and click OK.

      After a VSwitch is created, you can view it on the VSwitches page.

    • If no security group is available, click Create Security Group to create a security group on the Security Groups page of the ECS console.

      Click Create Security Group. In the Create Security Group dialog box that appears, set the parameters and click OK.

      After a security group is created, you can view it on the Security Groups page.

  3. After the configuration is completed, click Create.

Purchase an ApsaraDB for RDS instance

  1. Move the pointer over the icon in the upper-left corner of the DataWorks console and click Relational Database Service to log on to the ApsaraDB for RDS console. The Instances page appears by default.
  2. Click Create Instance in the upper-right corner.
  3. On the purchase page, set the parameters in Basic Configurations step and click Next:Instance Configuration.
  4. Set the parameters in the Instance Configuration step and click Next:Confirm Order.
    Note The values of Version, Zone, and Network Type must be the same as those specified for the exclusive resource group for data integration.
  5. Confirm the settings in the Parameters section, specify Purchase Plan and Duration, and read and select the check box for Terms of Service. Note that you only need to specify Duration if you create a subscription instance.
  6. Click Pay Now.

Configure a whitelist for an ApsaraDB for RDS instance

  1. On the Instances page, click the ID of the created ApsaraDB for RDS instance.
  2. On the page that appears, click Data Security in the left-side navigation pane.
  3. On the Whitelist Settings tab, click Create Whitelist.
  4. In the Create Whitelist dialog box that appears, enter the IP addresses or Classless Inter-Domain Routing (CIDR) blocks of the region and the IP address of the created VPC.
  5. In the left-side navigation pane, click Accounts and Databases to create an account and a database. For more information, see Create databases and accounts for an ApsaraDB RDS MySQL instance.

Create a MySQL connection

  1. Move the pointer over the icon in the upper-left corner of the ApsaraDB for RDS console and click DataWorks to log on to the DataWorks console.
  2. Find the target workspace and click Data Integration in the Actions column.
  3. In the left-side navigation pane, click Connection to go to the Workspace Manage > Data Source page.
  4. On the Data Source page, click Add Connection in the upper-right corner.
  5. In the Add Connection dialog box that appears, click MySQL in the Relational Database section.
  6. In the Add MySQL Connection dialog box that appears, set the parameters.
    The MySQL connection type can be set to ApsaraDB for RDS or JDBC Connection Mode. In this example, set Connect To to ApsaraDB for RDS.
    Parameter Description
    Connect To The type of the connection. Here, set the value to ApsaraDB for RDS.
    Connection Name The name of the connection. The name can contain letters, digits, and underscores (_) and must start with a letter.
    Description The description of the connection. The description cannot exceed 80 characters in length.
    Applicable Environment The environment in which the connection is used. Valid values: Development and Production.
    Note This parameter is available only when the workspace is in standard mode.
    Region The region of the ApsaraDB for RDS instance.
    RDS Instance ID The ID of the ApsaraDB for RDS instance. You can view the ID in the ApsaraDB for RDS console.
    RDS Instance Account ID The ID of the Alibaba Cloud account used to purchase the ApsaraDB for RDS instance. You can view your account ID on the Security Settings page after logging on to the Alibaba Cloud console with your Alibaba Cloud account.
    Database Name The name of the database.
    Username The username for logging on to the database.
    Password The password for logging on to the database.
    Note You must add the IP addresses or CIDR blocks that you use to access the ApsaraDB for RDS instance to a whitelist of the instance. For more information, see Configure a whitelist.
  7. Click Test Connection.
  8. After the connectivity test is passed, click Complete.

Change the workspace to which an exclusive resource group is bound

You must bind an exclusive resource group to a workspace so that the exclusive resource group can be used by nodes in the workspace. You can bind an exclusive resource group to multiple workspaces.

  1. Log on to the DataWorks console. In the left-side navigation pane, click Resource Groups.
  2. Click Change Workspace in the Actions column of the exclusive resource group for data integration.
  3. In the Change Workspace dialog box that appears, select the desired workspace and click OK.
    After you bind the exclusive resource group for data integration to a workspace, you can use the exclusive resource group for data integration to run sync nodes in the workspace.