This topic uses an ApsaraDB RDS for MySQL instance as an example to describe how to establish a network connection between a data source and DataWorks when they belong to different Alibaba Cloud accounts.
Use cases
If your data source and DataWorks workspace meet the following conditions at the same time, we recommend that you use this solution.
The data source belongs to an Alibaba Cloud service.
The data source and DataWorks workspace belong to different Alibaba Cloud accounts.
Solution description
If your data source belongs to Alibaba Cloud Account A and your resource group belongs to Alibaba Cloud Account B, we recommend that you use Cloud Enterprise Network (CEN) or a VPC peering connection to establish a network connection between the accounts. This way, the resource group can access the data source over a VPC.
Prerequisites
An Alibaba Cloud data source that DataWorks supports is available.
The data source and DataWorks workspace meet the requirements in the Use cases section.
Cross-account authorization is configured for the account to which the data source belongs. For more information, see Cross-account authorization.
Billing
The billing method varies based on the network connectivity tool that you use. For more information, see Billing of CEN or Billing of VPC peering connections.
If your data source and DataWorks resource group belong to different accounts but reside in the same region, you are not charged for using a VPC peering connection.
Configure network connectivity
The following section describes the general procedure for configuring network connectivity between a data source and a DataWorks resource group. This procedure helps you quickly understand the core logic. For more information about the configuration, see the Configuration example section in this topic.
Step 1: Obtain basic information
On the data source side
Account information: This example uses Account A.
Region information: This example uses an ApsaraDB RDS for MySQL instance in the China (Hangzhou) region.
VPC and vSwitch information:
NoteThis example uses an ApsaraDB RDS for MySQL instance. For information about how to obtain the VPC information for other Alibaba Cloud instances, see the official documentation for the specific instance.
Go to the RDS Management Console, find the target instance, and click the Instance Name to go to the Basic Information page.
In the navigation pane on the left, click Database Connection to obtain the VPC and vSwitch information of the RDS for MySQL instance.
On the DataWorks side
Account information: This example uses Account B.
Region information: This example uses a DataWorks workspace and resource group in the China (Shanghai) region.
Information about the VPC and vSwitch attached to the resource group:
Go to the Resource Groups page of the DataWorks console, find the target resource group, and then click Network Settings in the Actions column.
In the corresponding feature module, view the attached VPC and vSwitch information.
For example, to connect an RDS for MySQL instance to DataWorks for data synchronization, view the corresponding VPC and vSwitch information under Data Scheduling & Data Integration.
Step 2: Establish a network connection
To establish a network connection between VPCs that belong to different accounts, you can select one of the following network connectivity tools:
CEN: Suitable for complex enterprise network environments and for interconnecting multiple VPCs. For more information about the configuration, see Cross-account VPC-to-VPC connection.
VPC peering connection: Suitable for interconnecting two VPCs. For more information about the configuration, see Use a VPC peering connection to enable private connectivity between VPCs.
If you encounter issues when you configure the network connection, you can submit a ticket to contact technical support for the relevant cloud product.
Step 3: Add a route for the DataWorks resource group
When DataWorks accesses a data source across accounts, you must also add a route in the DataWorks resource group that points to the vSwitch CIDR block of the data source.
Go to the DataWorks Resource Groups page, find the target resource group, and then click Network Settings in the Actions column.
In the corresponding feature module, find the attached VPC and click Custom Route in the Actions column.
Click Add Route. Set Connection Method to Specify CIDR Block and Destination CIDR Block to the vSwitch CIDR block of the data source.
Step 4: (Optional) Add to the whitelist
If the data source uses a whitelist for access control, you must add the CIDR block of the vSwitch that is attached to the resource group to the data source whitelist. This allows the resource group to access the data source.
This topic uses setting an IP whitelist for an RDS for MySQL instance as an example to show how to add the vSwitch CIDR Block that is bound to the DataWorks resource group in Alibaba Cloud Account B to Whitelists And Security Groups.
For information about how to add other Alibaba Cloud instances to a whitelist, see the official documentation for the specific instance.
Test network connectivity
Log on to the DataWorks console. In the top navigation bar, select the desired region. In the left-side navigation pane, choose . On the page that appears, select the desired workspace from the drop-down list and click Go to Data Integration.
In the navigation pane on the left, click Data Source. On the Data Sources page, click Add Data Source. Select a data source and configure its connection parameters.
In the resource group list at the bottom, select the resource group that is connected to the data source and click Test Connectivity.
NoteIf the connectivity test Fails, you can use the Connectivity Diagnosis Tool to troubleshoot the issue. If you still cannot connect the resource group to the data source, submit a ticket.
Configuration example
This section provides an example of how to configure a network connection between an ApsaraDB RDS for MySQL instance in Account A in the China (Hangzhou) region and a DataWorks workspace in Account B in the China (Shanghai) region.
1. Basic information
Parameter | Data source (RDS for MySQL) | DataWorks resource group |
Account | Account A | Account B |
Region | China (Hangzhou) | China (Shanghai) |
VPC |
|
|
2. Establish a network connection
This solution supports using CEN and VPC peering connections to establish a network connection between the data source and DataWorks. You can select a method as needed.
If you encounter issues when you configure the network connection, you can submit a ticket to contact technical support for the relevant cloud product.
Configure a network connection using CEN
Log on with Account B, go to the CEN console, and click Create CEN Instance. In the dialog box that appears, set the instance Name and click Confirm.
NoteAs a big data processing platform, DataWorks may need to connect to data sources that belong to different accounts and are deployed in different VPCs. We recommend that you create a CEN instance in the account where DataWorks is located to facilitate unified management.
In the dialog box, click Create Network Instance Connection and configure the network information for the DataWorks resource group.
The following table describes the key parameters. You can keep the default values for other parameters.
Parameter
Description and example
Instance Type
This solution describes how to connect VPCs across accounts. Select Virtual Private Cloud (VPC).
Region
Select the region of the data source. In this example, select China (Shanghai).
Resource Ownership UID
Select Same Account.
Network Instance
Select the VPC instance where the DataWorks resource group is located.
VSwitch
Select the vSwitch where the resource group is located. For example, select
Account_B_Switch_sh_e.NoteNetwork interconnection using CEN requires zone-level disaster recovery. You must configure at least two vSwitches in different zones. After you ensure that the vSwitch of the resource group is included, configure another vSwitch in any zone. If you have fewer than two vSwitches, go to the vSwitch console to create one. Then, select the created vSwitch.
Click Create.
Grant cross-account authorization for the VPC instance.
Log on to Alibaba Cloud Account A and go to the VPC console. Find the VPC that contains the data source (
Account_A_hangzhou_VPCin this example) and click its name to go to the Basic Information page.Click the Cross-account Authorization tab and click CEN Authorization. Configure the information as follows.
Parameter
Description and example
Peer Account UID
The UID of Alibaba Cloud account B.
Peer CEN Instance ID
The ID of the CEN instance that you created in Step 1.
Payer
Select the payer.
CEN Instance Owner Pays Bills (default): The connection fees and traffic processing fees that are generated by the VPC instance are paid by the account to which the CEN instance belongs.
VPC User Pays Bills: The connection fees and traffic processing fees that are generated by the VPC instance are paid by the account to which the VPC instance belongs.
This example uses the default value.
ImportantSelect the payer carefully. Changing the payer later may affect your services. For more information, see Grant a transit router permissions on a network instance that belongs to another Alibaba Cloud account.
Click OK.
Create a cross-account VPC connection.
Log on with Account B, go to the CEN console, and click the ID of the created CEN instance to go to the Basic Information page.
On the Transit Router tab, find the created Transit Router and click Create Network Instance Connection in the Operation column. Configure the network information for the data source.
The following table describes the key parameters. You can keep the default values for other parameters.
Parameter
Description and example
Instance Type
This solution describes how to connect VPCs across accounts. Select Virtual Private Cloud (VPC).
Region
Select the region of the data source. Select China (Hangzhou).
Resource Ownership UID
Select Cross-account and enter the UID of Alibaba Cloud account A in the UID field.
Network Instance
Select the VPC instance where the data source is located.
VSwitch
Select the vSwitch where the data source is located. For example, select
Account_A_Switch_hz_h.NoteNetwork interconnection using CEN requires zone-level disaster recovery. You must configure at least two vSwitches in different zones. After you ensure that the vSwitch of the data source is included, configure another vSwitch in any zone. If you have fewer than two vSwitches, go to the vSwitch console to create one. Then, select the created vSwitch.
Click Create.
Create an inter-region connection.
NoteIn this example, the data source and DataWorks are in different accounts and different regions. Therefore, you must also configure an inter-region connection. If your data source and DataWorks are in different accounts but in the same region, skip this step.
Log on with Account B, go to the CEN console, and click the ID of the created CEN instance to go to the Basic Information page.
On the Transit Router tab, find the Transit Router in China (Hangzhou) (the data source region) and click Create Network Instance Connection in the Operation column. Configure the inter-region connection information.
Parameter
Description and example
Region
Select China (Hangzhou).
Peer Region
Select China (Shanghai).
Click Create.
Configure a network connection using a VPC peering connection
Log on with Account A, go to the VPC Peering Connection console, switch the region to China (Hangzhou) in the top navigation bar, and then click Create Peering Connection. Configure the parameters.
The following table describes the key parameters. You can keep the default values for other parameters.
Parameter
Description and example
Peering Connection Name
Specify a name. In this example, set it to
Account_A to Account_B.Requester VPC Instance
Select the VPC where the RDS for MySQL data source in Account A is located. In this example, select
Account_A_hangzhou_VPC.Accepter Account Type
In this example, select
Cross-account.Accepter Alibaba Cloud Account UID
Enter the UID of Account B.
Accepter Region Type
In this example, select
Inter-region.Accepter Region
Select the region where the DataWorks workspace and resource group in Account B are located. Select
China (Shanghai).Accepter VPC Instance
Enter the ID of the VPC that contains the DataWorks resource group in Alibaba Cloud Account B, such as
Account_B_shanghai_VPC.Click OK to complete the peering connection configuration. You are automatically redirected to the basic information page of the peering connection. The Status of the peering connection is Accepting.
Log on with Account B, go to the VPC Peering Connection console, and switch the region to China (Shanghai) in the top navigation bar. You can see a peering connection that is the same as the one in Account A. Click Accept in the Operation column. After you accept the connection, the Status of the peering connection changes to Activated.
Click Configure Route Entry under Accepter VPC Instance. In the Configure Route Entry dialog box, specify a Name for the route entry and set Destination CIDR Block to the vSwitch CIDR block of the requester VPC. In this example, the vSwitch CIDR block is
192.168.6.0/24.
Log on with Account A, go to the VPC Peering Connection console, switch the region to China (Hangzhou) in the top navigation bar, and find the created peering connection.
Click Configure Route Entry under Requester VPC Instance. In the Configure Route Entry dialog box, specify a Name for the route entry and set Destination CIDR Block to the vSwitch CIDR block of the accepter VPC. In this example, the vSwitch CIDR block is
172.16.66.0/24.
3. Add a route for the DataWorks resource group
Log on to the DataWorks console with Alibaba Cloud Account B and go to the DataWorks resource group list page. Find the target resource group and click Network Settings in the Actions column.
In the corresponding feature module, find the attached VPC and click Custom Route in the Actions column.
Click Add Route, select Specified CIDR Block as the Connection Method, and set Destination CIDR Block to the vSwitch CIDR block of the RDS for MySQL instance in Alibaba Cloud Account A (in this example,
192.168.6.0/24).
4. Configure the whitelist
Log on to Alibaba Cloud Account A. Add the vSwitch CIDR Block that is attached to the DataWorks resource group to the Whitelist And Security Group of the RDS for MySQL instance. In this example, the CIDR block is 172.16.66.0/24.
5. Test network connectivity
Before you perform this step, configure cross-account authorization in the account to which the data source belongs (Account A in this example).
Log on with Account B.
Log on to the DataWorks console. In the top navigation bar, select the desired region. In the left-side navigation pane, choose . On the page that appears, select the desired workspace from the drop-down list and click Go to Data Integration.
In the navigation pane on the left, click Data Source to go to the Data Source List page. Then, click Add Data Source.
Select the MySQL data source type and configure the data source information.
Set Configuration Mode to Alibaba Cloud Instance Mode.
Set Owner Account to Other Alibaba Cloud Account.
Set Other Alibaba Cloud Account UID to the UID of Account A.
For RAM Role Name For Authorization, enter the RAM role that is configured in Account A. For more information, see Cross-account authorization.
Set Region to China (Hangzhou).
For Instance, select the RDS for MySQL instance that is created in the China (Hangzhou) region in Account A and for which the network connection is established.
In the Connection Configuration section, click Test Connectivity for the resource group attached to the workspace and verify that the result is Connected.
NoteIf the connectivity test Fails, you can use the Connectivity Diagnosis Tool to identify and resolve the connection problem. If the problem persists, submit a ticket.
References
For answers to common questions about network connectivity, see Resource group operations and network connectivity.