Before DataWorks can run data synchronization, scheduling, or other tasks, the virtual private cloud (VPC) attached to your resource group must be able to reach your data source. This topic describes the available network connectivity solutions for different network environments.
Background
Most DataWorks features — such as creating data sources, data synchronization, data analysis, metadata collection, and DataService — operate on connected data sources or compute resources. If the data source you want to access is not in the VPC attached to your DataWorks resource group (for example, the data source is in another VPC or an on-premises IDC), you must select an appropriate network connectivity solution to connect the VPC attached to the resource group to the network where the data source resides.
For example, in a data synchronization flow, the VPC attached to the resource group must be connected to the networks of both the source and the destination.
Prerequisites
You have purchased a resource group with the required specifications. For details, see Use a serverless resource group.
For more information about resource groups, see Overview of resource groups.
The solutions in this topic apply to serverless resource groups and the following legacy resource groups: exclusive resource groups for Data Integration, exclusive resource groups for scheduling, and exclusive resource groups for Data Service.
Notes
Serverless resource groups can access data sources in complex network environments over a private network by associating a VPC. However, serverless resource groups cannot access the public network by default. To reach a data source over the Internet, configure an Internet NAT Gateway and EIPs for the VPC attached to the resource group. For details, see Connect to a data source over the Internet.
Public network connections offer no speed or stability guarantees. For data synchronization, use a private network or Cloud Enterprise Network (CEN).
Network connectivity between a resource group and a data source is required before tasks can run.
Resource groups do not support classic network environments. Migrate your data sources and services from a classic network to a VPC before proceeding.
Choose a solution
Your choice depends on where your data source is relative to your DataWorks workspace resource group. Use the information below to identify the right solution based on your scenario:
Solution 1: Connect to an Alibaba Cloud data source (same account, same region)
Use cases
This solution is recommended if your data source and DataWorks workspace meet all of the following conditions:
The data source is an Alibaba Cloud product.
The data source and the DataWorks workspace belong to the same Alibaba Cloud account.
The data source and the DataWorks workspace are in the same region.
Solution description
For the same-account, same-region scenario, we recommend that you use a VPC (private network) connection. Deploy the resource group of your DataWorks workspace and the data source in the same VPC to enable network connectivity.
Network connectivity diagram
Configure network connectivity
For the detailed solution description and configuration steps, see Connect to a data source in the same account and region.
Solution 2: Connect to an Alibaba Cloud data source (same account, different regions)
Use cases
This solution is recommended if your data source and DataWorks workspace meet all of the following conditions:
The data source is an Alibaba Cloud product.
The data source and the DataWorks workspace belong to the same Alibaba Cloud account.
The data source and the DataWorks workspace are in different regions.
Solution description
For the same-account, different-region scenario, we recommend that you use a VPC (private network) connection. Use a network connectivity tool (Cloud Enterprise Network or VPC Peering Connection) to connect the VPC of your DataWorks workspace resource group with the VPC where the data source resides.
Network connectivity diagram
Configure network connectivity
For the detailed solution description and configuration steps, see Connect to a data source in the same account but a different region.
Solution 3: Connect to an Alibaba Cloud data source (different accounts)
Use cases
This solution is recommended if your data source and DataWorks workspace meet all of the following conditions:
The data source is an Alibaba Cloud product.
The data source and the DataWorks workspace belong to different Alibaba Cloud accounts.
Solution description
For the different-account scenario, we recommend that you use a VPC (private network) connection. Use a network connectivity tool (Cloud Enterprise Network or VPC Peering Connection) to connect the data source under Account A with the DataWorks workspace resource group under Account B.
Network connectivity diagram
Configure network connectivity
For the detailed solution description and configuration steps, see Connect to a data source under a different account.
Solution 4: Connect to a data source deployed on ECS
Use case
This solution is recommended if your data source meets the following condition:
The data source is deployed on an Alibaba Cloud ECS instance.
Solution description
If the ECS instance hosting the data source and DataWorks are under the same account and in the same region, we recommend that you use a VPC (private network) connection. Deploy the resource group of your DataWorks workspace and the ECS instance in the same VPC to enable network connectivity.
If the ECS instance hosting the data source and DataWorks are under different accounts or under the same account but in different regions, we recommend that you use a VPC (private network) connection. Use a network connectivity tool (Cloud Enterprise Network or VPC Peering Connection) to connect the VPC of your DataWorks workspace resource group with the VPC where the ECS instance resides.
Network connectivity diagram
Same account, same region
Same account, different regions
Different accounts
Configure network connectivity
For the detailed solution description and configuration steps, see Connect to a data source deployed on ECS.
Solution 5: Connect to a data source in an on-premises IDC
Use case
This solution is recommended if your data source meets the following condition:
The data source is deployed in an on-premises IDC.
Solution description
For data sources deployed in an on-premises IDC, we recommend that you use a VPC (private network) connection. Use a network connectivity tool (Express Connect) to connect the on-premises network where the data source resides with the VPC of your DataWorks workspace resource group.
Network connectivity diagram
Configure network connectivity
For the detailed solution description and configuration steps, see Connect to a data source in an on-premises IDC.
Solution 6: Connect to a data source over the Internet
Use case
This solution is recommended if your data source meets the following condition:
The data source has a public endpoint.
Solution description
Serverless resource groups cannot access the public network by default. You must configure an Internet NAT Gateway and EIPs for the VPC attached to the resource group before you can access data sources over the Internet.
Legacy resource groups have public network access by default and can connect directly.
NoteLegacy resource groups are being phased out. We recommend that you use serverless resource groups.
Network connectivity diagram
The diagram applies only to serverless resource groups. Legacy resource groups already have EIPs attached by default and can connect directly.
Configure network connectivity
For the detailed solution description and configuration steps, see Connect to a data source over the Internet.
References
For more information about resource groups, see Overview of resource groups.
To create and use a resource group, see Use a serverless resource group.
To associate a VPC with a resource group, see Associate a VPC.
To configure an Internet NAT Gateway for the VPC and vSwitch attached to a resource group, see Create and manage an Internet NAT Gateway.
For common questions about network connectivity, see FAQ about network connectivity.