To ensure that tasks such as data synchronization and scheduling run smoothly in DataWorks, you must establish a network connection between the virtual private cloud (VPC) attached to your resource group and your data source. Data sources can include databases, DataService Studio, or other data in the target network environment. This topic describes network connectivity solutions for different network environments.
Background information
Most DataWorks features, such as creating data sources, data synchronization, DataAnalysis, data ingestion, and DataService Studio, operate on connected data sources or computing resources. If a data source is not in the VPC that is attached to your DataWorks resource group, for example, if the data source is in another VPC or an on-premises data center, you must select an appropriate network connectivity solution to connect the VPC attached to the DataWorks resource group to the network of the data source.
For example, in a data synchronization flow, the VPC attached to the resource group must be connected to the networks of both the source and the destination.
Prerequisites
You have purchased a resource group with the required specifications. For more information about how to purchase a resource group, see Use a Serverless resource group.
For more information about resource groups, see Overview of DataWorks resource groups.
The network connectivity solutions in this topic apply only to Serverless resource groups and the following legacy resource groups: exclusive resource groups for Data Integration, exclusive resource groups for scheduling, and exclusive resource groups for DataService Studio.
Precautions
You can attach a VPC to a Serverless resource group to access data sources or network addresses in complex network environments over a private network. However, Serverless resource groups cannot access the public network by default. To access a data source or network over the public network, you must configure an Internet NAT gateway and elastic IP addresses (EIPs) for the VPC that is attached to the Serverless resource group. For more information, see Connect to a data source over the Internet.
The speed and stability of tasks that run over the public network cannot be guaranteed. Use a private network or Cloud Enterprise Network (CEN) for data synchronization.
Network connectivity between a resource group and a data source is a prerequisite for running tasks.
Resource groups do not support interaction with classic network environments. You must migrate your data sources or services from a classic network to a VPC.
Network connectivity solutions
The network connectivity solution that you choose depends on the network environments of your data source and your DataWorks resource group. Select the solution that meets your needs.
Solution 1: Connect to an Alibaba Cloud data source in the same account and region
Scenarios
Use this solution if your data source and DataWorks workspace meet all the following conditions:
The data source is an Alibaba Cloud product.
The data source and the DataWorks workspace belong to the same Alibaba Cloud account.
The data source and the DataWorks workspace are in the same region.
Solution description
In a scenario where the data source and workspace are in the same account and region, you can use a VPC (private network) connection. You can deploy the resource group of the DataWorks workspace and the data source in the same VPC to enable network communication.
Network connection diagram
Configure the network connection
For a detailed solution description and procedure, see Connect to a data source in the same account and region.
Solution 2: Connect to an Alibaba Cloud data source in the same account but a different region
Scenarios
Use this solution if your data source and DataWorks workspace meet all the following conditions:
The data source is an Alibaba Cloud product.
The data source and the DataWorks workspace belong to the same Alibaba Cloud account.
The data source and the DataWorks workspace are in different regions.
Solution description
In a scenario where the data source and workspace are in the same account but different regions, you can use a VPC (private network) connection. You can use a network connectivity tool, such as CEN or a VPC peering connection, to connect the VPC of the DataWorks resource group to the VPC of the data source. This enables network communication.
Network connection diagram
Configure the network connection
For a detailed solution description and procedure, see Connect to a data source in the same account but a different region.
Solution 3: Connect to an Alibaba Cloud data source in a different account
Scenarios
Use this solution if your data source and DataWorks workspace meet all the following conditions:
The data source is an Alibaba Cloud product.
The data source and the DataWorks workspace belong to different Alibaba Cloud accounts.
Solution description
In a scenario where the data source and workspace are in different accounts, you can use a VPC (private network) connection. You can use a network connectivity tool, such as CEN or a VPC peering connection, to connect the VPC of the data source in Account A to the VPC of the DataWorks resource group in Account B. This enables network communication.
Network connection diagram
Configure the network connection
For a detailed solution description and procedure, see Connect to a data source in a different account.
Solution 4: Connect to a data source deployed on an ECS instance
Scenarios
Use this solution if your data source meets the following condition:
The data source is deployed on an Alibaba Cloud ECS instance.
Solution description
If the ECS instance where the data source is deployed and the DataWorks workspace are in the same account and region, you can use a VPC (private network) connection. You can deploy the resource group of the DataWorks workspace and the ECS instance in the same VPC to enable network communication.
If the ECS instance where the data source is deployed and the DataWorks workspace are in different accounts or in the same account but different regions, you can use a VPC (private network) connection. You can use a network connectivity tool, such as CEN or a VPC peering connection, to connect the VPC of the DataWorks resource group to the VPC of the ECS instance. This enables network communication.
Network connection diagrams
Same account and region
Same account, different region
Different accounts
Configure the network connection
For a detailed solution description and procedure, see Connect to a self-managed data source on an ECS instance.
Solution 5: IDC data source network connectivity
Scenarios
Use this solution if your data source meets the following condition:
The data source is deployed in an on-premises data center.
Solution description
In a scenario where the data source is deployed in an on-premises data center, you can use a VPC (private network) connection. You can use a network connectivity tool, such as Express Connect, to connect the on-premises network of the data source to the VPC of the DataWorks resource group. This enables network communication.
Network connection diagram
Configure the network connection
For a detailed solution description and procedure, see Connect to a data source in an on-premises data center.
Solution 6: Connect to a data source over the Internet
Scenarios
Use this solution if your data source meets the following condition:
The data source has a public endpoint.
Solution description
By default, Serverless resource groups cannot access the public network. To access a data source over the public network, you must configure an Internet NAT gateway and EIPs for the VPC that is attached to the resource group.
Legacy resource groups can access the public network and can connect directly.
NoteLegacy resource groups are being phased out. Use Serverless resource groups.
Network connection diagram
The diagram applies only to Serverless resource groups. Legacy resource groups have EIPs attached by default and can connect directly to the Internet.
Configure the network connection
For a detailed solution description and procedure, see Connect to a data source over the Internet.
References
For more information about resource groups, see Overview of DataWorks resource groups.
To create and use a resource group, see Use a Serverless resource group.
To attach a VPC to a resource group, see Attach a VPC.
You can configure an Internet NAT gateway for the VPC and vSwitch that are attached to the resource group. For more information, see Use the SNAT feature of an Internet NAT gateway to access the Internet.
For answers to frequently asked questions about network connectivity, see Resource group operations and network connectivity.