To ensure tasks like data synchronization and scheduling run correctly in DataWorks, you must connect your resource group's VPC to the network where your data source resides. A data source can be a database, data service, or another resource in the target network. This document describes various network connectivity solutions for different network environments.
Background
Most features in DataWorks, such as creating data sources, data synchronization, data analytics, data collection, and DataService Studio, require a connection to data sources or computing resources. If a data source is not in the VPC associated with your DataWorks resource group (for example, it is in another VPC or an on-premises IDC), choose a suitable network solution to connect the resource group's VPC to the data source's network.
For example, during data synchronization, the resource group's VPC must be able to connect to both the source and destination data sources.
Prerequisites
A resource group with the required specifications. For more information, see Use a serverless resource group.
For more information about resource groups, see Overview of DataWorks resource groups.
The network connectivity solutions described in this document apply only to serverless resource groups and the following legacy resource groups: exclusive resource groups for Data Integration, exclusive resource groups for scheduling, and exclusive resource groups for DataService Studio.
Usage notes
A serverless resource group can be associated with a VPC to access data sources and services in complex network environments over an internal network. However, serverless resource groups do not have public network access by default. To access a data source or network over the public network, configure an Internet NAT gateway and an EIP for the VPC associated with the serverless resource group. For more information, see Connect to a data source over the Internet.
The speed and stability of tasks that run over the public network cannot be guaranteed. Therefore, using an internal network or Cloud Enterprise Network (CEN) is recommended for data synchronization.
Connectivity between the resource group and the data source is a prerequisite for tasks to run successfully.
DataWorks resource groups do not support connections to classic networks. We recommend migrating data sources or services from a classic network to a VPC environment.
Network connectivity solutions
The right network solution depends on the location of your data source relative to your DataWorks resource group. Select the solution that matches your case.
Solution 1: Connect to an Alibaba Cloud data source (same account and region)
Use cases
Use this solution if your data source and DataWorks workspace meet these conditions:
The data source is an Alibaba Cloud product.
The data source and the DataWorks workspace belong to the same Alibaba Cloud account.
The data source and the DataWorks workspace are in the same region.
Solution description
For same-account, same-region setups, a VPC connection is recommended. Deploying the DataWorks resource group and the data source in the same VPC establishes network connectivity.
Network connectivity diagram
Configure network connectivity
For details and configuration steps, see Connect to a data source in the same account and region.
Solution 2: Connect to an Alibaba Cloud data source (same account, different regions)
Use cases
Use this solution if your data source and DataWorks workspace meet these conditions:
The data source is an Alibaba Cloud product.
The data source and the DataWorks workspace belong to the same Alibaba Cloud account.
The data source and the DataWorks workspace are in different regions.
Solution description
For same-account, different-region setups, a VPC connection is recommended. Use a network connectivity tool, such as CEN or a VPC peering connection, to connect the VPC of the DataWorks resource group with the VPC of the data source.
Network connectivity diagram
Configure network connectivity
For details and configuration steps, see Connect to a data source in the same account but a different region.
Solution 3: Connect to an Alibaba Cloud data source (different accounts)
Use cases
Use this solution if your data source and DataWorks workspace meet these conditions:
The data source is an Alibaba Cloud product.
The data source and the DataWorks workspace belong to different Alibaba Cloud accounts.
Solution description
In a cross-account scenario, a VPC connection is recommended. Use a network connectivity tool, such as CEN or a VPC peering connection, to connect the VPC that hosts the data source in Account A with the VPC for the DataWorks resource group in Account B.
Network connectivity diagram
Configure network connectivity
For details and configuration steps, see Connect to a data source in a different account.
Solution 4: Connect to a data source deployed on an ECS instance
Use case
Use this solution if your data source is deployed on an ECS instance.
Solution description
If the ECS instance is in the same account and region as DataWorks, a VPC connection is recommended. Deploy the DataWorks resource group and the ECS instance in the same VPC to enable network connectivity.
If the ECS instance is in a different account or a different region from DataWorks, a VPC connection is recommended. Use a network connectivity tool, such as CEN or a VPC peering connection, to connect the VPC of the DataWorks resource group with the VPC of the ECS instance.
Network connectivity diagram
Same account and region
Same account, different regions
Different accounts
Configure network connectivity
For details and configuration steps, see Connect to a self-managed data source on an ECS instance.
Solution 5: Connect to a data source in an on-premises IDC
Use case
Use this solution if your data source is deployed in an on-premises IDC.
Solution description
If your data source is deployed in an on-premises IDC, a VPC connection is recommended. Use a network connectivity tool such as Express Connect to connect your on-premises network with the VPC of the DataWorks resource group. This establishes network connectivity.
Network connectivity diagram
Configure network connectivity
For details and configuration steps, see Connect to a data source in an on-premises IDC.
Solution 6: Connect to a data source over the Internet
Use case
Use this solution if your data source has a public endpoint address.
Solution description
Serverless resource groups do not have public network access by default. To enable public network access to the data source, configure an Internet NAT gateway and an EIP for the VPC attached to the resource group.
Legacy resource groups have public network access and can connect directly.
NoteLegacy resource groups are being deprecated. We recommend that you use serverless resource groups.
Network connectivity diagram
This diagram applies only to serverless resource groups. Legacy resource groups have an EIP attached by default and can connect directly.
Configure network connectivity
For details and configuration steps, see Connect to a data source over the Internet.