Network connectivity solutions

Prerequisites

Before you begin, make sure you have:

• A resource group with appropriate specifications purchased. For setup instructions, see Create and use a serverless resource group.

The solutions in this topic apply to serverless resource groups and the following old-version resource group types: exclusive resource groups for Data Integration, exclusive resource groups for scheduling, and exclusive resource groups for DataService Studio. For an overview of resource group types, see Overview.

Limitations

• Serverless resource groups cannot access the Internet by default. To connect a serverless resource group to an Internet-accessible data source, configure an Internet NAT gateway for the associated VPC and attach an elastic IP address (EIP) to the gateway. For details, see Solution 6: Internet data source.

• Synchronizing data over the Internet reduces efficiency and stability. Use an internal network or Cloud Enterprise Network (CEN) instead.

• Resource groups cannot connect to data sources deployed in the classic network. If your data source uses the classic network, migrate it to a VPC.

How it works

For data synchronization tasks, the resource group must be able to reach both the source and destination.

Choose a connectivity solution

Use the following table to identify the right solution for your setup. Match your data source location and account relationship to find the recommended approach.

Solution 1: Same account, same region

When to use: Your data source belongs to an Alibaba Cloud service, and both the data source and DataWorks workspace are under the same Alibaba Cloud account in the same region.

Deploy the resource group and data source in the same VPC. The resource group then reaches the data source over the internal network.

When the DataWorks resource group and the data source (such as ApsaraDB RDS for MySQL or SQL Server) are deployed in the same VPC, you can access the data source directly over the internal network. No additional network connectivity tools such as CEN or NAT gateways are required.

If the data source also supports public endpoint access, you can use the connection string mode to configure the data source. However, we recommend that you use the VPC internal network connection for better stability and performance.

For configuration steps, see Solution 1: Establish a network connection between a resource group and a data source that belong to the same Alibaba Cloud account and reside in the same region.

Solution 2: Same account, different regions

When to use: Your data source belongs to an Alibaba Cloud service, both the data source and DataWorks workspace are under the same Alibaba Cloud account, but they reside in different regions.

Use Cloud Enterprise Network (CEN) or a VPC peering connection to connect the resource group VPC with the VPC where the data source is deployed.

For configuration steps, see Solution 2: Establish a network connection between a resource group and a data source that belong to the same Alibaba Cloud account but reside in different regions.

Solution 3: Different accounts

When to use: The data source and DataWorks workspace belong to different Alibaba Cloud accounts (regardless of region).

Use Cloud Enterprise Network (CEN) or a VPC peering connection to connect the VPCs across accounts. For example, if the data source is in Account A and the resource group is in Account B, establish cross-account VPC connectivity between the two.

For configuration steps, see Solution 3: Establish a network connection between a resource group and a data source that belong to different Alibaba Cloud accounts.

Solution 4: ECS-hosted data source

When to use: Your data source is hosted on an Elastic Compute Service (ECS) instance.

The approach depends on whether the ECS instance and DataWorks share the same account and region:

• Same account and same region: Deploy the resource group and the ECS instance in the same VPC.

• Different accounts or different regions: Use Cloud Enterprise Network (CEN) or a VPC peering connection to connect the resource group VPC with the VPC where the ECS instance is deployed.

Same account and same region

When the resource group and the ECS-hosted data source (such as a self-managed MySQL or SQL Server database) are deployed in the same VPC, the resource group can access the data source directly over the internal network. No additional network configuration is needed.

Same account, different regions

Different accounts

For configuration steps, see Solution 4: Establish a network connection between a resource group and a data source that is hosted on an ECS instance.

Solution 5: On-premises data center

When to use: Your data source is deployed in an on-premises data center.

Use Express Connect to link the data center network to the VPC associated with the resource group.

For configuration steps, see Solution 5: Establish a network connection between a resource group and a data source that is deployed in a data center.

Solution 6: Internet data source

When to use: Your data source has a public endpoint.

• Serverless resource groups: Configure an Internet NAT gateway for the VPC associated with the resource group, then attach an EIP to the gateway.

• Old-version resource groups: These can access the Internet directly, as EIPs are associated by default.

Old-version resource groups are being phased out. Use serverless resource groups for new setups.

Configure IP whitelisting for external data sources

When you use a serverless resource group to access an external data source such as Microsoft Azure Databricks or SQL Server, you must add the resource group's egress IP address to the data source's access whitelist. To determine the egress IP address:

• Public Internet access: Find the elastic IP address (EIP) that is attached to the Internet NAT gateway associated with the resource group's VPC. This EIP is the egress IP address for all outbound Internet traffic from the resource group.

• Internal network access (same VPC): Use the CIDR block of the VPC where the resource group is deployed. No EIP is needed for internal communication.

After you obtain the EIP address or VPC CIDR block, add it to the IP whitelist of your external data source. For example, add the EIP to the IP allowlist of a Databricks workspace or the firewall rules of a SQL Server instance.

The following diagram applies to serverless resource groups only.

For configuration steps, see Solution 6: Establish a network connection between a resource group and a data source that is deployed on the Internet.

What's next

• To associate a resource group with a VPC, see Associate a resource group with a VPC.

• To configure SNAT so your resource group can reach the Internet, see Use the SNAT feature of an Internet NAT gateway to access the Internet.

• To troubleshoot network connectivity issues, see Network connectivity and operations on resource groups.

FAQ