DataWorks Data Integration uses resource groups to run data synchronization tasks. The resource group type and its network configuration determine whether your task can reach the target database. Use this guide to set up network connectivity for common scenarios and resolve resource group issues.
Resource group types
DataWorks provides three resource group types. Identify the correct type before configuring network access or troubleshooting.
| Resource group type | Log identifier | Network stability |
|---|---|---|
| Default resource group | basecommon_ group_xxxxxxxxx |
Shared, unstable |
| Custom resource group | basecommon_xxxxxxxxx |
Dedicated |
| Exclusive resource group for Data Integration | basecommon_S_res_group_xxx |
Dedicated, stable |
To identify which resource group a task is using, check the task log for one of the identifiers above.
Before you configure network connectivity
Before setting up network connectivity for batch synchronization, gather the following information:
-
The VPC, vSwitch, and region of your database
-
The region of your DataWorks workspace
-
Whether the database and workspace are in different Alibaba Cloud accounts or regions
If you encounter issues during data synchronization, see Supported data source types, Reader plug-ins, and Writer plug-ins for troubleshooting.
If a data source connectivity test fails, see Network connectivity solutions for a full list of options.
When using an exclusive resource group for Data Integration, complete these steps in order: purchase the resource group, associate it with the VPC, evaluate whether to add a custom route, configure the database IP address whitelist, then associate the resource group with your workspace. For details, see Use exclusive resource groups for Data Integration.
Network connectivity scenarios
Self-managed ECS database (same account, internal network)
To sync data from a self-managed database on an Elastic Compute Service (ECS) instance over an internal network:
-
Associate the exclusive resource group for Data Integration with the VPC where the ECS instance resides. A route to the VPC CIDR block is added automatically — do not delete it manually. Deleting this route can cause access failures for other databases and lead to task errors.
-
Add the vSwitch CIDR block associated with the exclusive resource group to the database IP address whitelist. For details, see Configure an IP address whitelist.
Cross-region database (same account)
To sync data from a database in a different region, choose a solution from Network connectivity solutions first.
For Internet-based sync: add the Elastic IP (EIP) of the exclusive resource group to the database IP address whitelist. For details, see Configure an IP address whitelist.
Data transfer over the Internet incurs fees. For more information, see Internet traffic billing.
Cross-account database (different Alibaba Cloud account)
Choose a solution from Network connectivity solutions before proceeding.
Internet sync: Add the EIP of the exclusive resource group to the database IP address whitelist.
Data transfer over the Internet incurs fees. For more information, see Internet traffic billing.
Internal network sync: Complete the following four steps:
-
Connect the two accounts' networks using VPN Gateway or Express Connect.
-
Associate the exclusive resource group for Data Integration with the VPC connected to the other account's network.
-
Add a custom route: set the next hop to Local IDC and specify the IP address of the target database.
-
Add the vSwitch CIDR block of the exclusive resource group to the database IP address whitelist. For details, see Configure an IP address whitelist.
Common issues
VPC connectivity test fails
If you added the data source using a VPC endpoint:
-
Confirm the exclusive resource group for Data Integration is associated with the VPC where the database resides.
-
Confirm the vSwitch CIDR block associated with the exclusive resource group is added to the database IP address whitelist. For details, see Configure an IP address whitelist.
If you added the data source using a public endpoint:
Confirm the EIP of the exclusive resource group is added to the database IP address whitelist. For details, see Configure an IP address whitelist.
Data transfer over the Internet incurs fees. For more information, see Internet traffic billing.
Intermittent connectivity failures
Check whether the task is running on a shared resource group — shared resource groups have unstable network connectivity. Switch to an exclusive resource group for Data Integration to get a stable connection.
Exclusive resource group not found
Make sure the exclusive resource group is associated with the DataWorks workspace. For details, see Use exclusive resource groups for Data Integration.
Custom resource group waiting for a gateway
-
Go to the DataWorks console. In the left navigation pane, click Resource Groups and go to the Custom Resource Groups tab.
-
Click Server Management for the task's scheduling resource group.
-
Check whether the server is stopped or occupied by other running tasks.
If the issue persists, restart the service:
su - admin /home/admin/alisatasknode/target/alisatasknode/bin/serverctl restart"Insufficient resources" error despite available resources
Remaining resources are likely insufficient to start a new task — other tasks may be queued. Check the resource group details to confirm current usage.
Resource group operations
Find the public IP for whitelisting
-
In the DataWorks console, click Resource Groups in the left navigation pane.
-
Go to the Exclusive Resource Groups tab and click Details next to the Data Integration resource group.
-
In the Basic Information section, copy the EIP Address.
-
Add the EIP to your database IP address whitelist.
Change the resource group for a scheduled task
In Operation Center: Switch both the scheduling resource group and the execution resource group for the Data Integration task directly in the task settings.
In DataStudio: Modify the resource group in the task configuration, then deploy the changes to the production environment.
In standard mode, a commit only takes effect in the development environment. Deploy the changes to apply them to scheduled tasks in production. After deployment, verify the resource group change in Operation Center.
-
Modify the scheduling resource group:
-
Modify the execution resource group for the Data Integration task:
