All Products
Search

This Product

    DataWorks:Resource group operations and network connectivity

    Document Center

    DataWorks:Resource group operations and network connectivity

    Last Updated:Aug 24, 2026

    What should I know about DataWorks and its network capabilities before running a batch synchronization task?

    Before you synchronize data using Data Integration, make sure that you are familiar with the following information:

    • The VPC, vSwitch, and region of the database from which you want to synchronize data, and the region of the DataWorks workspace.

    • Whether special conditions exist, such as the database and the DataWorks workspace being under different Alibaba Cloud accounts or in different regions.

    If you encounter issues during task synchronization, you can first refer to Troubleshoot data synchronization issues for self-service troubleshooting.

    If you encounter issues when testing data source connectivity, you can first refer to Troubleshoot data source connectivity issues to select the appropriate solution for troubleshooting.

    When you use an exclusive resource group for Data Integration, follow these steps: purchase the resource group, associate it with the VPC where the database is connected, evaluate whether to add routes, configure database allowlists, and associate the exclusive resource group with the workspace. For more information, see Configure an exclusive resource group for Data Integration.

    How do I ensure network connectivity between DataWorks and a self-managed database on an ECS instance?

    If you want to use an exclusive resource group for Data Integration to synchronize data from a self-managed database on an ECS instance over the internal network, you must configure network settings for the exclusive resource group for Data Integration. For more information, see Configure an exclusive resource group for Data Integration. The key points of the connectivity solution are as follows:

    • Associate the exclusive resource group for Data Integration with the VPC where the ECS instance resides. A route pointing to the VPC CIDR block is automatically added. We recommend that you do not manually delete this route, as doing so may cause access failures to other databases and task errors.

    • Add the vSwitch CIDR block associated with the exclusive resource group for Data Integration to the database allowlist. For more information, see Add allowlist entries.

    How do I ensure network connectivity when my database and DataWorks workspace are in different regions?

    Before you begin, refer to Troubleshoot data source connectivity issues to select the appropriate network connectivity solution. The key points of the connectivity solution are as follows:

    If you use the Internet to synchronize data from a database in a different region, you must add the EIP of the exclusive resource group to the database allowlist. For more information, see Add allowlist entries.

    Note

    Internet-based data synchronization incurs Internet traffic fees. For more information, see Pricing.

    How do I ensure network connectivity when my database and DataWorks workspace are under different Alibaba Cloud accounts?

    Before you begin, refer to Troubleshoot data source connectivity issues to select the appropriate solution for troubleshooting.

    • If you use the Internet to synchronize data from a database under a different account, you must add the EIP of the exclusive resource group to the database allowlist. For more information, see Add allowlist entries.

      Note

      Internet-based data synchronization incurs Internet traffic fees. For more information, see Pricing.

    • If you want to synchronize data from a database under a different account over the internal network, perform the following steps:

      1. Connect the networks of the two Alibaba Cloud accounts by using a networking service such as VPN Gateway or Express Connect.

      2. Associate the exclusive resource group for Data Integration with the VPC that is already connected to the network of the other Alibaba Cloud account.

      3. After you associate the VPC, add a custom route, select "On-premises IDC", and add the IP address of the destination database.

      4. Add the vSwitch CIDR block associated with the exclusive resource group to the database allowlist. Note that the vSwitch CIDR block is the one associated with the resource group. For more information, see Add allowlist entries.

    What should I do if a connectivity test fails for a database in a VPC?

    • If you added the data source by using a VPC endpoint:

      1. Make sure that you have associated the exclusive resource group for Data Integration with the VPC where the database resides.

      2. Make sure that the vSwitch CIDR block associated with the exclusive resource group for Data Integration has been added to the database allowlist. For more information, see Add allowlist entries.

    • If you added the data source by using a public endpoint and the connectivity test fails with the exclusive resource group for Data Integration, check whether the EIP of the exclusive resource group for Data Integration has been added to the database allowlist. For more information, see Add allowlist entries.

      Note

      Internet-based data synchronization incurs Internet traffic fees. For more information, see Pricing.

    What should I do if data source connectivity is intermittent?

    Check whether you are using a shared resource group. The network of shared resource groups is unstable. We recommend that you use an exclusive resource group for Data Integration to ensure connection stability.

    How do I ensure network connectivity between a serverless resource group and Hologres when syncing data with Data Integration?

    The serverless resource group must be associated with the VPC where the Hologres instance resides. Perform the following steps:

    1. Log on to the DataWorks console and find the target serverless resource group in the resource group list.

    2. Click Network Settings. In the Data Scheduling and Data Integration section, click Add VPC Association.

    3. Select the VPC, zone, and vSwitch where the Hologres instance resides, and then click OK.

    4. After the association is complete, create a Hologres data source in the data source management section of Data Integration, and click Test Connectivity to verify that the connection is successful.

    If the CIDR block of the VPC already associated with the resource group conflicts with the Hologres VPC CIDR block, disassociate the conflicting VPC first and then add the Hologres VPC. Associating a VPC creates elastic network interfaces (ENIs) in the VPC that consume quota. Do not delete these ENIs.

    For detailed operations on switching Hologres networks, see Switch Hologres network configurations.

    How do I troubleshoot connection errors or connectivity failures when DataWorks connects to Kafka due to an advertised.listeners mismatch?

    If DataWorks fails to connect to a Kafka cluster or reports connection errors, troubleshoot the issue based on the following scenarios:

    1. In the ECS security group, allow access from the CIDR block of the vSwitch that is bound to the DataWorks resource group used for the task (for example, 172.17.112.0/20).

    2. Make sure that the Kafka cluster address matches the value of the advertised.listeners parameter in the server.properties file of each broker, and that all brokers in the cluster are in a normal running state.

    3. If you connect to Kafka by using an internal IP address, make sure that the VPC bound to the DataWorks resource group can access this internal IP address.

    4. If you connect to Kafka by using a public IP address, configure a NAT Gateway and an Elastic IP Address (EIP) for the VPC bound to the DataWorks resource group.

    5. If domain name resolution fails, configure internal DNS resolution by using PrivateZone.

    What should I do if I purchased an exclusive resource group but cannot find it when testing data source connectivity or running a task?

    Make sure that the exclusive resource group is associated with the DataWorks workspace. For more information, see Associate a resource group with a workspace.

    How do I check logs to determine the type of resource group a task is running on?

    • If a task is running on the default resource group, the following information appears in the logs: running in Pipeline[basecommon_ group_xxxxxxxxx].

    • If a task is running on a custom resource group for Data Integration, the following information appears in the logs: running in Pipeline[basecommon_xxxxxxxxx].

    • If a task is running on an exclusive resource group for Data Integration, the following information appears in the logs: running in Pipeline[basecommon_S_res_group_xxx].

    How do I switch the resource group used by a task in the scheduling system?

    • Switch the scheduling resource group and Data Integration resource group for task execution in Operation Center: On the Operation Center > Scheduled Task O&M > Scheduled Tasks page, select the tasks for which you want to switch the resource group, and click Change Scheduling Resource Group or Change Data Integration Resource Group at the bottom of the page to make batch changes.

    • Go through the deployment process in Data Studio to change the resource group used by tasks in the production environment.

      Note

      When you change the resource group by using the following method, remember to deploy the changes. In standard mode, submitting changes only takes effect in the development environment. Changes take effect for automatic scheduling of scheduled tasks in the production environment only after deployment. After deployment, you can check whether the resource group change has taken effect in the scheduled tasks section of Operation Center.

      1. Change the resource group for task scheduling: Open the Schedule Settings panel of the task node. In the Resource Properties section, click the Scheduling Resource Group drop-down list and select the target resource group (for example, Shared Scheduling Resource Group).

      2. Change the Data Integration resource group for task execution: Click the Data Integration Resource Group Configuration tab on the right sidebar. In the Exclusive Resource Group for Data Integration drop-down list, select the target resource group (for example, xiangcui_vpc).

    How do I troubleshoot a custom resource group that is waiting for a gateway?

    Log on to the DataWorks console. In the left-side navigation pane, click Resource Group List and go to the Custom Resource Groups tab. Click Server Management next to the scheduling resource group used by the task, and check whether the server is stopped or occupied by other tasks.

    If the preceding troubleshooting does not resolve the issue, run the following command to restart the service.

    su - admin /home/admin/alisatasknode/target/alisatasknode/bin/serverctl restart

    How do I find the public IP address of a resource group for an IP address allowlist?

    If you use an exclusive resource group for Data Integration to synchronize data over the Internet, add the EIP of the exclusive resource group for Data Integration to the database allowlist. You can obtain the EIP as follows:

    On the Resource Groups page of the DataWorks console, click the Exclusive Resource Groups tab. Click Details next to the Data Integration resource group, and copy the EIP from the basic information section to the database allowlist.

    Why do I get an "insufficient resources" error when the resource group has available resources?

    Check the resource group details. In most cases, the remaining resources of the resource group are insufficient for new tasks. For example, the resource group may have queued tasks at this time.

    How many VPCs can be associated with a serverless resource group, and what should I do if I reach the quota?

    Each serverless resource group supports a maximum of two VPC associations.

    Log on to the DataWorks console. On the Resource Groups page, find the target serverless resource group and click Details. In the basic information section of the resource group details page, the VPCs That Can Be Associated field displays the current quota usage in the format X associated, Y remaining.

    When the quota of two VPCs is fully used (2 associated, 0 remaining), you cannot add more VPC associations. You can resolve this issue by using the following methods:

    • Evaluate whether you can reuse an already-associated VPC. Deploy the data source that requires connectivity in an already-associated VPC, or connect the VPC where the data source resides to the already-associated VPC by using Cloud Enterprise Network (CEN).

    • Use Cloud Enterprise Network (CEN) for cross-VPC connectivity. Add the already-associated VPC and the target VPC to the same CEN instance to establish network connectivity. This way, you can access data sources in the target VPC without consuming additional VPC association quota.

    VPC associations in the network settings of a resource group are divided into two sections: Data Service and Data Scheduling. The quota of two VPCs is a resource-group-level total quota shared by the two sections.

    What should I do if a DataWorks resource group and an ApsaraDB RDS for MySQL instance are in the same VPC and an allowlist is configured, but the connection still fails?

    This issue usually occurs because the task did not run on the resource group that is associated with the VPC of the database. When you configure or run the task, explicitly select the resource group that is bound to the VPC where the ApsaraDB RDS for MySQL instance resides and that already has the required database allowlist entries configured.

    If you only need to develop and debug the task in Data Integration, it is sufficient to make sure that the data source connectivity test passes. You do not need to consider the connectivity of the Data Service module in this scenario.

    What should I do if data sources are uploaded successfully in a batch but the entries do not appear in the list, or the page stops responding?

    If you upload data sources in a batch but the entries do not appear in the data source list, or the page stops responding, troubleshoot the issue as follows:

    1. Confirm that you clicked Start Creation after the upload, and then refresh the page.

    2. Check whether the uploaded table contains entries whose names duplicate existing data sources. Duplicate entries are skipped during creation, which can also cause the page to show no feedback. Remove the duplicate entries and upload the table again.

    3. Check the mode and environment field of the workspace. If the workspace is in Basic Mode, it supports only Production Environment. If the environment field in the uploaded table is set to dev, change it to Prod and upload the table again. Only in Standard Mode do you need to distinguish between Development Environment and Production Environment.

    4. Try uploading in the incognito or private browsing mode of your browser to rule out interference from cached data.

    Note

    The exact implementation of the Start Creation button and the duplicate-entry handling logic could not be located in the main console codebase reviewed for this FAQ. The preceding steps reflect the troubleshooting steps that have been confirmed to work.

    What should I do if I cannot find the target instance when I create a data source from an ApsaraDB RDS instance?

    If the instance that you want to use does not appear in the instance list when you create a data source from an ApsaraDB RDS instance, first confirm whether the RDS instance type is MySQL or PostgreSQL. Then, set the data source type to match the actual RDS instance type, and refresh the page to reload the full instance list.

    What should I do if DataWorks fails to connect to an external database, such as MongoDB Atlas, over the Internet and reports an UnknownHostException error?

    If a connectivity failure or an UnknownHostException error occurs when you use DataWorks to connect to an external database over the Internet, troubleshoot the issue as follows:

    1. Confirm that a NAT Gateway is configured for the VPC of the serverless resource group and that an Elastic IP Address (EIP) is bound to the NAT Gateway. Verify that this EIP can be accessed from the Internet, for example, by running a ping test against it.

    2. On the external database side (for example, MongoDB Atlas), check whether the EIP of the NAT Gateway has been added to the access allowlist, or whether the database is configured to allow access from all IP addresses.

    3. If the connection still fails after you complete the preceding checks, the target database usually resides in another cloud environment, such as MongoDB Atlas or Microsoft Azure. Internet connections between different cloud environments may be affected by cross-border network restrictions or instability. In this case, use IPsec VPN, Express Connect, or a leased line to establish an internal network connection between your Alibaba Cloud VPC and the target cloud environment. Then, configure the data source by using an internal network address to improve connectivity and stability.