Resource planning and configuration

Calling an API consumes resources from a resource group. To prevent issues such as failed access to data sources, API call failures due to insufficient CPU or memory, or rate limiting during high-frequency requests, ensure your resource group has proper network connectivity and sufficient performance. This topic describes key considerations for resource group planning and provides recommendations for using different types of resource groups.

Terms

A resource group is the computing resource used to run API calls in DataService Studio. It typically appears as a machine with specific amounts of CPU, memory, and bandwidth. The API call flow is as follows: a user sends an API request, which is first received by the API gateway, then forwarded to a DataService Studio server, and finally sent to the target data source to execute a query.

Resource group types

Resource groups fall into two categories: public resource groups and exclusive resource groups.

Exclusive resource group (recommended):
An exclusive resource group is dedicated to your use after purchase. For high-concurrency or high-frequency API calls, use a Serverless resource group (recommended) or an exclusive resource group for DataService Studio.
Note
Exclusive resource groups for DataService Studio are available only in China (Shanghai).
Public resource group (for testing only):
A public resource group is shared among all users and provided by DataWorks. During peak usage periods, users may compete for resources. For more information, see Use a public resource group.

Key aspects of resource planning: connectivity and performance

When using a resource group, focus on two aspects: connectivity and performance.

Connectivity:
When you send an API request, it is routed to a DataService Studio server and then forwarded to the data source to run a query. Ensure the resource group can access the target data source and its network. Use DataService Studio only after confirming network connectivity. Without it, API requests cannot execute.
Performance:
Running an API task consumes CPU, memory, and network resources from the machine hosting the resource group. Insufficient resources can cause API failures, rate limiting during high-frequency calls, or delayed query responses. Allocate enough resources to ensure smooth API execution. We recommend using an exclusive resource group to run API tasks in an isolated environment, free from interference by shared public resources. For performance metrics of exclusive resource groups, see Billing for exclusive resource groups for DataService Studio: subscription.

Resource group comparison and recommendations

Different scenarios call for different resource group types. The following table compares public and exclusive resource groups across dimensions such as ownership, network connectivity, billing, and performance. Choose the appropriate resource group when creating your API based on your needs.

Category	Exclusive resource group	Public resource group
Resource ownership	Maintained by DataWorks and exclusively used by each tenant.	Maintained by DataWorks and shared competitively among all tenants.
Network	Supports data sources in the public network, Alibaba Cloud VPC (instance mode and connection string mode), and IDC networks.	Supports data sources in the public network, Alibaba Cloud VPC (instance mode), and Alibaba Cloud classic network. Note Finance Cloud in Shenzhen does not support accessing classic network data sources using public resource groups.
Billing method	Subscription billing based on resource group specifications.	Pay-as-you-go billing based on call count and duration, with tiered pricing.
Supported data sources	ClickHouse, Hologres, RDS, MySQL, PostgreSQL, SQL Server, Oracle, Table Store, AnalyticDB for MySQL 2.0, AnalyticDB for MySQL 3.0, AnalyticDB for PostgreSQL, MongoDB, DRDS, StarRocks, Doris (continuously expanding)	Hologres, RDS, MySQL, PostgreSQL, SQL Server, Oracle, Table Store, AnalyticDB for MySQL 2.0, AnalyticDB for MySQL 3.0, AnalyticDB for PostgreSQL, MongoDB, DRDS
Maximum queries per second (QPS)^①	QPS thresholds vary by exclusive resource group specification: Serverless resource groups support a minimum of 125 QPS. Exclusive resource groups for DataService Studio support a minimum of 500 QPS. Select a specification that matches your required QPS level. Note A single exclusive resource group can be attached to multiple workspaces and APIs. If API traffic exceeds the QPS threshold for your resource group specification, the system enforces rate limiting and API calls fail.	Each tenant account supports up to 200 QPS per region. To increase this limit, switch to an exclusive resource group. If API traffic exceeds 200 QPS, the system enforces rate limiting and API calls fail.
Timeout limit	30s	10s
Reliability	High	Low
Security	High	High
Scenarios	High-concurrency, high-frequency online calls, complex query statements, and large data returns.	Low-concurrency, low-frequency calls.
Recommendation rating	★★★★★	★★★

Note

You can no longer purchase resource groups of the api.s1.small, api.s1.medium, or api.s1.large specifications. You can only upgrade, downgrade, or release existing resource groups of these types of specifications.
Note1: The maximum queries per second (QPS) for exclusive resource groups is calculated based on actual business scenarios. You can estimate the QPS threshold by using the following information:
- Whether to generate an API in script mode.
- Whether the pagination feature is enabled for an API call so that the returned results are displayed on multiple pages.
- The average runtime of SQL statements configured for an API call is 100 milliseconds in a data source.
- The average size of data returned by a single API call is 3,000 bytes.
If your business scenario is different from the preceding scenario, join the DataWorks DingTalk group to obtain the appropriate specifications that suit your business scenarios.

Based on the comparison above, we recommend using an exclusive resource group for API calls.

Resource group configuration guidance

When using a Serverless resource group, ensure proper network connectivity between the resource group and your selected data source. For details, see: Configure network connectivity.
When using an exclusive service resource group, select a network connectivity solution based on your data source’s network environment and configure a whitelist. See: Get whitelist IP addresses: exclusive resource group for DataService Studio.
When using a public resource group, add the corresponding region’s whitelist to your data source. For whitelist configuration, see Get whitelist IP addresses: public resource group.

Configure DataService Studio quota for a Serverless resource group

If your DataService Studio uses a Serverless resource group, configure a DataService Studio quota before selecting Exclusive Resource Group for DataService Studio in Configure service resource group.

Note

If you do not configure a DataService Studio quota for your Serverless resource group, you may encounter the following issues:

Network connectivity tests between the data source and DataService Studio resource group fail.
You cannot select the target resource group when configuring an exclusive service resource group.

Log on to the DataWorks console. In the target region, click Resource Group in the left-side navigation pane to open the resource group list.

Click the action icon next to your Serverless resource group and select Manage Quota . Assign either Occupied CUs (for pay-as-you-go resource groups) or Minimum CUs (for subscription resource groups) to Data Services.

Pay-as-you-go resource group	Subscription resource group
In the Quota Management dialog box, find the DataService Studio row. Click the edit icon in the Current CU Usage column and set the CU value to your desired amount (for example, `1`). After editing, a message appears: "Detected a change in DataService Studio CU guarantee. Click Confirm to reinitialize. Estimated wait time: 3 minutes." Click Confirm to complete the configuration.	In the Quota Management dialog box, the current specification is 4 CU. Set the CU Guarantee for DataService Studio to `4`, then click Confirm.

After assigning Current CU Usage, click Confirm to save.
Go to the DataService Studio page and reconfigure the exclusive service resource group.
Select the Serveless_Resource (pay-as-you-go) resource group.