When you use DataService Studio to make an API call, the call consumes resources in a resource group. You must ensure that the resource group has sufficient network connectivity and performance. This helps prevent issues, such as the resource group failing to access a data source, API call exceptions, or the throttling of frequent API calls due to insufficient CPU or memory resources. This topic describes what to consider when you plan resource groups and provides suggestions for using different types of resource groups.
Terms
A resource group provides computing resources for API calls in DataService Studio. A resource group is a set of servers with specific CPU, memory, and bandwidth resources. When a user makes an API call, API Gateway receives the request. The request is then forwarded to a DataService Studio server. Finally, the server forwards the request to the destination data source to execute a data query.
Types of resource groups
Resource groups are classified into public resource groups and exclusive resource groups.
Exclusive resource group (Recommended)
An exclusive resource group is a resource group that you purchase for your exclusive use. To ensure high concurrency and high-frequency API calls, you can use a Serverless resource group (Recommended) or an exclusive resource group for DataService Studio.
NoteExclusive resource groups for DataService Studio are available only in the China (Shanghai) region.
Public resource group (For testing only)
A public resource group is provided by DataWorks and shared by all users. During peak hours, users may compete for resources. For more information about public resource groups, see Use a public resource group.
Key to resource planning: connectivity and performance
When you use resource groups, consider their connectivity and performance.
Connectivity
When you make an API call, the request is sent to a DataService Studio server. The server then forwards the request to the destination data source to execute a data query. Therefore, you must ensure that the resource group can access the data source over the network before you use DataService Studio. If the network is disconnected, API calls will fail.
Performance
API calls consume CPU, memory, and network resources on the servers in the resource group. Insufficient resources can cause API call exceptions, throttling of frequent API calls, or delayed query results. You must allocate sufficient resources to ensure that API calls run smoothly. Use exclusive resource groups to run API calls in an isolated environment. This prevents competition for resources in the public resource pool. For more information about the performance metrics of exclusive resource groups, see Subscription billing for exclusive resource groups for DataService Studio.
Comparison and recommendations
Different types of resource groups are suitable for different scenarios. The following table compares them based on resource ownership, network connectivity, billing methods, and performance. When you create an API, select a resource group that meets your needs.
Item | Exclusive resource group | Public resource group |
Ownership of resources | The resources are maintained by DataWorks and exclusively used by each tenant. | The resources are maintained by DataWorks and shared among all tenants. |
Network | This type of resource group can connect to data sources that are deployed over the Internet, in Alibaba Cloud VPCs (instance and connection string modes), and in data centers. | This type of resource group can connect to data sources that are deployed over the Internet, in Alibaba Cloud VPCs (instance mode), and in the classic network. Note You cannot use a public resource group to access a data source that is deployed in the classic network in the China (Shenzhen) Finance Cloud. |
Billing method | This type of resource group is charged by resource group specifications based on the subscription billing method. | This type of resource group is charged by the number of calls and the call duration based on different billing tiers. |
Supported data sources | ClickHouse, Hologres, RDS, MySQL, PostgreSQL, SQL Server, Oracle, Tablestore, AnalyticDB for MySQL 2.0, AnalyticDB for MySQL 3.0, AnalyticDB for PostgreSQL, MongoDB, DRDS, StarRocks, and Doris. More data sources will be supported in the future. | Hologres, RDS, MySQL, PostgreSQL, SQL Server, Oracle, Tablestore, AnalyticDB for MySQL 2.0, AnalyticDB for MySQL 3.0, AnalyticDB for PostgreSQL, MongoDB, and DRDS |
Maximum queries per second (QPS)¹ | The QPS thresholds vary based on the specifications of exclusive resource groups:
You can select resource groups of different specifications to meet different QPS requirements. Note One exclusive resource group can be attached to multiple workspaces and APIs. If the number of API calls exceeds the QPS threshold of a resource group, throttling is triggered and the API calls fail. | A maximum of 200 QPS is supported for each tenant in each region. To increase the QPS threshold, switch to an exclusive resource group. If the number of API calls exceeds 200 QPS, throttling is triggered and the API calls fail. |
Timeout limit | 30s | 10s |
Reliability | High | Low |
Security | High | High |
Scenarios | This type of resource group is used for highly concurrent and frequent online API calls in which complex query statements are used and a large volume of data needs to be returned. | This type of resource group is used for low-concurrency or low-frequency API calls. |
Recommendation rating | ★★★★★ | ★★★ |
Note1: The maximum queries per second (QPS) for exclusive resource groups is calculated based on actual business scenarios. You can estimate the QPS threshold by using the following information:
Whether to generate an API in script mode.
Whether the pagination feature is enabled for an API call so that the returned results are displayed on multiple pages.
The average runtime of SQL statements configured for an API call is 100 milliseconds in a data source.
The average size of data returned by a single API call is 3,000 bytes.
If your business scenario is different from the preceding scenario, join the DataWorks DingTalk group to obtain the appropriate specifications that suit your business scenarios.
Based on the preceding comparison, we recommend that you use exclusive resource groups for API calls.
Resource group configuration guide
If you use a Serverless resource group, ensure that the resource group can connect to the selected data source. For more information, see Network connectivity configuration.
If you use an exclusive resource group for DataService Studio, select a network connectivity solution based on the network environment of the data source and configure a whitelist. For more information, see Obtain IP addresses for a whitelist: Exclusive resource groups for DataService Studio.
If you use a public resource group, add the CIDR blocks of the corresponding region to the whitelist of the data source. For more information about how to configure the whitelist, see Obtain IP addresses for a whitelist: Public resource groups.
Configure a DataService Studio quota for a Serverless resource group
If you use a Serverless resource group for DataService Studio, you must configure a quota for DataService Studio on the Serverless resource group before you set the resource group to Exclusive Resource Group for DataService Studio.
If you do not configure a quota for DataService Studio on the Serverless resource group, the following issues may occur:
The network connection fails when you test the connectivity between the data source and the DataService Studio resource group.
You cannot select the target resource group when you configure an exclusive resource group for DataService Studio.
Go to the Resource Groups page.
Log on to the DataWorks console. In the top navigation bar, select the desired region. In the left-side navigation pane, click Resource Group to go to the Resource Groups page.
In the Actions column of the Serverless resource group, click the
icon and select Manage Quota. To allocate resources to DataService Studio, set Occupied CUs for a pay-as-you-go resource group or Minimum CUs for a subscription resource group.Pay-as-you-go resource group
Subscription resource group
After you allocate the CUs, click OK to save the configuration.
Go to the DataService Studio page and configure the exclusive resource group again.
