DataWorks introduces serverless resource groups, which combine the core functionalities of the previous exclusive resource groups for scheduling, Data Integration, and DataService Studio. Now, you can use a single serverless resource group to run all core operations such as data synchronization, periodic scheduling tasks, and API services, which greatly simplifies resource management. Serverless resource groups offer two billing methods to meet different needs:
Subscription: Provides stable and predictable exclusive computing resources. This billing method is ideal for production environments.
Pay-as-you-go: Provides on-demand and auto-scaling computing resources for flexibility and cost-effectiveness.
When you use a serverless resource group, task scheduling fees are generated for all tasks that are deployed to the production environment for periodic scheduling.
Billing scenarios
The fees for a DataWorks serverless resource group include resource usage fees and task scheduling fees.
Resource usage fees: Some DataWorks tasks consume compute units (CUs) of a serverless resource group when they run. The system charges you based on the total CUs consumed.
1 CU = 1 vCPU + 4 GiB of memory.Task scheduling fees: If you deploy a task to the production environment for periodic scheduling, the task relies on a serverless resource group. However, only task scheduling fees are generated, and no resource usage fees are generated. Task scheduling fees are charged based on the number of successfully run instances (excluding dry-run instances).
A serverless resource group supports a maximum of 200 concurrently running instances. This meets the maximum concurrency requirements of all specifications of earlier-version resource groups.
The following table describes the relationship between task types supported by serverless resource groups and the fees generated.
Task type | Description | Fee type |
Data Integration | Run a data synchronization task (such as an offline synchronization task) in the Data Integration or Data Studio module. | Resource usage fee |
Data Compute |
Important For information about data compute tasks, see Appendix 1: Task Types and CU Consumption. | |
DataService Studio | Call the Generate API operation in DataService Studio. | |
Individual development environment | Debug tasks in the individual development environment. | |
Task Scheduling | Run a periodically scheduled task in the production environment. | Task scheduling fee |
Notes
If you use a pay-as-you-go serverless resource group, resource preemption may occur during peak hours, and timely resource availability cannot be guaranteed.
When you use serverless resource groups, you cannot convert a subscription serverless resource group to a pay-as-you-go serverless resource group.
When a new user activates DataWorks, a pay-as-you-go serverless resource group is provisioned by default. You are not charged for the resource group if you do not use it. For more information about billing, see Pay-as-you-go resource group billing.
Performance metrics
Serverless resource groups are billed based on the number of CUs. In this context, 1 CU = 1 vCPU + 4 GiB memory. When you use a serverless resource group, plan the resource group specifications based on your development scenarios and task types.
Use these specifications as general guidelines. Adjust the resources based on your business requirements to ensure that tasks run efficiently and stably.
Data Integration
Offline synchronization
Concurrency configuration for offline synchronization tasks | Recommended specifications | Minimum specifications for running tasks |
<4 | 0.5 CU | 0.5 CU |
>=4 |
|
Real-time synchronization
Synchronization task type | Recommended specifications | Minimum specifications for running tasks | |
Real-time synchronization for MySQL | 1 database | 2 CUs | Minimum specifications for running a real-time synchronization task: 1 CU |
2 to 5 databases | 2 CUs | ||
6 or more databases | 2 CUs | ||
Real-time synchronization for Kafka | 1 CU | ||
Other types of single-table real-time tasks | 1 CU | ||
Real-time synchronization for an entire database | - | Minimum specifications for running an entire-database synchronization task: 2 CUs | |
Data compute
Each data compute task has a default number of CUs. For more information, see Task types and CU consumption.
Task scheduling
A Serverless resource group supports a maximum of 200 concurrently running instances. The concurrency limit is independent of the CU configuration. The default number of concurrently running instances is 50. You can set the maximum number of concurrent scheduling tasks to 200 on the resource group details page.
DataService Studio
Maximum queries per second (QPS) | Minimum specifications | Service level agreement (SLA) |
500 | 4 CUs | 99.95% |
1000 | 8 CUs | |
2000 | 16 CUs |
Individual development environment
CPU-based individual development environments provide resource quotas ranging from 2 to 100 CUs. GPU-based individual development environments provide resource quotas ranging from 21 to 60 CUs. Select a resource quota based on the task type:
Lightweight tasks (such as simple SQL queries and Python script debugging): A low resource quota is recommended, for example, 2 CUs.
Moderately complex tasks (such as data processing and Notebook analysis): A medium resource quota is recommended, for example, 4 CUs.
Deep learning tasks (such as TensorFlow and PyTorch model training): GPU resources are recommended. Select an appropriate video memory size and number of CUs based on the model size.
Billing methods
Serverless resource groups are available in subscription and pay-as-you-go billing methods.
Subscription: With the subscription plan, you commit to a specific number of CUs for a set duration and pay upfront. This fee covers all resource usage for data synchronization, data computing, and API calls that run on the resource group.
Pay-as-you-go: Use the features and then pay for the total amount of CUs that you use. If you use a pay-as-you-go Serverless resource group to run specific tasks, such as offline synchronization tasks, DataService Studio tasks, and Data Studio tasks, resource usage fees are generated.
The following table compares the features of the billing methods.
Item | Pay-as-you-go | Subscription |
Total CUs available in the resource group | Calculated based on actual usage. | The number of CUs specified at the time of purchase. |
Scaling (up or down) and renewal | Not applicable | Supported |
Quota management | Used to control the maximum number of CUs that can be used in different scenarios. Supported for Data Compute, Data Integration, and DataService Studio. | |
Set maximum task scheduling concurrency | Supported. A maximum of 200 task instances can run concurrently. | |
Number of bound VPCs |
| The number of VPCs you can bind depends on the number of CUs purchased.
|
Billing standards
Billing of subscription resource groups
Fees are based on CU usage. Fee = Monthly unit price × Number of months × Number of CUs purchased per month.
For the subscription billing method, a minimum of 2 CUs can be purchased each month. There is no upper limit on the purchase specification, but it may be affected by inventory. If the inventory is insufficient, refer to the notifications on the purchase page.
If the specifications do not meet your requirements after purchase, you can scale up at any time. For more information, see Using serverless resource groups.
For the minimum resource specifications required for different types of tasks when using a serverless resource group, see Performance metrics.
Region | Monthly unit price (USD/Month/CU) |
China (Shanghai), China (Hangzhou), China (Beijing), China (Shenzhen) | 37.1517 |
UK (London) | 51.01286 |
US (Virginia) | 53.92014 |
Malaysia (Kuala Lumpur) | 63.36534 |
China (Hong Kong), Singapore, Germany (Frankfurt), Indonesia (Jakarta) | 67.61327 |
US (Silicon Valley) | 72.74794 |
Japan (Tokyo) | 77.45584 |
Billing of pay-as-you-go resource groups
Fees are charged based on the formula: Fee = CU-hours × CU unit price. Bills are generated on an hourly basis.
In the resource group quota management, if you configure 1 CU for DataService Studio, the CU consumption continues regardless of whether the DataService Studio feature is used, until you adjust the CU quota occupied by DataService Studio to 0.
Region | Unit price (USD/CU-hour) | Example |
China (Shanghai), China (Hangzhou), China (Beijing), China (Shenzhen) | 0.077399 | Example: A data synchronization task in the China (Shanghai) region is configured with 2 CUs and runs successfully after 0.5 hours. The unit price of a CU in the China (Shanghai) region is USD 0.077399/CU-hour. The CU-hours and fees for the task are calculated as follows:
|
UK (London) | 0.106277 | |
US (Virginia) | 0.112334 | |
Malaysia (Kuala Lumpur) | 0.132011 | |
Germany (Frankfurt), Indonesia (Jakarta), China (Hong Kong), Singapore | 0.140861 | |
US (Silicon Valley) | 0.151558 | |
Japan (Tokyo) | 0.161366 |
View bill details
When you view bill details, the billable items and codes for serverless resource groups are as follows:
Pay-as-you-go: The billable item is
General Resource Group CU*H (Pay-as-you-go), and the code isexresource_cu_hour_post.Subscription: The billable item is
General Exclusive Resource Group (Subscription and Pay-as-you-go), and the code iscu_number.
For more information, see View bill details.
Expiration and renewal
If a subscription serverless resource group is about to expire, renew it. If you do not renew the resource group, the service for the resource group is suspended or the resource group is released. For more information about renewal, see Expiration and renewal.
Next steps
Purchase a resource group and use it for tasks in Data Integration, Data Studio, and DataService Studio. For information about how to purchase a resource group, bind a resource group to a workspace, and establish network connectivity for a resource group, see Use serverless resource groups.
More information
Appendix 1: Task types and CU consumption
DataWorks tasks are categorized into data compute tasks (which consume CUs) and scheduling tasks (which do not consume CUs).
Determine the task type
Go to the node editing page in Data Studio and in the right-side navigation pane, choose to view the task type.
Compute task: In the Scheduling Policies section, specify the CUs required for the task.
Scenario 1: You can customize the number of CUs.
Scenario 2: Use the default CU quota.
Task scheduling: In the Scheduling Policies section, you can only select a resource group for scheduling. You do not need to configure CUs for the task.
CU configuration list for compute tasks
Running data compute tasks by using a serverless resource group consumes CUs. The default CUs and running CUs are described as follows:
Default CUs: The recommended number of CUs allocated by the platform for each task run based on the task type. f you configure a lower value, efficient task execution cannot be guaranteed.
Running CUs: The number of CUs actually configured for running a task. By default, this is set to the default CU quota, which you can adjust as needed. Follow these principles for configuration:
The minimum value is 0.25 CU, configurable in increments of 0.25 CU. If the interface prompts The CU quota for the current resource group is insufficient, adjust the CU quota for the data compute task.
To avoid insufficient or excessive resource configuration, refer to the default CU quota and the CU quota of the data compute task for proper configuration. For more information, see Allocate CU quotas to tasks.
Only some tasks support adjusting running CUs. For example:
The running CUs for a Hologres SQL task cannot be adjusted and can only be configured to 0.25 (the default CU value).
The default running CUs for a PyODPS 2 task is 0.5. You can adjust it as needed (for example, to 0.4 or 0.6).
Node type | Node name | Default CUs (Unit: CU) | Can running CUs be modified? |
Notebook | 0.5 | Yes | |
MaxCompute | 0.5 | Yes | |
0.5 | Yes | ||
0.5 | Yes | ||
0.25 | Yes | ||
0.25 | Yes | ||
Hologres | 0.25 | - | |
0.25 | - | ||
0.25 | Yes | ||
0.25 | Yes | ||
EMR | 0.25 | - | |
0.25 | - | ||
0.25 | Yes | ||
0.25 | - | ||
0.25 | Yes | ||
0.5 | Yes | ||
0.5 | Yes | ||
0.5 | Yes | ||
0.25 | - | ||
0.25 | - | ||
ADB | 0.25 | Yes | |
0.25 | Yes | ||
0.25 | - | ||
0.25 | - | ||
CDH | 0.25 | - | |
0.5 | Yes | ||
0.25 | - | ||
0.25 | - | ||
0.25 | - | ||
0.25 | - | ||
Lindorm | 0.25 | - | |
0.25 | - | ||
ClickHouse | 0.25 | - | |
Data Quality | 0.25 | - | |
0.5 | Yes | ||
General | 0.25 | Yes | |
0.25 | Yes | ||
0.25 | - | ||
0.5 | Yes | ||
0.25 | Yes | ||
0.25 | Yes | ||
0.25 | - | ||
0.25 | - | ||
Data push node | 0.25 | - | |
MySQL node | 0.25 | - | |
SQL Server | |||
Oracle node | |||
PostgreSQL node | |||
StarRocks node | |||
DRDS node | |||
PolarDB MySQL node | |||
PolarDB PostgreSQL node | |||
Doris node | |||
MariaDB node | |||
SelectDB node | |||
Redshift node | |||
Saphana node | |||
Vertica node | |||
DM node | |||
KingbaseES node | |||
OceanBase node | |||
DB2 node | |||
GBase 8a node | |||
Algorithm | 0.25 | - |
Configurations for scheduling tasks
Scheduling tasks do not consume CUs from the serverless resource group.
Node type | Node name |
Data Integration | |
MaxCompute | |
Flink | |
General | |
Algorithm |
Appendix 2: Billing models for task execution
When you run a node task in DataWorks, the computing fees for the task may not be charged by DataWorks. You need to identify the compute engine or resource on which the task is ultimately executed. Three scenarios exist:
When a task is published to the production environment for periodic scheduling, task scheduling fees are generated.
Execution method | Representative task nodes | Compute resource provider | Fee composition |
Method 1: Compute tasks executed directly by a serverless resource group | PyODPS, Shell, Data Integration, Data Quality | Serverless resource group | Serverless resource group fees only |
Method 2: Compute tasks submitted to a third-party engine via a serverless resource group | EMR Hive, Hologres SQL | Serverless resource group + third-party engine | Serverless resource group fees + third-party engine fees |
Method 3: Scheduling tasks submitted to a third-party engine | MaxCompute SQL, Flink SQL | Third-party engine | Third-party engine fees |
Appendix 3: Fee breakdown for specific modules
When using a serverless resource group in the following modules, the specific fees for the serverless resource group are generated as follows:
Data Integration: When synchronizing data, data integration tasks run in the Data Integration, Data Studio, and Operation Center modules. This consumes serverless resources, incurring data integration fees.
Data Studio: When using Data Studio for task development, data compute tasks and scheduling tasks run in Data Studio, Data Quality, and Operation Center modules. This consumes serverless resources, incurring data compute fees and task scheduling fees. When using the individual development environment, individual development environment fees are also generated.
DataAnalysis: When using DataAnalysis for SQL query analysis and downloading query results, data compute tasks run in the DataAnalysis module. This consumes serverless resources, incurring data compute fees. When using data exploration, task scheduling fees are also generated.
DataService Studio: When using DataService Studio to generate APIs, configure the CUs occupied by DataService Studio through resource group quota management. This consumes serverless resources, incurring DataService Studio fees. When using data push, task scheduling fees are also generated.