Create and manage a resource group - AnalyticDB - Alibaba Cloud Documentation Center

AnalyticDB for MySQL uses resource groups to isolate computing resources for batch processing and interactive queries. You can configure scheduled or automatic scaling of resources and determine the priority for executing queries. This topic describes how to create and manage a resource group.

Prerequisites

An AnalyticDB for MySQL Data Lakehouse Edition cluster is created.

Billing rules

You are charged AnalyticDB compute unit (ACU) elastic resource fees for the elastic resources used by interactive resource groups and job resource groups.

Enterprise Edition and Basic Edition: In the left-side navigation pane, choose Cluster Management > Resource Management. On the Resource Overview tab, you can view the total resources and reserved resources of all resource groups. The amount of elastic resources can be calculated by using the following formula: Amount of total resources - Amount of reserved resources.
Data Lakehouse Edition: In the left-side navigation pane, choose Cluster Management > Resource Management. On the Resource Overview tab, you can view the total computing resources and reserved computing resources of all resource groups. The amount of elastic resources can be calculated by using the following formula: Amount of total computing resources - Amount of reserved computing resources.

Create a resource group

Each cluster has a default interactive resource group named user_default. New AnalyticDB for MySQL clusters of V3.2.2.8 or later also provide an additional default job resource group named serverless. If no other resource groups exist, all XIHE queries are executed by the user_default resource group, and all Spark jobs, such as Spark JAR and Spark SQL jobs, are executed by the serverless resource group. If you have resource isolation requirements among queries, you can create additional resource groups.

Log on to the AnalyticDB for MySQL console. In the upper-left corner of the console, select a region. In the left-side navigation pane, click Clusters. On the Clusters page, click an edition tab. Find the cluster that you want to manage and click the cluster ID.
In the left-side navigation pane, choose Cluster Management > Resource Management. On the Resource Management page, click the Resource Groups tab. In the upper-right corner of the Resource Groups tab, click Create Resource Group.
In the Create Resource Group panel, enter a resource group name and select an option from the Job Type drop-down list.
- If you want to perform real-time analysis with high queries per second (QPS) and low response time (RT), select Interactive.
  An interactive resource group uses resident computing resources to execute queries in massively parallel processing (MPP) mode. The query response is fast, within milliseconds.
- If you want to perform batch processing with high throughput, select Job.
  A job resource group pulls up temporary computing resources to execute queries in bulk synchronous parallel (BSP) mode. The query response is slow, within seconds or minutes. The amount of temporary computing resources that can be pulled up varies based on the size of the job that you run, ranging from 0 ACUs to the maximum resource amount of the job resource group.
Important
After you create a resource group, you cannot change the job type of the resource group.

Configure the property parameters of the resource group based on the job type that you selected. Then, click OK.

Properties of an interactive resource group

Parameter	Description
Engine	XIHE: The resource group supports only XIHE SQL jobs. Spark: The resource group supports only Spark SQL jobs. Spark SQL jobs are run in an interactive manner. Important After you create a resource group, you cannot change the engine of the resource group.
Automatic Stop	Specifies whether to enable the automatic stop feature for the resource group. If an interactive resource group remains idle for a specific period of time after all SQL statements are executed, the system uses the automatic stop feature to automatically release the compute clusters that are started in the resource group. The automatic stop feature prevents resource waste and reduces resource costs. However, if you execute queries after the started compute clusters are released, resources need to be re-pulled up and latency occurs. Important This parameter is available only if you set the Engine parameter to Spark.
Cluster Size	The size of a single compute cluster when the Engine parameter is set to XIHE. Unit: AnalyticDB compute units (ACUs). The minimum value is 16 ACUs. The size of a single compute cluster when the Engine parameter is set to Spark. Set this parameter to the number of ACUs that you want to allocate to each Spark application. The minimum value is 24 ACUs. You can run multiple Spark applications in each Spark interactive resource group. You can specify the Minimum Clusters and Maximum Clusters parameters to define the range of Spark applications that can run in the resource group. For more information, see the "Mappings between the compute cluster size and Spark driver and executor specifications" section of the Multi-cluster scaling models topic.
Minimum Clusters Maximum Clusters	Minimum Clusters: the minimum number of compute clusters that you must run in the resource group. The minimum value is 1. Maximum Clusters: the maximum number of compute clusters that you can run in the resource group. The maximum value is 10. If you specify different values for the Minimum Clusters and Maximum Clusters parameters, AnalyticDB for MySQL dynamically scales the number of compute clusters available for the resource group within the range of the minimum and maximum values based on query loads. If you specify the same value for the Minimum Clusters and Maximum Clusters parameters, AnalyticDB for MySQL starts the specified compute clusters after the resource group is created to statically maintain the total amount of computing resources available for the resource group. Note If you set the Minimum Clusters or Maximum Clusters parameter to a value that is greater than or equal to 2, the multi-cluster feature is enabled for the resource group. For information about the multi-cluster feature, see Multi-cluster scaling models.
Job Resubmission Rules	The rules that resubmit the queries executed for a period of time longer than the specified execution duration threshold to the destination resource group. For more information, see Job resubmission. Important This parameter is available only if you set the Engine parameter to XIHE.
Spark Configuration	The Spark application configuration parameters that can be applied to all Spark jobs executed in the resource group. If you want to configure parameters for a specific Spark job, you can specify values for the parameters in the code editor when you submit the job. For information about the Spark configuration parameters, see Spark application configuration parameters. Important This parameter is available only if you set the Engine parameter to Spark.

Properties of a job resource group

Parameter	Description
Minimum Computing Resources	The minimum amount of computing resources. Set the value to 0 ACUs. Important After you create a job resource group, you cannot change the minimum computing resources.
Maximum Computing Resources	The maximum amount of computing resources. Set this parameter within the range of 8 ACUs to 1,024 ACUs in increments of 8 ACUs. If you want to change the maximum value of the Maximum Computing Resources parameter, submit a ticket.
Spot Instance	Specifies whether to enable the spot instance feature for the job resource group. After you enable the spot instance feature for the job resource group, Spark jobs that run in the job resource group attempt to use the spot instance resources. For more information, see Spot instances.
Spark Configuration	The Spark application configuration parameters that can be applied to all Spark jobs executed in the resource group. If you want to configure parameters for a specific Spark job, you can specify values for the parameters in the code editor when you submit the job. For information about the Spark configuration parameters, see Spark application configuration parameters.

Modify a resource group

Properties that can be modified

You can modify the following properties for custom resource groups:
- Interactive resource group: Automatic Stop, Cluster Size, Minimum Clusters, Maximum Clusters, Job Resubmission Rules, and Spark Configuration.
- Job resource group: Maximum Computing Resources, Spot Instance, and Spark Configuration.
You cannot modify other properties, including Resource Group Name, Job Type, Engine of interactive resource groups, and Minimum Computing Resources of job resource groups.
For the default resource groups named user_default and serverless:
- In Enterprise Edition and Basic Edition, you can modify only the Job Resubmission Rules property for the user_default resource group and cannot modify properties for the serverless resource group.
- In Lakehouse Edition, you can modify the Reserved Computing Resources and Job Resubmission Rules for the user_default resource group and cannot modify properties for the serverless resource group.

Procedure

On the Resource Groups tab, find the resource group that you want to modify and click Modify in the Actions column.
In the Modify Resource Group panel, modify the parameter values and click OK.
When the modification takes effect, the status of the resource group changes to Running.

Delete a resource group

You can delete custom resource groups, but you cannot delete the default resource groups named user_default and serverless.

Impacts

If you delete a resource group, jobs that are running in the resource group are interrupted.
After you delete the resource group that is used to run a XIHE SQL job or a Spark job, specify a different resource group for the job. If you do not specify a different resource group, the default resource group is used to run the XIHE SQL job, or an error occurs on the Spark job.

Procedure

On the Resource Groups tab, find the resource group that you want to delete and click Delete in the Actions column. In the Delete Resource Group message, click OK.

Monitor the resource usage

You can view the resource usage at the cluster level, the resources and workloads at the resource group level, and the resource consumption at the job level. For more information about the resource usage metrics, see the "Resource group monitoring" section of the Metric overview topic.

View the reserved and elastic resources of a cluster

Enterprise Edition and Basic Edition: In the left-side navigation pane, choose Cluster Management > Resource Management. On the Resource Overview tab, you can view the total resources and reserved resources of all resource groups in the cluster at a specific point in time. The amount of elastic resources can be calculated by using the following formula: Amount of total resources - Amount of reserved resources.
Data Lakehouse Edition: In the left-side navigation pane, choose Cluster Management > Resource Management. On the Resource Overview tab, you can view the total computing resources and reserved computing resources of all resource groups in the cluster at a specific point in time. The amount of elastic resources can be calculated by using the following formula: Amount of total computing resources - Amount of reserved computing resources.

View the resources and workloads of a resource group

You can view the computing resources that are actually used by a resource group. You can also view the following metrics related to the workloads of a resource group: the number of running XIHE SQL statements, the number of queuing XIHE SQL statements, the number of Spark engines, and the number of connections.

In the left-side navigation pane of an AnalyticDB for MySQL cluster, choose Cluster Management > Resource Management. On the Resource Management page, click the Resource Groups tab. On the Resource Groups tab, find the resource group that you want to manage and click Monitoring in the Actions column to view the computing resources that are actually used by the resource group.

View the resources consumed by a job

The Job Usage Statistics tab provides statistical information about the resource consumption of the following jobs: XIHE BSP jobs, Spark jobs, and data synchronization and migration jobs from Simple Log Service or ApsaraMQ for Kafka in the AnalyticDB for MySQL console.

In the left-side navigation pane of an AnalyticDB for MySQL cluster, choose Cluster Management > Resource Management. On the Resource Management page, click the Job Usage Statistics tab. On the Job Usage Statistics tab, view the total resources, reserved resources, elastic resources, and spot instance resources that are consumed by each job.

Associate with or disassociate from a database account

If you want to isolate resources among queries that are initiated by different database accounts, you can associate different resource groups with the database accounts. After you associate a resource group with a database account, the system routes the SQL queries to the associated resource group if you do not specify a resource group for SQL queries initiated by the database account.

Association rules

You can associate each database account with only one resource group. You can associate each resource group with one or more database accounts.
If a database account is not associated with a custom resource group, the database account is associated with the default resource group.
You cannot associate the user_default or serverless resource group with a database account.
When you delete a resource group or a database account, the associated relationship is dissolved.

Procedure

Associate with or disassociate from a database account in the AnalyticDB for MySQL console

In the left-side navigation pane of an AnalyticDB for MySQL cluster, choose Cluster Management > Resource Management. On the Resource Management page, click the Resource Groups tab. On the Resource Groups tab, find the resource group that you want to manage and click Associate with Account in the Actions column.
In the Associate with Account panel, select a database account from the Database Account drop-down list and click Associate with Account. In the Associate with Account message, click OK.
You can associate a resource group with only a single database account at a time. To associate the resource group with multiple database accounts, repeat Step 2.
If no database accounts are created, the drop-down list is empty. In this case, create a database account first. For more information, see Create a database account.
To disassociate the resource group from a database account, find the database account and click Disassociate in the Actions column in the Associate with Account panel. In the Disassociate from Database Account message, click OK.

Call API operations to associate with or disassociate from a database account

FAQ

An AnalyticDB for MySQL cluster has 32 ACUs of reserved resources, and you create a job resource group for the cluster. Does the job resource group consume 32 ACUs of reserved resources together with the default resource groups?

If you use an AnalyticDB for MySQL Enterprise Edition or Basic Edition cluster, the reserved resources of the cluster can be allocated only to the default resource group named user_default. The default resource group named serverless and the job or interactive resource groups that you created can consume only elastic resources.

If you use an AnalyticDB for MySQL Data Lakehouse Edition cluster, the reserved resources of the cluster can be allocated to the default resource group named user_default, the default resource group named serverless, and the job or interactive resource groups that you created. The minimum or maximum computing resources of the default resource group are the reserved resources allocated to the default resource group named user_default. The difference between the reserved resources and the minimum or maximum resources of the default resource group named user_default represents the resources that can be allocated to the default resource group named serverless and the job or interactive resource groups that you created.

References

For information about the functionalities of resource groups and how to specify a resource group to execute queries, see Resource group management (Data Lakehouse Edition).
For information about how to create, modify, and delete resource groups and associate resource groups with database accounts by calling API operations, see Resource group management.