Each fully managed Flink namespace has a default queue named default-queue to run deployments. If you want to isolate and manage resources, you can add queues. This topic describes how to add, edit, and delete a queue.
Scenarios and methods
You can configure queues for resource isolation and management. Queues help prevent a deployment or a user from occupying excessive resources and reduce execution failures caused by insufficient resources for other deployments or users. You can deploy batch drafts in a specific queue and use the Workflows feature to specify scheduling priorities of the batch drafts to ensure that critical or important deployments have higher priorities and more resources. This improves system performance and resource utilization.
You can use a queue by using the following methods:
Set the Deployment Target parameter to the queue that you want to use in the Deploy draft dialog box when you deploy a draft. For more information, see Create a deployment.
NoteIf you do not add a queue, the default queue named default-queue is used to run deployments.
Set the Deployment Target parameter to the queue that you want to use in the Standard section of the Create Session Cluster page when you create a session cluster. For more information, see Step 1: Create a session cluster.
Set the Resource Queue parameter to the queue that you want to use in the Create Workflow panel when you create a workflow. For more information, see Manage workflows.
Limits
Queues can run only in exclusive mode. Queues cannot run in preemption mode. If a queue runs in preemption mode, the resources in the queue cannot be preempted. After specific resources in the queue are released, the resources can be used by a new deployment.
Add a queue
You can add queues to isolate and manage resources. By default, the default queue named default-queue is used to run deployments if you do not add a queue.
Log on to the Realtime Compute for Apache Flink console.
In the left-side navigation pane, click Queue Management.
In the upper-left corner of the Queue Management page, click Add queue.
In the Add Queue dialog box, configure the Queue Name and Resources (CU) parameters. The following table describes the parameters.
Reconfigure the number of CUs that can be used in other queues. For more information, see Reconfigure the number of CUs used for a queue.
Increase resources in a namespace. For more information, see Reconfigure resources.
Click OK.
Parameter | Description |
Queue Name | The name of the queue. The queue name must be unique in a namespace. The name must be 1 to 32 characters in length and can contain only lowercase letters, digits, and hyphens (-). Note After a queue is added, the queue name cannot be modified. |
Resources (CU) | The number of compute units (CUs) that can be used in the queue. The maximum value of this parameter is determined based on the remaining available resources in the namespace. If the available resources in the namespace are insufficient, you can use one of the following methods to adjust the number of resources: |
Reconfigure the number of CUs that can be used in a queue
Log on to the Realtime Compute for Apache Flink console.
In the left-side navigation pane, click Queue Management.
Find the desired queue and click Edit queue in the Actions column. In the Edit Queue dialog box, change the value of the Resources (CU) parameter.
NoteFor more information about the Resources (CU) parameter, see Add a queue.
You cannot set the Resources (CU) parameter to a value that is less than the number of CUs that have been used in the queue.
Click OK.
Delete a queue
You cannot delete the default queue named default-queue.
A queue in which a deployment runs cannot be deleted.
If you delete a queue in which a deployment runs, an error is reported when you restart the deployment.
Log on to the Realtime Compute for Apache Flink console.
In the left-side navigation pane, click Queue Management.
Find the desired queue and click Delete in the Actions column.
In the message that appears, click OK.
References
For more information about how to reconfigure the resources of a namespace or change the upper limit of the quota for a namespace, see Reconfigure resources.
For more information about how to create a deployment of the SQL, JAR, or Python type, see Create a deployment.
For more information about how to create a session cluster, see Step 1: Create a session cluster.
For more information about the Workflows feature that is used to specify the scheduling priorities of batch deployments, see Workflows (public preview).