Container Service for Kubernetes (ACK) allows you to configure priority-based resource scheduling. A ResourcePolicy specifies the priorities of nodes in descending order for pod scheduling. When the system deploys or scales out pods for an application, pods are scheduled to nodes based on the priorities of the nodes that are listed in the ResourcePolicy. When the system scales in pods for an application, pods are removed from nodes based on the priorities of the nodes in ascending order. This topic describes how to configure priority-based resource scheduling.
Prerequisites
- An ACK Pro cluster that runs Kubernetes 1.20.11 or later is created. For more information about how to update the Kubernetes version of your cluster, see Update the Kubernetes version of an ACK cluster.
- Specific versions of the scheduler are required based on the Kubernetes version of the cluster. The following table describes the scheduler versions that are required for each Kubernetes version.
Kubernetes version Scheduler version 1.20 1.20.4-ack-7.0 or later 1.22 1.22.15-ack-2.0 or later 1.24 v1.24.3-ack-2.0 or later - ack-virtual-node is deployed if you want to use elastic container instances. For more information, see Use Elastic Container Instance in ACK clusters.
Limits
- Priority-based resource scheduling is mutually exclusive with the pod deletion cost feature. For more information about the pod deletion cost feature, see Pod deletion cost.
- You cannot use priority-based resource scheduling and Elastic Container Instance-based scheduling at the same time. For more information, see Use Elastic Container Instance-based scheduling.
- This feature uses the BestEffort policy and does not ensure that pods are removed from nodes based on the priorities of the nodes in ascending order when the system scales in pods for an application.
- Before you create a ResourcePolicy, make sure that no pod is scheduled. Pods that are scheduled before you create a ResourcePolicy are deleted first in scale-in activities.
- After you modify a ResourcePolicy, only the pods that are scheduled based on the modified ResourcePolicy are deleted based on the modified ResourcePolicy. The pods that are scheduled based on the original ResourcePolicy are deleted based on the original ResourcePolicy.
How to configure priority-based resource scheduling
Create a ResourcePolicy with the following template:
apiVersion: scheduling.alibabacloud.com/v1alpha1
kind: ResourcePolicy
metadata:
name: xxx
namespace: xxx
spec:
selector:
key1: value1
strategy: prefer
units:
- resource: ecs
nodeSelector:
key2: value2
- resource: ecs
nodeSelector:
key3: value3
- resource: eci
selector
: the selector that is used to select pods in a namespace. The ResourcePolicy is applied to the selected pods. In this example, pods that have thekey1=value1
label
are selected.strategy
: the scheduling policy. Set the value toprefer
.units
: the schedulable units. In a scale-out activity, pods are scheduled to nodes based on the priorities of the nodes that are listed underunits
in descending order. In a scale-in activity, pods are deleted from the nodes based on the priorities of the nodes in ascending order.resource
: the type of resource for pod scheduling. Valid values:eci
andecs
.nodeSelector
: the selector that is used to select nodes with a specifiednode
label
. This parameter takes effect only if the resource parameter is set toecs
.
Examples
Example 1: Schedule pods to specified node poolsYou want to schedule the pods of a Deployment to specific node pools, for example, Node Pool A and Node Pool B. You want to prioritize the use of Node Pool A and schedule pods to Node Pool B only if the computing resources of Node Pool A are insufficient. In scale-in activities, you want to delete pods from the nodes in Node Pool B first. In this example, Node Pool A contains the following nodes: cn-beijing.10.0.3.137
and cn-beijing.10.0.3.138
. Node Pool B contains the following nodes: cn-beijing.10.0.6.47
and cn-beijing.10.0.6.46
. Each of these nodes has two vCPUs and 4 GB of memory. Perform the following steps to configure priority-based resource scheduling:
- Create a ResourcePolicy with the following template:
apiVersion: scheduling.alibabacloud.com/v1alpha1 kind: ResourcePolicy metadata: name: nginx namespace: default spec: selector: app: nginx # You must specify the label of the pods to which you want to apply the ResourcePolicy. strategy: prefer units: - resource: ecs nodeSelector: alibabacloud.com/nodepool-id: np7ec79f2235954e879de07b780058**** - resource: ecs nodeSelector: alibabacloud.com/nodepool-id: npab2df797738644e3a7b7cbf532bb****
Note To view the ID of a node pool, choose Nodes > Node Pools on the details page of a cluster in the ACK console. For more information, see Manage node pools. - Use the following template to create a Deployment that provisions two pods:
apiVersion: apps/v1 kind: Deployment metadata: name: nginx labels: app: nginx spec: replicas: 2 selector: matchLabels: app: nginx template: metadata: name: nginx labels: app: nginx # The pod label must be the same as the one that you specified for the selector in the ResourcePolicy. spec: containers: - name: nginx image: nginx resources: limits: cpu: 2 requests: cpu: 2
- Deploy an NGINX application and query the pods.
- Scale out pods for the NGINX application.
- Scale in pods for the NGINX application.
You want to schedule the pods of a Deployment to multiple types of resources, such as subscription Elastic Compute Service (ECS) instances, pay-as-you-go ECS instances, and elastic container instances. To reduce the resource cost, you want to schedule pods to resources based on the following priorities: subscription ECS instances > pay-as-you-go ECS instances > elastic container instances. In scale-in activities, you want to delete pods from these resources based on the following sequence: elastic container instances, pay-as-you-go ECS instances, and subscription ECS instances. In this example, each of the ECS instances has two vCPUs and 4 GB of memory. Perform the following steps to configure priority-based resource scheduling:
- Run the following command to add
labels
that indicate different billing methods to the nodes. You can also use node pools to automatically addlabels
to the nodes.kubectl label node cn-beijing.10.0.3.137 paidtype=subscription kubectl label node cn-beijing.10.0.3.138 paidtype=subscription kubectl label node cn-beijing.10.0.6.46 paidtype=pay-as-you-go kubectl label node cn-beijing.10.0.6.47 paidtype=pay-as-you-go
- Create a ResourcePolicy with the following template:
apiVersion: scheduling.alibabacloud.com/v1alpha1 kind: ResourcePolicy metadata: name: nginx namespace: default spec: selector: app: nginx # You must specify the label of the pods to which you want to apply the ResourcePolicy. strategy: prefer units: - resource: ecs nodeSelector: paidtype: subscription - resource: ecs nodeSelector: paidtype: pay-as-you-go - resource: eci
- Use the following template to create a Deployment that provisions two pods:
apiVersion: apps/v1 kind: Deployment metadata: name: nginx labels: app: nginx spec: replicas: 2 selector: matchLabels: app: nginx template: metadata: name: nginx labels: app: nginx # The pod label must be the same as the one that you specified for the selector in the ResourcePolicy. spec: containers: - name: nginx image: nginx resources: limits: cpu: 2 requests: cpu: 2
- Deploy an NGINX application and query the pods.
- Scale out pods for the NGINX application.
- Scale in pods for the NGINX application.