Work with capacity scheduling - Container Compute Service

Container Compute Service (ACS) uses the ResourceQuota mechanism of open source Kubernetes to limit and control resource usage in namespaces. ResourceQuotas can be used to limit and monitor the usage of different resources in a namespace, such as the usage of CPU, memory, and storage resources, number of replicated pods, and number of Services. This helps prevent other applications from occupying excessive amounts of resources in the namespace and ensures the stability and reliability of your application.

ResourceQuota

You can use the ResourceQuota mechanism of open source Kubernetes in ACS clusters to control the total resource usage of each namespace. You can define a ResourceQuota to limit the upper limit of a resource in a namespace, such as CPU, memory, and extended resources. When you create Kubernetes resources such as pods and Services in a namespace, the quota system of Kubernetes tracks the resource usage of the cluster and ensures that the total resource usage does not exceed the hard resource limit defined in the ResourceQuota of the namespace.

When you use a ResourceQuota, take note of the following items:

Each team or application needs a separate namespace.
The cluster administrator needs to create one or more ResourceQuotas for each namespace.
When the resource usage exceeds the hard resource limit of a ResourceQuota, subsequent Kubernetes resource creation requests are rejected.
If you configure CPU and memory quotas in a ResourceQuota, you must configure resource requests and limits for the pods that you want to create. Otherwise, the pods cannot be created.

Note

You do not need to configure resource requests or limits for other resources, such as extended resources. Note: You can use the LimitRanger admission controller to configure default resource limits for pods that are not configured with resource requests.
The name of a ResourceQuota must be a valid DNS subdomain.
After you modify the ResourceQuota, the modification does not apply to resources that have already used the ResourceQuota.

Enable the ResourceQuota feature

By default, the ResourceQuota feature is enabled for Container Service for Kubernetes (ACK) clusters created in the ACS console. You only need to create a ResourceQuota in the namespace. Then, the resourceQuota feature is enabled for the namespace.

Supported resource types

ACS is completely compatible with the ResourceQuota mechanism of open source Kubernetes. Therefore, both standard resources and extended resources support ResourceQuotas.

Standard resources

Resource	Description
limits.cpu	The CPU usage of a pod that is not in the Terminated state must not exceed this limit.
limits.memory	The memory usage of a pod that is not in the Terminated state must not exceed this limit.
requests.cpu	The CPU request of a pod that is not in the Terminated state must not exceed this limit.
requests.memory	The memory request of a pod that is not in the Terminated state must not exceed this limit.
hugepages-<size>	The size of HugePage requests of a pod that is not in the Terminated state must not exceed this limit.
cpu	Equivalent to `requests.cpu`.
memory	Equivalent to `requests.memory`.

Extended resources

Kubernetes does not support extended resource overcommitment (limit > request). Therefore, you need to configure only the resource quota that starts with requests.. For more information, see Resource quotas.

Examples

This example shows how to use kubectl to view and create ResourceQuotas.

Run the following command to create a namespace named test:

kubectl create namespace test

Create a ResourceQuota to set the CPU limit of the Namespace named test to 4000 millicores.

cat << EOF | kubectl apply -f -
apiVersion: v1
kind: ResourceQuota
metadata:
  name: test-quota
  namespace: test
spec:
  hard:
    requests.cpu: "4000m"
    limits.cpu: "4000m"
EOF

Run the following command to query the ResourceQuota:

kubectl -n test describe resourcequota test-quota

Expected results:

Namespace:    test
Resource      Used  Hard
--------      ----  ----
limits.cpu    0     4
requests.cpu  0     4

Create four pods. Each pod can use up to 1000 millicores.

cat << EOF | kubectl apply -f -
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: test-app
  name: test-app
  namespace: test
spec:
  replicas: 5
  selector:
    matchLabels:
      app: test-app
  template:
    metadata:
      labels:
        app: test-app
    spec:
      containers:
      - image: registry.cn-hangzhou.aliyuncs.com/acs/busybox:v1.29.2
        imagePullPolicy: IfNotPresent
        name: test
        command:
        - sleep
        - "360000000"
        resources:
          limits:
            cpu: "1"
EOF

Run the following command to query the pods:

kubectl -n test get pod

Expected results:

NAME                        READY   STATUS    RESTARTS   AGE
test-app-5ddxxxxx94-jdv4m   1/1     Running   0          35s
test-app-5ddxxxxx94-jhmtb   1/1     Running   0          35s
test-app-5ddxxxxx94-mr8vq   1/1     Running   0          35s
test-app-5ddxxxxx94-pjdfn   1/1     Running   0          35s

The output indicates that the four pods are in the Running state.

Run the following command to query the status of the ResourceQuota:

kubectl -n test describe resourcequota

Expected results:

Name:         test-quota
Namespace:    test
Resource      Used  Hard
--------      ----  ----
limits.cpu    4     4
requests.cpu  4     4

The values in the Used column are the same as those in the Hard column.

Create another pod. The pod creation request is intercepted by the ResourceQuota admission controller.

Create a pod.

kubectl -n test scale deploy test-app --replicas 5

Query the ReplicaSet. The DESIRED column displays 5 but the CURRENT column displays 4. This indicates that a pod fails to be created.
```
kubectl -n test get rs
```
Expected results:
```
NAME                  DESIRED   CURRENT   READY   AGE
test-app-5ddxxxxx94   5         4         4       3m10s
```

Query the events of the ReplicaSet. The output indicates that the new pod is intercepted by the ResourceQuota admission controller.

kubectl -n test describe rs test-app-5ddxxxxx94

Expected results:

Name:           test-app-5ddc68c994
Namespace:      test
Selector:       app=test-app,pod-template-hash=5ddc68c994
Labels:         app=test-app
                pod-template-hash=5ddc68c994
Annotations:    deployment.kubernetes.io/desired-replicas: 5
                deployment.kubernetes.io/max-replicas: 7
                deployment.kubernetes.io/revision: 1
Controlled By:  Deployment/test-app
Replicas:       4 current / 5 desired
Pods Status:    4 Running / 0 Waiting / 0 Succeeded / 0 Failed
Pod Template:
  Labels:  app=test-app
           pod-template-hash=5ddc68c994
  Containers:
   test:
    Image:      busybox:latest
    Port:       <none>
    Host Port:  <none>
    Command:
      sleep
      360000000
    Limits:
      cpu:        1
    Environment:  <none>
    Mounts:       <none>
  Volumes:        <none>
Conditions:
  Type             Status  Reason
  ----             ------  ------
  ReplicaFailure   True    FailedCreate
Events:
  Type     Reason            Age                    From                   Message
  ----     ------            ----                   ----                   -------
  Normal   SuccessfulCreate  3m18s                  replicaset-controller  Created pod: test-app-5ddc68c994-pjdfn
  Normal   SuccessfulCreate  3m18s                  replicaset-controller  Created pod: test-app-5ddc68c994-jdv4m
  Normal   SuccessfulCreate  3m18s                  replicaset-controller  Created pod: test-app-5ddc68c994-jhmtb
  Normal   SuccessfulCreate  3m18s                  replicaset-controller  Created pod: test-app-5ddc68c994-mr8vq
  Warning  FailedCreate      3m18s                  replicaset-controller  Error creating: pods "test-app-5ddc68c994-5s4ph" is forbidden: exceeded quota: test-quota, requested: limits.cpu=1,requests.cpu=1, used: limits.cpu=4,requests.cpu=4, limited: limits.cpu=4,requests.cpu=