Container Service for Kubernetes:Enable CPU QoS for containers

Billing

No fee is charged when you install or use the ack-koordinator component. However, fees may be charged in the following scenarios:

• ack-koordinator is a non-managed component that occupies worker node resources after it is installed. You can specify the amount of resources requested by each module when you install the component.

• By default, ack-koordinator exposes the monitoring metrics of features such as resource profiling and fine-grained scheduling as Prometheus metrics. If you enable Prometheus metrics for ack-koordinator and use Managed Service for Prometheus, these metrics are considered custom metrics and fees are charged for these metrics. The fee depends on factors such as the size of your cluster and the number of applications. Before you enable Prometheus metrics, we recommend that you read the Billing topic of Managed Service for Prometheus to learn about the free quota and billing rules of custom metrics. For more information about how to monitor and manage resource usage, see Query the amount of observable data and bills.

Procedure

You can use a ConfigMap to enable CPU QoS in a cluster and configure CPU group identity priorities for LS and BE pods. You can use the group identity feature to specify an identifier for each CPU cgroup. When the kernel schedules the tasks for which identities are configured, the kernel processes the tasks based on the priorities of the tasks.

After the configuration is complete, you can specify the CPU QoS class of a pod by adding the koordinator.sh/qosClass label to the YAML file of the pod. If you do not add the koordinator.sh/qosClass label to the pod, ack-koordinator selects a Kubernetes-native QoS class. BE indicates the BestEffort QoS class and LS indicates the Burstable or Guaranteed QoS class.

1. Create a file named configmap.yaml and copy the following content to the file. The file is used to create a ConfigMap that allows you to enable the CPU QoS feature.

   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: ack-slo-config
     namespace: kube-system
   data:
     # Enable the CPU QoS feature. 
     resource-qos-config: |
       {
         "clusterStrategy": {
           "lsClass": {
             "cpuQOS": {
               "enable": true,
               "groupIdentity": 2
             }
           },
           "beClass": {
             "cpuQOS": {
               "enable": true,
               "groupIdentity": -1
             }
           }
         }
       }

   Specify the lsClass and beClass parameters to assign the LS and BE classes to different pods. The cpuQOS field is used to configure the CPU QoS feature. The following table describes the key parameters.

   Parameter	Type	Value range	Description
   enable	Boolean	true false	true: enables the CPU QoS feature for all containers in the cluster. false: disables the CPU QoS feature for all containers in the cluster.
   groupIdentity	Int	[-1, 2]	The priority of the CPU group identity. A greater group identity value indicates a higher priority in CPU scheduling. For more information, see Group identity feature. The default value for an LS pod is 2 and the default value for a BE pod is -1. A value of 0 disables the group identity feature.

2. Check whether the ConfigMap named ack-slo-config exists in the kube-system namespace.

   • If the ack-slo-config ConfigMap exists, we recommend that you run the kubectl patch command to update the ConfigMap. This avoids changing other settings in the ConfigMap.

     kubectl patch cm -n kube-system ack-slo-config --patch "$(cat configmap.yaml)"

   • If the ack-slo-config ConfigMap does not exist, run the following command to create a ConfigMap:

     kubectl apply -f configmap.yaml

3. Create a file named ls-pod-demo.yaml and copy the following content to the file. The YAML file assigns the LS class to the pod. Then, deploy the YAML file in the cluster.

   Note

   To apply configurations to a workload, such as a deployment, set the appropriate annotations for the pod in the template.metadata field.

   apiVersion: v1
   kind: Pod
   metadata:
     name: ls-pod-demo
     labels:
       koordinator.sh/qosClass: 'LS' # Set the QoS class of the pod to LS. 
   spec:
     containers:
     - command:
       - httpd
       - -D
       - FOREGROUND
       image: registry.cn-zhangjiakou.aliyuncs.com/acs/apache-2-4-51-for-slo-test:v0.1
       imagePullPolicy: Always
       name: apache
       resources:
         limits:
           cpu: "4"
           memory: 10Gi
         requests:
           cpu: "4"
           memory: 10Gi
     restartPolicy: Never
     schedulerName: default-scheduler

   kubectl apply -f ls-pod-demo.yaml

4. Run the following command to check whether the CPU group identity of the LS pod in the control group (cgroup) of the node takes effect:

   cat /sys/fs/cgroup/cpu/kubepods.slice/kubepods-pod1c20f2ad****.slice/cpu.bvt_warp_ns

   Expected output:

   # The group identity of the LS pod is 2 (high priority). 
   2

5. Create a file named ls-pod-demo.yaml and copy the following content to the file. The YAML file assigns the BE class to the pod. Then, deploy the YAML file in the cluster.

   apiVersion: v1
   kind: Pod
   metadata:
     name: be-pod-demo
     labels:
       koordinator.sh/qosClass: 'BE' # Set the QoS class of the pod to BE. 
   spec:
     containers:
       - args:
           - '-c'
           - '1'
           - '--vm'
           - '1'
         command:
           - stress
         image: polinux/stress
         imagePullPolicy: Always
         name: stress
     restartPolicy: Always
     schedulerName: default-scheduler

   kubectl apply -f be-pod-demo.yaml

6. Run the following command to check whether the CPU group identity of the BE pod in the cgroup of the node takes effect:

   cat /sys/fs/cgroup/cpu/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b6e96c8****.slice/cpu.bvt_warp_ns

   Expected output:

   # The group identity of the BE pod is -1 (low priority). 
   -1

   The output shows that the group identity of the LS pod has a high priority and the group identity of the BE pod has a low priority. CPU resources are preferably scheduled to the LS pod to ensure service quality.

FAQ

Is the CPU QoS feature that is enabled based on the earlier version of the ack-slo-manager protocol still supported after I upgrade from ack-slo-manager to ack-koordinator?

In an earlier version (≤ 0.8.0) of the ack-slo-manager protocol, the alibabacloud.com/qosClass pod annotation is used to enable CPU QoS.

ack-koordinator is compatible with earlier versions of the ack-slo-manager protocol. You can seamlessly upgrade from ack-slo-manager to ack-koordinator and gradually change the protocol used by the pod to koordinator.sh. ack-koordinator is compatible with the earlier protocol versions no later than July 30, 2023. We recommend that you upgrade the resource parameters in an earlier protocol version to the latest version.