Container Service for Kubernetes:CPU topology-aware scheduling

How it works

By default, Kubernetes relies on the kernel's Completely Fair Scheduler (CFS) to balance the CPU load. This scheduler distributes the load across all cores by fairly allocating CPU time slices. However, this can cause performance jitter in sensitive applications because the scheduler ignores the physical CPU topology.

The Kubernetes CPU Manager (with the static policy) can bind pods to exclusive CPU cores, but it has the following limitations:

• Lack of scheduler awareness: The native kube-scheduler makes decisions at the node level only. It cannot perceive the CPU topology of the entire cluster. As a result, it cannot find the optimal physical core layout for a pod on a global scale.

• Lack of topology awareness: When allocating cores on a node, the static policy is unaware of the NUMA architecture. This can lead to memory access across NUMA nodes, which introduces extra latency.

• Lack of flexibility: This policy requires that the pod's QoS class is <a href="https://kubernetes.io/docs/concepts/workloads/pods/pod-qos/#guaranteed" id="5b2a90624cgtc">Guaranteed</a>. It cannot be applied to <a href="https://kubernetes.io/docs/concepts/workloads/pods/pod-qos/#burstable" id="174c05a5978cf">Burstable</a> or <a href="https://kubernetes.io/docs/concepts/workloads/pods/pod-qos/#burstable" id="6cada2573fokq">BestEffort</a> pods.

To address these limitations, ACK provides enhanced CPU topology-aware scheduling that is based on the new Scheduling Framework. This feature is achieved through the collaboration between the ACK kube-scheduler and ack-koordinator.

1. Node topology reporting: The ack-koordinator component detects the local physical CPU topology, such as sockets, NUMA nodes, and caches, in real time and reports the topology to the scheduling center.

2. Global topology-aware scheduling: The kube-scheduler uses the global topology information to select the optimal node for a pod from the entire cluster and plans the core allocation scheme. For example, when selecting the optimal node, the scheduler finds the core with the fewest bound applications by default. This allocation scheme is then written to the pod's annotation as part of the scheduling result.

3. Local core pinning: After the pod is scheduled to the target node, ack-koordinator reads the pod's annotation and modifies the cpuset.cpus file in the corresponding Cgroup of the pod to bind the pod to the physical core.

Step 1: Deploy a sample application

This topic uses an Nginx application as an example to demonstrate how to enable CPU topology-aware scheduling and achieve CPU pinning.

1. Create an nginx-app.yaml file.

   View YAML

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: nginx-deployment
     namespace: default
     labels:
       app: nginx
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: nginx
     template:
       metadata:
         labels:
           app: nginx
       spec:
         containers:
         - name: nginx
           image: alibaba-cloud-linux-3-registry.cn-hangzhou.cr.aliyuncs.com/alinux3/nginx_optimized:20240221-1.20.1-2.3.0
           ports:
           - containerPort: 80
           command:
           - "sleep"
           - "infinity"
           resources:
             requests:
               cpu: 4
               memory: 8Gi
             limits:
               # Set the CPU value. It must be an integer.
               cpu: 4 
               memory: 8Gi

2. Deploy the application.

   kubectl apply -f nginx-app.yaml

3. Log on to the node where the pod is located to obtain the pod UID and container ID.

   • Obtain the pod name.
     Run the kubectl get pods -n <your-namespace> command to view the pod name, such as nginx-deployment-6f5899*****.
     
     

   • Obtain the pod UID.

     # Replace <your-pod-name> with the actual pod name
     kubectl get pod <your-pod-name> -n default -o jsonpath='{.metadata.uid}{"\n"}'

     Expected output:

       uid: a78a02b5-c87f-4e74-9ddd-254c163*****

   • Obtain the container ID.

     # Replace <your-pod-name> with the actual pod name
     kubectl describe pod <your-pod-name> -n default

     In the output, find the Containers field, locate the Container ID, and remove the prefix, such as containerd://.

     Expected output:

     Containers:
       nginx:
         Container ID:   containerd://b8b88a70096aabb0aea197dd2aba78d15bcbe9145198ef46a0474b31*****

4. Check the cgroup version of the node.

   stat -fc %T /sys/fs/cgroup/

   • If the output is cgroup_root, the system uses cgroup v1.

   • If the output is cgroup2fs, the system uses cgroup v2.

5. Run the corresponding verification command based on the cgroup version to check the CPU pinning status.

   In the cgroup path, replace the hyphens (-) in the pod UID with underscores (_). For example, if the original pod UID is a78a02b5-c87f-4e74-9ddd-254c163*****, the format used in the path is a78a02b5_c87f_4e74_9ddd_254c163*****.

   • cgroup v1:

     # Replace <POD_UID> and <CONTAINER_ID> with the actual values
     cat /sys/fs/cgroup/cpuset/kubepods.slice/kubepods-pod<POD_UID>.slice/cri-containerd-<CONTAINER_ID>.scope/cpuset.cpus

   • cgroup v2:

     # Replace <POD_UID> and <CONTAINER_ID> with the actual values
     cat /sys/fs/cgroup/kubepods.slice/kubepods-pod<POD_UID>.slice/cri-containerd-<CONTAINER_ID>.scope/cpuset.cpus.effective

   The expected output is a range, which means the container can use all cores on the node and is not pinned.

   0-31

Step 2: Enable CPU topology-aware scheduling

You can enable CPU topology-aware scheduling by adding an annotation to the pod.

• General pinning policy: A general policy that follows a 1:1 pinning principle. It binds the number of cores specified by resources.limits.cpu to the pod. It prioritizes CPU cores within the same NUMA node to ensure optimal memory access performance.

• Automatic pinning policy: A policy that is optimized for specific hardware. It prioritizes binding a complete physical core cluster (such as a CCX/CCD on an AMD CPU) to maximize CPU locality and improve concurrency. This is recommended for specific large-scale (32 cores or more) AMD machine types.

Related operations

Recommendations for production environments

• Observability: Before and after you enable core pinning, integrate with Alibaba Cloud Prometheus monitoring. Closely monitor key application performance metrics (such as response time (RT) and QPS) and node metrics such as CPU usage and CPU throttling to observe the performance changes that are caused by core pinning.

• Phased rollout: For applications with multiple replicas, use canary releases or phased updates to gradually enable or disable the pinning policy. This helps control the risks that are associated with the change.

Billing

The ack-koordinator component is free to install and use, but additional fees may apply in the following scenarios:

• ack-koordinator is a non-managed component and consumes worker node resources after installation. You can configure the resource requests for each module during installation.

• By default, ack-koordinator exposes monitoring metrics for features such as resource profiling and fine-grained scheduling in Prometheus format. If you select the Enable Prometheus monitoring for ACK-Koordinator option and use Alibaba Cloud Prometheus, these metrics are considered custom metrics and will incur fees. The cost depends on your cluster size and the number of applications. Before you enable this feature, review the Prometheus billing information to understand the free quota and pricing for custom metrics. You can monitor and manage your resource usage by querying your usage data.

References

• ACK supports GPU topology-aware scheduling, which can select the optimal combination of GPUs on a node for the best training speed.

• You can quantify the allocated but unused resources in a cluster and provide them to low-priority jobs. This achieves dynamic resource overcommitment.