Load-aware pod scheduling is a scheduling policy that can be used by Container Service for Kubernetes (ACK) to monitor the loads on node resources based on the scheduling framework. When load-aware pod scheduling is enabled, the system reviews the historical statistics of loads on nodes. Then, the system schedules pods to nodes with lower loads to implement load balancing. This prevents applications or node crashes caused by an overloaded node. This topic describes how to use load-aware pod scheduling.

Prerequisites

  • This feature is suitable only for professional Kubernetes clusters. For more information about how to create a professional Kubernetes cluster, see Create an ACK Pro cluster.
  • ack-slo-manager is installed in the cluster. For more information, see Install ack-so-manager.
  • The following table describes the versions that are required for the system components.
    Component Version
    Kubernetes ≥ V1.18
    Helm ≥ V3.0

Procedure

You can specify whether to enable load-aware pod scheduling by adding an annotation.

      annotations:
        alibabacloud.com/loadAwareScheduleEnabled: "true" # Enable load-aware pod scheduling. 

Use load-aware pod scheduling

The cluster used in this example contains three nodes. Each node has 4 CPU cores and 8 GiB of memory.

  1. Use the following YAML file to create a Job for video transcoding:
    apiVersion: batch/v1
    kind: Job
    metadata:
      generateName: ffmpeg
      namespace: default
    spec:
      backoffLimit: 1
      parallelism: 1
      template:
        metadata:
        spec:
          containers:
          - image: registry.cn-hangzhou.aliyuncs.com/kubernetes-image-hub/ffmpeg:latest
            name: ffmpeg
            resources:
              requests:
                cpu: 1
                memory: 2Gi
              limits:
                cpu: 2
                memory: 4Gi
            command: ["/bin/sh"]
            args: ["-c","while true;do ffmpeg -i /root/mp4/statefulset.mp4 -c:v libx264 -crf 100 -threads 200 destination.flv -y;sleep 1;done"]
          restartPolicy: OnFailure
  2. Run the following command to query the status and deployment of the pod:
    # The image size is large and it may require about 2 minutes to start the pod.
    kubectl get pods -o wide

    Expected output:

    NAME                READY   STATUS    RESTARTS   AGE    IP             NODE                   NOMINATED NODE   READINESS GATES
    ffmpegjjpv2-xcl6t   1/1     Running   0          2m6s   172.20.177.8   cn-beijing.10.0.0.46   <none>           <none>

    The preceding output indicates that the pod is running on the node cn-beijing.10.0.0.46.

  3. Run the following command to view the loads on all of the cluster nodes:
    kubectl top node

    Expected output:

    NAME                   CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%
    cn-beijing.10.0.0.44   251m         6%     1531Mi          22%
    cn-beijing.10.0.0.45   352m         8%     1881Mi          27%
    cn-beijing.10.0.0.46   1802m        45%    1689Mi          25%

    The preceding output indicates that the load on the node cn-beijing.10.0.0.46 is the highest.

  4. Use the following YAML file to create pods for which load-aware pod scheduling is disabled:
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: nginx
      namespace: default
      labels:
        app: nginx
    spec:
      replicas: 6
      selector:
        matchLabels:
          app: nginx
      template:
        metadata:
          name: nginx
          labels:
            app: nginx
        spec:
          containers:
          - name: nginx
            image: nginx
            resources:
              limits:
                cpu: 500m
              requests:
                cpu: 500m
  5. Run the following command to view details of the pods:
    kubectl get pods | grep nginx

    Expected output:

    nginx-6976ccb95-96xmf   1/1     Running   0          108s    172.20.177.10    cn-beijing.10.0.0.46   <none>           <none>
    nginx-6976ccb95-b6xtp   1/1     Running   0          108s    172.20.177.70    cn-beijing.10.0.0.44   <none>           <none>
    nginx-6976ccb95-hmqcw   1/1     Running   0          108s    172.20.177.69    cn-beijing.10.0.0.44   <none>           <none>
    nginx-6976ccb95-l2gnn   1/1     Running   0          108s    172.20.177.9     cn-beijing.10.0.0.46   <none>           <none>
    nginx-6976ccb95-rzsfc   1/1     Running   0          108s    172.20.176.200   cn-beijing.10.0.0.45   <none>           <none>
    nginx-6976ccb95-shv52   1/1     Running   0          108s    172.20.177.71    cn-beijing.10.0.0.44   <none>           <none>

    The preceding output indicates that the pods are evenly scheduled to the nodes in the cluster. In this scenario, load-aware pod scheduling is disabled for the pods, and the cluster cannot monitor the loads on the nodes.

  6. Run the following command to delete the pods for which load-aware pod scheduling is disabled:
    kubectl delete deployment nginx
  7. Use the following YAML file to create pods that have load-aware pod scheduling enabled:
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: nginx
      namespace: default
      labels:
        app: nginx
    spec:
      replicas: 6
      selector:
        matchLabels:
          app: nginx
      template:
        metadata:
          name: nginx
          labels:
            app: nginx
          annotations:
            alibabacloud.com/loadAwareScheduleEnabled: "true" # Enable load-aware pod scheduling. 
        spec:
          containers:
          - name: nginx
            image: nginx
            resources:
              limits:
                cpu: 500m
              requests:
                cpu: 500m
  8. Run the following command to view details of the pods:
    kubectl get pods | grep nginx

    Expected output:

    nginx-7759585d4c-hfblh   1/1     Running   0          72s     172.20.177.74    cn-beijing.10.0.0.44   <none>           <none>
    nginx-7759585d4c-hzzt9   1/1     Running   0          72s     172.20.176.203   cn-beijing.10.0.0.45   <none>           <none>
    nginx-7759585d4c-m4gmm   1/1     Running   0          72s     172.20.177.72    cn-beijing.10.0.0.44   <none>           <none>
    nginx-7759585d4c-v7wfb   1/1     Running   0          72s     172.20.176.201   cn-beijing.10.0.0.45   <none>           <none>
    nginx-7759585d4c-wx59r   1/1     Running   0          72s     172.20.177.73    cn-beijing.10.0.0.44   <none>           <none>
    nginx-7759585d4c-xbwh6   1/1     Running   0          72s     172.20.176.202   cn-beijing.10.0.0.45   <none>           <none>

    The preceding output indicates that after load-aware pod scheduling is enabled for the pods, the cluster can monitor the loads on the nodes and use the scheduling policy to schedule the pods to nodes other than the node cn-beijing.10.0.0.46.