All Products
Search
Document Center

Container Service for Kubernetes:Configure QoS for pods

Last Updated:Dec 24, 2024

Online applications are sensitive to the network latency while offline applications consume large amounts of bandwidth resources. If you run both types of applications without limits, unexpected traffic spikes may overwhelm your online applications and adversely affect the entire network. To avoid this issue, you can configure quality of service (QoS) for pods. This topic describes the pod annotations defined by Kubernetes and the relevant bandwidth limits. This topic also describes how to configure QoS classes.

Bandwidth limits

You can use the Terway network plug-in to control the network bandwidth of pods. The following table describes the pod annotations that you can use to specify the inbound and outbound bandwidth of pods.

Annotation

Mean

kubernetes.io/ingress-bandwidth: 10M

Ingress bandwidth

kubernetes.io/egress-bandwidth: 10M

Egress bandwidth

The following table describes features supported in different Terway modes for limiting bandwidth.

Terway Mode

Egress Shaping

Ingress Shaping

VPC mode (legacy)

对

对

Shared ENI

对

对

Shared ENI + IPVLAN eBPF (legacy)

对

错

Shared ENI + DataPath V2

对

对

Note

In DataPathv2 mode:

  • Egress and ingress bandwidth limitations only apply to inbound and outbound traffic on nodes, and do not affect intra-node traffic.

  • Ingress bandwidth limitations are supported starting from Terway version 1.13.0.

Use pod annotations defined by Kubernetes

For Container Service for Kubernetes (ACK) clusters that use a Terway version earlier than V1.3.0, you need to modify the ConfigMap of Terway to enable the pod annotations defined by Kubernetes.

  1. Run the following command to modify the ConfigMap of Terway:

    kubectl edit cm -n kube-system eni-config
  2. Add capabilities to the data.10-terway.conf section.

    apiVersion: v1
    data:
      10-terway.conf: |
        {
          "cniVersion": "0.3.1",
          "name": "terway",
          "capabilities": {"bandwidth": true},
          "type": "terway"
        }
  3. Press Esc. Then, enter :wq! and press Enter to save the ConfigMap and exit.

  4. Run the following command to recreate the Terway pod for the modified ConfigMap to take effect:

    kubectl delete -n kube-system pod -l app=terway-eniip

Configure QoS classes

By default, the elastic network interface (ENI) queue follows the first-in, first-out (FIFO) principle when processing packets. This allows you to prioritize the packets of latency-sensitive applications. By default, traffic prioritization is disabled. You need to manually enable this feature.

Note

This feature is not available for clusters in DataPath V2 mode.

The following table describes the pod annotations that can be used to configure QoS classes for pods when Terway is used.

Annotation

Mean

k8s.aliyun.com/network-priority: "guaranteed"

Latency-sensitive applications

k8s.aliyun.com/network-priority: "best-effort"

Regular applications

k8s.aliyun.com/network-priority: "burstable"

Applications that consume large amounts of bandwidth

Perform the following steps to enable traffic prioritization.

  1. Run the following command to modify the ConfigMap of Terway:

    kubectl edit cm -n kube-system eni-config
  2. Add enable_network_priority to the data.10-terway.conf section to enable traffic prioritization.

    apiVersion: v1
    data:
      10-terway.conf: |
        {
          "cniVersion": "0.3.1",
          "name": "terway",
          "enable_network_priority": true,
          "type": "terway"
        }
  3. After you modify the ConfigMap, press Esc. Then, enter :wq! and press Enter to save the modified file and exit the edit mode.

  4. Recreate the Terway pod for the modified ConfigMap to take effect.

    kubectl delete -n kube-system pod -l app=terway-eniip
    Important

    After traffic prioritization is enabled, the ENI queue is replaced by mq+prio.