Online applications are sensitive to the network latency while offline applications consume large amounts of bandwidth resources. If you run both types of applications without limits, unexpected traffic spikes may overwhelm your online applications and adversely affect the entire network. To avoid this issue, you can configure quality of service (QoS) for pods. This topic describes the pod annotations defined by Kubernetes and the relevant bandwidth limits. This topic also describes how to configure QoS classes.
Bandwidth limits
You can use the Terway network plug-in to control the network bandwidth of pods. The following table describes the pod annotations that you can use to specify the inbound and outbound bandwidth of pods.
Annotation | Mean |
kubernetes.io/ingress-bandwidth: 10M | Ingress bandwidth |
kubernetes.io/egress-bandwidth: 10M | Egress bandwidth |
The following table describes features supported in different Terway modes for limiting bandwidth.
Terway Mode | Egress Shaping | Ingress Shaping |
VPC mode (legacy) | ||
Shared ENI | ||
Shared ENI + IPVLAN eBPF (legacy) | ||
Shared ENI + DataPath V2 |
In DataPathv2 mode:
Egress and ingress bandwidth limitations only apply to inbound and outbound traffic on nodes, and do not affect intra-node traffic.
Ingress bandwidth limitations are supported starting from Terway version 1.13.0.
Use pod annotations defined by Kubernetes
For Container Service for Kubernetes (ACK) clusters that use a Terway version earlier than V1.3.0, you need to modify the ConfigMap of Terway to enable the pod annotations defined by Kubernetes.
Run the following command to modify the ConfigMap of Terway:
kubectl edit cm -n kube-system eni-config
Add
capabilities
to thedata.10-terway.conf
section.apiVersion: v1 data: 10-terway.conf: | { "cniVersion": "0.3.1", "name": "terway", "capabilities": {"bandwidth": true}, "type": "terway" }
Press Esc. Then, enter :wq! and press Enter to save the ConfigMap and exit.
Run the following command to recreate the Terway pod for the modified ConfigMap to take effect:
kubectl delete -n kube-system pod -l app=terway-eniip
Configure QoS classes
By default, the elastic network interface (ENI) queue follows the first-in, first-out (FIFO) principle when processing packets. This allows you to prioritize the packets of latency-sensitive applications. By default, traffic prioritization is disabled. You need to manually enable this feature.
This feature is not available for clusters in DataPath V2 mode.
The following table describes the pod annotations that can be used to configure QoS classes for pods when Terway is used.
Annotation | Mean |
k8s.aliyun.com/network-priority: "guaranteed" | Latency-sensitive applications |
k8s.aliyun.com/network-priority: "best-effort" | Regular applications |
k8s.aliyun.com/network-priority: "burstable" | Applications that consume large amounts of bandwidth |
Perform the following steps to enable traffic prioritization.
Run the following command to modify the ConfigMap of Terway:
kubectl edit cm -n kube-system eni-config
Add
enable_network_priority
to thedata.10-terway.conf
section to enable traffic prioritization.apiVersion: v1 data: 10-terway.conf: | { "cniVersion": "0.3.1", "name": "terway", "enable_network_priority": true, "type": "terway" }
After you modify the ConfigMap, press Esc. Then, enter :wq! and press Enter to save the modified file and exit the edit mode.
Recreate the Terway pod for the modified ConfigMap to take effect.
kubectl delete -n kube-system pod -l app=terway-eniip
ImportantAfter traffic prioritization is enabled, the ENI queue is replaced by mq+prio.