An Ingress is an API object that provides Layer 7 load balancing to manage external access to Services in a Kubernetes cluster. The NGINX Ingress controller is used to implement the features of Ingresses. This allows Ingresses to perform load balancing for external access based on Ingress rules. In high-load scenarios, insufficient CPU resources and network connections may downgrade application performance. This topic describes how to improve application performance in high-load scenarios by using the NGINX Ingress controller.
Prerequisites
- The NGINX Ingress controller in your Container Service for Kubernetes (ACK) cluster runs as normal.
- kubectl is installed.
Description
Take note of the following items when you deploy the NGINX Ingress controller in a high-load scenario.- Elastic Compute Service (ECS) instance specifications
When the cluster receives a large number of concurrent requests, Ingresses consume a large amount of CPU resources and network connections. We recommend that you use ECS instance types with enhanced performance, such as:
- ecs.c6e.8xlarge (32 Core - 64 GB): compute optimized instance type with enhanced performance. This instance type supports up to 6,000,000 packets per second (PPS).
- ecs.g6e.8xlarge (32 Core - 128 GB): general purpose instance type with enhanced performance. This instance type supports up to 6,000,000 packets per second (PPS).
- Kubernetes configurations
- Use exclusive nodes to deploy the NGINX Ingress controller. Run the following commands
to add labels and taints to the nodes:
kubectl label nodes $node_name ingress-pod="yes" kubectl taint nodes $node_name ingress-pod="yes":NoExecute
- Set CPU Policy to
static
. - We recommend that you select Super I (slb.s3.large) as the Server Load Balancer (SLB) specification for the ingress-controller Service.
- We recommend that you use Terway as the network plug-in and use the exclusive ENI mode.
- Use exclusive nodes to deploy the NGINX Ingress controller. Run the following commands
to add labels and taints to the nodes:
- NGINX Ingress controller configurations
- Configure Guaranteed pods for the NGINX Ingress controller.
- Set the
requests
andlimits
parameters of the nginx-ingress-controller containers to 15 Core and 20 GiB. - Set the
requests
andlimits
parameters of the init-sysctl init container to 100 m and 70 MiB.
- Set the
- Delete the
podAntiAffinity
parameters from the configurations of the pods that are created for the NGINX Ingress controller. This way, a node can host two pods of the NGINX Ingress controller. - Set the number of the replicated pods of the NGINX Ingress controller Deployment to a value that is twice the number of newly added nodes.
- Set
worker-processes
in the ConfigMap of the NGINX Ingress controller to 15. This reserves 15 worker processes for the system. - Set
keepalive
in the ConfigMap of the NGINX Ingress controller to specify the maximum number of requests through a connection. - Disable logging.
- Configure Guaranteed pods for the NGINX Ingress controller.
Step 1: Add nodes
Create a node pool in the ACK cluster and add two nodes to the node pool.
Configure the node pool based on the following description. For more information, see Manage node pools.
- Set Operating System to Alibaba Cloud Linux 2.1903.
- Set Node Label and Taints.
- Add a taint. Set Key to ingress-pod, set Value to yes, and set Effect to NoExecute.
- Add a node label. Set Key to ingress-pod and set Value to yes.
- Set CPU Policy to Static.
Step 2: Update the NGINX Ingress controller
Run the kubectl edit deploy nginx-ingress-controller -n kube-system
command to edit the configuration file of the NGINX Ingress controller based on the
following description.
- Delete the pod anti-affinity settings.
podAntiAffinity: requiredDuringSchedulingIgnoredDuringExecution: - labelSelector: matchExpressions: - key: app operator: In values: - ingress-nginx topologyKey: kubernetes.io/hostname
- Set the
requests
andlimits
parameters for the init container.resources: limits: cpu: 100m memory: 70Mi requests: cpu: 100m memory: 70Mi
- Set the
requests
andlimits
parameters of the nginx-ingress-controller containers to 15 Core and 20 GiB.resources: limits: cpu: "15" memory: 20Gi requests: cpu: "15" memory: 20Gi
- Set node affinity settings and tolerations.
nodeSelector: ingress-pod: "yes" tolerations: - effect: NoExecute key: ingress-pod operator: Equal value: "yes"
- Set the number of the replicated pods of the NGINX Ingress controller Deployment to a value that is twice the number of the newly added nodes.
- Disable metric collection by adding
--enable-metrics=false
to the startup parameters.Note If you do not need metrics, we recommend that you disable metric collection.containers: - args: - /nginx-ingress-controller - --configmap=$(POD_NAMESPACE)/nginx-configuration - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services - --udp-services-configmap=$(POD_NAMESPACE)/udp-services - --annotations-prefix=nginx.ingress.kubernetes.io - --publish-service=$(POD_NAMESPACE)/nginx-ingress-lb - --enable-metrics=false - --v=1