Container Service for Kubernetes:Configure NGINX Ingress controller for high loads

Deploy on a dedicated node pool

A dedicated node pool isolates the NGINX Ingress controller from other workloads, preventing resource contention.

Adjust the CLB instance specification

The CLB instance specification determines the maximum connections and QPS for the NGINX Ingress controller. Use a high-specification instance.

Edit the Service to specify the CLB instance specification with the service.beta.kubernetes.io/alibaba-cloud-loadbalancer-spec annotation:

kubectl edit service -n kube-system nginx-ingress-lb

apiVersion: v1
kind: Service
metadata:
  annotations:
    ...
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-spec: "slb.s3.large" # Specify the CLB instance specification
  name: nginx-ingress-lb
  namespace: kube-system
  ...
spec:
  ...

Only pay-by-specification CLB instances created before June 2025 support this operation. See Performance specifications.

Use HPA for autoscaling

If dedicated node pool pods cannot absorb traffic bursts, configure HPA to autoscale the NGINX Ingress controller.

Save the following as nginx-hpa.yaml and apply with kubectl apply -f nginx-hpa.yaml:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: nginx-ingress-controller-hpa
  namespace: kube-system
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: nginx-ingress-controller
  minReplicas: 2
  maxReplicas: 5
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 50

SLS logs

• In the ACK console, go to the NGINX Ingress Overview tab on the Network > Ingresses page. Here, you can view the log dashboard to check client access data. You can also go to Operations > Log Center > Application Logs > Logstore and select nginx-ingress to view specific log entries.

• If the log dashboard shows null data points, or the nginx-ingress logs in Logstore are empty, logging was not enabled during cluster creation. See Collect and analyze NGINX Ingress access logs.

Prometheus monitoring

Log on to the ACK console, and navigate to Operations > Prometheus Monitoring to view the monitoring dashboard.

• If a component is not installed, follow the prompts to install it and check the dashboard.

• Select the Ingresses tab. In the Controller Class dropdown, select k8s.io/ingress-nginx to view NGINX Ingress metrics. If k8s.io/ingress-nginx is not listed, the NGINX Ingress controller is not installed.

Add a host field to your Ingress resources. Resources without host are not scraped for metrics by default. To skip per-host monitoring, add --metrics-per-host=false to the controller args in the NGINX Ingress Deployment.

Configure automatic log rotation

NGINX Ingress controller pods log to both /dev/stdout and /var/log/nginx/. As log files grow, writing new entries consumes more resources. Automatic rotation mitigates this by periodically archiving and clearing logs.

1. Log on to the node running the NGINX Ingress controller pod.

2. Create nginx-log-rotate.sh in /root.

   Containerd node

   #!/bin/bash
   # The maximum number of log files to keep. Adjust as needed.
   keep_log_num=5
   
   # Get the IDs of all running ingress-nginx containers.
   ingress_nginx_container_ids=$(crictl ps | grep nginx-ingress-controller | grep -v pause | awk '{print $1}')
   if [[ -z "$ingress_nginx_container_ids" ]]; then
    echo "error: failed to get ingress nginx container ids"
    exit 1
   fi
   
   # Sleep for a random interval between 5 and 10 seconds.
   sleep $(( RANDOM % (10 - 5 + 1 ) + 5 ))
   for id in $ingress_nginx_container_ids; do
    crictl exec $id bash -c "cd /var/log/nginx; if [[ \$(ls access.log-* | wc -l) -gt $keep_log_num ]]; then rm -f \$(ls -t access.log-* | tail -1); fi ; mv access.log access.log-\$(date +%F:%T) ; kill -USR1 \$(cat /tmp/nginx/nginx.pid)"
   done

   Docker node

   #!/bin/bash
   # The maximum number of log files to keep. Adjust as needed.
   keep_log_num=5
   
   # Get the IDs of all running ingress-nginx containers.
   ingress_nginx_container_ids=$(docker ps | grep nginx-ingress-controller | grep -v pause | awk '{print $1}')
   if [[ -z "$ingress_nginx_container_ids" ]]; then
    echo "error: failed to get ingress nginx container ids"
    exit 1
   fi
   
   # Sleep for a random interval between 5 and 10 seconds.
   sleep $(( RANDOM % (10 - 5 + 1 ) + 5 ))
   for id in $ingress_nginx_container_ids; do
    docker exec $id bash -c "cd /var/log/nginx; if [[ \$(ls access.log-* | wc -l) -gt $keep_log_num ]]; then rm -f \$(ls -t access.log-* | tail -1); fi ; mv access.log access.log-\$(date +%F:%T) ; kill -USR1 \$(cat /tmp/nginx/nginx.pid)"
   done

3. Make nginx-log-rotate.sh executable.

   chmod 755 /root/nginx-log-rotate.sh

4. Add the following to /etc/crontab:

   This rotates logs every 15 minutes. Adjust the schedule as needed.

   */15 * * * *  root /root/nginx-log-rotate.sh

Disable metrics collection

Metrics collection is enabled by default and consumes CPU. Disable it if you do not need metrics.

Edit the controller Deployment (kubectl edit deploy nginx-ingress-controller -n kube-system). Add --enable-metrics=false to the container args to disable metrics.

spec:
  containers:
  - args:
    - ...
    - --enable-metrics=false

spec:
  containers:
  - args:
    - ...
    - --enable-metrics=true
    - --exclude-socket-metrics # Effective only when --enable-metrics=true

Enable Brotli compression

The Brotli algorithm typically achieves 15%–30% better compression than gzip for text-based web resources.

Actual improvement depends on your scenario.

Edit the NGINX Ingress ConfigMap (kubectl edit cm -n kube-system nginx-configuration) to enable Brotli compression.

data:
  enable-brotli: "true"
  brotli-level: "6"
  brotli-types: "text/xml image/svg+xml application/x-font-ttf image/vnd.microsoft.icon application/x-font-opentype application/json font/eot application/vnd.ms-fontobject application/javascript font/otf application/xml application/xhtml+xml text/javascript application/x-javascript text/plain application/x-font-truetype application/xml+rss image/x-icon font/opentype text/css image/x-win-bitmap"

• enable-brotli: Specifies whether to enable Brotli. Valid values: true, false.

• brotli-level: Compression level. Valid values: 1–11. Default: 4. Higher values consume more CPU.

• brotli-types: MIME types to compress with Brotli.

Adjust the timeout policy

Reducing FIN_WAIT2 and TIME_WAIT timeouts allows the NGINX Ingress controller to recycle finished connections faster, freeing resources.

Edit the NGINX Ingress controller Deployment (kubectl edit deploy nginx-ingress-controller -n kube-system).

Add the following to initContainers:

• net.ipv4.tcp_fin_timeout: FIN_WAIT2 timeout. Default: 60 seconds.

• net.netfilter.nf_conntrack_tcp_timeout_time_wait: TIME_WAIT timeout. Default: 60 seconds.

dnsPolicy: ...
initContainers:
- command:  
  - /bin/sh
  - -c
  - |
    if [ "$POD_IP" != "$HOST_IP" ]; then
    mount -o remount rw /proc/sys
    sysctl -w net.core.somaxconn=65535
    sysctl -w net.ipv4.ip_local_port_range="1024 65535"
    sysctl -w kernel.core_uses_pid=0
    sysctl -w net.ipv4.tcp_fin_timeout=15 # Set the timeout for the FIN_WAIT2 state to 15 seconds
    sysctl -w net.netfilter.nf_conntrack_tcp_timeout_time_wait=30 # Set the timeout for the TIME_WAIT state to 30 seconds
    fi
  env:                                                           
    ...                                                     
  image: ...
  imagePullPolicy: IfNotPresent                           
  name: init-sysctl            

Optimize HTTPS performance

Adjust the following parameters in the nginx-configuration ConfigMap.

kubectl edit cm -n kube-system nginx-configuration

• SSL session cache and timeout

  Setting the SSL session cache size and timeout reduces handshake overhead.

  data:
    ssl-session-cache-size: "10m"
    ssl-session-timeout: "10m"

  Corresponding nginx.conf configuration

  You can adjust this configuration in the NGINX nginx.conf file based on your scenario.

  ssl_session_cache shared:SSL:10m; # 1 MB can store about 4,000 sessions.
  ssl_session_timeout 10m;

• Enable OCSP stapling to reduce certificate verification time.

  data:
    enable-ocsp: "true"

• Enable TLS 1.3 0-RTT to let clients send data before the handshake completes, reducing connection setup latency.

  data:
    ssl-early-data: "true"
    ssl-protocols: "TLSv1.3"

• Adjust cipher priority (no manual tuning required)

  Cipher suite priority affects latency. The NGINX Ingress controller default is already optimized.

  ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384';
  ssl_prefer_server_ciphers on;    # Prioritize the server's cipher configuration.

Configure other performance-related options

The following nginx-configuration ConfigMap options also affect performance. Run kubectl edit cm -n kube-system nginx-configuration to edit.