Container Service for Kubernetes:Horizontal Pod Autoscaling Based on Nginx Ingress Component Metrics

This topic walks you through the following steps to configure horizontal pod autoscaling (HPA) using Ingress QPS metrics from Simple Log Service (SLS). The example deploys a Deployment named nginx, a Service, and an Ingress.

Step 1: Deploy the ack-alibaba-cloud-metrics-adapter component

The ack-alibaba-cloud-metrics-adapter component enables Kubernetes to retrieve monitoring data from Alibaba Cloud services—including ECS, SLB, and RDS—through the External Metrics API. This enhances cluster monitoring and autoscaling.

1. Log on to the Container Service Management Console . In the navigation pane on the left, click Clusters.

2. On the Clusters page, click the name of your cluster. In the left navigation pane, choose Applications > Helm.

3. On the Helm page, click Create. Complete the Basic Information configuration. Select ack-alibaba-cloud-metrics-adapter and click Next.

4. On the Parameter Settings page, select a Chart Version and click OK.

Step 2: Create an application and a service

1. Create a file named nginx-test.yaml.

   Expand to view the YAML example

   apiVersion: apps/v1 
   kind: Deployment
   metadata:
     name: nginx-deployment-basic
     labels:
       app: nginx
   spec:
     replicas: 2
     selector:
       matchLabels:
         app: nginx
     template:
       metadata:
         labels:
           app: nginx
       spec:
         containers:
         - name: nginx
           image: nginx:1.7.9 
           ports:
           - containerPort: 80
   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: nginx
     namespace: default
   spec:
     ports:
       - port: 80
         protocol: TCP
         targetPort: 80
     selector:
       app: nginx
     type: ClusterIP

2. Run the following command to create the Deployment and its Service.

   kubectl apply -f nginx-test.yaml

Step 3: Create a route

1. In the left navigation pane of the cluster management page, choose Network > Ingress. On the Ingress page, click Create Ingress in the upper-left corner.

2. Fill in the required fields in the Create panel and click OK. After creation, the system automatically redirects you to the Ingress page.

3. In the Name column, click the name of the Ingress you created to view its routing rules. For more information, see Manage Ingress.

Step 4: Configure HPA

You can configure two SLS Project metrics for scaling in HPA: sls_ingress_qps and sls_ingress_latency_p9999.

• sls_ingress_qps: Use AverageValue. The QPS value is divided by the number of pods before comparison.

• sls_ingress_latency_p9999: Use Value. The raw value is used without division.

1. Create a file named ingress-hpa.yaml and paste the following content into it.

   Expand to view the YAML example

   apiVersion: autoscaling/v2
   kind: HorizontalPodAutoscaler
   metadata:
     name: ingress-hpa
   spec:
     scaleTargetRef:
       apiVersion: apps/v1
       kind: Deployment
       name: nginx-deployment-basic
     minReplicas: 2
     maxReplicas: 10
     metrics:
       - type: External
         external:
           metric:
             name: sls_ingress_qps
             selector:
               matchLabels:
                 sls.project: "***" # Replace *** with your actual SLS project name.
                 sls.logstore: "nginx-ingress"
                 sls.ingress.route: "default-nginx-80"
           target:
             type: AverageValue
             averageValue: 10
       - type: External
         external:
           metric:
             name: sls_ingress_latency_p9999
             selector:
               matchLabels:
                 # Default Ingress log project is k8s-log-clusterId
                 sls.project: "***" 
                 # Default Ingress logstore is nginx-ingress
                 sls.logstore: "nginx-ingress"
                 # Format: namespace-svc-port
                 sls.ingress.route: "default-nginx-80"
                 # SLS VPC endpoint. Default is true.
                 # sls.internal.endpoint:true
           target:
             type: Value
             # If sls_ingress_latency_p9999 > 10 ms, HPA increases the number of pods in nginx-deployment-basic.
             value: 10

   The following table describes the parameters used in the HPA configuration.

   Parameter name	Required	Description
   sls.ingress.route	Yes	Format: <namespace>-<svc>-<port>. <namespace> is the namespace where the Ingress resides. <svc> is the name of the Service associated with the Ingress. <port> is the port name of that Service. Example: default-nginx-80.
   sls.logstore	Yes	The name of the Logstore in Simple Log Service. By default, sls.logstore is nginx-ingress.
   sls.project	Yes	The name of the Project in Simple Log Service. By default, sls.project is k8s-log-<cluster-id>.
   sls.internal.endpoint	No	Set whether to access Simple Log Service over the internal network or the public network. Default is true. true: Access SLS over the internal network. false: Access SLS over the public network.

2. Run the following command to create the HPA.

   kubectl apply -f ingress-hpa.yaml

Step 5: Verify the result

1. After configuring HPA, run the following command to perform stress testing.

   ab -t 300 -c 10 <domain-name-configured-for-the-ingress> # Use Apache Benchmark to send 10 concurrent requests to the Ingress-exposed service for 300 seconds.

2. Verify the scaling status.

   Run the following command to check the status.

   kubectl get hpa ingress-hpa

   Expected output:

   NAME            REFERENCE                              TARGETS           MINPODS    MAXPODS    REPLICAS   AGE
   ingress-hpa     Deployment/nginx-deployment-basic      21/10 (avg)       2          10         10         7m49s

   If the REPLICAS value equals the MAXPODS value, scaling succeeded.

FAQ

kubectl get --raw  /apis/external.metrics.k8s.io/v1beta1/namespaces/*/sls_ingress_qps?labelSelector=sls.project={{SLS_Project}},sls.logstore=nginx-ingress

Error from server: {
    "httpCode": 400,
    "errorCode": "ParameterInvalid",
    "errorMessage": "key (slb_pool_name) is not config as key value config,if symbol : is  in your log,please wrap : with quotation mark \"",
    "requestID": "xxxxxxx"
}

{
  "kind": "ExternalMetricValueList",
  "apiVersion": "external.metrics.k8s.io/v1beta1",
  "metadata": {},
  "items": [
    {
      "metricName": "sls_ingress_qps",
      "timestamp": "2025-02-26T16:45:00Z", 
      "value": "50",   # QPS value
      "metricLabels": {
        "sls.project": "your-sls-project-name",
        "sls.logstore": "nginx-ingress"
      }
    }
  ]
}

References

• Horizontal Pod Autoscaling Based on Alibaba Cloud Prometheus Metrics

• Multi-application Horizontal Pod Autoscaling Based on Nginx Ingress Traffic Metrics