Microservices Engine:Auto-scale ACK applications based on gateway QPS

How it works

The auto-scaling pipeline consists of four components:

MSE cloud-native gateway  -->  Simple Log Service (SLS)  -->  Metrics adapter  -->  HPA  -->  Scale Deployment

1. The MSE cloud-native gateway processes incoming requests and generates access logs.

2. Log shipping sends these access logs to an SLS logstore in NGINX Ingress-compatible format.

3. The ack-alibaba-cloud-metrics-adapter reads QPS metrics from SLS and exposes them as Kubernetes external metrics.

4. The HPA evaluates the external QPS metric against a target threshold and adjusts the replica count of the backend Deployment.

The HPA uses the External metric type because gateway QPS originates outside the Kubernetes metrics pipeline, unlike Resource (CPU/memory) or Pods metrics.

Prerequisites

Before you begin, make sure that you have:

• An MSE cloud-native gateway. For more information, see Create a cloud-native gateway

• An ACK managed cluster. For more information, see Create an ACK managed cluster

Step 1: Install the metrics adapter

The metrics adapter bridges SLS metrics and the Kubernetes metrics API. Install it from the ACK Marketplace.

1. Log on to the ACK console.

2. In the left-side navigation pane, choose Marketplace > Marketplace.

3. Search for ack-alibaba-cloud-metrics-adapter and click its card.

   

4. Click Deploy in the upper-right corner. In the Deploy panel, configure the settings and click OK.

Step 2: Deploy a sample backend application

Deploy a sample Deployment and Service in your ACK cluster. If you already have a backend application, skip to Step 3.

1. In the left-side navigation pane of the ACK console, click Clusters.

2. On the Clusters page, click the name of your cluster.

3. In the left-side navigation pane, choose Workloads > Deployments.

4. Click Create Resources in YAML. Select Custom from the Sample Template drop-down list, paste the following YAML, and click Create.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: httpbin-deploy
  labels:
    app: httpbin-deploy
spec:
  replicas: 1
  selector:
    matchLabels:
      app: httpbin
  template:
    metadata:
      labels:
        app: httpbin
    spec:
      containers:
      - image: kennethreitz/httpbin
        imagePullPolicy: IfNotPresent
        name: httpbin
        ports:
        - name: http
          containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: httpbin-svc
  namespace: default
  labels:
    app: httpbin-svc
spec:
  ports:
    - port: 8080
      name: http
      protocol: TCP
      targetPort: 80
  selector:
    app: httpbin
  type: ClusterIP

This creates:

• A Deployment named httpbin-deploy with one replica running the httpbin container on port 80.

• A Service named httpbin-svc that exposes the Deployment on port 8080 within the cluster.

Step 3: Connect the ACK service to the gateway

Register your ACK cluster as a service source in the MSE gateway, then add the backend service.

Step 4: Enable log shipping

Log shipping sends gateway access logs to SLS, which the metrics adapter reads to calculate QPS.

1. In the left-side navigation pane of your gateway, click Parameter Settings.

2. In the Observability Parameters section, click the icon next to Log Shipping.

3. In the Log Shipping Settings dialog box, turn on Enable Log Shipping (Ship Gateway Access Logs to Log Service) and Compatible with NGINX Ingress.

   

Step 5: Create an HPA with QPS-based scaling

Create an HPA that scales the backend Deployment based on per-pod QPS.

Run the following command or use the ACK console to apply this YAML:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: higress-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1beta2
    kind: Deployment
    name: httpbin-deploy                  # Deployment to scale
  minReplicas: 1                          # Minimum pod count
  maxReplicas: 10                         # Maximum pod count
  metrics:
    - type: External                      # External metric (from SLS, not from K8s)
      external:
        metric:
          name: sls_ingress_qps           # QPS metric exposed by the metrics adapter
          selector:
            matchLabels:
              sls.project: "aliyun-product-data-xxxxxxxxxxxxx-cn-hangzhou"   # SLS project name
              sls.logstore: "nginx-ingress"                                  # SLS logstore name
              sls.ingress.route: "default-httpbin-svc-8080"                  # <namespace>-<service>-<port>
        target:
          type: AverageValue
          averageValue: 10                # Scale out when per-pod QPS exceeds 10

Replace the following placeholders with your actual values:

Step 6: Verify auto-scaling

Generate traffic against the gateway route and confirm that the HPA scales the backend Deployment.

1. Run a load test against the gateway route to generate sustained QPS above the configured threshold (10 in the example).

2. Check the HPA status:

   kubectl describe hpa higress-hpa

3. In the Events section of the output, look for SuccessfulRescale events: "above target" means the average per-pod QPS exceeded the averageValue threshold, which triggered scale-out.

   Normal  SuccessfulRescale  9m     horizontal-pod-autoscaler  New size: 3; reason: external metric sls_ingress_qps(...) above target
   Normal  SuccessfulRescale  8m45s  horizontal-pod-autoscaler  New size: 4; reason: external metric sls_ingress_qps(...) above target

4. Stop the load test. After the stabilization window (default: 300 seconds), the HPA scales the Deployment back down: "All metrics below target" means QPS dropped below the threshold and the HPA reduced the replica count.

   Normal  SuccessfulRescale  3m41s  horizontal-pod-autoscaler  New size: 3; reason: All metrics below target
   Normal  SuccessfulRescale  2m55s  horizontal-pod-autoscaler  New size: 1; reason: All metrics below target

Tune scaling behavior (optional)

By default, the HPA uses a 300-second stabilization window for scale-down and no stabilization window for scale-up. To customize this behavior -- for example, to prevent rapid scale-down after a traffic spike -- add a behavior block to the HPA spec:

spec:
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0       # Scale up immediately (default)
      policies:
      - type: Percent
        value: 100
        periodSeconds: 15
    scaleDown:
      stabilizationWindowSeconds: 300     # Wait 5 minutes before scaling down (default)
      policies:
      - type: Percent
        value: 100
        periodSeconds: 15

For more information about scaling policies, see Horizontal Pod Autoscaling in the Kubernetes documentation.