Customize Waypoint resource usage and HPA - Alibaba Cloud Service Mesh

When Waypoint proxies handle high L7 traffic volumes or require graceful shutdown for long-lived connections, the default resource allocations may not be sufficient. Service Mesh (ASM) enables you to customize CPU and memory requests, Horizontal Pod Autoscaler (HPA) scaling boundaries, and proxy drain behavior for Waypoint proxies deployed through the Gateway API in Ambient mode.

Prerequisites

Before you begin, make sure that you have:

Version 1.2.1 or later of the Gateway API component installed in your cluster

How it works

To customize a Waypoint, define the target settings in a ConfigMap and reference it from the Gateway resource. ASM reads the ConfigMap and merges your overrides with the default resource configuration using a Strategic Merge Patch. Your ConfigMap only needs to contain the fields you want to change -- ASM handles the merge and creates or updates the Waypoint Deployment and HPA accordingly.

The ConfigMap supports three configuration sections:

Section	Purpose
`data.horizontalPodAutoscaler`	HPA settings such as minimum and maximum replica counts
`data.deployment`	Pod-level settings such as container resource requests and graceful shutdown timing
`data.proxyConfig`	Istio proxy runtime settings such as drain duration and termination behavior

Create a ConfigMap with custom settings

The ConfigMap must be in the same namespace as the Gateway resource it configures.

Save the following content as gw-options.yaml.

apiVersion: v1
kind: ConfigMap
metadata:
  name: gw-options
data:
  horizontalPodAutoscaler: |
    spec:
      minReplicas: 2
      maxReplicas: 2
  deployment: |
    spec:
      # Do not configure replicas when HPA is enabled.
      # replicas: 4
      template:
        spec:
          containers:
          - name: istio-proxy
            resources:
              requests:
                cpu: 1000m
          # The maximum wait time before a pod goes offline.
          terminationGracePeriodSeconds: 120
  proxyConfig: |
    # The proxy drain duration.
    drainDuration: 30s
    # The maximum duration for the proxy to stop draining. Ensure that terminationGracePeriodSeconds > terminationDrainDuration > drainDuration.
    terminationDrainDuration: 60s

Apply the ConfigMap.
```
kubectl apply -f gw-options.yaml
```

ConfigMap parameter reference

HPA parameters (`data.horizontalPodAutoscaler`)

Parameter	Description
`spec.minReplicas`	Minimum number of Waypoint pod replicas. In the preceding example, set to `2`.
`spec.maxReplicas`	Maximum number of Waypoint pod replicas. In the preceding example, set to `2` (fixed replica count).

Important

Do not set replicas in the Deployment section when HPA is enabled. The HPA controller manages replica counts automatically, and a static replicas value conflicts with HPA scaling decisions.

Deployment parameters (`data.deployment`)

Parameter	Description
`spec.template.spec.containers[].name`	Container name. Set to `istio-proxy` for the Waypoint proxy container.
`spec.template.spec.containers[].resources.requests.cpu`	CPU request for the Waypoint proxy. In the preceding example, set to `1000m` (1 vCPU). Increase this value for namespaces with high L7 traffic.
`spec.template.spec.terminationGracePeriodSeconds`	Maximum time (in seconds) that Kubernetes waits before forcefully terminating a pod during shutdown. In the preceding example, set to `120`.

ProxyConfig parameters (`data.proxyConfig`)

Parameter	Description
`drainDuration`	Duration that the Envoy proxy drains active connections during shutdown. In the preceding example, set to `30s`.
`terminationDrainDuration`	Maximum duration the proxy waits after initiating graceful shutdown before terminating remaining connections. In the preceding example, set to `60s`.

Important

The three timing parameters must satisfy this ordering constraint: terminationGracePeriodSeconds > terminationDrainDuration > drainDuration. If this constraint is violated, connections may be terminated before the proxy finishes draining, causing request failures.

Link the ConfigMap to the Gateway resource

After creating the ConfigMap, reference it from the Gateway resource through the infrastructure.parametersRef field.

Save the following content as gateway.yaml. The infrastructure.parametersRef block references the gw-options ConfigMap. ASM reads this reference and applies the ConfigMap values to the Waypoint Deployment and HPA.

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: waypoint
  namespace: default
spec:
  infrastructure:
    parametersRef:
      group: ""
      kind: ConfigMap
      name: gw-options
  gatewayClassName: istio-waypoint
  listeners:
  - allowedRoutes:
      namespaces:
        from: Same
    name: mesh
    port: 15008
    protocol: HBONE

Apply the Gateway resource.
```
kubectl apply -f gateway.yaml
```

Verify the configuration

After applying both resources, verify that ASM merged the ConfigMap settings into the Waypoint Deployment and HPA.

Check the Waypoint Deployment. In the output, confirm that:
- The istio-proxy container has cpu: 1000m under resources.requests.
- terminationGracePeriodSeconds is 120.
```
kubectl get deployment waypoint -n default -o yaml
```
Check the HPA. Confirm that MINPODS is 2 and MAXPODS is 2 for the Waypoint HPA.
```
kubectl get hpa -n default
```