Improve availability for the ingress gateway of an ASM instance - Alibaba Cloud Service Mesh

Ingress gateways are the traffic entry point for services managed by Service Mesh (ASM). If all gateway pods run on the same node or in the same zone, a single failure can make your services unreachable. Distribute gateway pods across nodes or availability zones to eliminate this single point of failure.

The scheduling mechanism depends on your cluster type:

Cluster type	Scheduling mechanism	Why
ACK cluster	Pod anti-affinity (`podAntiAffinity`)	Spreads pods across nodes or zones using Kubernetes-native scheduling
ACK Serverless cluster	ECI scheduling annotations	Pod anti-affinity is not supported on ACK Serverless. ECI annotations distribute pods across zones through vSwitch selection

Prerequisites

Before you begin, ensure that you have:

An ASM instance is created. For more information, see Create an ASM instance.
A Container Service for Kubernetes (ACK) cluster or an ACK Serverless cluster. For more information, see Create an ACK managed cluster or Create an ACK Serverless cluster
The cluster is added to the ASM instance.

Distribute gateway pods in an ACK cluster

Add a podAntiAffinity rule to the IstioGateway YAML to prevent the scheduler from placing multiple gateway pods on the same node or in the same zone.

The examples in this section use preferredDuringSchedulingIgnoredDuringExecution (soft anti-affinity). With soft anti-affinity, the scheduler tries to honor the constraint but may place pods on the same node or zone when no other nodes are available.

In all examples, replace <your-cluster-id> with the ID of the ACK cluster added to the ASM instance (for example, c954ee9df88f64f229591f0ea4c61****).

Spread pods across nodes

Set topologyKey to kubernetes.io/hostname to place each gateway pod on a different node.

apiVersion: istio.alibabacloud.com/v1beta1
kind: IstioGateway
metadata:
  name: ingressgateway-1
  namespace: istio-system
spec:
  clusterIds:
    - "<your-cluster-id>"
  cpu:
    targetAverageUtilization: 80
  externalTrafficPolicy: Local
  maxReplicas: 4
  minReplicas: 2
  ports:
  - name: status-port
    port: 15020
    targetPort: 15020
  - name: http2
    port: 80
    targetPort: 80
  - name: https
    port: 443
    targetPort: 80
  - name: tls
    port: 15443
    targetPort: 15443
  replicaCount: 1
  resources:
    limits:
      cpu: '2'
      memory: 2G
    requests:
      cpu: 200m
      memory: 256Mi
  sds:
    enabled: true
    resources:
      requests:
        cpu: 100m
        memory: 128Mi
      limits:
        cpu: 2000m
        memory: 1024Mi
  serviceType: LoadBalancer
  affinity:
    podAntiAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
      - podAffinityTerm:
          labelSelector:
            matchExpressions:
            - key: app
              operator: In
              values:
              - istio-ingressgateway-1
          topologyKey: kubernetes.io/hostname
        weight: 100
  rollingMaxSurge: "100%"
  rollingMaxUnavailable: "25%"

Key fields in the affinity section:

Field	Value	Description
`topologyKey`	`kubernetes.io/hostname`	Anti-affinity scope: individual nodes. Each node runs at most one gateway pod with the label `app=istio-ingressgateway-1`.
`matchExpressions`	`key: app`, `operator: In`, `values: istio-ingressgateway-1`	Selects existing pods to apply the anti-affinity rule against. The scheduler avoids placing a new pod on a node that already runs a pod with this label.
`weight`	`100`	Priority weight for the soft anti-affinity rule (range: 1--100). A value of 100 gives this rule the highest priority.

Spread pods across zones

Set topologyKey to topology.kubernetes.io/zone to place each gateway pod in a different availability zone. This protects against zone-level failures.

To switch from node-level to zone-level spreading, change the topologyKey value in the affinity section of the preceding YAML:

  affinity:
    podAntiAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
      - podAffinityTerm:
          labelSelector:
            matchExpressions:
            - key: app
              operator: In
              values:
              - istio-ingressgateway-1
          topologyKey: topology.kubernetes.io/zone
        weight: 100

The following table compares the two topologyKey values:

topologyKey value	Anti-affinity scope	Use case
`kubernetes.io/hostname`	Node	Protects against single-node failures. Suitable when all nodes are in one zone.
`topology.kubernetes.io/zone`	Availability zone	Protects against zone-level failures. Requires nodes in multiple zones.

Distribute gateway pods in an ACK Serverless cluster

ACK Serverless clusters run pods as Elastic Container Instances (ECIs) and do not support the podAntiAffinity scheduling policy. Instead, distribute gateway pods across zones using ECI-specific annotations.

Configure multiple zones in your ACK Serverless cluster. Each zone must have a vSwitch in the cluster's Virtual Private Cloud (VPC). For more information, see Create ECIs across zones.

Add podAnnotations to the IstioGateway YAML to specify the vSwitches and scheduling strategy. Replace the following placeholders: Key annotations:

Placeholder	Description	Example
`<your-cluster-id>`	ID of the ACK Serverless cluster added to the ASM instance	`c954ee9df88f64f229591f0ea4c61****`
`<vswitch-id-1>,<vswitch-id-2>`	Comma-separated vSwitch IDs, each in a different zone within the cluster's VPC	`vsw-bp1b07j0miob3khtn**,vsw-bp12b85hh323se8ft**`

Annotation	Value	Description
`k8s.aliyun.com/eci-vswitch`	Comma-separated vSwitch IDs	Zones where ECI pods can be created. Each vSwitch maps to a zone.
`k8s.aliyun.com/eci-schedule-strategy`	`VSwitchRandom`	Distributes ECI pods randomly across the specified zones to balance load.

   apiVersion: istio.alibabacloud.com/v1beta1
   kind: IstioGateway
   metadata:
     name: ingressgateway
     namespace: istio-system
   spec:
     clusterIds:
       - "<your-cluster-id>"
     cpu:
       targetAverageUtilization: 80
     externalTrafficPolicy: Local
     maxReplicas: 4
     minReplicas: 2
     ports:
     - name: status-port
       port: 15020
       targetPort: 15020
     - name: http2
       port: 80
       targetPort: 80
     - name: https
       port: 443
       targetPort: 80
     - name: tls
       port: 15443
       targetPort: 15443
     replicaCount: 1
     resources:
       limits:
         cpu: '2'
         memory: 2G
       requests:
         cpu: 200m
         memory: 256Mi
     sds:
       enabled: true
       resources:
         requests:
           cpu: 100m
           memory: 128Mi
         limits:
           cpu: 2000m
           memory: 1024Mi
     serviceType: LoadBalancer
     podAnnotations:
       k8s.aliyun.com/eci-vswitch: "<vswitch-id-1>,<vswitch-id-2>"
       k8s.aliyun.com/eci-schedule-strategy: "VSwitchRandom"
     rollingMaxSurge: "100%"
     rollingMaxUnavailable: "25%"

Rolling update parameters

All examples in this topic include rollingMaxSurge and rollingMaxUnavailable to control how pods are replaced during updates. These values follow a "scale out before scale in" strategy: new gateway pods are fully running before old ones are removed, minimizing downtime risk during updates.

Parameter	Value	Description
`rollingMaxSurge`	`100%`	During an update, up to 100% additional pods can be created beyond the desired count. New pods start before old pods terminate.
`rollingMaxUnavailable`	`25%`	At most 25% of pods can be unavailable during an update. This maintains capacity while the rollout progresses.