Container Service for Kubernetes:Implement zone-based pod spreading and affinity scheduling for virtual nodes

Example 1: Spread pods evenly across zones using topology spread constraints

The topologySpreadConstraints field controls the maximum difference in pod count between zones (maxSkew). With maxSkew: 1 across three zones, no zone can have more than one extra pod compared to any other zone.

The field supports these parameters:

topologySpreadConstraints:
  - maxSkew: <integer>
    minDomains: <integer>               # Optional. Beta in Kubernetes 1.25 and later.
    topologyKey: <string>
    whenUnsatisfiable: <string>
    labelSelector: <object>
    matchLabelKeys: <list>              # Optional. Beta in Kubernetes 1.27 and later.
    nodeAffinityPolicy: [Honor|Ignore]  # Optional. Beta in Kubernetes 1.26 and later.
    nodeTaintsPolicy: [Honor|Ignore]    # Optional. Beta in Kubernetes 1.26 and later.

For parameter details, see topologySpreadConstraints field.

The following Deployment spreads 10 replicas evenly across available zones. whenUnsatisfiable: DoNotSchedule prevents scheduling a pod if placing it would violate the maxSkew: 1 constraint.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: with-pod-topology-spread
  labels:
    app: with-pod-topology-spread
spec:
  replicas: 10
  selector:
    matchLabels:
      app: with-pod-topology-spread
  template:
    metadata:
      labels:
        app: with-pod-topology-spread
    spec:
      topologySpreadConstraints:
        - maxSkew: 1
          topologyKey: topology.kubernetes.io/zone
          whenUnsatisfiable: DoNotSchedule
          labelSelector:
            matchLabels:
              app: with-pod-topology-spread
      containers:
      - name: with-pod-topology-spread
        image: registry.k8s.io/pause:2.0
        resources:
          requests:
            cpu: "1"
            memory: "256Mi"

Deploy and verify

1. Save the YAML to deployment.yaml, then apply it:

   kubectl apply -f deployment.yaml

2. Query the nodes on which the pods are running:

   kubectl get po -lapp=with-pod-topology-spread -ocustom-columns=NAME:.metadata.name,NODE:.spec.nodeName --no-headers | grep -v "<none>"

3. Count pods per zone to confirm even distribution:

   kubectl get po -lapp=with-pod-topology-spread -ocustom-columns=NODE:.spec.nodeName --no-headers | grep -v "<none>" | xargs -I {} kubectl get no {} -ojson | jq '.metadata.labels["topology.kubernetes.io/zone"]' | sort | uniq -c

Example 2: Pin pods to a specific zone using affinities

Use podAffinity to co-locate pods in the same zone as existing matching pods, or nodeAffinity to pin pods to a zone by node label.

Option A: Co-locate pods in one zone using pod affinity

The following Deployment uses podAffinity to schedule all replicas to the same zone as existing pods with the label app: with-affinity.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: with-affinity
  labels:
    app: with-affinity
spec:
  replicas: 3
  selector:
    matchLabels:
      app: with-affinity
  template:
    metadata:
      labels:
        app: with-affinity
    spec:
      affinity:
        podAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchExpressions:
              - key: app
                operator: In
                values:
                - with-affinity
            topologyKey: topology.kubernetes.io/zone
      containers:
      - name: with-affinity
        image: registry.k8s.io/pause:2.0

Option B: Pin pods to a specific zone using node affinity

To deploy pods exclusively in a specific zone, replace podAffinity with nodeAffinity. The following configuration pins pods to Beijing Zone A (cn-beijing-a).

apiVersion: apps/v1
kind: Deployment
metadata:
  name: with-affinity
  labels:
    app: with-affinity
spec:
  replicas: 3
  selector:
    matchLabels:
      app: with-affinity
  template:
    metadata:
      labels:
        app: with-affinity
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: topology.kubernetes.io/zone
                operator: In
                values:
                - cn-beijing-a
      containers:
      - name: with-affinity
        image: registry.k8s.io/pause:2.0

Deploy and verify

1. Save the YAML to deployment.yaml, then apply it:

   kubectl apply -f deployment.yaml

2. Query the nodes on which the pods are running:

   kubectl get po -lapp=with-affinity -ocustom-columns=NAME:.metadata.name,NODE:.spec.nodeName --no-headers | grep -v "<none>"

3. Confirm that all pods land in the expected zone:

   kubectl get po -lapp=with-affinity -ocustom-columns=NODE:.spec.nodeName --no-headers | grep -v "<none>" | xargs -I {} kubectl get no {} -ojson | jq '.metadata.labels["topology.kubernetes.io/zone"]' | sort | uniq -c