In large-scale business scenarios, thousands of services may call each other across different regions to implement required features. By default, Kubernetes implements load balancing in a round-robin manner. However, to achieve the best service performance, we recommend that you route traffic to the closest services and limit the traffic to the same region as much as possible. Alibaba Cloud Service Mesh (ASM) is based on Istio and provides locality-based routing. You can use ASM to route traffic to pods that are closest to the originating pod. This reduces service latency and saves traffic fees because services call each other within the same region as much as possible. This topic describes how to use ASM to route traffic based on the locality. You can use this feature to improve service performance and save costs.

Prerequisites

  • Create an ASM instance.
  • A managed multi-zone cluster is created in the Container Service for Kubernetes (ACK) console. Nodes in the ACK cluster reside in different zones of the same region. In this topic, Zone G and Zone H of the China (Beijing) region are used. For more information, see Create a managed Kubernetes cluster.

Deploy a sample backend service

Deploy the version 1 and version 2 of the backend service that is named nginx in the ACK cluster. Configure node selectors to assign the pod of the version 1 to the node in Zone G and the pod of the version 2 to the node in Zone H, as described in the following steps:

  1. Deploy the version 1 of the nginx backend service in the ACK cluster. The following YAML code provides an example:
    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: mynginx-configmap-v1
      namespace: backend
    data:
      default.conf: |-
        server {
            listen       80;
            server_name  localhost;
    
            #charset koi8-r;
            #access_log  /var/log/nginx/host.access.log  main;
    
            location / {
                return 200 'v1\n';
            }
        } 
  2. Configure a node selector to assign the pod of the version 1 of the nginx backend service to the node in Zone G. The following YAML code provides an example:
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: nginx-v1
      namespace: backend
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: nginx
          version: v1
      template:
        metadata:
          labels:
            app: nginx
            version: v1
        spec:
          containers:
          - image: docker.io/nginx:1.15.9
            imagePullPolicy: IfNotPresent
            name: nginx
            ports:
            - containerPort: 80
            volumeMounts:
            - name: nginx-config
              mountPath: /etc/nginx/conf.d
              readOnly: true
          volumes:
          - name: nginx-config
            configMap:
              name: mynginx-configmap-v1    
          nodeSelector:
            failure-domain.beta.kubernetes.io/zone: "cn-beijing-g"
  3. Deploy the version 2 of the nginx backend service in the ACK cluster. The following YAML code provides an example:
    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: mynginx-configmap-v2
      namespace: backend
    data:
      default.conf: |-
        server {
            listen       80;
            server_name  localhost;
    
            #charset koi8-r;
            #access_log  /var/log/nginx/host.access.log  main;
    
            location / {
                return 200 'v2\n';
            }
        } 
  4. Configure a node selector to assign the pod of the version 2 of the nginx backend service to the node in Zone H. The following YAML code provides an example:
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: nginx-v2
      namespace: backend
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: nginx
          version: v2
      template:
        metadata:
          labels:
            app: nginx
            version: v2
        spec:
          containers:
          - image: docker.io/nginx:1.15.9
            imagePullPolicy: IfNotPresent
            name: nginx
            ports:
            - containerPort: 80
            volumeMounts:
            - name: nginx-config
              mountPath: /etc/nginx/conf.d
              readOnly: true
          volumes:
          - name: nginx-config
            configMap:
              name: mynginx-configmap-v2 
          nodeSelector:
            failure-domain.beta.kubernetes.io/zone: "cn-beijing-h"     
  5. Deploy the nginx backend service. The following YAML code provides an example:
    apiVersion: v1
    kind: Service
    metadata:
      name: nginx
      namespace: backend
      labels:
        app: nginx
    spec:
      ports:
      - name: http
        port: 8000
        targetPort: 80
      selector:
        app: nginx

Deploy a sample client service

Deploy the version 1 and version 2 of the client service that is named sleep in the ACK cluster. Configure node selectors to assign the pod of the version 1 to the node in Zone G and the pod of the version 2 to the node in Zone H, as described in the following steps:

  1. Deploy the version 1 of the sleep client service in the ACK cluster and configure a node selector to assign the pod of the version 1 to the node in Zone G. The following YAML code provides an example:
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: sleep-cn-beijing-g
      namespace: backend
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: sleep
          version: v1
      template:
        metadata:
          labels:
            app: sleep
            version: v1
        spec:
          containers:
          - name: sleep
            image: tutum/curl
            command: ["/bin/sleep","infinity"]
            imagePullPolicy: IfNotPresent
          nodeSelector:
            failure-domain.beta.kubernetes.io/zone: "cn-beijing-g"  
  2. Deploy the version 2 of the sleep client service in the ACK cluster and configure a node selector to assign the pod of the version 2 to the node in Zone H. The following YAML code provides an example:
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: sleep-cn-beijing-h
      namespace: backend
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: sleep
          version: v2
      template:
        metadata:
          labels:
            app: sleep
            version: v2
        spec:
          containers:
          - name: sleep
            image: tutum/curl
            command: ["/bin/sleep","infinity"]
            imagePullPolicy: IfNotPresent
          nodeSelector:
            failure-domain.beta.kubernetes.io/zone: "cn-beijing-h" 
  3. Deploy the sample client service. The following YAML code provides an example:
    apiVersion: v1
    kind: Service
    metadata:
      name: sleep
      namespace: backend
      labels:
        app: sleep
    spec:
      ports:
      - name: http
        port: 80
      selector:
        app: sleep
  4. Execute the following script to access the nginx backend service from the two pods of the sleep client service:
    echo "entering into the 1st container"
    export SLEEP_ZONE_1=$(kubectl get pods -lapp=sleep,version=v1 -n backend  -o 'jsonpath={.items[0].metadata.name}')
    for i in {1..20}
    do
      kubectl exec -it $SLEEP_ZONE_1 -c sleep -n backend -- sh -c 'curl  http://nginx.backend:8000'
    done
    echo "entering into the 2nd container"
    export SLEEP_ZONE_2=$(kubectl get pods -lapp=sleep,version=v2 -n backend  -o 'jsonpath={.items[0].metadata.name}')
    for i in {1..20}
    do
      kubectl exec -it $SLEEP_ZONE_2 -c sleep -n backend -- sh -c 'curl  http://nginx.backend:8000'
    done
    Execution result

    The following execution result indicates that traffic is sent to the two pods of the nginx backend service in round-robin mode:

    entering into the 1st container
    v2
    v1
    v1
    v2
    v2
    v1
    v1
    v2
    v2
    v1
    v1
    v2
    v2
    v1
    v2
    v1
    v1
    v2
    v2
    v1
    entering into the 2nd container
    v2
    v2
    v1
    v1
    v2
    v1
    v2
    v1
    v2
    v2
    v1
    v1
    v2
    v2
    v1
    v2
    v1
    v2
    v1
    v1

Locality-prioritized load balancing

By default, the algorithm of locality-prioritized load balancing guides traffic from a client service pod to a backend service pod in the same zone. If no backend service pod in the same zone is available, the traffic is guided to a pod in another zone.

To enable locality-prioritized load balancing, you must configure a virtual service and a destination rule with an outlier policy. The outlier policy checks the health status of pods and makes decisions on routing.

  1. Log on to the ASM console.
  2. In the left-side navigation pane, choose Service Mesh > Mesh Management.
  3. On the Mesh Management page, find the ASM instance that you want to configure. Click the name of the ASM instance or click Manage in the Actions column of the ASM instance.
  4. On the details page of the ASM instance, choose Traffic Management > VirtualService in the left-side navigation pane. On the VirtualService page, click Create.
  5. In the Create panel, perform the following steps to define a virtual service and click OK.
    1. Select a namespace as required. In this example, the backend namespace is selected.
    2. Copy the following YAML code to the code editor to define a virtual service:
      apiVersion: networking.istio.io/v1alpha3
      kind: VirtualService
      metadata:
        name: nginx
        namespace: backend
      spec:
        hosts:
          - nginx
        http:
        - route:
          - destination:
              host: nginx
  6. On the details page of the ASM instance, choose Traffic Management > DestinationRule in the left-side navigation pane. On the DestinationRule page, click Create.
  7. Choose Traffic Management > DestinationRule in the left-side navigation pane. On the DestinationRule page, click Create. In the Create panel, perform the following steps to define a destination rule and click OK.
    1. Select a namespace as required. In this example, the backend namespace is selected.
    2. Copy the following YAML code to the code editor to define a destination rule:
      apiVersion: networking.istio.io/v1alpha3
      kind: DestinationRule
      metadata:
        name: nginx
        namespace: backend
      spec:
        host: nginx
        trafficPolicy:
          outlierDetection:
            consecutiveErrors: 7
            interval: 30s
            baseEjectionTime: 30s
  8. Execute the following script to access the nginx backend service from the two pods of the sleep client service:
    echo "entering into the 1st container"
    export SLEEP_ZONE_1=$(kubectl get pods -lapp=sleep,version=v1 -n backend  -o 'jsonpath={.items[0].metadata.name}')
    for i in {1..20}
    do
      kubectl exec -it $SLEEP_ZONE_1 -c sleep -n backend -- sh -c 'curl  http://nginx.backend:8000'
    done
    echo "entering into the 2nd container"
    export SLEEP_ZONE_2=$(kubectl get pods -lapp=sleep,version=v2 -n backend  -o 'jsonpath={.items[0].metadata.name}')
    for i in {1..20}
    do
      kubectl exec -it $SLEEP_ZONE_2 -c sleep -n backend -- sh -c 'curl  http://nginx.backend:8000'
    done
    Execution result

    The following execution result indicates that traffic is sent to the two pods of the nginx backend service in locality-prioritized mode:

    entering into the 1st container
    v1
    v1
    v1
    v1
    v1
    v1
    v1
    v2
    v1
    v1
    v2
    v1
    v1
    v1
    v1
    v1
    v1
    v1
    v1
    v1
    entering into the 2nd container
    v2
    v2
    v2
    v2
    v2
    v2
    v2
    v2
    v2
    v2
    v2
    v2
    v2
    v2
    v2
    v2
    v2
    v2
    v2
    v2

Locality-weighted load balancing

Most use cases work well with locality-prioritized load balancing. However, sometimes you may want to split traffic and send the traffic to multiple zones. For example, if all traffic comes from a zone, you can split traffic to prevent the zone from being overloaded. In this case, you can use locality-weighted load balancing.

In this example, the following rules are applied:

  • Route 80% of traffic from China (Beijing) Zone G to China (Beijing) Zone G and 20% to China (Beijing) Zone H.
  • Route 20% of traffic from China (Beijing) Zone H to China (Beijing) Zone G and 80% to China (Beijing) Zone H.
  1. Log on to the ASM console.
  2. In the left-side navigation pane, choose Service Mesh > Mesh Management.
  3. On the Mesh Management page, find the ASM instance that you want to configure. Click the name of the ASM instance or click Manage in the Actions column of the ASM instance.
  4. On the details page of the ASM instance, choose Traffic Management > VirtualService in the left-side navigation pane. On the VirtualService page, click Create.
  5. In the Create panel, perform the following steps to define a virtual service and click OK.
    1. Select a namespace as required. In this example, the backend namespace is selected.
    2. Copy the following YAML code to the code editor to define a virtual service:
      apiVersion: networking.istio.io/v1alpha3
      kind: VirtualService
      metadata:
        name: nginx
        namespace: backend
      spec:
        hosts:
          - nginx
        http:
        - route:
          - destination:
              host: nginx
  6. On the details page of the ASM instance, choose Traffic Management > DestinationRule in the left-side navigation pane. On the DestinationRule page, click Create.
  7. In the Create panel, perform the following steps to define a destination rule and click OK.
    1. Select a namespace as required. In this example, the backend namespace is selected.
    2. Copy the following YAML code to the code editor to define a destination rule:
      apiVersion: networking.istio.io/v1alpha3
      kind: DestinationRule
      metadata:
        name: nginx
        namespace: backend
      spec:
        host: nginx
        trafficPolicy:
          outlierDetection:
            consecutiveErrors: 7
            interval: 30s
            baseEjectionTime: 30s
          loadBalancer:
            localityLbSetting:
              enabled: true
              distribute:
              - from: cn-beijing/cn-beijing-g/*
                to:
                  "cn-beijing/cn-beijing-g/*": 80
                  "cn-beijing/cn-beijing-h/*": 20
              - from: cn-beijing/cn-beijing-h/*
                to:
                  "cn-beijing/cn-beijing-g/*": 20
                  "cn-beijing/cn-beijing-h/*": 80
  8. Execute the following script to access the nginx backend service from the two pods of the sleep client service:
    echo "entering into the 1st container"
    export SLEEP_ZONE_1=$(kubectl get pods -lapp=sleep,version=v1 -n backend  -o 'jsonpath={.items[0].metadata.name}')
    for i in {1..20}
    do
      kubectl exec -it $SLEEP_ZONE_1 -c sleep -n backend -- sh -c 'curl  http://nginx.backend:8000'
    done
    echo "entering into the 2nd container"
    export SLEEP_ZONE_2=$(kubectl get pods -lapp=sleep,version=v2 -n backend  -o 'jsonpath={.items[0].metadata.name}')
    for i in {1..20}
    do
      kubectl exec -it $SLEEP_ZONE_2 -c sleep -n backend -- sh -c 'curl  http://nginx.backend:8000'
    done
    Execution result
    The following execution result indicates that traffic is sent to the two pods of the nginx backend service in locality-weighted mode:
    entering into the 1st container
    v1
    v1
    v1
    v1
    v2
    v1
    v1
    v2
    v1
    v1
    v1
    v2
    v1
    v1
    v1
    v1
    v1
    v2
    v1
    v1
    entering into the 2nd container
    v2
    v2
    v2
    v2
    v2
    v2
    v2
    v2
    v2
    v2
    v2
    v2
    v2
    v2
    v2
    v1
    v2
    v1
    v2
    v2