Use ASM to route traffic based on the locality - Alibaba Cloud Service Mesh

In large-scale business scenarios, thousands of services may call each other across different regions to implement required features. By default, Kubernetes implements load balancing in a round-robin manner. However, to achieve the best service performance, we recommend that you route traffic to the closest services and limit the traffic to the same region as much as possible. Service Mesh (ASM) is based on Istio and provides locality-based routing. You can use ASM to route traffic to pods that are closest to the originating pod. This reduces service latency and saves traffic fees because services call each other within the same region as much as possible. This topic shows you how to use ASM to route traffic based on the locality. You can use this feature to improve service performance and save costs.

Prerequisites

An ASM instance is created. For more information, see Create an ASM instance.
A managed multi-zone cluster is created in the Container Service for Kubernetes (ACK) console. Nodes in the ACK cluster reside in different zones of the same region. In this topic, Zone G and Zone H of the China (Beijing) region are used. For more information, see Create an ACK managed cluster.

Deploy a sample backend service

Deploy the version 1 and version 2 of the backend service that is named nginx in the ACK cluster. Configure node selectors to assign the pod of the version 1 to the node in Zone G and the pod of the version 2 to the node in Zone H, as described in the following steps:

Deploy the version 1 of the nginx backend service in the ACK cluster. The following YAML code provides an example:

apiVersion: v1
kind: ConfigMap
metadata:
  name: mynginx-configmap-v1
  namespace: backend
data:
  default.conf: |-
    server {
        listen       80;
        server_name  localhost;

        #charset koi8-r;
        #access_log  /var/log/nginx/host.access.log  main;

        location / {
            return 200 'v1\n';
        }
    }

Configure a node selector to assign the pod of the version 1 of the nginx backend service to the node in Zone G. The following YAML code provides an example:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-v1
  namespace: backend
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
      version: v1
  template:
    metadata:
      labels:
        app: nginx
        version: v1
    spec:
      containers:
      - image: docker.io/nginx:1.15.9
        imagePullPolicy: IfNotPresent
        name: nginx
        ports:
        - containerPort: 80
        volumeMounts:
        - name: nginx-config
          mountPath: /etc/nginx/conf.d
          readOnly: true
      volumes:
      - name: nginx-config
        configMap:
          name: mynginx-configmap-v1    
      nodeSelector:
        failure-domain.beta.kubernetes.io/zone: "cn-beijing-g"

Deploy the version 2 of the nginx backend service in the ACK cluster. The following YAML code provides an example:

apiVersion: v1
kind: ConfigMap
metadata:
  name: mynginx-configmap-v2
  namespace: backend
data:
  default.conf: |-
    server {
        listen       80;
        server_name  localhost;

        #charset koi8-r;
        #access_log  /var/log/nginx/host.access.log  main;

        location / {
            return 200 'v2\n';
        }
    }

Configure a node selector to assign the pod of the version 2 of the nginx backend service to the node in Zone H. The following YAML code provides an example:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-v2
  namespace: backend
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
      version: v2
  template:
    metadata:
      labels:
        app: nginx
        version: v2
    spec:
      containers:
      - image: docker.io/nginx:1.15.9
        imagePullPolicy: IfNotPresent
        name: nginx
        ports:
        - containerPort: 80
        volumeMounts:
        - name: nginx-config
          mountPath: /etc/nginx/conf.d
          readOnly: true
      volumes:
      - name: nginx-config
        configMap:
          name: mynginx-configmap-v2 
      nodeSelector:
        failure-domain.beta.kubernetes.io/zone: "cn-beijing-h"

Deploy the nginx backend service. The following YAML code provides an example:

apiVersion: v1
kind: Service
metadata:
  name: nginx
  namespace: backend
  labels:
    app: nginx
spec:
  ports:
  - name: http
    port: 8000
    targetPort: 80
  selector:
    app: nginx

Deploy a sample client service

Deploy the version 1 and version 2 of the client service that is named sleep in the ACK cluster. Configure node selectors to assign the pod of the version 1 to the node in Zone G and the pod of the version 2 to the node in Zone H, as described in the following steps:

Deploy the version 1 of the sleep client service in the ACK cluster and configure a node selector to assign the pod of the version 1 to the node in Zone G. The following YAML code provides an example:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sleep-cn-beijing-g
  namespace: backend
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sleep
      version: v1
  template:
    metadata:
      labels:
        app: sleep
        version: v1
    spec:
      containers:
      - name: sleep
        image: tutum/curl
        command: ["/bin/sleep","infinity"]
        imagePullPolicy: IfNotPresent
      nodeSelector:
        failure-domain.beta.kubernetes.io/zone: "cn-beijing-g"

Deploy the version 2 of the sleep client service in the ACK cluster and configure a node selector to assign the pod of the version 2 to the node in Zone H. The following YAML code provides an example:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sleep-cn-beijing-h
  namespace: backend
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sleep
      version: v2
  template:
    metadata:
      labels:
        app: sleep
        version: v2
    spec:
      containers:
      - name: sleep
        image: tutum/curl
        command: ["/bin/sleep","infinity"]
        imagePullPolicy: IfNotPresent
      nodeSelector:
        failure-domain.beta.kubernetes.io/zone: "cn-beijing-h"

Deploy the sample client service. The following YAML code provides an example:

apiVersion: v1
kind: Service
metadata:
  name: sleep
  namespace: backend
  labels:
    app: sleep
spec:
  ports:
  - name: http
    port: 80
  selector:
    app: sleep

Execute the following script to access the nginx backend service from the two pods of the sleep client service:

echo "entering into the 1st container"
export SLEEP_ZONE_1=$(kubectl get pods -lapp=sleep,version=v1 -n backend  -o 'jsonpath={.items[0].metadata.name}')
for i in {1..20}
do
  kubectl exec -it $SLEEP_ZONE_1 -c sleep -n backend -- sh -c 'curl  http://nginx.backend:8000'
done
echo "entering into the 2nd container"
export SLEEP_ZONE_2=$(kubectl get pods -lapp=sleep,version=v2 -n backend  -o 'jsonpath={.items[0].metadata.name}')
for i in {1..20}
do
  kubectl exec -it $SLEEP_ZONE_2 -c sleep -n backend -- sh -c 'curl  http://nginx.backend:8000'
done

Verify the result

The following execution result indicates that traffic is sent to the two pods of the nginx backend service in round-robin mode:

entering into the 1st container
v2
v1
v1
v2
v2
v1
v1
v2
v2
v1
v1
v2
v2
v1
v2
v1
v1
v2
v2
v1
entering into the 2nd container
v2
v2
v1
v1
v2
v1
v2
v1
v2
v2
v1
v1
v2
v2
v1
v2
v1
v2
v1
v1

Locality-prioritized load balancing

By default, the algorithm of locality-prioritized load balancing guides traffic from a client service pod to a backend service pod in the same zone. If no backend service pod in the same zone is available, the traffic is guided to a pod in another zone.

To enable locality-prioritized load balancing, you must configure a virtual service and a destination rule with an outlier policy. The outlier policy checks the health status of pods and makes decisions on routing.

Log on to the ASM console.
In the left-side navigation pane, choose Service Mesh > Mesh Management.
On the Mesh Management page, find the ASM instance that you want to configure. Click the name of the ASM instance or click Manage in the Actions column.
On the details page of the ASM instance, choose Traffic Management Center > VirtualService in the left-side navigation pane. On the page that appears, click Create from YAML.
On the Create page, perform the following steps to define a virtual service. Then, click Create.
1. Select a namespace as required. In this example, the backend namespace is selected.
2. Copy the following YAML code to the code editor to define a virtual service:
```
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginx
  namespace: backend
spec:
  hosts:
    - nginx
  http:
  - route:
    - destination:
        host: nginx
```
On the details page of the ASM instance, choose Traffic Management Center > DestinationRule in the left-side navigation pane. On the page that appears, click Create from YAML.
On the Create page, perform the following steps to define a destination rule. Then, click Create.
1. Select a namespace as required. In this example, the backend namespace is selected.
2. Copy the following YAML code to the code editor to define a destination rule:
```
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: nginx
  namespace: backend
spec:
  host: nginx
  trafficPolicy:
    outlierDetection:
      consecutiveErrors: 7
      interval: 30s
      baseEjectionTime: 30s
```

Execute the following script to access the nginx backend service from the two pods of the sleep client service:

echo "entering into the 1st container"
export SLEEP_ZONE_1=$(kubectl get pods -lapp=sleep,version=v1 -n backend  -o 'jsonpath={.items[0].metadata.name}')
for i in {1..20}
do
  kubectl exec -it $SLEEP_ZONE_1 -c sleep -n backend -- sh -c 'curl  http://nginx.backend:8000'
done
echo "entering into the 2nd container"
export SLEEP_ZONE_2=$(kubectl get pods -lapp=sleep,version=v2 -n backend  -o 'jsonpath={.items[0].metadata.name}')
for i in {1..20}
do
  kubectl exec -it $SLEEP_ZONE_2 -c sleep -n backend -- sh -c 'curl  http://nginx.backend:8000'
done

Verify the result

The following execution result indicates that traffic is sent to the two pods of the nginx backend service in locality-prioritized mode:

entering into the 1st container
v1
v1
v1
v1
v1
v1
v1
v2
v1
v1
v2
v1
v1
v1
v1
v1
v1
v1
v1
v1
entering into the 2nd container
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2

Locality-weighted load balancing

Most use cases work well with locality-prioritized load balancing. However, sometimes you may want to split traffic and send the traffic to multiple zones. For example, if all traffic comes from a zone, you can split traffic to prevent the zone from being overloaded. In this case, you can use locality-weighted load balancing.

In this example, the following rules are applied:

Route 80% of traffic from China (Beijing) Zone G to China (Beijing) Zone G and 20% to China (Beijing) Zone H.
Route 20% of traffic from China (Beijing) Zone H to China (Beijing) Zone G and 80% to China (Beijing) Zone H.

Log on to the ASM console.
In the left-side navigation pane, choose Service Mesh > Mesh Management.
On the Mesh Management page, find the ASM instance that you want to configure. Click the name of the ASM instance or click Manage in the Actions column.
On the details page of the ASM instance, choose Traffic Management Center > VirtualService in the left-side navigation pane. On the page that appears, click Create from YAML.
On the Create page, perform the following steps to define a virtual service. Then, click Create.
1. Select a namespace as required. In this example, the backend namespace is selected.
2. Copy the following YAML code to the code editor to define a virtual service:
```
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginx
  namespace: backend
spec:
  hosts:
    - nginx
  http:
  - route:
    - destination:
        host: nginx
```
On the details page of the ASM instance, choose Traffic Management Center > DestinationRule in the left-side navigation pane. On the page that appears, click Create from YAML.

On the Create page, perform the following steps to define a destination rule. Then, click Create.

Select a namespace as required. In this example, the backend namespace is selected.

Copy the following YAML code to the code editor to define a destination rule:

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: nginx
  namespace: backend
spec:
  host: nginx
  trafficPolicy:
    outlierDetection:
      consecutiveErrors: 7
      interval: 30s
      baseEjectionTime: 30s
    loadBalancer:
      localityLbSetting:
        enabled: true
        distribute:
        - from: cn-beijing/cn-beijing-g/*
          to:
            "cn-beijing/cn-beijing-g/*": 80
            "cn-beijing/cn-beijing-h/*": 20
        - from: cn-beijing/cn-beijing-h/*
          to:
            "cn-beijing/cn-beijing-g/*": 20
            "cn-beijing/cn-beijing-h/*": 80

Execute the following script to access the nginx backend service from the two pods of the sleep client service:

echo "entering into the 1st container"
export SLEEP_ZONE_1=$(kubectl get pods -lapp=sleep,version=v1 -n backend  -o 'jsonpath={.items[0].metadata.name}')
for i in {1..20}
do
  kubectl exec -it $SLEEP_ZONE_1 -c sleep -n backend -- sh -c 'curl  http://nginx.backend:8000'
done
echo "entering into the 2nd container"
export SLEEP_ZONE_2=$(kubectl get pods -lapp=sleep,version=v2 -n backend  -o 'jsonpath={.items[0].metadata.name}')
for i in {1..20}
do
  kubectl exec -it $SLEEP_ZONE_2 -c sleep -n backend -- sh -c 'curl  http://nginx.backend:8000'
done

Verify the result

The following execution result indicates that traffic is sent to the two pods of the nginx backend service in locality-weighted mode:

entering into the 1st container
v1
v1
v1
v1
v2
v1
v1
v2
v1
v1
v1
v2
v1
v1
v1
v1
v1
v2
v1
v1
entering into the 2nd container
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2
v1
v2
v1
v2
v2