Use ASM to route traffic based on the locality - Alibaba Cloud Service Mesh

In large-scale business scenarios, thousands of services may call each other across different regions to implement required features. By default, Kubernetes implements load balancing in a round-robin manner. However, to achieve the best service performance, we recommend that you route traffic to the closest services and limit the traffic to the same region as much as possible. Service Mesh (ASM) is based on Istio and provides locality-based routing. You can use ASM to route traffic to the pods that are closest to the originating pod. This reduces service latency and saves traffic fees because services call each other within the same region as much as possible. This topic describes how to use ASM to route traffic based on the locality. You can use this feature to improve service performance and save costs.

Prerequisites

An ASM instance is created. For more information, see Create an ASM instance.
A managed multi-zone cluster is created in the Container Service for Kubernetes (ACK) console. Nodes in the ACK cluster reside in different zones of the same region. In this topic, Zone G and Zone H of the China (Beijing) region are used. For more information, see Create an ACK managed cluster.

Deploy a sample backend service

Deploy version 1 and version 2 of the backend service that is named nginx in the ACK cluster. Configure node selectors to assign the pod of version 1 to the node in Zone G and the pod of version 2 to the node in Zone H, as described in the following steps:

Deploy version 1 of the nginx backend service in the ACK cluster. The following YAML code provides an example:

apiVersion: v1
kind: ConfigMap
metadata:
  name: mynginx-configmap-v1
  namespace: backend
data:
  default.conf: |-
    server {
        listen       80;
        server_name  localhost;

        #charset koi8-r;
        #access_log  /var/log/nginx/host.access.log  main;

        location / {
            return 200 'v1\n';
        }
    }

Configure a node selector to assign the pod of version 1 of the nginx backend service to the node in Zone G. The following YAML code provides an example:

Show YAML content

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-v1
  namespace: backend
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
      version: v1
  template:
    metadata:
      labels:
        app: nginx
        version: v1
    spec:
      containers:
      - image: docker.io/nginx:1.15.9
        imagePullPolicy: IfNotPresent
        name: nginx
        ports:
        - containerPort: 80
        volumeMounts:
        - name: nginx-config
          mountPath: /etc/nginx/conf.d
          readOnly: true
      volumes:
      - name: nginx-config
        configMap:
          name: mynginx-configmap-v1    
      nodeSelector:
        failure-domain.beta.kubernetes.io/zone: "cn-beijing-g"

Deploy version 2 of the nginx backend service in the ACK cluster. The following YAML code provides an example:

apiVersion: v1
kind: ConfigMap
metadata:
  name: mynginx-configmap-v2
  namespace: backend
data:
  default.conf: |-
    server {
        listen       80;
        server_name  localhost;

        #charset koi8-r;
        #access_log  /var/log/nginx/host.access.log  main;

        location / {
            return 200 'v2\n';
        }
    }

Configure a node selector to assign the pod of version 2 of the nginx backend service to the node in Zone H. The following YAML code provides an example:

Show YAML content

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-v2
  namespace: backend
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
      version: v2
  template:
    metadata:
      labels:
        app: nginx
        version: v2
    spec:
      containers:
      - image: docker.io/nginx:1.15.9
        imagePullPolicy: IfNotPresent
        name: nginx
        ports:
        - containerPort: 80
        volumeMounts:
        - name: nginx-config
          mountPath: /etc/nginx/conf.d
          readOnly: true
      volumes:
      - name: nginx-config
        configMap:
          name: mynginx-configmap-v2 
      nodeSelector:
        failure-domain.beta.kubernetes.io/zone: "cn-beijing-h"

Deploy the nginx backend service. The following YAML code provides an example:

apiVersion: v1
kind: Service
metadata:
  name: nginx
  namespace: backend
  labels:
    app: nginx
spec:
  ports:
  - name: http
    port: 8000
    targetPort: 80
  selector:
    app: nginx

Deploy a sample client service

Deploy version 1 and version 2 of the client service that is named sleep in the ACK cluster. Configure node selectors to assign the pod of version 1 to the node in Zone G and the pod of version 2 to the node in Zone H, as described in the following steps:

Deploy version 1 of the sleep client service in the ACK cluster and configure a node selector to assign the pod of version 1 to the node in Zone G. The following YAML code provides an example:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sleep-cn-beijing-g
  namespace: backend
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sleep
      version: v1
  template:
    metadata:
      labels:
        app: sleep
        version: v1
    spec:
      containers:
      - name: sleep
        image: tutum/curl
        command: ["/bin/sleep","infinity"]
        imagePullPolicy: IfNotPresent
      nodeSelector:
        failure-domain.beta.kubernetes.io/zone: "cn-beijing-g"

Deploy version 2 of the sleep client service in the ACK cluster and configure a node selector to assign the pod of version 2 to the node in Zone H. The following YAML code provides an example:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sleep-cn-beijing-h
  namespace: backend
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sleep
      version: v2
  template:
    metadata:
      labels:
        app: sleep
        version: v2
    spec:
      containers:
      - name: sleep
        image: tutum/curl
        command: ["/bin/sleep","infinity"]
        imagePullPolicy: IfNotPresent
      nodeSelector:
        failure-domain.beta.kubernetes.io/zone: "cn-beijing-h"

Deploy the sample client service. The following YAML code provides an example:

apiVersion: v1
kind: Service
metadata:
  name: sleep
  namespace: backend
  labels:
    app: sleep
spec:
  ports:
  - name: http
    port: 80
  selector:
    app: sleep

Execute the following script to access the nginx backend service from the two pods of the sleep client service:

echo "entering into the 1st container"
export SLEEP_ZONE_1=$(kubectl get pods -lapp=sleep,version=v1 -n backend  -o 'jsonpath={.items[0].metadata.name}')
for i in {1..20}
do
  kubectl exec -it $SLEEP_ZONE_1 -c sleep -n backend -- sh -c 'curl  http://nginx.backend:8000'
done
echo "entering into the 2nd container"
export SLEEP_ZONE_2=$(kubectl get pods -lapp=sleep,version=v2 -n backend  -o 'jsonpath={.items[0].metadata.name}')
for i in {1..20}
do
  kubectl exec -it $SLEEP_ZONE_2 -c sleep -n backend -- sh -c 'curl  http://nginx.backend:8000'
done

Expected output:

Show the expected output

entering into the 1st container
v2
v1
v1
v2
v2
v1
v1
v2
v2
v1
v1
v2
v2
v1
v2
v1
v1
v2
v2
v1
entering into the 2nd container
v2
v2
v1
v1
v2
v1
v2
v1
v2
v2
v1
v1
v2
v2
v1
v2
v1
v2
v1
v1

The output indicates that traffic is sent to the two pods of the nginx backend service in round-robin mode:

Locality-prioritized load balancing

By default, the algorithm of locality-prioritized load balancing routes traffic from a client service pod to a backend service pod in the same zone. If no backend service pod in the same zone is available, the traffic is routed to a pod in another zone.

To enable locality-prioritized load balancing, you must configure a virtual service and a destination rule with an outlier policy. The outlier policy checks the health status of pods and makes decisions on routing.

Log on to the ASM console.
In the left-side navigation pane, choose Service Mesh > Mesh Management.
On the Mesh Management page, find the ASM instance that you want to configure. Click the name of the ASM instance or click Manage in the Actions column.
On the details page of the ASM instance, choose Traffic Management Center > VirtualService in the left-side navigation pane. On the page that appears, click Create from YAML.
On the Create page, perform the following steps to define a virtual service. Then, click Create.
1. Select a namespace from the Namespace drop-down list. In this example, the backend namespace is selected.
2. Copy the following YAML code to the code editor to define a virtual service:
```
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginx
  namespace: backend
spec:
  hosts:
    - nginx
  http:
  - route:
    - destination:
        host: nginx
```
On the details page of the ASM instance, choose Traffic Management Center > DestinationRule in the left-side navigation pane. On the page that appears, click Create from YAML.
On the Create page, perform the following steps to define a destination rule. Then, click Create.
1. Select a namespace from the Namespace drop-down list. In this example, the backend namespace is selected.
2. Copy the following YAML code to the code editor to define a destination rule:
```
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: nginx
  namespace: backend
spec:
  host: nginx
  trafficPolicy:
    outlierDetection:
      consecutiveErrors: 7
      interval: 30s
      baseEjectionTime: 30s
```

Execute the following script to access the nginx backend service from the two pods of the sleep client service:

echo "entering into the 1st container"
export SLEEP_ZONE_1=$(kubectl get pods -lapp=sleep,version=v1 -n backend  -o 'jsonpath={.items[0].metadata.name}')
for i in {1..20}
do
  kubectl exec -it $SLEEP_ZONE_1 -c sleep -n backend -- sh -c 'curl  http://nginx.backend:8000'
done
echo "entering into the 2nd container"
export SLEEP_ZONE_2=$(kubectl get pods -lapp=sleep,version=v2 -n backend  -o 'jsonpath={.items[0].metadata.name}')
for i in {1..20}
do
  kubectl exec -it $SLEEP_ZONE_2 -c sleep -n backend -- sh -c 'curl  http://nginx.backend:8000'
done

Expected output:

Show the expected output

entering into the 1st container
v1
v1
v1
v1
v1
v1
v1
v2
v1
v1
v2
v1
v1
v1
v1
v1
v1
v1
v1
v1
entering into the 2nd container
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2

The output indicates that traffic is sent to the two pods of the nginx backend service in locality-prioritized mode.

Locality-weighted load balancing

Most use cases work well with locality-prioritized load balancing. However, sometimes you may want to split traffic and send the traffic to multiple zones. For example, if all traffic comes from a zone, you can split traffic to prevent the zone from being overloaded. In this case, you can use locality-weighted load balancing.

In this example, the following rules are applied:

Route 80% of traffic from China (Beijing) Zone G to China (Beijing) Zone G and 20% to China (Beijing) Zone H.
Route 20% of traffic from China (Beijing) Zone H to China (Beijing) Zone G and 80% to China (Beijing) Zone H.

Log on to the ASM console.
In the left-side navigation pane, choose Service Mesh > Mesh Management.
On the Mesh Management page, find the ASM instance that you want to configure. Click the name of the ASM instance or click Manage in the Actions column.
On the details page of the ASM instance, choose Traffic Management Center > VirtualService in the left-side navigation pane. On the page that appears, click Create from YAML.
On the Create page, perform the following steps to define a virtual service. Then, click Create.
1. Select a namespace from the Namespace drop-down list. In this example, the backend namespace is selected.
2. Copy the following YAML code to the code editor to define a virtual service:
```
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: nginx
  namespace: backend
spec:
  hosts:
    - nginx
  http:
  - route:
    - destination:
        host: nginx
```
On the details page of the ASM instance, choose Traffic Management Center > DestinationRule in the left-side navigation pane. On the page that appears, click Create from YAML.

On the Create page, perform the following steps to define a destination rule. Then, click Create.

Select a namespace from the Namespace drop-down list. In this example, the backend namespace is selected.

Copy the following YAML code to the code editor to define a destination rule:

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: nginx
  namespace: backend
spec:
  host: nginx
  trafficPolicy:
    outlierDetection:
      consecutiveErrors: 7
      interval: 30s
      baseEjectionTime: 30s
    loadBalancer:
      localityLbSetting:
        enabled: true
        distribute:
        - from: cn-beijing/cn-beijing-g/*
          to:
            "cn-beijing/cn-beijing-g/*": 80
            "cn-beijing/cn-beijing-h/*": 20
        - from: cn-beijing/cn-beijing-h/*
          to:
            "cn-beijing/cn-beijing-g/*": 20
            "cn-beijing/cn-beijing-h/*": 80

Execute the following script to access the nginx backend service from the two pods of the sleep client service:

echo "entering into the 1st container"
export SLEEP_ZONE_1=$(kubectl get pods -lapp=sleep,version=v1 -n backend  -o 'jsonpath={.items[0].metadata.name}')
for i in {1..20}
do
  kubectl exec -it $SLEEP_ZONE_1 -c sleep -n backend -- sh -c 'curl  http://nginx.backend:8000'
done
echo "entering into the 2nd container"
export SLEEP_ZONE_2=$(kubectl get pods -lapp=sleep,version=v2 -n backend  -o 'jsonpath={.items[0].metadata.name}')
for i in {1..20}
do
  kubectl exec -it $SLEEP_ZONE_2 -c sleep -n backend -- sh -c 'curl  http://nginx.backend:8000'
done

Expected output:

Show the expected output

entering into the 1st container
v1
v1
v1
v1
v2
v1
v1
v2
v1
v1
v1
v2
v1
v1
v1
v1
v1
v2
v1
v1
entering into the 2nd container
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2
v2
v1
v2
v1
v2
v2

The output indicates that traffic is sent to the two pods of the nginx backend service in locality-weighted mode.

References

If you want to ensure that traffic flows only within specific regions, you can create ACK clusters in different regions and enable the feature of keeping traffic in-cluster to limit traffic to their respective clusters. For more information, see Enable the feature of keeping traffic in-cluster in multi-cluster scenarios.