Use ASM to achieve gRPC load balancing - Alibaba Cloud Service Mesh

When you use HTTP 2-based gRPC to call services in Kubernetes, a connection is always pointed to a specific pod. If the gRPC client sends multiple requests, all of the requests are routed to the pod for processing, which causes unbalanced load. This topic describes the unbalanced load issue between gRPC service calls. This topic also shows you how to use Alibaba Cloud Service Mesh (ASM) to achieve gRPC load balancing.

Background information

gRPC is an HTTP 2-based communication protocol for services. You can specify service definitions in a format that is called protocol buffers. The data of service calls is serialized into a small binary format for transmission. gRPC allows you to generate boilerplate code from .proto files into multiple programming languages. This makes gRPC an ideal choice for polyglot microservices.

HTTP 1.1-based remote procedure calls (RPCs) build temporary connections between the client and pods. In this case, a TCP load balancer is enough to balance the load between RPCs. However, for an HTTP 2-based gRPC service call, the TCP connection between the client and a pod is persistent. If the pod expires, the load in the cluster becomes unbalanced.

Unbalanced load between gRPC service calls

The following example shows unbalanced load between gRPC service calls.

Prerequisites:

A Container Service for Kubernetes (ACK) cluster is created.
The kubectl client is connected to the ACK cluster. For more information, see Connect to ACK clusters by using kubectl.

Procedure

Log on to the ACK console.
In the left-side navigation pane of the ACK console, click Clusters.
On the Clusters page, find the cluster that you want to manage and click the name of the cluster or click Details in the Actions column. The details page of the cluster appears.
In the left-side navigation pane of the details page, click Namespaces and Quotas.
On the Namespace page, click Create in the upper-right corner. In the Create Namespace dialog box, enter a name for the namespace, such as grpc-nosidecar, and click OK.

Deploy a gRPC server that is named istio-grpc-server in the created grpc-nosidecar namespace.

Run the following command to deploy the gRPC server by using the description YAML file istio-grpc-server.yaml:

kubectl apply -n grpc-nosidecar -f istio-grpc-server.yaml

The istio-grpc-server.yaml file contains the following content:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: istio-grpc-server-v1
  labels:
    app: istio-grpc-server
    version: v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: istio-grpc-server
      version: v1
  template:
    metadata:
      labels:
        app: istio-grpc-server
        version: v1
    spec:
      containers:
      - args:
        - --address=0.0.0.0:8080
        image: registry.cn-hangzhou.aliyuncs.com/aliacs-app-catalog/istio-grpc-server
        imagePullPolicy: Always
        livenessProbe:
          exec:
            command:
            - /bin/grpc_health_probe
            - -addr=:8080
          initialDelaySeconds: 2
        name: istio-grpc-server
        ports:
        - containerPort: 8080
        readinessProbe:
          exec:
            command:
            - /bin/grpc_health_probe
            - -addr=:8080
          initialDelaySeconds: 2
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: istio-grpc-server-v2
  labels:
    app: istio-grpc-server
    version: v2
spec:
  replicas: 1
  selector:
    matchLabels:
      app: istio-grpc-server
      version: v2
  template:
    metadata:
      labels:
        app: istio-grpc-server
        version: v2
    spec:
      containers:
        - args:
            - --address=0.0.0.0:8080
          image: registry.cn-hangzhou.aliyuncs.com/aliacs-app-catalog/istio-grpc-server
          imagePullPolicy: Always
          livenessProbe:
            exec:
              command:
                - /bin/grpc_health_probe
                - -addr=:8080
            initialDelaySeconds: 2
          name: istio-grpc-server
          ports:
            - containerPort: 8080
          readinessProbe:
            exec:
              command:
                - /bin/grpc_health_probe
                - -addr=:8080
            initialDelaySeconds: 2
---
apiVersion: v1
kind: Service
metadata:
  name: istio-grpc-server
  labels:
    app: istio-grpc-server
spec:
  ports:
  - name: grpc-backend
    port: 8080
    protocol: TCP
  selector:
    app: istio-grpc-server
  type: ClusterIP
---

Deploy a gRPC client that is named istio-grpc-client in the created grpc-nosidecar namespace.

Run the following command to deploy the gRPC client by using the description YAML file istio-grpc-client.yaml:

kubectl apply -n grpc-nosidecar -f istio-grpc-client.yaml

The istio-grpc-client.yaml file contains the following content:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: istio-grpc-client
  labels:
    app: istio-grpc-client
spec:
  replicas: 1
  selector:
    matchLabels:
      app: istio-grpc-client
  template:
    metadata:
      labels:
        app: istio-grpc-client
    spec:
      containers:
      - image: registry.cn-hangzhou.aliyuncs.com/aliacs-app-catalog/istio-grpc-client
        imagePullPolicy: Always
        command: ["/bin/sleep", "3650d"]
        name: istio-grpc-client
---
apiVersion: v1
kind: Service
metadata:
  name: istio-grpc-client
spec:
  ports:
  - name: grpc
    port: 8080
    protocol: TCP
  selector:
    app: istio-grpc-client
  type: ClusterIP
---

Run the following command to query the pod status:

kubectl get pod -n grpc-nosidecar

The command output is similar to the following example:

NAME                                    READY   STATUS    RESTARTS   AGE
istio-grpc-client-dd56bcb45-hvmjt       1/1     Running   0          95m
istio-grpc-server-v1-546d9876c4-j2p9r   1/1     Running   0          95m
istio-grpc-server-v2-66d9b8847-276bd    1/1     Running   0          95m

Run the following command to log on to the default container of the pod of the gRPC client:
```
kubectl exec -it -n grpc-nosidecar istio-grpc-client-dd56bcb45-hvmjt sh
```

After you log on to the container, run the following command:

/bin/greeter-client --insecure=true --address=istio-grpc-server:8080 --repeat=100

The command output indicates that all requests are routed to the same pod:

2020/01/14 14:37:14 Hello world from istio-grpc-server-v2-66d9b8847-276bd
2020/01/14 14:37:14 Hello world from istio-grpc-server-v2-66d9b8847-276bd
2020/01/14 14:37:14 Hello world from istio-grpc-server-v2-66d9b8847-276bd
2020/01/14 14:37:14 Hello world from istio-grpc-server-v2-66d9b8847-276bd
2020/01/14 14:37:14 Hello world from istio-grpc-server-v2-66d9b8847-276bd
2020/01/14 14:37:14 Hello world from istio-grpc-server-v2-66d9b8847-276bd
2020/01/14 14:37:14 Hello world from istio-grpc-server-v2-66d9b8847-276bd
2020/01/14 14:37:14 Hello world from istio-grpc-server-v2-66d9b8847-276bd
2020/01/14 14:37:14 Hello world from istio-grpc-server-v2-66d9b8847-276bd
2020/01/14 14:37:14 Hello world from istio-grpc-server-v2-66d9b8847-276bd
2020/01/14 14:37:14 Hello world from istio-grpc-server-v2-66d9b8847-276bd

All requests from the gRPC client are processed by the same pod, which causes unbalanced load.

Use ASM to balance the load between gRPC service calls

The following example describes how to use ASM to achieve gRPC load balancing.

Prerequisites:

An ASM instance is created and an ACK cluster is added to the instance.
The kubectl client is connected to the ACK cluster. For more information, see Connect to ACK clusters by using kubectl.

Procedure

Log on to the ACK console.
In the left-side navigation pane of the ACK console, click Clusters.
On the Clusters page, find the cluster that you want to manage and click the name of the cluster or click Details in the Actions column. The details page of the cluster appears.
In the left-side navigation pane of the details page, click Namespaces and Quotas.
On the Namespace page, click Create in the upper-right corner. In the Create Namespace dialog box, enter a namespace name, such as grpc-sidecar, create a tag istio-injection:enabled, and then click OK.
Deploy a gRPC server that is named istio-grpc-server in the created grpc-sidecar namespace.
Run the following command to deploy the gRPC server by using the description YAML file istio-grpc-server.yaml:
```
kubectl apply -n grpc-sidecar -f istio-grpc-server.yaml
```
For more information about the istio-grpc-server.yaml file, see the sample file in the preceding section.
Deploy a gRPC client that is named istio-grpc-client in the created grpc-sidecar namespace.
Run the following command to deploy the gRPC client by using the description YAML file istio-grpc-client.yaml:
```
kubectl apply -n grpc-sidecar -f istio-grpc-client.yaml
```
For more information about the istio-grpc-client.yaml file, see the sample file in the preceding section.

Run the following command to query the pod status:

kubectl get pod -n grpc-sidecar

The command output indicates that each pod contains two containers, one of which is the container for the injected sidecar proxy. The command output is similar to the following example:

NAME                                    READY   STATUS    RESTARTS   AGE
istio-grpc-client-dd56bcb45-zhfsg       2/2     Running   0          1h15m
istio-grpc-server-v1-546d9876c4-tndsm   2/2     Running   0          1h15m
istio-grpc-server-v2-66d9b8847-99v62    2/2     Running   0          1h15m

Run the following command to log on to the default container of the pod of the gRPC client:
```
kubectl exec -it -n grpc-sidecar istio-grpc-client-dd56bcb45-zhfsg sh
```

After you log on to the container, run the following command:

/bin/greeter-client --insecure=true --address=istio-grpc-server:8080 --repeat=100

The command output indicates that requests are routed to the two pods at a ratio of almost 50 to 50 in round-robin scheduling mode:

2020/01/14 14:53:16 Hello world from istio-grpc-server-v1-546d9876c4-tndsm
2020/01/14 14:53:16 Hello world from istio-grpc-server-v2-66d9b8847-99v62
2020/01/14 14:53:16 Hello world from istio-grpc-server-v1-546d9876c4-tndsm
2020/01/14 14:53:16 Hello world from istio-grpc-server-v2-66d9b8847-99v62
2020/01/14 14:53:16 Hello world from istio-grpc-server-v1-546d9876c4-tndsm
2020/01/14 14:53:16 Hello world from istio-grpc-server-v2-66d9b8847-99v62
2020/01/14 14:53:16 Hello world from istio-grpc-server-v1-546d9876c4-tndsm
2020/01/14 14:53:16 Hello world from istio-grpc-server-v2-66d9b8847-99v62
2020/01/14 14:53:16 Hello world from istio-grpc-server-v1-546d9876c4-tndsm
2020/01/14 14:53:16 Hello world from istio-grpc-server-v2-66d9b8847-99v62
2020/01/14 14:53:16 Hello world from istio-grpc-server-v1-546d9876c4-tndsm
2020/01/14 14:53:16 Hello world from istio-grpc-server-v2-66d9b8847-99v62

Configure the control plane for the ASM instance.
1. Log on to the ASM console.
2. In the left-side navigation pane, choose Service Mesh > Mesh Management.
3. On the Mesh Management page, find the ASM instance that you want to configure. Click the name of the ASM instance or click Manage in the Actions column.
4. On the details page of the ASM instance, choose ASM Instance > Global Namespace in the left-side navigation pane. On the Global Namespace page, click Create.
5. In the Create Namespace dialog box, enter a name for the namespace, such as grpc-sidecar, and click OK.
6. Create a destination rule by using the following content. For more information, see Manage destination rules.
```
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: dr-istio-grpc-server
spec:
  host: istio-grpc-server
  trafficPolicy:
    loadBalancer:
      simple: ROUND_ROBIN
  subsets:
    - name: v1
      labels:
        version: "v1"
    - name: v2
      labels:
        version: "v2"
```
7. Create a virtual service by using the following content. For more information, see Manage virtual services.
```
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: vs-istio-grpc-server
spec:
  hosts:
    - "istio-grpc-server"
  http:
    - match:
        - port: 8080
      route:
        - destination:
            host: istio-grpc-server
            subset: v1
          weight: 90
        - destination:
            host: istio-grpc-server
            subset: v2
          weight: 10
```
8. Run the following command to log on to the default container of the pod of the gRPC client:
```
kubectl exec -it -n grpc-sidecar istio-grpc-client-dd56bcb45-zhfsg sh
```
9. After you log on to the container, run the following command:
```
/bin/greeter-client --insecure=true --address=istio-grpc-server:8080 --repeat=100
```
  The command output indicates that requests are routed to the two pods at a ratio of 90 to 10, instead of 50 to 50 in round-robin scheduling mode.
10. On the details page of the ASM instance, choose Traffic Management > VirtualService in the left-side navigation pane. Click YAML in the Actions column of the vs-istio-grpc-server virtual service that is deployed in the grpc-sidecar namespace to modify the weight assigned to each service version.
  Use the following content to modify the weight:
```
     route:
        - destination:
            host: istio-grpc-server
            subset: v1
          weight: 0
        - destination:
            host: istio-grpc-server
            subset: v2
          weight: 100
```
11. Use the kubectl client to log on to the default container of the pod of the gRPC client and run the following command:
```
/bin/greeter-client --insecure=true --address=istio-grpc-server:8080 --repeat=100
```
  The command output indicates that all requests are routed to the pod of the version v2.