When you use HTTP 2-based gRPC to call services in Kubernetes, a connection is always pointed to a specific pod. If the gRPC client sends multiple requests, all of the requests are routed to the pod for processing, which causes unbalanced load. This topic describes the unbalanced load issue between gRPC service calls. This topic also shows you how to use Alibaba Cloud Service Mesh (ASM) to achieve gRPC load balancing.

Background information

gRPC is an HTTP 2-based communication protocol for services. You can specify service definitions in a format that is called protocol buffers. The data of service calls is serialized into a small binary format for transmission. gRPC allows you to generate boilerplate code from .proto files into multiple programming languages. This makes gRPC an ideal choice for polyglot microservices.

HTTP 1.1-based remote procedure calls (RPCs) build temporary connections between the client and pods. In this case, a TCP load balancer is enough to balance the load between RPCs. However, for an HTTP 2-based gRPC service call, the TCP connection between the client and a pod is persistent. If the pod expires, the load in the cluster becomes unbalanced.

Unbalanced load between gRPC service calls

The following example shows unbalanced load between gRPC service calls.

Prerequisites:

Procedure

  1. Log on to the Container Service console.
  2. In the left-side navigation pane of the ACK console, click Clusters.
  3. On the Clusters page, find the cluster that you want to manage and click the name of the cluster or click Details in the Actions column. The details page of the cluster appears.
  4. In the left-side navigation pane of the details page, click Namespaces and Quotas.
  5. On the Namespace page, click Create in the upper-right corner. In the Create Namespace dialog box, enter a name for the namespace, such as grpc-nosidecar, and click OK.
  6. Deploy a gRPC server that is named istio-grpc-server in the created grpc-nosidecar namespace.
    Run the following command to deploy the gRPC server by using the description YAML file istio-grpc-server.yaml:
    kubectl apply -n grpc-nosidecar -f istio-grpc-server.yaml
    The istio-grpc-server.yaml file contains the following content:
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: istio-grpc-server-v1
      labels:
        app: istio-grpc-server
        version: v1
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: istio-grpc-server
          version: v1
      template:
        metadata:
          labels:
            app: istio-grpc-server
            version: v1
        spec:
          containers:
          - args:
            - --address=0.0.0.0:8080
            image: registry.cn-hangzhou.aliyuncs.com/aliacs-app-catalog/istio-grpc-server
            imagePullPolicy: Always
            livenessProbe:
              exec:
                command:
                - /bin/grpc_health_probe
                - -addr=:8080
              initialDelaySeconds: 2
            name: istio-grpc-server
            ports:
            - containerPort: 8080
            readinessProbe:
              exec:
                command:
                - /bin/grpc_health_probe
                - -addr=:8080
              initialDelaySeconds: 2
    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: istio-grpc-server-v2
      labels:
        app: istio-grpc-server
        version: v2
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: istio-grpc-server
          version: v2
      template:
        metadata:
          labels:
            app: istio-grpc-server
            version: v2
        spec:
          containers:
            - args:
                - --address=0.0.0.0:8080
              image: registry.cn-hangzhou.aliyuncs.com/aliacs-app-catalog/istio-grpc-server
              imagePullPolicy: Always
              livenessProbe:
                exec:
                  command:
                    - /bin/grpc_health_probe
                    - -addr=:8080
                initialDelaySeconds: 2
              name: istio-grpc-server
              ports:
                - containerPort: 8080
              readinessProbe:
                exec:
                  command:
                    - /bin/grpc_health_probe
                    - -addr=:8080
                initialDelaySeconds: 2
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: istio-grpc-server
      labels:
        app: istio-grpc-server
    spec:
      ports:
      - name: grpc-backend
        port: 8080
        protocol: TCP
      selector:
        app: istio-grpc-server
      type: ClusterIP
    ---
  7. Deploy a gRPC client that is named istio-grpc-client in the created grpc-nosidecar namespace.
    Run the following command to deploy the gRPC client by using the description YAML file istio-grpc-client.yaml:
    kubectl apply -n grpc-nosidecar -f istio-grpc-client.yaml
    The istio-grpc-client.yaml file contains the following content:
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: istio-grpc-client
      labels:
        app: istio-grpc-client
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: istio-grpc-client
      template:
        metadata:
          labels:
            app: istio-grpc-client
        spec:
          containers:
          - image: registry.cn-hangzhou.aliyuncs.com/aliacs-app-catalog/istio-grpc-client
            imagePullPolicy: Always
            command: ["/bin/sleep", "3650d"]
            name: istio-grpc-client
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: istio-grpc-client
    spec:
      ports:
      - name: grpc
        port: 8080
        protocol: TCP
      selector:
        app: istio-grpc-client
      type: ClusterIP
    ---
  8. Run the following command to query the pod status:
    kubectl get pod -n grpc-nosidecar
    The command output is similar to the following example:
    NAME                                    READY   STATUS    RESTARTS   AGE
    istio-grpc-client-dd56bcb45-hvmjt       1/1     Running   0          95m
    istio-grpc-server-v1-546d9876c4-j2p9r   1/1     Running   0          95m
    istio-grpc-server-v2-66d9b8847-276bd    1/1     Running   0          95m
  9. Run the following command to log on to the default container of the pod of the gRPC client:
    kubectl exec -it -n grpc-nosidecar istio-grpc-client-dd56bcb45-hvmjt sh
  10. After you log on to the container, run the following command:
    /bin/greeter-client --insecure=true --address=istio-grpc-server:8080 --repeat=100
    The command output indicates that all requests are routed to the same pod:
    2020/01/14 14:37:14 Hello world from istio-grpc-server-v2-66d9b8847-276bd
    2020/01/14 14:37:14 Hello world from istio-grpc-server-v2-66d9b8847-276bd
    2020/01/14 14:37:14 Hello world from istio-grpc-server-v2-66d9b8847-276bd
    2020/01/14 14:37:14 Hello world from istio-grpc-server-v2-66d9b8847-276bd
    2020/01/14 14:37:14 Hello world from istio-grpc-server-v2-66d9b8847-276bd
    2020/01/14 14:37:14 Hello world from istio-grpc-server-v2-66d9b8847-276bd
    2020/01/14 14:37:14 Hello world from istio-grpc-server-v2-66d9b8847-276bd
    2020/01/14 14:37:14 Hello world from istio-grpc-server-v2-66d9b8847-276bd
    2020/01/14 14:37:14 Hello world from istio-grpc-server-v2-66d9b8847-276bd
    2020/01/14 14:37:14 Hello world from istio-grpc-server-v2-66d9b8847-276bd
    2020/01/14 14:37:14 Hello world from istio-grpc-server-v2-66d9b8847-276bd
    All requests from the gRPC client are processed by the same pod, which causes unbalanced load.

Use ASM to balance the load between gRPC service calls

The following example describes how to use ASM to achieve gRPC load balancing.

Prerequisites:

Procedure

  1. Log on to the Container Service console.
  2. In the left-side navigation pane of the ACK console, click Clusters.
  3. On the Clusters page, find the cluster that you want to manage and click the name of the cluster or click Details in the Actions column. The details page of the cluster appears.
  4. In the left-side navigation pane of the details page, click Namespaces and Quotas.
  5. On the Namespace page, click Create in the upper-right corner. In the Create Namespace dialog box, enter a namespace name, such as grpc-nosidecar, create a tag istio-injection:enabled, and then click OK.
  6. Deploy a gRPC server that is named istio-grpc-server in the created grpc-sidecar namespace.
    Run the following command to deploy the gRPC server by using the description YAML file istio-grpc-server.yaml:
    kubectl apply -n grpc-sidecar -f istio-grpc-server.yaml
    For more information about the istio-grpc-server.yaml file, see the sample file in the preceding section.
  7. Deploy a gRPC client that is named istio-grpc-client in the created grpc-sidecar namespace.
    Run the following command to deploy the gRPC client by using the description YAML file istio-grpc-client.yaml:
    kubectl apply -n grpc-sidecar -f istio-grpc-client.yaml
    For more information about the istio-grpc-client.yaml file, see the sample file in the preceding section.
  8. Run the following command to query the pod status:
    kubectl get pod -n grpc-sidecar
    The command output indicates that each pod contains two containers, one of which is the container for the injected sidecar proxy. The command output is similar to the following example:
    NAME                                    READY   STATUS    RESTARTS   AGE
    istio-grpc-client-dd56bcb45-zhfsg       2/2     Running   0          1h15m
    istio-grpc-server-v1-546d9876c4-tndsm   2/2     Running   0          1h15m
    istio-grpc-server-v2-66d9b8847-99v62    2/2     Running   0          1h15m
  9. Run the following command to log on to the default container of the pod of the gRPC client:
    kubectl exec -it -n grpc-sidecar istio-grpc-client-dd56bcb45-zhfsg sh
  10. After you log on to the container, run the following command:
    /bin/greeter-client --insecure=true --address=istio-grpc-server:8080 --repeat=100
    The command output indicates that requests are routed to the two pods at a ratio of almost 50 to 50 in round-robin scheduling mode:
    2020/01/14 14:53:16 Hello world from istio-grpc-server-v1-546d9876c4-tndsm
    2020/01/14 14:53:16 Hello world from istio-grpc-server-v2-66d9b8847-99v62
    2020/01/14 14:53:16 Hello world from istio-grpc-server-v1-546d9876c4-tndsm
    2020/01/14 14:53:16 Hello world from istio-grpc-server-v2-66d9b8847-99v62
    2020/01/14 14:53:16 Hello world from istio-grpc-server-v1-546d9876c4-tndsm
    2020/01/14 14:53:16 Hello world from istio-grpc-server-v2-66d9b8847-99v62
    2020/01/14 14:53:16 Hello world from istio-grpc-server-v1-546d9876c4-tndsm
    2020/01/14 14:53:16 Hello world from istio-grpc-server-v2-66d9b8847-99v62
    2020/01/14 14:53:16 Hello world from istio-grpc-server-v1-546d9876c4-tndsm
    2020/01/14 14:53:16 Hello world from istio-grpc-server-v2-66d9b8847-99v62
    2020/01/14 14:53:16 Hello world from istio-grpc-server-v1-546d9876c4-tndsm
    2020/01/14 14:53:16 Hello world from istio-grpc-server-v2-66d9b8847-99v62
  11. Configure the control plane for the ASM instance.
    1. Log on to the ASM console.
    2. In the left-side navigation pane, choose Service Mesh > Mesh Management.
    3. On the Mesh Management page, find the ASM instance that you want to configure. Click the name of the ASM instance or click Manage in the Actions column of the ASM instance.
    4. On the details page of the ASM instance, click Namespaces in the left-side navigation pane. On the Namespaces page, click Create.
    5. In the Create Namespace dialog box, enter a name for the namespace, such as grpc-sidecar, and click OK.
    6. Create a destination rule by using the following content. For more information, see Manage destination rules.
      apiVersion: networking.istio.io/v1alpha3
      kind: DestinationRule
      metadata:
        name: dr-istio-grpc-server
      spec:
        host: istio-grpc-server
        trafficPolicy:
          loadBalancer:
            simple: ROUND_ROBIN
        subsets:
          - name: v1
            labels:
              version: "v1"
          - name: v2
            labels:
              version: "v2"
    7. Create a virtual service by using the following content. For more information, see Manage virtual services.
      apiVersion: networking.istio.io/v1alpha3
      kind: VirtualService
      metadata:
        name: vs-istio-grpc-server
      spec:
        hosts:
          - "istio-grpc-server"
        http:
          - match:
              - port: 8080
            route:
              - destination:
                  host: istio-grpc-server
                  subset: v1
                weight: 90
              - destination:
                  host: istio-grpc-server
                  subset: v2
                weight: 10
    8. Run the following command to log on to the default container of the pod of the gRPC client:
      kubectl exec -it -n grpc-sidecar istio-grpc-client-dd56bcb45-zhfsg sh
    9. After you log on to the container, run the following command:
      /bin/greeter-client --insecure=true --address=istio-grpc-server:8080 --repeat=100

      The command output indicates that requests are routed to the two pods at a ratio of 90 to 10, instead of 50 to 50 in round-robin scheduling mode.

    10. On the details page of the ASM instance, choose Traffic Management > VirtualService in the left-side navigation pane. Click YAML in the Actions column of the vs-istio-grpc-server virtual service that is deployed in the grpc-sidecar namespace to modify the weight assigned to each service version.
      Use the following content to modify the weight:
           route:
              - destination:
                  host: istio-grpc-server
                  subset: v1
                weight: 0
              - destination:
                  host: istio-grpc-server
                  subset: v2
                weight: 100
    11. Use the kubectl client to log on to the default container of the pod of the gRPC client and run the following command:
      /bin/greeter-client --insecure=true --address=istio-grpc-server:8080 --repeat=100

      The command output indicates that all requests are routed to the pod of the version v2.