When you use HTTP 2-based gRPC to call services in Kubernetes, a connection is always pointed to a specific pod. If the gRPC client sends multiple requests, all of the requests are routed to the pod for processing, which causes an unbalanced load. This topic describes the unbalanced load issue between gRPC service calls. This topic also shows you how to use Service Mesh (ASM) to achieve gRPC load balancing.
Background information
gRPC is an HTTP 2-based communication protocol for services. You can specify service definitions in a format that is called protocol buffers. The data of service calls is serialized into a small binary format for transmission. gRPC allows you to generate boilerplate code from .proto files into multiple programming languages. This makes gRPC an ideal choice for polyglot microservices.
HTTP 1.1-based remote procedure calls (RPCs) build temporary connections between the client and pods. In this case, a TCP load balancer is enough to balance the load between RPCs. However, for an HTTP 2-based gRPC service call, the TCP connection between the client and a pod is persistent. If the pod expires, the load in the cluster becomes unbalanced.
Unbalanced load between gRPC service calls
The following example shows unbalanced load between gRPC service calls.
Prerequisites
A Container Service for Kubernetes (ACK) cluster is created. For more information, see Create an ACK managed cluster.
A namespace named grpc-nosidecar is created. For more information, see Create a namespace.
The cluster is connected by using kubectl. For more information, see Obtain the kubeconfig file of a cluster and use kubectl to connect to the cluster.
Procedure
Deploy a gRPC server that is named istio-grpc-server in the grpc-nosidecar namespace.
Create an istio-grpc-server.yaml file that contains the following content:
Run the following command to deploy the gRPC server istio-grpc-server:
kubectl apply -n grpc-nosidecar -f istio-grpc-server.yaml
Deploy a gRPC client that is named istio-grpc-client in the grpc-nosidecar namespace.
Create an istio-grpc-client.yaml file that contains the following content:
Run the following command to deploy the gRPC client istio-grpc-client:
kubectl apply -n grpc-nosidecar -f istio-grpc-client.yaml
Run the following command to query the pod status:
kubectl get pod -n grpc-nosidecar
Expected output:
NAME READY STATUS RESTARTS AGE istio-grpc-client-dd56bcb45-hvmjt 1/1 Running 0 95m istio-grpc-server-v1-546d9876c4-j2p9r 1/1 Running 0 95m istio-grpc-server-v2-66d9b8847-276bd 1/1 Running 0 95m
Run the following command to log on to the default container of the pod of the gRPC client:
kubectl exec -it -n grpc-nosidecar istio-grpc-client-dd56bcb45-hvmjt sh
After you log on to the container, run the following command:
/bin/greeter-client --insecure=true --address=istio-grpc-server:8080 --repeat=100
Expected output:
2024/02/07 14:37:14 Hello world from istio-grpc-server-v2-66d9b8847-276bd 2024/02/07 14:37:14 Hello world from istio-grpc-server-v2-66d9b8847-276bd 2024/02/07 14:37:14 Hello world from istio-grpc-server-v2-66d9b8847-276bd 2024/02/07 14:37:14 Hello world from istio-grpc-server-v2-66d9b8847-276bd 2024/02/07 14:37:14 Hello world from istio-grpc-server-v2-66d9b8847-276bd 2024/02/07 14:37:14 Hello world from istio-grpc-server-v2-66d9b8847-276bd 2024/02/07 14:37:14 Hello world from istio-grpc-server-v2-66d9b8847-276bd 2024/02/07 14:37:14 Hello world from istio-grpc-server-v2-66d9b8847-276bd 2024/02/07 14:37:14 Hello world from istio-grpc-server-v2-66d9b8847-276bd 2024/02/07 14:37:14 Hello world from istio-grpc-server-v2-66d9b8847-276bd 2024/02/07 14:37:14 Hello world from istio-grpc-server-v2-66d9b8847-276bd
The command output indicates that all requests are routed to the same pod. All requests from the gRPC client are processed by the same pod, which causes unbalanced load.
Use ASM to balance the load between gRPC service calls
The following example describes how to use ASM to achieve gRPC load balancing:
Prerequisites
An ASM instance is created and an ACK cluster is added to the instance. For more information, see Add a cluster to an ASM instance.
A namespace named grpc-sidecar is created and the
istio-injection:enabled
label is added to the namespace. For more information, see Create a namespace.You are connected to the cluster by using kubectl. For more information, see Obtain the kubeconfig file of a cluster and use kubectl to connect to the cluster.
Procedure
Deploy a gRPC server that is named istio-grpc-server in the grpc-sidecar namespace.
Create an istio-grpc-server.yaml file that contains the following content:
Run the following command to deploy the gRPC server istio-grpc-server:
kubectl apply -n grpc-sidecar -f istio-grpc-server.yaml
Deploy a gRPC client that is named istio-grpc-client in the grpc-sidecar namespace.
Create an istio-grpc-client.yaml file that contains the following content:
Run the following command to deploy the gRPC client istio-grpc-client:
kubectl apply -n grpc-sidecar -f istio-grpc-client.yaml
Run the following command to query the pod status:
kubectl get pod -n grpc-sidecar
Expected output:
NAME READY STATUS RESTARTS AGE istio-grpc-client-dd56bcb45-zhfsg 2/2 Running 0 1h15m istio-grpc-server-v1-546d9876c4-tndsm 2/2 Running 0 1h15m istio-grpc-server-v2-66d9b8847-99v62 2/2 Running 0 1h15m
The command output indicates that each pod contains two containers, one of which runs the injected sidecar proxy.
Run the following command to log on to the default container of the pod of the gRPC client:
kubectl exec -it -n grpc-sidecar istio-grpc-client-dd56bcb45-zhfsg -c istio-grpc-client sh
After you log on to the container, run the following command:
/bin/greeter-client --insecure=true --address=istio-grpc-server:8080 --repeat=100
Expected output:
2024/02/07 14:53:16 Hello world from istio-grpc-server-v1-546d9876c4-tndsm 2024/02/07 14:53:16 Hello world from istio-grpc-server-v2-66d9b8847-99v62 2024/02/07 14:53:16 Hello world from istio-grpc-server-v1-546d9876c4-tndsm 2024/02/07 14:53:16 Hello world from istio-grpc-server-v2-66d9b8847-99v62 2024/02/07 14:53:16 Hello world from istio-grpc-server-v1-546d9876c4-tndsm 2024/02/07 14:53:16 Hello world from istio-grpc-server-v2-66d9b8847-99v62 2024/02/07 14:53:16 Hello world from istio-grpc-server-v1-546d9876c4-tndsm 2024/02/07 14:53:16 Hello world from istio-grpc-server-v2-66d9b8847-99v62 2024/02/07 14:53:16 Hello world from istio-grpc-server-v1-546d9876c4-tndsm 2024/02/07 14:53:16 Hello world from istio-grpc-server-v2-66d9b8847-99v62 2024/02/07 14:53:16 Hello world from istio-grpc-server-v1-546d9876c4-tndsm 2024/02/07 14:53:16 Hello world from istio-grpc-server-v2-66d9b8847-99v62
The command output indicates that requests are routed to the two pods at a ratio of almost 50 to 50 in round-robin scheduling mode:
Configure the control plane for the ASM instance.
Create a namespace named grpc-sidecar in the ASM instance. For more information, see Create a namespace.
Create a destination rule in the grpc-sidecar namespace by using the following content. For more information, see Manage destination rules.
apiVersion: networking.istio.io/v1alpha3 kind: DestinationRule metadata: name: dr-istio-grpc-server spec: host: istio-grpc-server trafficPolicy: loadBalancer: simple: ROUND_ROBIN subsets: - name: v1 labels: version: "v1" - name: v2 labels: version: "v2"
Create a virtual service in the grpc-sidecar namespace by using the following content. For more information, see Manage virtual services.
apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: vs-istio-grpc-server spec: hosts: - "istio-grpc-server" http: - match: - port: 8080 route: - destination: host: istio-grpc-server subset: v1 weight: 90 - destination: host: istio-grpc-server subset: v2 weight: 10
Run the following command to log on to the default container of the pod of the gRPC client:
kubectl exec -it -n grpc-sidecar istio-grpc-client-dd56bcb45-zhfsg -c istio-grpc-client sh
After you log on to the container, run the following command:
/bin/greeter-client --insecure=true --address=istio-grpc-server:8080 --repeat=100
Expected output:
2024/02/07 14:56:16 Hello world from istio-grpc-server-v2-66d9b8847-99v62 2024/02/07 14:56:16 Hello world from istio-grpc-server-v1-546d9876c4-tndsm 2024/02/07 14:56:16 Hello world from istio-grpc-server-v1-546d9876c4-tndsm 2024/02/07 14:56:16 Hello world from istio-grpc-server-v1-546d9876c4-tndsm 2024/02/07 14:56:16 Hello world from istio-grpc-server-v1-546d9876c4-tndsm 2024/02/07 14:56:16 Hello world from istio-grpc-server-v1-546d9876c4-tndsm 2024/02/07 14:56:16 Hello world from istio-grpc-server-v1-546d9876c4-tndsm 2024/02/07 14:56:16 Hello world from istio-grpc-server-v1-546d9876c4-tndsm 2024/02/07 14:56:16 Hello world from istio-grpc-server-v1-546d9876c4-tndsm 2024/02/07 14:56:16 Hello world from istio-grpc-server-v1-546d9876c4-tndsm
The command output indicates that requests are routed to the two pods at a ratio of 90 to 10, instead of 50 to 50 in round-robin scheduling mode.
Adjust the values of the weight parameters in the virtual service and observe the distribution of requests.
Log on to the ASM console. In the left-side navigation pane, choose .
On the Mesh Management page, click the name of the ASM instance. In the left-side navigation pane, choose .
In the upper part of the VirtualService page, select the grpc-sidecar namespace from the Namespace drop-down list. Find the vs-istio-grpc-server virtual service and click YAML in the Actions column. In the Edit dialog box, change the values of the weight parameters in the virtual service and click OK.
Change the values of the weight parameters according to the values shown in the following code block:
route: - destination: host: istio-grpc-server subset: v1 weight: 0 - destination: host: istio-grpc-server subset: v2 weight: 100
Run the following command to log on to the default container of the pod of the gRPC client:
kubectl exec -it -n grpc-sidecar istio-grpc-client-dd56bcb45-zhfsg -c istio-grpc-client sh
After you log on to the container, run the following command:
/bin/greeter-client --insecure=true --address=istio-grpc-server:8080 --repeat=100
Expected output:
2024/02/07 14:58:30 Hello world from istio-grpc-server-v2-66d9b8847-99v62 2024/02/07 14:58:30 Hello world from istio-grpc-server-v2-66d9b8847-99v62 2024/02/07 14:58:30 Hello world from istio-grpc-server-v2-66d9b8847-99v62 2024/02/07 14:58:30 Hello world from istio-grpc-server-v2-66d9b8847-99v62 2024/02/07 14:58:30 Hello world from istio-grpc-server-v2-66d9b8847-99v62 2024/02/07 14:58:30 Hello world from istio-grpc-server-v2-66d9b8847-99v62 2024/02/07 14:58:30 Hello world from istio-grpc-server-v2-66d9b8847-99v62 2024/02/07 14:58:30 Hello world from istio-grpc-server-v2-66d9b8847-99v62 2024/02/07 14:58:30 Hello world from istio-grpc-server-v2-66d9b8847-99v62 2024/02/07 14:58:30 Hello world from istio-grpc-server-v2-66d9b8847-99v62
The command output indicates that all requests are routed to the pod of the version v2.