Implement load balancing among gRPC servers

Sample project

Step 1: Create a gRPC service on the gRPC servers

1. Create a YAML file named grpc-server-svc.
   e

   apiVersion: v1
   kind: Service
   metadata:
     namespace: grpc-best
     name: grpc-server-svc
     labels:
       app: grpc-server-svc
   spec:
     ports:
       - port: 9996
         name: grpc-port
     selector:
       app: grpc-server-deploy

2. Run the following command to create the gRPC service:

   kubectl apply -f grpc-server-svc.yaml

Step 2: Create a Deployment on each gRPC server

1. Create a YAML file named grpc-server-node.

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     namespace: grpc-best
     name: grpc-server-node
     labels:
       app: grpc-server-deploy
       version: v3
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: grpc-server-deploy
         version: v3
     template:
       metadata:
         labels:
           app: grpc-server-deploy
           version: v3
       spec:
         containers:
           - name: grpc-server-deploy
             image: registry.cn-hangzhou.aliyuncs.com/aliacs-app-catalog/asm-grpc-server-node:1.0.0
             imagePullPolicy: Always
             ports:
               - containerPort: 9996
                 name: grpc-port

2. Run the following command to create the Deployment:

   kubectl apply -f grpc-server-node.yaml

Step 3: Create a Deployment on each gRPC client

In this step, you must create a Deployment on each of the four gRPC clients. The following example shows you how to use the grpc-client-go.yaml file of a Go-based gRPC client to create a Deployment on the gRPC client.

1. Create a YAML file named grpc-client-go.

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     namespace: grpc-best
     name: grpc-client-go
     labels:
       app: grpc-client-go
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: grpc-client-go
     template:
       metadata:
         labels:
           app: grpc-client-go
       spec:
         containers:
           - name: grpc-client-go
             image: registry.cn-hangzhou.aliyuncs.com/aliacs-app-catalog/asm-grpc-client-go:1.0.0
             command: ["/bin/sleep", "3650d"]
             env:
               - name: GRPC_SERVER
                 value: "grpc-server-svc.grpc-best.svc.cluster.local"
             imagePullPolicy: Always

2. Run the following command to create the Deployment:

   kubectl apply -f grpc-client-go.yaml

The command: ["/bin/sleep", "3650d"] line keeps the container running in sleep mode after the pod of the Go-based gRPC client is started. The GRPC_SERVER variable in env is set to grpc-server-svc.grpc-best.svc.cluster.local.

Step 4: Deploy the gRPC service and the Deployments

1. Run the following commands to create a namespace named grpc-best in the ACK cluster:

   alias k="kubectl --kubeconfig $USER_CONFIG"
   k create ns grpc-best

2. Run the following command to enable automatic sidecar injection for the namespace:

   k label ns grpc-best istio-injection=enabled

3. Run the following commands to deploy the gRPC service and the eight Deployments:

   kubectl apply -f grpc-svc.yaml
   kubectl apply -f deployment/grpc-server-java.yaml
   kubectl apply -f deployment/grpc-server-python.yaml
   kubectl apply -f deployment/grpc-server-go.yaml
   kubectl apply -f deployment/grpc-server-node.yaml
   kubectl apply -f deployment/grpc-client-java.yaml
   kubectl apply -f deployment/grpc-client-python.yaml
   kubectl apply -f deployment/grpc-client-go.yaml
   kubectl apply -f deployment/grpc-client-node.yaml

Verify the result

Use pods to verify the load balancing of the gRPC service

You can check load balancing among gRPC servers by sending requests to the gRPC service on the gRPC servers from the pods of the gRPC clients.

1. Run the following commands to obtain the names of the pods of the four gRPC clients:

   client_java_pod=$(k get pod -l app=grpc-client-java -n grpc-best -o jsonpath={.items..metadata.name})

   client_go_pod=$(k get pod -l app=grpc-client-go -n grpc-best -o jsonpath={.items..metadata.name})

   client_node_pod=$(k get pod -l app=grpc-client-node -n grpc-best -o jsonpath={.items..metadata.name})

   client_python_pod=$(k get pod -l app=grpc-client-python -n grpc-best -o jsonpath={.items..metadata.name})

2. Run the following commands to send requests from the pods of the gRPC clients to the gRPC service on the four gRPC servers:

   k exec "$client_java_pod" -c grpc-client-java -n grpc-best -- java -jar /grpc-client.jar

   k exec "$client_go_pod" -c grpc-client-go -n grpc-best -- ./grpc-client

   k exec "$client_node_pod" -c grpc-client-node -n grpc-best -- node proto_client.js

   k exec "$client_python_pod" -c grpc-client-python -n grpc-best -- sh /grpc-client/start_client.sh

3. Use a FOR loop to verify the load balancing among the gRPC servers. In this example, the Node.js-based gRPC client is used.

   for ((i = 1; i <= 100; i++)); do
   kubectl exec "$client_node_pod" -c grpc-client-node -n grpc-best -- node kube_client.js > kube_result
   done
   sort kube_result  grep -v "^[[:space:]]*$"  uniq -c  sort -nrk1

   Expected output:

     26 Talk:PYTHON
     25 Talk:NODEJS
     25 Talk:GOLANG
     24 Talk:JAVA

   The output indicates that the four gRPC servers on which the gRPC service is deployed receive an approximate number of requests. The load balancing result indicates that ASM can route external requests to the four gRPC servers on which the gRPC service is deployed based on a load balancing policy.

Use an ingress gateway to verify the load balancing of the gRPC service

You can verify load balancing among gRPC servers by using the Istio ingress gateway.

1. Log on to the ASM console.
2. In the left-side navigation pane, choose Service Mesh > Mesh Management.
3. On the Mesh Management page, find the ASM instance that you want to configure. Click the name of the ASM instance or click Manage in the Actions column.
4. On the details page of the ASM instance, click ASM Gateways in the left-side navigation pane. On the ASM Gateways page, click Create from YAML.
5. On the Create page, select a namespace as required, copy the following content to the code editor, and then click Create.

   apiVersion: networking.istio.io/v1alpha3
   kind: Gateway
   metadata:
     namespace: grpc-best
     name: grpc-gateway
   spec:
     selector:
       istio: ingressgateway
     servers:
       - port:
           number: 9996
           name: grpc
           protocol: GRPC
         hosts:
           - "*"

6. Run the following command to obtain the IP address of the Istio ingress gateway:

   INGRESS_IP=$(k -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')

7. Use a FOR loop to verify the load balancing among the gRPC servers.

   docker run -d --name grpc_client_node -e GRPC_SERVER="${INGRESS_IP}" registry.cn-hangzhou.aliyuncs.com/aliacs-app-catalog/asm-grpc-client-node:1.0.0 /bin/sleep 3650d
   client_node_container=$(docker ps -q)
   
   docker exec -e GRPC_SERVER="${INGRESS_IP}" -it "$client_node_container" node kube_client.js
   
   for ((i = 1; i <= 100; i++)); do
   docker exec -e GRPC_SERVER="${INGRESS_IP}" -it "$client_node_container" node kube_client.js >> kube_result
   done
   sort kube_result  grep -v "^[[:space:]]*$"  uniq -c  sort -nrk1

   Expected output:

     26 Talk:PYTHON
     25 Talk:NODEJS
     25 Talk:GOLANG
     24 Talk:JAVA

   The output indicates that the four gRPC servers on which the gRPC service is deployed receive an approximate number of requests. The load balancing result indicates that ASM can route external requests to the four gRPC servers on which the gRPC service is deployed based on a load balancing policy.