:Implement load balancing among gRPC servers

Sample project

Step 1: Create a gRPC service on the gRPC servers

1. Create a grpc-server-svc file that contains the following content:

   apiVersion: v1
   kind: Service
   metadata:
     namespace: grpc-best
     name: grpc-server-svc
     labels:
       app: grpc-server-svc
   spec:
     ports:
       - port: 9996
         name: grpc-port
     selector:
       app: grpc-server-deploy

2. Run the following command to create the gRPC service:

   kubectl apply -f grpc-server-svc.yaml

Step 2: Create a Deployment on each gRPC server

1. Create a grpc-server-node YAML file that contains the following content:

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     namespace: grpc-best
     name: grpc-server-node
     labels:
       app: grpc-server-deploy
       version: v3
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: grpc-server-deploy
         version: v3
     template:
       metadata:
         labels:
           app: grpc-server-deploy
           version: v3
       spec:
         containers:
           - name: grpc-server-deploy
             image: registry.cn-hangzhou.aliyuncs.com/aliacs-app-catalog/asm-grpc-server-node:1.0.0
             imagePullPolicy: Always
             ports:
               - containerPort: 9996
                 name: grpc-port

2. Run the following command to create the Deployment:

   kubectl apply -f grpc-server-node.yaml

Step 3: Create a Deployment on each gRPC client

You must create a Deployment on each of the four gRPC clients. The following steps show you how to use the grpc-client-go.yaml file of a Go language gRPC client to create a Deployment on the gRPC client.

1. Create a grpc-client-go file that contains the following content:

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     namespace: grpc-best
     name: grpc-client-go
     labels:
       app: grpc-client-go
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: grpc-client-go
     template:
       metadata:
         labels:
           app: grpc-client-go
       spec:
         containers:
           - name: grpc-client-go
             image: registry.cn-hangzhou.aliyuncs.com/aliacs-app-catalog/asm-grpc-client-go:1.0.0
             command: ["/bin/sleep", "3650d"]
             env:
               - name: GRPC_SERVER
                 value: "grpc-server-svc.grpc-best.svc.cluster.local"
             imagePullPolicy: Always

2. Run the following command to create the Deployment:

   kubectl apply -f grpc-client-go.yaml

The command: ["/bin/sleep", "3650d"] line indicates the command that is executed to keep the pod of the Go language gRPC client running in sleep mode after the pod is started. The GRPC_SERVER variable in env is set to grpc-server-svc.grpc-best.svc.cluster.local.

Step 4: Deploy the gRPC service and the Deployments

1. Run the following commands to create a namespace named grpc-best in the ACK cluster:

   alias k="kubectl --kubeconfig $USER_CONFIG"
   k create ns grpc-best

2. Run the following command to enable automatic sidecar injection for the namespace:

   k label ns grpc-best istio-injection=enabled

3. Run the following commands to deploy the gRPC service and the eight Deployments:

   kubectl apply -f grpc-svc.yaml
   kubectl apply -f deployment/grpc-server-java.yaml
   kubectl apply -f deployment/grpc-server-python.yaml
   kubectl apply -f deployment/grpc-server-go.yaml
   kubectl apply -f deployment/grpc-server-node.yaml
   kubectl apply -f deployment/grpc-client-java.yaml
   kubectl apply -f deployment/grpc-client-python.yaml
   kubectl apply -f deployment/grpc-client-go.yaml
   kubectl apply -f deployment/grpc-client-node.yaml

Verify the result

Use pods to verify the load balancing of the gRPC service

You can verify load balancing among gRPC servers by sending requests from the pods of the gRPC clients to the gRPC service on the gRPC servers.

1. Run the following commands to obtain the names of the pods of the four gRPC clients:

   client_java_pod=$(k get pod -l app=grpc-client-java -n grpc-best -o jsonpath={.items..metadata.name})

   client_go_pod=$(k get pod -l app=grpc-client-go -n grpc-best -o jsonpath={.items..metadata.name})

   client_node_pod=$(k get pod -l app=grpc-client-node -n grpc-best -o jsonpath={.items..metadata.name})

   client_python_pod=$(k get pod -l app=grpc-client-python -n grpc-best -o jsonpath={.items..metadata.name})

2. Run the following commands to send requests from the pods of the gRPC clients to the gRPC service on the four gRPC servers:

   k exec "$client_java_pod" -c grpc-client-java -n grpc-best -- java -jar /grpc-client.jar

   k exec "$client_go_pod" -c grpc-client-go -n grpc-best -- ./grpc-client

   k exec "$client_node_pod" -c grpc-client-node -n grpc-best -- node proto_client.js

   k exec "$client_python_pod" -c grpc-client-python -n grpc-best -- sh /grpc-client/start_client.sh

3. Use a FOR loop to verify load balancing among the gRPC servers. In this example, the Node.js language gRPC client is used.

   for ((i = 1; i <= 100; i++)); do
   kubectl exec "$client_node_pod" -c grpc-client-node -n grpc-best -- node kube_client.js > kube_result
   done
   sort kube_result | grep -v "^[[:space:]]*$" | uniq -c | sort -nrk1

   Expected output:

     26 Talk:PYTHON
     25 Talk:NODEJS
     25 Talk:GOLANG
     24 Talk:JAVA

   The output indicates that the four gRPC servers where the gRPC service is deployed receive an approximate number of requests. The load balancing result indicates that ASM can route external requests to the four gRPC servers on which the gRPC service is deployed based on a load balancing policy.

Use an ingress gateway to verify load balancing of the gRPC service

You can verify load balancing among gRPC servers by using an ingress gateway.

1. Create an ingress gateway that exposes the port 9996.

   For more information, see Create an ingress gateway service.

2. Use the following YAML code to create an Istio gateway:

   For more information, see Manage Istio gateways.

   apiVersion: networking.istio.io/v1alpha3
   kind: Gateway
   metadata:
     namespace: grpc-best
     name: grpc-gateway
   spec:
     selector:
       istio: ingressgateway
     servers:
       - port:
           number: 9996
           name: grpc
           protocol: GRPC
         hosts:
           - "*"

3. Use the following YAML code to create a virtual service:

   For more information, see Manage virtual services.

   apiVersion: networking.istio.io/v1alpha3
   kind: VirtualService
   metadata:
     namespace: grpc-best
     name: grpc-vs
   spec:
     hosts:
     - "*"
     gateways:
     - grpc-gateway
     http:
     - route:
       - destination:
           host: grpc-server-svc.grpc-best.svc.cluster.local
           port:
             number: 9996

4. Run the following command to obtain the IP address of the ingress gateway:

   INGRESS_IP=$(k -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')

5. Use a FOR loop to verify load balancing among the gRPC servers.

   docker run -d --name grpc_client_node -e GRPC_SERVER="${INGRESS_IP}" registry.cn-hangzhou.aliyuncs.com/aliacs-app-catalog/asm-grpc-client-node:1.0.0 /bin/sleep 3650d
   client_node_container=$(docker ps -q)
   
   docker exec -e GRPC_SERVER="${INGRESS_IP}" -it "$client_node_container" node kube_client.js
   
   for ((i = 1; i <= 100; i++)); do
   docker exec -e GRPC_SERVER="${INGRESS_IP}" -it "$client_node_container" node kube_client.js >> kube_result
   done
   sort kube_result | grep -v "^[[:space:]]*$" | uniq -c | sort -nrk1

   Expected output:

     26 Talk:PYTHON
     25 Talk:NODEJS
     25 Talk:GOLANG
     24 Talk:JAVA

   The output indicates that the four gRPC servers where the gRPC service is deployed receive an approximate number of requests. The load balancing result indicates that ASM can route external requests to the four gRPC servers on which the gRPC service is deployed based on a load balancing policy.