After gRPC clients send requests to access the grpc-server-svc.grpc-best.svc.cluster.local service that is specified by the GRPC_SERVER variable, Alibaba Cloud Service Mesh (ASM) can route the requests to gRPC servers in round robin mode. This topic describes how to deploy a gRPC service in a Container Service for Kubernetes (ACK) cluster to implement load balancing among gRPC servers. This topic also describes how to verify the load balancing of the gRPC service.

Background information

In this topic, four gRPC clients and four gRPC servers in Java, Go, Node.js, and Python are used. Assume that the gRPC clients call the grpc-server-svc.grpc-best.svc.cluster.local service that is specified by the GRPC_SERVER variable. When ASM receives the internal requests, ASM routes the requests to the four gRPC servers in round robin mode. In addition, you can configure an ingress gateway to route external requests to the four gRPC servers based on a load balancing policy. Examples

Sample project

For information about the sample project of gRPC, visit hello-servicemesh-grpc. The directories in this topic are directories of hello-servicemesh-grpc.
Note The image repository in this topic is for reference only. Use an image script to build and push images to your self-managed image repository. For more information about the image script, visit hello-servicemesh-grpc.

Step 1: Create a gRPC service on the gRPC servers

In this example, a gRPC service that is named grpc-server-svc is created on all gRPC servers.
Note The value of the spec.ports.name parameter must start with grpc.
  1. Create a YAML file that is named grpc-server-svc.
    apiVersion: v1
    kind: Service
    metadata:
      namespace: grpc-best
      name: grpc-server-svc
      labels:
        app: grpc-server-svc
    spec:
      ports:
        - port: 9996
          name: grpc-port
      selector:
        app: grpc-server-deploy
  2. Run the following command to create the gRPC service:
    kubectl apply -f grpc-server-svc.yaml

Step 2: Create a deployment on each gRPC server

In this step, you must create a deployment on each of the four gRPC servers. The following example describes how to use the grpc-server-node.yaml file of a Node.js-based gRPC server to create a deployment on the gRPC server. For more information about all the deployments for gRPC servers in other languages, visit https://github.com/AliyunContainerService/hello-servicemesh-grpc/tree/main/kube/deployment.
Note You must set the app label to grpc-server-deploy for the four deployments on the gRPC servers to match the selector of the gRPC service that you create in Step 1. The deployments on the four gRPC servers in different languages must have a unique version label.
  1. Create a YAML file that is named grpc-server-node.
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      namespace: grpc-best
      name: grpc-server-node
      labels:
        app: grpc-server-deploy
        version: v3
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: grpc-server-deploy
          version: v3
      template:
        metadata:
          labels:
            app: grpc-server-deploy
            version: v3
        spec:
          serviceAccountName: grpc-best-sa
          containers:
            - name: grpc-server-deploy
              image: registry.cn-beijing.aliyuncs.com/asm_repo/grpc_server_node:1.0.0
              imagePullPolicy: Always
              ports:
                - containerPort: 9996
                  name: grpc-port
  2. Run the following command to create the deployment:
    kubectl apply -f grpc-server-node.yaml

Step 3: Create a deployment on each gRPC client

The deployments for the gRPC clients and gRPC servers are different in the following aspects:
  • The gRPC servers continuously run after they are started. The gRPC clients stop running when the requests are complete. Therefore, an endless loop is required to keep client-side containers from stopping.
  • You must set the GRPC_SERVER variable on the gRPC clients. When the pod of a gRPC client is started, the value of the GRPC_SERVER variable is passed to the gRPC clients.

In this step, you must create a deployment on each of the four gRPC clients. The following example describes how to use the grpc-client-go.yaml file of a Go-based gRPC client to create a deployment on the gRPC client.

  1. Create a YAML file that is named grpc-client-go.
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      namespace: grpc-best
      name: grpc-client-go
      labels:
        app: grpc-client-go
    spec:
      replicas: 1
      selector:
        matchLabels:
          app: grpc-client-go
      template:
        metadata:
          labels:
            app: grpc-client-go
        spec:
          serviceAccountName: grpc-best-sa
          containers:
            - name: grpc-client-go
              image: registry.cn-beijing.aliyuncs.com/asm_repo/grpc_client_go:1.0.0
              command: ["/bin/sleep", "3650d"]
              env:
                - name: GRPC_SERVER
                  value: "grpc-server-svc.grpc-best.svc.cluster.local"
              imagePullPolicy: Always
  2. Run the following command to create the deployment:
    kubectl apply -f grpc-client-go.yaml

The command: ["/bin/sleep", "3650d"] line keeps the container running in sleep mode after the pod of the Go-based gRPC client is started. The GRPC_SERVER variable in env is set to grpc-server-svc.grpc-best.svc.cluster.local.

Step 4: Deploy the gRPC service and the deployments

  1. Run the following commands to create a namespace that is named grpc-best in the ACK instance:
    alias k="kubectl --kubeconfig $USER_CONFIG"
    k create ns grpc-best
  2. Run the following command to enable automatic sidecar injection for the namespace:
    k label ns grpc-best istio-injection=enabled
  3. Run the following command to deploy the ServiceAccount, the gRPC service, and the eight deployments:
    k apply -f grpc-sa.yaml
    k apply -f grpc-svc.yaml
    k apply -f deployment/grpc-server-java.yaml
    k apply -f deployment/grpc-server-python.yaml
    k apply -f deployment/grpc-server-go.yaml
    k apply -f deployment/grpc-server-node.yaml
    k apply -f deployment/grpc-client-java.yaml
    k apply -f deployment/grpc-client-python.yaml
    k apply -f deployment/grpc-client-go.yaml
    k apply -f deployment/grpc-client-node.yaml

View the details of the custom egress gateway service

Use pods to verify the load balancing of the gRPC service

You can check load balancing among gRPC servers by sending requests to the gRPC service on the gRPC servers from the pods of the gRPC clients.

  1. Run the following commands to obtain the names of the pods of the four gRPC clients:
    client_java_pod=$(k get pod -l app=grpc-client-java -n grpc-best -o jsonpath={.items..metadata.name})
    client_go_pod=$(k get pod -l app=grpc-client-go -n grpc-best -o jsonpath={.items..metadata.name})
    client_node_pod=$(k get pod -l app=grpc-client-node -n grpc-best -o jsonpath={.items..metadata.name})
    client_python_pod=$(k get pod -l app=grpc-client-python -n grpc-best -o jsonpath={.items..metadata.name})
  2. Run the following commands to send requests from the pods of the gRPC clients to the gRPC service on the four gRPC servers:
    k exec "$client_java_pod" -c grpc-client-java -n grpc-best -- java -jar /grpc-client.jar
    k exec "$client_go_pod" -c grpc-client-go -n grpc-best -- ./grpc-client
    k exec "$client_node_pod" -c grpc-client-node -n grpc-best -- node proto_client.js
    k exec "$client_python_pod" -c grpc-client-python -n grpc-best -- sh /grpc-client/start_client.sh
  3. Use a FOR loop to verify the load balancing among the gRPC servers. In this example, the Node.js gRPC client is used.
    for ((i = 1; i <= 100; i++)); do
      k exec "$client_node_pod" -c grpc-client-node -n grpc-best -- node kube_client.js > kube_result
    done
    sort kube_result | grep -v "^[[:space:]]*$" | uniq -c | sort -nrk1
    The following output is expected:
      26 Talk:PYTHON
      25 Talk:NODEJS
      25 Talk:GOLANG
      24 Talk:JAVA
    The output indicates that the four gRPC servers where the gRPC service is deployed receive an approximate number of requests. The load balancing result indicates that ASM can route external requests to the four gRPC servers where the gRPC service is deployed.

Use an ingress gateway to verify the load balancing

You can verify load balancing among gRPC servers by using the Istio ingress gateway.

  1. Log on to the ASM console.
  2. In the left-side navigation pane, choose Service Mesh > Mesh Management.
  3. On the Mesh Management page, find the ASM instance that you want to configure. Click the name of the ASM instance or click Manage in the Actions column of the ASM instance.
  4. In the Data Plane (Service Discovery) section, click the ASM Gateway tab.
  5. On the ASM Gateway tab, click Deploy Custom Ingress/Egress Gateway.
  6. In the Deploy Ingress Gateway panel, set Namespaces to the namespace that you created. Copy and paste the following code into the code editor. Then, click OK.
    apiVersion: networking.istio.io/v1alpha3
    kind: Gateway
    metadata:
      namespace: grpc-best
      name: grpc-gateway
    spec:
      selector:
        istio: ingressgateway
      servers:
        - port:
            number: 9996
            name: grpc
            protocol: GRPC
          hosts:
            - "*"
  7. Run the following command to obtain the IP address of the Istio ingress gateway:
    alias k="kubectl --kubeconfig $USER_CONFIG"
    INGRESS_IP=$(k -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
  8. Use a FOR loop to verify the load balancing among gRPC servers.
    docker run -d --name grpc_client_node -e GRPC_SERVER="${INGRESS_IP}" registry.cn-beijing.aliyuncs.com/asm_repo/grpc_client_node:1.0.0 /bin/sleep 3650d
    client_node_container=$(docker ps -q)
    
    docker exec -e GRPC_SERVER="${INGRESS_IP}" -it "$client_node_container" node kube_client.js
    
    for ((i = 1; i <= 100; i++)); do
      docker exec -e GRPC_SERVER="${INGRESS_IP}" -it "$client_node_container" node kube_client.js >> kube_result
    done
    sort kube_result | grep -v "^[[:space:]]*$" | uniq -c | sort -nrk1
    The following output is expected:
      26 Talk:PYTHON
      25 Talk:NODEJS
      25 Talk:GOLANG
      24 Talk:JAVA
    The output indicates that the four gRPC servers where the gRPC service is deployed receive an approximate number of requests. The load balancing result indicates that ASM can route external requests to the four gRPC servers where the gRPC service is deployed.