Route traffic to multi-cluster gRPC services - Container Service for Kubernetes

The Application Load Balancer (ALB) multi-cluster gateways of Distributed Cloud Container Platform for Kubernetes (ACK One) can be used to route traffic to gRPC services. This topic describes how to use ALB multi-cluster gateways to route traffic to gRPC services.

Prerequisites

ALB is activated.
The Fleet management feature is enabled. For more information, see Enable multi-cluster management.
The ACK One Fleet instance is associated with two ACK clusters that are deployed in the same virtual private cloud (VPC) as the ACK One Fleet instance. For more information, see Manage associated clusters.
The kubeconfig file of the Fleet instance is obtained in the Distributed Cloud Container Platform for Kubernetes (ACK One) console and a kubectl client is connected to the Fleet instance.
grpccurl is installed.
A trusted certificate is obtained. You can obtain a certificate in one of the following ways:
- Purchase a certificate in the Certificate Management Service console. For more information, see Purchase an SSL certificate.
- Purchase a certificate from other certificate authorities (CAs).
- Generate a self-signed CA certificate by following the operations in Step 1.
  Important
  By default, self-signed certificates are not trusted by browsers or clients. If you use a self-signed certificate, you may receive security warnings. The self-signed certificate generated in this example is for reference only. Do not use the certificate in the production environment. If you have purchased a certificate, proceed to Step 2.

Step 1: Generate a self-signed CA certificate

If you do not own a certificate, you can generate a self-signed certificate. In this example, the grpc.example.com domain name is bound to the certificate. For more information about how to configure certificates for ALB instances, see Configure HTTPS certificates for encrypted communication. In this example, a Secret is used to configure a certificate for an ALB instance.

Create a file named /tmp/openssl.cnf and copy the following content to the file:

[ req ]
#default_bits           = 2048
#default_md             = sha256
#default_keyfile        = privkey.pem
distinguished_name      = req_distinguished_name
attributes              = req_attributes
req_extensions          = v3_req

[ req_distinguished_name ]
countryName                     = Country Name (2 letter code)
countryName_min                 = 2
countryName_max                 = 2
stateOrProvinceName             = State or Province Name (full name)
localityName                    = Locality Name (eg, city)
0.organizationName              = Organization Name (eg, company)
organizationalUnitName          = Organizational Unit Name (eg, section)
commonName                      = Common Name (eg, fully qualified host name)
commonName_max                  = 64
emailAddress                    = Email Address
emailAddress_max                = 64

[ req_attributes ]
challengePassword               = A challenge password
challengePassword_min           = 4
challengePassword_max           = 20

[v3_req]
# Extensions to add to a certificate request
basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName = @alt_names

[alt_names]
DNS.1 = grpc.example.com

Run the following command to generate a certificate signing request:

openssl req -new -nodes -keyout grpc.key -out grpc.csr -config /tmp/openssl.cnf -subj "/C=CN/ST=Zhejiang/L=Hangzhou/O=AlibabaCloud/OU=ContainerService/CN=grpc.example.com"

Run the following command to sign the certificate:
```
openssl x509 -req -days 3650 -in grpc.csr -signkey grpc.key -out grpc.crt -extensions v3_req -extfile /tmp/openssl.cnf
```
After the command is executed, the certificate file grpc.crt and the private key file grpc.key are generated.

Run the following command to create a Secret named grpc-secret in the gateway-demo namespace. The Secret is used to store the certificate.

kubectl create secret tls grpc-secret --key grpc.key --cert grpc.crt -ngateway-demo   # Replace grpc.key with the actual name of your certificate file and grpc.crt with the actual name of your private key file.

Step 2: Deploy a gRPC service to multiple clusters

ACK One allows you to use GitOps and the application distribution feature to deploy an application to multiple clusters. For more information, see Getting started with GitOps, Create a multi-cluster application, and Get started with application distribution. In this step, GitOps is used.

Log on to the ACK One console. In the left-side navigation pane, choose Fleet > Multi-cluster Applications.
In the upper-left corner of the Multi-cluster Applications page, click to the right of the Fleet instance name and select your Fleet instance from the drop-down list.
Choose Create Multi-cluster Application > GitOps to go to the Create Multi-cluster Application - GitOps page.

On the Create from YAML tab, copy the following YAML template to the code editor. Then, click OK to deploy the application.

Note

The following YAML template is used to deploy the application in the gateway-demo namespace in each associated cluster. You can also select the clusters where you want to deploy the application on the Quick Create tab. The configuration changes that you make on the Quick Create tab will be automatically synchronized to the YAML template on the Create from YAML tab.
If the gateway-demo namespace does not exist, you must create it.

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: appset-grpc-demo
  namespace: argocd
spec:
  template:
    metadata:
      name: '{{.metadata.annotations.cluster_name}}-grpc'
      namespace: argocd
    spec:
      destination:
        name: '{{.name}}'
        namespace: gateway-demo
      project: default
      source:
        repoURL: https://github.com/ivan-cai/gitops-demo.git
        path: manifests/directory/grpc
        targetRevision: main
      syncPolicy:
        automated: {}
        syncOptions:
          - CreateNamespace=true
  generators:
    - clusters:
        selector:
          matchExpressions:
            - values:
                - cluster
              key: argocd.argoproj.io/secret-type
              operator: In
            - values:
                - in-cluster
              key: name
              operator: NotIn
  goTemplateOptions:
    - missingkey=error
  syncPolicy:
    preserveResourcesOnDeletion: false
  goTemplate: true

Step 3: Create an ALB multi-cluster gateway and an Ingress

You can use an AlbConfig to create an ALB multi-cluster gateway from the ACK One Fleet instance. You can associate clusters with the gateway. You must use HTTPS for gRPC services. Therefore, you must specify an HTTPS port in the listeners parameter of the AlbConfig configurations. In this example, port 443 is used.

Obtain the IDs of two vSwitches that belong to the virtual private cloud (VPC) where the ACK One Fleet instance resides.

Create a file named gateway.yaml and copy the following content to the file.

Note

Replace ${vsw-id1} and ${vsw-id2} with the vSwitch IDs that you obtained. Replace ${cluster1} and ${cluster2} with the IDs of the associated clusters.
Configure the security groups of ${cluster1} and ${cluster2} to open all ports and allow access from IP addresses in the CIDR block of the vSwitch.

apiVersion: alibabacloud.com/v1
kind: AlbConfig
metadata:
  name: ackone-gateway-demo
  annotations:
    # Specify the IDs of the clusters that you want to associate with the ALB instance. 
    alb.ingress.kubernetes.io/remote-clusters: ${cluster1},${cluster2}
spec:
  config:
    name: one-alb-demo
    addressType: Internet
    addressAllocatedMode: Fixed
    zoneMappings:
    - vSwitchId: ${vsw-id1}
    - vSwitchId: ${vsw-id2}
  listeners:
  - port: 8001
    protocol: HTTP
  - port: 443
    protocol: HTTPS
---
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  name: alb
spec:
  controller: ingress.k8s.alibabacloud/alb
  parameters:
    apiGroup: alibabacloud.com
    kind: AlbConfig
    name: ackone-gateway-demo

Create an Ingress by using the following template.

Note

You must add the alb.ingress.kubernetes.io/backend-protocol: grpc and alb.ingress.kubernetes.io/listen-ports annotations in the Ingress configurations. In addition, you must specify the Secret certificate you created in Step 1.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    alb.ingress.kubernetes.io/listen-ports: |
     [{"HTTPS": 443}]
    alb.ingress.kubernetes.io/backend-protocol: grpc
  name: grpc-ingress
  namespace: gateway-demo
spec:
  ingressClassName: alb
  rules:
  - host: grpc.example.com
    http:
      paths:
      - backend:
          service:
            name: grpc-service
            port:
              number: 50051
        path: /
        pathType: Prefix
  tls:
  - hosts:
    - grpc.example.com
    secretName: grpc-secret

Step 4: Check whether traffic is forwarded as expected

Run the following command to view the Ingress:

kubectl get ingress -ngateway-dmeo

Expected output:

NAME           CLASS   HOSTS              ADDRESS         PORTS     AGE
grpc-ingress   alb     grpc.example.com   alb-*****       80, 443   3m51s

Use grpcurl to access the service.

# Replace <ADDRESS> with the domain name that is associated with the ABL Ingress. 
grpcurl -insecure -authority grpc.example.com <ADDRESS>:443 list

The following output indicates that the request is forwarded to the backend gRPC service.

grpc.reflection.v1alpha.ServerReflection
helloworld.Greeter