Microservices Engine:Implement an end-to-end canary release based on Message Queue for Apache RocketMQ

Demo architecture

This topic uses an application deployed in the Container Service for Kubernetes (ACK) console as an example. You can also use kubectl to deploy the applications and simulate a real call chain. The following figure shows the architecture of the demo application. The calls between applications include both Spring Cloud and Dubbo calls, which are two of the most common microservice frameworks. Application C produces RocketMQ messages that are consumed by Application A. When Application A consumes the messages, it also initiates new calls. These applications are based on the standard usage of Spring Cloud, Dubbo, and RocketMQ.

Step 1: Connect applications to MSE Microservices Governance

• Enable Microservices Governance for microservice applications in the Container Service for Kubernetes (ACK) cluster. You can select an appropriate method based on your business requirements. For more information, see Enable Microservices Governance for Java microservice applications in an ACK or ACS cluster.

  Enable Microservices Governance for applications in an ACK namespace

  1. Log on to the MSE console, and select a region in the top navigation bar.

  2. In the left-side navigation pane, choose Microservices Governance > Application Governance.

  3. On the Application list page, click ACK Application Access.

  4. In the ACK Application Access dialog box, configure the parameters and click OK.

     

     Parameter	Description
     Cluster type	Select ACK Cluster, ACK Serverless Cluster, or ACS Cluster. Note If you do not grant ACK the permissions to call MSE, you need to click the link to perform authorization.
     Cluster Name/ID	Select the Cluster Name/ID for which you want to enable Microservices Governance. You can use keywords to search for cluster names or IDs.
     ack-onepilot	Display the status of the ack-onepilot component. For information about the ack-onepilot component and how to update the ack-onepilot component, see ack-onepilot and Install and update the Microservices Governance component. If the ack-onepilot component is not installed, click Click Install on the right of ack-onepilot. After the installation is complete, the status of the ack-onepilot component is displayed as Installed. If you perform the operation as a RAM user and a message is displayed, indicating that you have no permissions to perform the operation, you can log on to the ACK console, and click the name of the cluster. On the page that appears, choose Operations > Add-ons. On the Add-ons page, search for ack-onepilot and click Install on the resource card. Note In this step, the ack-onepilot component is involved. You can log on to the ACK console, and click the name of the cluster. On the page that appears, choose Operations > Add-ons to view the component details. After the ack-onepilot component is installed, the agent is automatically injected. In this case, the startup time of the application may be increased by up to 10s. When you enable Microservices Governance for a cluster namespace, if the desired cluster is not in the following regions, you must make sure that the cluster can access the Internet and connect to acm.aliyun.com:8080: China (Qingdao), China (Hangzhou), China (Beijing), China (Shanghai), China East 2 Finance, China (Shenzhen), China (Hong Kong), Singapore, Germany (Frankfurt), Australia (Sydney), US (Silicon Valley), and US (Virginia).
     Access Type	Select Namespace Access.
     ACK Cluster Namespace	Select an ACK Cluster Namespace.
     Microservices Governance Namespace	Select a Microservices Governance Namespace.

  Enable Microservices Governance for an application

  1. Log on to the MSE console, and select a region in the top navigation bar.

  2. In the left-side navigation pane, choose Microservices Governance > Application Governance.

  3. On the Application list page, click ACK Application Access.

  4. In the ACK Application Access dialog box, configure the parameters and click OK.

     

     Parameter	Description
     Cluster type	Select ACK Cluster, ACK Serverless Cluster, or ACS Cluster. Note If you do not grant ACK the permissions to call MSE, you need to click the link to perform authorization.
     Cluster Name/ID	Select the Cluster Name/ID for which you want to enable Microservices Governance. You can use keywords to search for cluster names or IDs.
     ack-onepilot	Display the status of the ack-onepilot component. For information about the ack-onepilot component and how to update the ack-onepilot component, see ack-onepilot and Install and update the Microservices Governance component. If the ack-onepilot component is not installed, click Click Install on the right of ack-onepilot. After the installation is complete, the status of the ack-onepilot component is displayed as Installed. If you perform the operation as a RAM user and a message is displayed, indicating that you have no permissions to perform the operation, you can log on to the ACK console, and click the name of the cluster. On the page that appears, choose Operations > Add-ons. On the Add-ons page, search for ack-onepilot and click Install on the resource card. Note In this step, the ack-onepilot component is involved. You can log on to the ACK console, and click the name of the cluster. On the page that appears, choose Operations > Add-ons to view the component details. After the ack-onepilot component is installed, the agent is automatically injected. In this case, the startup time of the application may be increased by up to 10s.
     Access Type	Select Single Application Access.
     Access Procedure	Perform the following steps: Step 1: Go to the ACK console, and click the cluster name. In the left-side navigation pane of the page that appears, choose Workloads > Deployments. Then, select the namespace to which the application belongs. Step 2: Click the name of the application. On the page that appears, click View in YAML. Step 3: Edit labels in the following format and click Update. spec: template: metadata: labels: # The on value indicates that Microservices Governance is enabled. The on value must be enclosed in double quotation marks ("). msePilotAutoEnable: "on" # The MSE microservice namespace where your application resides. If the namespace that you specify does not exist, a namespace with the specified name is automatically created. mseNamespace: default # The actual name of the application for which Microservices Governance is enabled. The name must be enclosed in double quotation marks ("). msePilotCreateAppName: "your-deployment-name"

Step 2: Deploy the applications

Deploy the spring-cloud-zuul, spring-cloud-a, spring-cloud-b, and spring-cloud-c applications, along with the Nacos server and RocketMQ server. You can also obtain the source code directly from the Demo.

When the spring-cloud-zuul application receives a request for /A/dubbo, it forwards the request to spring-cloud-a. Then, spring-cloud-a uses the Dubbo protocol to call spring-cloud-b, which in turn calls spring-cloud-c. After receiving the request, spring-cloud-c produces a message and returns its environment tag and IP address. These messages are then consumed by the spring-cloud-a application. When consuming a message, spring-cloud-a uses Spring Cloud to call spring-cloud-b, which then calls spring-cloud-c. Finally, spring-cloud-a outputs the result to its logs.

# When /A/dubbo is accessed, the return value is: A[10.25.xx.xx] -> B[10.25.xx.xx] -> C[10.25.xx.xx].
# At the same time, spring-cloud-a receives the message and logs the following information:

2021-12-28 10:58:50.301  INFO 1 --- [essageThread_15] c.a.mse.demo.service.MqConsumer
          : topic:TEST_MQ,producer:C[10.25.xx.xx],invoke result:A[10.25.xx.xx] -> B[10.25.xx.xx] -> C[10.25.xx.xx]

1. Log on to the ACK console. In the navigation pane on the left, click Clusters.

2. On the Clusters page, find the cluster that you want to manage and click the name of the cluster or click Details in the Actions column. The details page of the cluster appears.

3. In the left-side navigation pane of the details page, choose Workloads > Deployments.

4. On the Deployments page, select a Namespace and click Create From YAML to deploy an application with the following YAML code:

   Expand to view the YAML file

   # Deploy the Nacos server.
   
   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: nacos-server
   spec:
     selector:
       matchLabels:
         app: nacos-server
     template:
       metadata:
         labels:
           app: nacos-server
       spec:
         containers:
           - env:
               - name: MODE
                 value: "standalone"
             image: registry.cn-hangzhou.aliyuncs.com/mse-governance-demo/nacos-server:v2.1.2
             imagePullPolicy: IfNotPresent
             name: nacos-server
             ports:
               - containerPort: 8848
   
   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: nacos-server
   spec:
     type: ClusterIP
     selector:
       app: nacos-server
     ports:
       - name: http
         port: 8848
         targetPort: 8848
   
   # Deploy the business applications.
   ---
   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: spring-cloud-zuul
   spec:
     selector:
       matchLabels:
         app: spring-cloud-zuul
     template:
       metadata:
         labels:
           app: spring-cloud-zuul
           msePilotCreateAppName: spring-cloud-zuul
       spec:
         containers:
           - env:
               - name: JAVA_HOME
                 value: /usr/lib/jvm/java-1.8-openjdk/jre
               - name: enable.mq.invoke
                 value: 'true'
             image: registry.cn-hangzhou.aliyuncs.com/mse-governance-demo/spring-cloud-zuul:3.0.1
             imagePullPolicy: Always
             name: spring-cloud-zuul
             ports:
               - containerPort: 20000
   
   ---
   apiVersion: v1
   kind: Service
   metadata:
     annotations:
       service.beta.kubernetes.io/alibaba-cloud-loadbalancer-spec: slb.s1.small
       service.beta.kubernetes.io/alicloud-loadbalancer-address-type: internet
     name: zuul-slb
   spec:
     ports:
       - port: 80
         protocol: TCP
         targetPort: 20000
     selector:
       app: spring-cloud-zuul
     type: LoadBalancer
   status:
     loadBalancer: {}
   
   ---
   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: spring-cloud-a
   spec:
     selector:
       matchLabels:
         app: spring-cloud-a
     template:
       metadata:
         labels:
           app: spring-cloud-a
           msePilotCreateAppName: spring-cloud-a
       spec:
         containers:
           - env:
               - name: JAVA_HOME
                 value: /usr/lib/jvm/java-1.8-openjdk/jre
             image: registry.cn-hangzhou.aliyuncs.com/mse-governance-demo/spring-cloud-a:3.0.1
             imagePullPolicy: Always
             name: spring-cloud-a
             ports:
               - containerPort: 20001
             livenessProbe:
               tcpSocket:
                 port: 20001
               initialDelaySeconds: 10
               periodSeconds: 30
   
   
   ---
   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: spring-cloud-b
   spec:
     selector:
       matchLabels:
         app: spring-cloud-b
     template:
       metadata:
         labels:
           app: spring-cloud-b
           msePilotCreateAppName: spring-cloud-b
       spec:
         containers:
           - env:
               - name: JAVA_HOME
                 value: /usr/lib/jvm/java-1.8-openjdk/jre
             image: registry.cn-hangzhou.aliyuncs.com/mse-governance-demo/spring-cloud-b:3.0.1
             imagePullPolicy: Always
             name: spring-cloud-b
             ports:
               - containerPort: 20002
             livenessProbe:
               tcpSocket:
                 port: 20002
               initialDelaySeconds: 10
               periodSeconds: 30
   
   ---
   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: spring-cloud-c
   spec:
     selector:
       matchLabels:
         app: spring-cloud-c
     template:
       metadata:
         labels:
           app: spring-cloud-c
           msePilotCreateAppName: spring-cloud-c
       spec:
         containers:
           - env:
               - name: JAVA_HOME
                 value: /usr/lib/jvm/java-1.8-openjdk/jre
             image: registry.cn-hangzhou.aliyuncs.com/mse-governance-demo/spring-cloud-c:3.0.1
             imagePullPolicy: Always
             name: spring-cloud-c
             ports:
               - containerPort: 20003
             livenessProbe:
               tcpSocket:
                 port: 20003
               initialDelaySeconds: 10
               periodSeconds: 30
   ---
   
   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: rockectmq-broker
   spec:
     selector:
       matchLabels:
         app: rockectmq-broker
     template:
       metadata:
         labels:
           app: rockectmq-broker
       spec:
         containers:
           - command:
               - sh
               - mqbroker
               - '-n'
               - 'mqnamesrv:9876'
               - '-c /home/rocketmq/rocketmq-4.9.7/conf/broker.conf'
             env:
               - name: ROCKETMQ_HOME
                 value: /home/rocketmq/rocketmq-4.9.7
             image: registry.cn-hangzhou.aliyuncs.com/mse-governance-demo/rocketmq:4.9.7
             imagePullPolicy: Always
             name: rockectmq-broker
             ports:
               - containerPort: 9876
                 protocol: TCP
               - containerPort: 10911
                 protocol: TCP
               - containerPort: 10912
                 protocol: TCP
               - containerPort: 10909
   
   ---
   
   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: rocketmq-name-server
   spec:
     selector:
       matchLabels:
         app: rocketmq-name-server
     template:
       metadata:
         labels:
           app: rocketmq-name-server
       spec:
         containers:
           - command:
               - sh
               - mqnamesrv
             env:
               - name: ROCKETMQ_HOME
                 value: /home/rocketmq/rocketmq-4.9.7
             image: registry.cn-hangzhou.aliyuncs.com/mse-governance-demo/rocketmq:4.9.7
             imagePullPolicy: Always
             name: rocketmq-name-server
             ports:
               - containerPort: 9876
                 protocol: TCP
               - containerPort: 10911
                 protocol: TCP
               - containerPort: 10912
                 protocol: TCP
               - containerPort: 10909
                 protocol: TCP
   
   ---
   
   apiVersion: v1
   kind: Service
   metadata:
     name: mqnamesrv
   spec:
     type: ClusterIP
     selector:
       app: rocketmq-name-server
     ports:
       - name: mqnamesrv-9876-9876
         port: 9876
         targetPort: 9876
                           

5. Run the following command to check whether the applications are deployed.

   kubectl get svc,deploy

   Expected output:

   NAME                   TYPE           CLUSTER-IP       EXTERNAL-IP    PORT(S)        AGE
   service/kubernetes     ClusterIP      192.168.xx.xx    <none>         4xx/TCP        7d
   service/mqnamesrv      ClusterIP      192.168.xx.xx    <none>         98xx/TCP       47h
   service/nacos-server   ClusterIP      192.168.xx.xx    <none>         88xx/TCP       47h
   service/zuul-slb       LoadBalancer   192.168.xx.xx   123.56.xx.xx   80:302xxx/TCP   47h
   
   NAME                                   READY   UP-TO-DATE   AVAILABLE   AGE
   deployment.apps/nacos-server           1/1     1            1           4m
   deployment.apps/rockectmq-broker       1/1     1            1           4m
   deployment.apps/rocketmq-name-server   1/1     1            1           5m
   deployment.apps/spring-cloud-a         1/1     1            1           5m
   deployment.apps/spring-cloud-b         1/1     1            1           5m
   deployment.apps/spring-cloud-c         1/1     1            1           5m
   deployment.apps/spring-cloud-zuul      1/1     1            1           5m

Step 3: Enable RocketMQ message-based canary release for the applications

Assume that spring-cloud-c is the message producer and spring-cloud-a is the message consumer. Enable RocketMQ message-based canary release for both applications.

1. Log on to the MSE console, and select a region in the top navigation bar.

2. In the left-side navigation pane, choose Microservices Governance > Application Governance.

3. On the Application list page, click the resource card of the desired application. In the left-side navigation pane, click Traffic management.

4. On the Message Grayscale tab, turn on the Enable Message Grayscale switch and click OK.

   

5. Log on to the ACK console ACK console and redeploy the application to apply the configuration.

Step 4: Deploy the new version applications

Deploy the new versions of the applications: spring-cloud-a-gray, spring-cloud-b-gray, and spring-cloud-c-gray.

1. Log on to the ACK console. In the navigation pane on the left, click Clusters.

2. On the Clusters page, find the cluster that you want to manage and click the name of the cluster or click Details in the Actions column. The details page of the cluster appears.

3. In the left-side navigation pane of the details page, choose Workloads > Deployments.

4. On the Deployments page, select a Namespace and click Create From YAML. Use the following YAML code to deploy the applications:

   Expand to view the YAML file

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: spring-cloud-a-gray
   spec:
     selector:
       matchLabels:
         app: spring-cloud-a-gray
     template:
       metadata:
         labels:
           alicloud.service.tag: gray
           app: spring-cloud-a-gray
           msePilotCreateAppName: spring-cloud-a
       spec:
         containers:
           - env:
               - name: JAVA_HOME
                 value: /usr/lib/jvm/java-1.8-openjdk/jre
             image: registry.cn-hangzhou.aliyuncs.com/mse-governance-demo/spring-cloud-a:3.0.1
             imagePullPolicy: Always
             name: spring-cloud-a-gray
             ports:
               - containerPort: 20001
             livenessProbe:
               tcpSocket:
                 port: 20001
               initialDelaySeconds: 10
               periodSeconds: 30
   ---
   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: spring-cloud-b-gray
   spec:
     selector:
       matchLabels:
         app: spring-cloud-b-gray
     template:
       metadata:
         labels:
           alicloud.service.tag: gray 
           app: spring-cloud-b-gray
           msePilotCreateAppName: spring-cloud-b
       spec:
         containers:
           - env:
               - name: JAVA_HOME
                 value: /usr/lib/jvm/java-1.8-openjdk/jre
             image: registry.cn-hangzhou.aliyuncs.com/mse-governance-demo/spring-cloud-b:3.0.1
             imagePullPolicy: Always
             name: spring-cloud-b-gray
             ports:
               - containerPort: 20002
             livenessProbe:
               tcpSocket:
                 port: 20002
               initialDelaySeconds: 10
               periodSeconds: 30
   
   ---
   
   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: spring-cloud-c-gray
   spec:
     selector:
       matchLabels:
         app: spring-cloud-c-gray
     template:
       metadata:
         labels:
           alicloud.service.tag: gray
           app: spring-cloud-c-gray
           msePilotCreateAppName: spring-cloud-c
       spec:
         containers:
           - env:
               - name: JAVA_HOME
                 value: /usr/lib/jvm/java-1.8-openjdk/jre
             image: registry.cn-hangzhou.aliyuncs.com/mse-governance-demo/spring-cloud-c:3.0.1
             imagePullPolicy: Always
             name: spring-cloud-c-gray
             ports:
               - containerPort: 20003
             livenessProbe:
               tcpSocket:
                 port: 20003
               initialDelaySeconds: 10
               periodSeconds: 30

Step 5: Route traffic and verify the result

1. Log on to the MSE console, and select a region in the top navigation bar.

2. In the left-side navigation pane, choose Microservices Governance > Application Governance.

3. Click the resource card for the spring-cloud-a application. On the Application Overview page, the QPS Trend Graph shows that all traffic is routed to the application's baseline version, which is its stable version.

4. In the navigation pane on the left, click Traffic Governance and then the Tag-based Routing tab. In the Traffic Rule column for the gray tag, click Add.

   

5. In the Create Label Route panel, configure a traffic rule and click OK.

   In this example, the traffic rule condition is set to name=xiaoming. For more information about how to configure traffic rules, see Configure tag-based routing.

   

   You can view the traffic distribution on the Application Overview page after the traffic rule takes effect.

6. On the Tag-based Routing tab, click the name of the destination rule for the gray tag. In the Rule Details panel, click Edit. In the Modify Tag-based Routing Rule panel, enable the Tag Propagation switch and click OK.

   Note

   After you enable tag propagation, grayscale traffic that matches name=xiaoming is passed through end-to-end, and you do not need to configure the rule repeatedly.

7. Log on to the ACK console. In the navigation pane on the left, click Clusters.

8. On the Clusters page, find the cluster you want and click its name. In the left-side pane, choose Network > Services.

9. Click the zuul-slb service. Under Basic Information, click the address next to External IP Address (External IP).

10. On the service call page, enter /A/dubbo?name=xiaoming and click Start Call.

    The end-to-end canary release is now in effect.

    

    The traffic data for the spring-cloud-b application is now as follows:

    

Step 6: Adjust the tag-based message filtering rule and verify the result

1. Log on to the MSE console.

2. In the navigation pane on the left, select Microservices Governance > Application Governance and click the resource card for the spring-cloud-a application.

3. In the navigation pane on the left, click Traffic Governance and then click the Message Grayscale tab.

4. On the Message Grayscale tab, click the Edit button next to Tags To Be Ignored By The Baseline Environment. Select gray in the input box and click OK.

   If you select gray for the Tags to Ignore in the Baseline Environment parameter, messages with the `gray` environment tag can be consumed only by `spring-cloud-a-gray`, not by `spring-cloud-a`. The Tags to Ignore in the Baseline Environment parameter takes effect dynamically after it is configured, and you do not need to restart the application.

   Note

   By default, baseline nodes consume messages from all environments. If you do not want a baseline node to consume messages produced by a tagged environment, you only need to configure the Tags Ignored in Baseline Environment parameter.

5. Log on to the ACK console to view the logs for the spring-cloud-a application and verify the configuration.

   The log in the following figure shows that the baseline environment can consume messages produced in both the gray and baseline environments. After you set the Tags to Ignore in the Baseline Environment parameter to `gray`, the baseline environment consumes only messages produced in the baseline environment.