Microservices Engine:Use Kruise Rollouts to implement MSE-based end-to-end canary releases

Canary release lifecycle

When a Deployment image changes, Kruise Rollouts follows this sequence:

1. Pause the Deployment's native rolling update.

2. Create canary pods with the new image version.

3. Configure traffic routing so that matching requests reach the canary pods.

4. Wait for manual approval (or automated metric checks).

5. On approval, promote the canary version to all pods using a rolling update.

6. Remove the canary pods and restore native traffic routing.

Install ack-kruise

1. Log on to the ACK console. In the left-side navigation pane, click Clusters.

2. On the Clusters page, find your target cluster and click its name. In the left-side navigation pane, choose Operations > Add-ons.

3. On the Manage Components page, click the Manage Applications tab.

4. Find the ack-kruise card and click Install.

5. In the confirmation dialog, click OK.

Install the MSE Ingress Controller

Create an MseIngressConfig and an IngressClass resource to enable MSE-based ingress for your cluster. For detailed steps, see Use MSE Ingresses to access applications in ACK clusters.

Enable Microservices Governance

Enable Microservices Governance for your target applications. For detailed steps, see Enable Microservices Governance for microservice applications in an ACK cluster.

Create the application resources

1. Create a file named mse-demo.yaml with the following content:

   View the mse-demo.yaml file

       # Service for application A
       apiVersion: v1
       kind: Service
       metadata:
         name: spring-cloud-a
       spec:
         ports:
           - name: http
             port: 20001
             protocol: TCP
             targetPort: 20001
         selector:
           app: spring-cloud-a
       ---
       # Deployment for application A (base version)
       apiVersion: apps/v1
       kind: Deployment
       metadata:
         name: spring-cloud-a
       spec:
         replicas: 2
         selector:
           matchLabels:
             app: spring-cloud-a
         template:
           metadata:
             annotations:
               msePilotCreateAppName: spring-cloud-a
             labels:
               app: spring-cloud-a
           spec:
             containers:
               - env:
                   - name: JAVA_HOME
                     value: /usr/lib/jvm/java-1.8-openjdk/jre
                 image: registry.cn-hangzhou.aliyuncs.com/mse-demo-hz/spring-cloud-a:mse-2.0.0
                 imagePullPolicy: Always
                 name: spring-cloud-a
                 ports:
                   - containerPort: 20001
       ---
       # Deployment for application B (base version)
       apiVersion: apps/v1
       kind: Deployment
       metadata:
         name: spring-cloud-b
       spec:
         replicas: 2
         selector:
           matchLabels:
             app: spring-cloud-b
         template:
           metadata:
             annotations:
               msePilotCreateAppName: spring-cloud-b
             labels:
               app: spring-cloud-b
           spec:
             containers:
               - env:
                   - name: JAVA_HOME
                     value: /usr/lib/jvm/java-1.8-openjdk/jre
                 image: registry.cn-hangzhou.aliyuncs.com/mse-demo-hz/spring-cloud-b:mse-2.0.0
                 imagePullPolicy: Always
                 name: spring-cloud-b
       ---
       # Deployment for application C (base version)
       apiVersion: apps/v1
       kind: Deployment
       metadata:
         name: spring-cloud-c
       spec:
         replicas: 2
         selector:
           matchLabels:
             app: spring-cloud-c
         template:
           metadata:
             annotations:
               msePilotCreateAppName: spring-cloud-c
             labels:
               app: spring-cloud-c
           spec:
             containers:
               - env:
                   - name: JAVA_HOME
                     value: /usr/lib/jvm/java-1.8-openjdk/jre
                 image: registry.cn-hangzhou.aliyuncs.com/mse-demo-hz/spring-cloud-c:mse-2.0.0
                 imagePullPolicy: Always
                 name: spring-cloud-c
       ---
       # Nacos service discovery server
       apiVersion: apps/v1
       kind: Deployment
       metadata:
         name: nacos-server
       spec:
         replicas: 1
         selector:
           matchLabels:
             app: nacos-server
         template:
           metadata:
             labels:
               app: nacos-server
           spec:
             containers:
             - env:
               - name: MODE
                 value: standalone
               image: registry.cn-hangzhou.aliyuncs.com/mse-governance-demo/nacos-server:v2.1.2
               imagePullPolicy: Always
               name: nacos-server
             dnsPolicy: ClusterFirst
             restartPolicy: Always
       ---
       apiVersion: v1
       kind: Service
       metadata:
         name: nacos-server
       spec:
         type: ClusterIP
         ports:
           - name: nacos-server-8848-8848
             port: 8848
             protocol: TCP
             targetPort: 8848
           - name: nacos-server-9848-9848
             port: 9848
             protocol: TCP
             targetPort: 9848
         selector:
           app: nacos-server
       ---
       # MySQL database
       apiVersion: v1
       kind: Service
       metadata:
         labels:
           app: demo-mysql
         name: demo-mysql
       spec:
         ports:
           - port: 3306
             targetPort: 3306
         selector:
           app: demo-mysql
       ---
       apiVersion: apps/v1
       kind: Deployment
       metadata:
         name: demo-mysql
       spec:
         selector:
           matchLabels:
             app: demo-mysql
         replicas: 1
         strategy:
           type: Recreate
         template:
           metadata:
             labels:
               app: demo-mysql
           spec:
             containers:
               - args:
                   - --character-set-server=utf8mb4
                   - --collation-server=utf8mb4_unicode_ci
                 env:
                   - name: MYSQL_ROOT_PASSWORD
                     value: root
                 image: registry.cn-hangzhou.aliyuncs.com/mse-demo-hz/demo-mysql:3.0.1
                 name: demo-mysql
                 ports:
                   - containerPort: 3306

2. Deploy the applications:

       kubectl apply -f mse-demo.yaml

Create the Ingress resource

1. Create a file named mse-ingress.yaml with the following content:

   View the mse-ingress.yaml file

       apiVersion: networking.k8s.io/v1
       kind: Ingress
       metadata:
         annotations:
           mse.ingress.kubernetes.io/service-subset: ""
         name: spring-cloud-a
       spec:
         ingressClassName: mse
         rules:
           - http:
               paths:
                 - backend:
                     service:
                       name: spring-cloud-a
                       port:
                         number: 20001
                   path: /
                   pathType: Prefix

2. Apply the Ingress configuration:

       kubectl apply -f mse-ingress.yaml

Verify the deployment

1. Get the external IP address of the Ingress: Expected output:

       kubectl get ingress

       NAME              CLASS    HOSTS   ADDRESS        PORTS   AGE
       spring-cloud-a    <none>   *       EXTERNAL_IP    80      12m

2. Send a test request. Replace <EXTERNAL_IP> with the IP address from the previous output: Expected output: This output confirms that the call chain A -> B -> C is working and all three services run the base version.

       curl http://<EXTERNAL_IP>/a

       A[192.168.42.115][config=base] -> B[192.168.42.118] -> C[192.168.42.101]

Define the Rollout and TrafficRouting resources

In this demo, the canary release is performed in three batches:

1. A/B testing is performed. Requests that contain header[User-Agent]=xiaoming are forwarded to the canary version. Other requests are forwarded to the base version.

2. Half of the pods run the canary version, and half of requests are forwarded to the canary version.

3. All the pods run the canary version, and all the requests are forwarded to the canary version.

Create and apply the Rollout resources as follows:

1. Create a file named rollout.yaml with the following content:

   View the rollout.yaml file

       # Rollout configuration for application A
       apiVersion: rollouts.kruise.io/v1alpha1
       kind: Rollout
       metadata:
         name: rollout-a
         annotations:
           rollouts.kruise.io/trafficrouting: mse-traffic
       spec:
         objectRef:
           workloadRef:
             apiVersion: apps/v1
             kind: Deployment
             name: spring-cloud-a
         strategy:
           canary:
             steps:
             - pause: {}
               replicas: 1
             patchPodTemplateMetadata:
               labels:
                 alicloud.service.tag: gray
                 opensergo.io/canary-gray: gray
       ---
       # Rollout configuration for application B
       apiVersion: rollouts.kruise.io/v1alpha1
       kind: Rollout
       metadata:
         name: rollout-b
         annotations:
           rollouts.kruise.io/trafficrouting: mse-traffic
       spec:
         objectRef:
           workloadRef:
             apiVersion: apps/v1
             kind: Deployment
             name: spring-cloud-b
         strategy:
           canary:
             steps:
               - pause: {}
                 replicas: 1
             patchPodTemplateMetadata:
               labels:
                 alicloud.service.tag: gray
                 opensergo.io/canary-gray: gray
       ---
       # Rollout configuration for application C
       apiVersion: rollouts.kruise.io/v1alpha1
       kind: Rollout
       metadata:
         name: rollout-c
         annotations:
           rollouts.kruise.io/trafficrouting: mse-traffic
       spec:
         objectRef:
           workloadRef:
             apiVersion: apps/v1
             kind: Deployment
             name: spring-cloud-c
         strategy:
           canary:
             steps:
               - pause: {}
                 replicas: 1
             patchPodTemplateMetadata:
               labels:
                 alicloud.service.tag: gray
                 opensergo.io/canary-gray: gray
       ---
       # TrafficRouting: defines how canary traffic is identified and routed
       apiVersion: rollouts.kruise.io/v1alpha1
       kind: TrafficRouting
       metadata:
         name: mse-traffic
       spec:
         objectRef:
         - service: spring-cloud-a
           ingress:
             classType: mse
             name: spring-cloud-a
         strategy:
           matches:
           # Route requests with User-Agent: xiaoming to the canary version
           - headers:
             - type: Exact
               name: User-Agent
               value: xiaoming
           requestHeaderModifier:
             set:
             - name: x-mse-tag
               value: gray

   What happens when you apply this configuration:

   Each Rollout resource watches a Deployment. When the Deployment image changes, Kruise Rollouts:

   1. Pauses the native rolling update.

   2. Creates one canary pod (replicas: 1) with the new image version.

   3. Applies the patchPodTemplateMetadata labels to the canary pod:

      • alicloud.service.tag: gray -- tells MSE Microservices Governance to recognize this pod as part of the canary (gray) group.

      • opensergo.io/canary-gray: gray -- enables OpenSergo-compliant traffic lane routing across the call chain.

   4. Pauses and waits for manual approval before promoting.

   The TrafficRouting resource defines the traffic-splitting rules:

   • Requests with the header User-Agent: xiaoming match the canary rule.

   • The requestHeaderModifier injects an x-mse-tag: gray header into matching requests. MSE propagates this header through the entire call chain, enabling end-to-end canary routing. Every service in the chain checks this header and routes to its canary pod if one exists.

2. Deploy the Rollout resources:

       kubectl apply -f rollout.yaml

3. Verify the Rollout status: If the output shows STATUS=Healthy, the Rollout resources are ready.

       kubectl get rollout

Trigger a canary release

Kruise Rollouts is a one-time configuration. After the Rollout resources are deployed, trigger a canary release by updating your Deployment image -- no additional Rollout configuration is needed. In addition to kubectl, you can use Helm or Vela to deploy the deployments to your cluster.

1. In mse-demo.yaml, update the image versions for spring-cloud-a and spring-cloud-c from mse-2.0.0 to mse-2.0.1: Application B stays at mse-2.0.0. Because B has no canary version, requests routed to the canary lane pass through B's base version but continue to C's canary version. This demonstrates the end-to-end traffic lane: even when a service in the middle of the chain has no canary deployment, the canary tag is preserved and downstream services still route correctly.

       # In the spring-cloud-a Deployment
       image: registry.cn-hangzhou.aliyuncs.com/mse-demo-hz/spring-cloud-a:mse-2.0.1
   
       # In the spring-cloud-c Deployment
       image: registry.cn-hangzhou.aliyuncs.com/mse-demo-hz/spring-cloud-c:mse-2.0.1

2. Apply the updated Deployments:

       kubectl apply -f mse-demo.yaml

3. Check the Rollout progress: Expected output:

   Field	Value	Meaning
   STATUS	Progressing	The canary release is in progress
   CANARY_STATE	StepPaused	The canary batch is deployed and the rollout is waiting for manual approval

       kubectl get rollouts rollout-a -n default
       kubectl get rollouts rollout-c -n default

       NAME         STATUS        CANARY_STEP   CANARY_STATE   MESSAGE                                                                        AGE
       rollout-a    Progressing   1             StepPaused     Rollout is in step(1/1), and you need manually confirm to enter the next step   41m
       rollout-c    Progressing   1             StepPaused     Rollout is in step(1/1), and you need manually confirm to enter the next step   41m

Verify and promote the canary version

1. Send a canary test request by including the User-Agent: xiaoming header: Canary traffic flows through the canary versions of A and C, while B uses its base version. Verify the behavior through application logs and monitoring metrics.

       curl http://<EXTERNAL_IP>/a -H "User-Agent: xiaoming"

2. After confirming the canary version works as expected, approve the rollout to promote it to production. Run the following command for each Rollout that has a canary version deployed (in this demo, rollout-a and rollout-c). Replace <rollouts-demo> with the name of the Rollouts resource:

       rollout.rollouts.kruise.io/<rollouts-demo> approved

Roll back a canary release

If the canary version does not meet expectations, revert the image versions in mse-demo.yaml to their original values and reapply:

kubectl apply -f mse-demo.yaml

No changes to the Rollout resources are needed. Kruise Rollouts detects the Deployment change and handles the rollback automatically.

The end-to-end canary release feature of Microservices Governance provides lanes, which greatly facilitate verification during testing and release. You can use Microservices Governance together with Kruise Rollouts to significantly improve the stability of online applications in a DevOps process.