How to use Kruise Rollout to implement canary, A/B testing, and blue-green deployments for workloads - Container Service for Kubernetes

Kruise Rollout is a Kubernetes extension for phased delivery with native and OpenKruise workloads (Deployment, StatefulSet, CloneSet, and Advanced StatefulSet). Use it for canary, A/B testing, and batch rollouts on ACK.Kruise Rollout.

Two use cases:

Canary and A/B testing — route partial traffic to the new version with Nginx Ingress or MSE Ingress before full promotion
Phased release for microservice apps — roll out pods in batches without Ingress, for apps (such as Nacos-based services) that handle traffic shifting internally

Kruise Rollout phased release architecture:

How it works

Kruise Rollout is an OpenKruise progressive delivery controller that supports canary releases, A/B testing, blue-green deployments, and partition-based phased rollouts for Deployment, StatefulSet, and OpenKruise workloads. It intercepts workload updates and executes them in steps. Each step specifies how many pods to update (replicas) and how long to pause (pause). For Ingress-based deployments, each step also controls the traffic weight (weight).

Configure a Rollout resource once. For every subsequent deployment update, Kruise Rollout automatically intercepts the change and progresses it through the defined steps. Rollout can also use Prometheus metrics to automate batch progression and pauses.

Prerequisites

Ensure you have:

An ACK managed cluster.
- Version 1.19+ for A/B testing or canary releases
- Version 1.16+ for phased releases
kubectl-kruise installed.

Preparations

Step 1: Install the ack-kruise add-on

Log on to the ACK console. In the left navigation pane, choose Clusters.
On the Clusters page, click the name of the target cluster. In the left navigation pane, click Add-ons.
On the Add-ons page, click the Manage Applications tab. On the ack-kruise card, click Install.
In the dialog box, click OK.

The ack-kruise add-on 1.8 or later supports the v1beta1 API. See API specifications.

Step 2: Deploy the sample application

Deploy an echoserver service with a Deployment and Service.echoserver service with a Deployment and Ingress. This sample is used in both use cases.

Create a file named echoserver.yaml.

Expand to view the YAML content

apiVersion: apps/v1
kind: Deployment
metadata:
  name: echoserver
  labels:
    app: echoserver
spec:
  replicas: 6
  selector:
    matchLabels:
      app: echoserver
  template:
    metadata:
      labels:
        app: echoserver
    spec:
      containers:
      - name: echoserver
        image: openkruise-registry.cn-shanghai.cr.aliyuncs.com/openkruise/demo:1.10.2
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 8080
        env:
        - name: NODE_NAME
          value: version1
        - name: PORT
          value: '8080'
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: POD_IP
          valueFrom:
            fieldRef:
              fieldPath: status.podIP
---
apiVersion: v1
kind: Service
metadata:
  name: echoserver
  labels:
    app: echoserver
spec:
  ports:
  - port: 80
    targetPort: 8080
    protocol: TCP
    name: http
  selector:
    app: echoserver

Apply the configuration.
```
kubectl apply -f echoserver.yaml
```

Use case 1: Canary release and A/B testing with Ingress

Configure Kruise Rollout for canary releases and A/B testing with Nginx Ingress or MSE Ingress. The trafficRoutings field connects the Rollout to the Service and Ingress for traffic weight control at each step. Each Rollout defines three steps:

Strategy	Step 1	Step 2	Step 3
Canary	20% traffic weight, 1 pod, manual pause	50% weight, 50% pods, 60s auto-pause	100% (full release)
A/B testing	Route `User-Agent: Android` traffic to new version, 1 pod, manual pause	50% pods, 60s auto-pause	100% pods

Step 1: Install the Ingress controller and create an Ingress

Select your Ingress type and follow the corresponding instructions.

Nginx Ingress

Install the Nginx Ingress Controller.
- New cluster: Select Nginx Ingress in the Ingress configuration section during cluster creation.
- Existing cluster: See Create and use an Nginx Ingress to expose services.

Create a file named echoserver-ingress.yaml.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: echoserver
spec:
  ingressClassName: nginx
  rules:
  - http:
      paths:
      - backend:
          service:
            name: echoserver
            port:
              number: 80
        path: /apis/echo
        pathType: Exact

Apply the Ingress.

kubectl apply -f echoserver-ingress.yaml

MSE Ingress

Install the MSE Ingress Controller and create an MseIngressConfig and an IngressClass. See Access container services through an MSE Ingress.

Create a file named echoserver-ingress.yaml.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: echoserver
spec:
  # You must set ingressClassName to mse.
  ingressClassName: mse
  rules:
    - http:
        paths:
          - backend:
              service:
                name: echoserver
                port:
                  number: 80
            path: /apis/echo
            pathType: Exact

Apply the Ingress.

kubectl apply -f echoserver-ingress.yaml

Step 2: Verify access to the application

Retrieve the external IP address or hostname. Nginx Ingress:

export EXTERNAL_IP=$(kubectl get ingress echoserver -o jsonpath="{.status.loadBalancer.ingress[0].ip}")

MSE Ingress:

export EXTERNAL_IP=$(kubectl get ingress echoserver -o jsonpath="{.status.loadBalancer.ingress[0].hostname}")

Test access.

curl http://${EXTERNAL_IP}/apis/echo

Expected output:

Hostname: echoserver-75d49c475c-ls2bs

Pod Information:
    node name:    version1
    pod name:    echoserver-75d49c475c-ls2bs
    pod namespace:    default

Server values:
    server_version=nginx: 1.13.3 - lua: 10008
...

Step 3: Define the Rollout resource

Create a file named rollout.yaml with the strategy for your use case. The trafficRoutings field links the Rollout to the echoserver Service and Ingress for traffic shifting at each step.

Canary

apiVersion: rollouts.kruise.io/v1alpha1
kind: Rollout
metadata:
  name: rollouts-demo
spec:
  objectRef:
    workloadRef:
      apiVersion: apps/v1
      kind: Deployment
      name: echoserver
  strategy:
    canary:
      steps:
      # Step 1: Route 20% of traffic to the new version (1 pod). Pause until manually approved.
      - weight: 20
        replicas: 1
        pause: {}
      # Step 2: Increase to 50% traffic and 50% pods. Auto-proceed after 60 seconds.
      - weight: 50
        replicas: 50%
        pause: {duration: 60}  # unit: seconds
      # Step 3: Full release.
      - weight: 100
        replicas: 100%
        pause: {duration: 60}  # unit: seconds
      trafficRoutings:
      - service: echoserver
        ingress:
          name: echoserver

A/B testing

apiVersion: rollouts.kruise.io/v1alpha1
kind: Rollout
metadata:
  name: rollouts-demo
spec:
  objectRef:
    workloadRef:
      apiVersion: apps/v1
      kind: Deployment
      name: echoserver
  strategy:
    canary:
      steps:
      # Step 1: Route Android traffic to the new version (1 pod). Pause until manually approved.
      - matches:
          - headers:
              - type: Exact
                name: User-Agent
                value: Android
        pause: {}
        replicas: 1
      # Step 2: 50% of pods, still matching Android traffic. Auto-proceed after 60 seconds.
      - matches:
          - headers:
              - type: Exact
                name: User-Agent
                value: Android
        pause: {duration: 60}  # unit: seconds
        replicas: 50%
      # Step 3: Full release.
      - matches:
          - headers:
              - type: Exact
                name: User-Agent
                value: Android
        pause: {duration: 60}  # unit: seconds
        replicas: 100%
      trafficRoutings:
        - service: echoserver
          ingress:
            name: echoserver

Apply the Rollout and verify it is healthy.

kubectl apply -f rollout.yaml
kubectl get rollout

Expected output:

NAME            STATUS    CANARY_STEP   CANARY_STATE   MESSAGE                            AGE
rollouts-demo   Healthy   3             Completed      workload deployment is completed   7s

STATUS=Healthy confirms the Rollout is ready.

Step 4: Trigger the upgrade

Kruise Rollout activates automatically when the Deployment spec changes. Update the image version in echoserver.yaml to trigger a release.

# echoserver.yaml (relevant section)
spec:
  containers:
  - name: echoserver
    image: openkruise-registry.cn-shanghai.cr.aliyuncs.com/openkruise/demo:1.10.3
    env:
    - name: NODE_NAME
      value: version2  # Optional: helps identify the new version in verification output

Apply the updated Deployment. You can also use tools such as Helm and Vela.

kubectl apply -f echoserver.yaml

Check the rollout status.

kubectl get rollouts rollouts-demo -n default

Expected output:

NAME            STATUS        CANARY_STEP   CANARY_STATE   MESSAGE                                                                         AGE
rollouts-demo   Progressing   1             StepPaused     Rollout is in step(1/3), and you need manually confirm to enter the next step   41m

The status fields show:

Field	Value	Meaning
`STATUS`	`Progressing`	Canary release is in progress
`CANARY_STEP`	`1`	Currently in step 1 of 3
`CANARY_STATE`	`StepPaused`	Step 1 is complete; waiting for manual approval

Step 5: Verify traffic distribution and proceed

Canary

Access the service 10 times and observe the node name in the output.

for i in {1..10}; do curl -s http://${EXTERNAL_IP}/apis/echo | grep 'node name'; sleep 1; done

Expected output (approximately 8 requests to version1, 2 to version2):

        node name:      version1
        node name:      version1
        node name:      version2
        node name:      version1
        node name:      version2
        node name:      version1
        node name:      version1
        node name:      version1
        node name:      version1
        node name:      version1

The 8:2 ratio matches the 20% weight configured in step 1.

Approve step 1 to proceed to step 2.

kubectl-kruise rollout approve rollouts/rollouts-demo -n default

Monitor the rollout through steps 2 and 3.
```
kubectl get rollouts rollouts-demo -n default -w
```
The rollout automatically advances through step 2 (50% weight, 60s pause) and step 3 (100%, 60s pause). The rollout is complete when STATUS=Healthy and CANARY_STATE=Completed.

A/B testing

Send requests with and without the User-Agent: Android header.

curl -s http://${EXTERNAL_IP}/apis/echo | grep 'Pod Information:' -A 3
curl -sH "User-Agent: Android" http://${EXTERNAL_IP}/apis/echo | grep 'Pod Information:' -A 3

Expected output:

Pod Information:
        node name:      version1
        pod name:       echoserver-69598f9458-7c66v
        pod namespace:  default
Pod Information:
        node name:      version2
        pod name:       echoserver-fvhzg-687b4b56-qbhc8
        pod namespace:  default

The request without the header reaches version1; the Android request reaches version2.

Approve step 1 to proceed to step 2.

kubectl-kruise rollout approve rollouts/rollouts-demo -n default

Monitor the rollout until it completes.
```
kubectl get rollouts rollouts-demo -n default -w
```
The rollout automatically advances through step 2 and step 3. The rollout is complete when STATUS=Healthy and CANARY_STATE=Completed.

Roll back a release

If the new version behaves unexpectedly at any step, restore the previous Deployment configuration and reapply. The Rollout resource requires no changes.

# echoserver.yaml (rollback version)
spec:
  containers:
  - name: echoserver
    image: openkruise-registry.cn-shanghai.cr.aliyuncs.com/openkruise/demo:1.10.2
    env:
    - name: NODE_NAME
      value: version1

kubectl apply -f echoserver.yaml

Use case 2: Phased release for microservice apps

Microservice frameworks such as Nacos handle traffic shifting internally without Ingress or Service-level routing. Use Kruise Rollout's partition-based phased release to roll out pods in batches without the trafficRoutings field.

Because the microservice framework manages traffic shifting, this use case covers only rollout batch progression, not traffic verification.

Step 1: Define and deploy the Rollout resource

This Rollout uses the partition rolling style to release pods in three batches: 1 pod, then 50%, then 100%. The trafficRoutings field is omitted because the microservice framework manages traffic.

# Save as rollout.yaml
apiVersion: rollouts.kruise.io/v1alpha1
kind: Rollout
metadata:
  name: rollouts-demo
  annotations:
    rollouts.kruise.io/rolling-style: partition
spec:
  objectRef:
    workloadRef:
      apiVersion: apps/v1
      kind: Deployment
      name: echoserver
  strategy:
    canary:
      steps:
      # Step 1: Update 1 pod. Pause until manually approved.
      - replicas: 1
        pause: {}
      # Step 2: Update 50% of pods. Auto-proceed after 60 seconds.
      - replicas: 50%
        pause: {duration: 60}  # unit: seconds
      # Step 3: Update all pods.
      - replicas: 100%
        pause: {duration: 60}  # unit: seconds

kubectl apply -f rollout.yaml

Step 2: Trigger the upgrade

Update the image version in echoserver.yaml.

# echoserver.yaml (relevant section)
spec:
  containers:
  - name: echoserver
    image: openkruise-registry.cn-shanghai.cr.aliyuncs.com/openkruise/demo:1.10.3
    env:
    - name: NODE_NAME
      value: version2  # Optional: helps identify the new version

kubectl apply -f echoserver.yaml

Step 3: Check rollout status

kubectl get rollouts rollouts-demo -n default

Expected output:

NAME            STATUS        CANARY_STEP   CANARY_STATE   MESSAGE                                                                         AGE
rollouts-demo   Progressing   1             StepPaused     Rollout is in step(1/3), and you need manually confirm to enter the next step   41m

Field	Value	Meaning
`STATUS`	`Progressing`	Release is in progress
`CANARY_STEP`	`1`	Currently in step 1 of 3
`CANARY_STATE`	`StepPaused`	Step 1 is complete; waiting for manual approval

Step 4: Approve batches and monitor progress

Approve step 1 to move to step 2.

kubectl-kruise rollout approve rollouts/rollouts-demo -n default

Monitor the rollout.

kubectl get rollouts rollouts-demo -n default -w

After approval, the rollout advances through step 2 (50% pods, 60s pause) and step 3 (all pods, 60s pause). Key state transitions:

`CANARY_STATE`	Meaning
`StepPaused`	Batch complete; either waiting for manual approval or timed auto-pause
`StepUpgrade`	Upgrading pods in this batch
`StepReady`	Batch upgrade complete
`Completed`	All steps finished

The rollout is complete when STATUS=Healthy and CANARY_STATE=Completed.

Roll back a release

If the new version behaves unexpectedly, restore the previous Deployment configuration and reapply. The Rollout resource requires no changes.

# echoserver.yaml (rollback version)
spec:
  containers:
  - name: echoserver
    image: openkruise-registry.cn-shanghai.cr.aliyuncs.com/openkruise/demo:1.10.2
    env:
    - name: NODE_NAME
      value: version1

kubectl apply -f echoserver.yaml

Next steps

Kruise Rollout API specifications
Kruise Rollout on GitHub