Implement phased releases and blue-green deployments using Nginx Ingress - Container Service for Kubernetes

Use the Nginx Ingress Controller in a Container Service for Kubernetes (ACK) cluster to gradually shift traffic from an old service version to a new one — by request header, cookie, or weight — without taking the old version offline until the new version is stable.

Background

Phased releases (also called canary releases) and blue-green deployments both involve running a new service version alongside the old one and directing a subset of traffic to it. Once the new version proves stable, all traffic is switched over and the old version is decommissioned.

The ACK Nginx Ingress Controller supports two annotation-based methods for traffic splitting:

Method	Status	Notes
`canary-*` annotations	Active	The official community method. Use this for all new configurations.
`service-*` annotations	Deprecated	No longer available in Nginx Ingress Controller v1.12 and later. Do not use it.

Use cases

Traffic splitting based on client requests: Route traffic to the new version only when a request contains a specific header or cookie — for example, foo=bar. All other requests continue reaching the old version. Once the new version is stable, switch all traffic.
Traffic splitting based on service weight: Route a fixed percentage of traffic — for example, 20% — to the new version while the old version handles the rest. Gradually increase the percentage until all traffic goes to the new version.

The Nginx Ingress Controller supports the following traffic-splitting dimensions:

Request header — suitable for phased releases and A/B testing
Cookie — suitable for phased releases and A/B testing
Query parameter — suitable for phased releases and A/B testing
Service weight — suitable for blue-green deployments

The canary-\* annotation method

Annotation reference

All canary configurations require the nginx.ingress.kubernetes.io/canary: "true" annotation. The remaining annotations define the routing logic.

<table> <thead> <tr> <td><b>Annotation</b></td> <td><b>Description</b></td> <td><b>Min. version</b></td> </tr> </thead> <tbody> <tr> <td><code>nginx.ingress.kubernetes.io/canary</code></td> <td> Enables the canary feature. Must be set to <code>true</code> for any other canary annotation to take effect.<br/> Valid values: <code>true</code> | <code>false</code> </td> <td>≥v0.22.0</td> </tr> <tr> <td><code>nginx.ingress.kubernetes.io/canary-by-header</code></td> <td> Routes requests to the canary service based on a request header.<br/> Special values for the header: <ul> <li><code>always</code>: Always routes to the canary service.</li> <li><code>never</code>: Never routes to the canary service.</li> </ul> If no value is specified, traffic is forwarded whenever the header is present. </td> <td>≥v0.22.0</td> </tr> <tr> <td><code>nginx.ingress.kubernetes.io/canary-by-header-value</code></td> <td> Routes requests to the canary service when the header specified by <code>canary-by-header</code> matches an exact value.<br/> Must be used together with <code>canary-by-header</code>. Has no effect if <code>canary-by-header</code> is not defined. </td> <td>≥v0.30.0</td> </tr> <tr> <td><code>nginx.ingress.kubernetes.io/canary-by-header-pattern</code></td> <td> Routes requests to the canary service when the header specified by <code>canary-by-header</code> matches a regular expression.<br/> Must be used together with <code>canary-by-header</code>. Has no effect if <code>canary-by-header</code> is not defined. </td> <td>≥v0.44.0</td> </tr> <tr> <td><code>nginx.ingress.kubernetes.io/canary-by-cookie</code></td> <td> Routes requests to the canary service based on a cookie. For example: <code>nginx.ingress.kubernetes.io/canary-by-cookie: foo</code>.<br/> Cookie values: <ul> <li><code>always</code>: When <code>foo=always</code>, routes to the canary service.</li> <li><code>never</code>: When <code>foo=never</code>, does not route to the canary service.</li> </ul> Traffic is forwarded only when the cookie exists and its value is <code>always</code>. </td> <td>≥v0.22.0</td> </tr> <tr> <td><code>nginx.ingress.kubernetes.io/canary-weight</code></td> <td> Routes a percentage of requests to the canary service based on weight.<br/> Range: <code>0</code> to <code>canary-weight-total</code> (default: 100). </td> <td>≥v0.22.0</td> </tr> <tr> <td><code>nginx.ingress.kubernetes.io/canary-weight-total</code></td> <td> Sets the total weight denominator. Default: <code>100</code>. </td> <td>≥v1.1.2</td> </tr> </tbody> </table>

Annotation priority (descending):

canary-by-header > canary-by-cookie > canary-weight

Each Ingress rule supports only one canary Ingress at a time. Additional canary Ingresses are ignored.

Step 1: Deploy the service

Deploy an Nginx service and expose it via Layer 7 domain access through the Nginx Ingress Controller.

Create a Deployment and a Service. Save the following content to nginx.yaml.

View YAML file

apiVersion: apps/v1
kind: Deployment
metadata:
  name: old-nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      run: old-nginx
  template:
    metadata:
      labels:
        run: old-nginx
    spec:
      containers:
      - image: registry.cn-hangzhou.aliyuncs.com/acs-sample/old-nginx
        imagePullPolicy: Always
        name: old-nginx
        ports:
        - containerPort: 80
          protocol: TCP
      restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  name: old-nginx
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: old-nginx
  sessionAffinity: None
  type: NodePort

Apply the manifest:

kubectl apply -f nginx.yaml

Create the Ingress. Save the following content to ingress.yaml.

For clusters of v1.19 and later

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: gray-release
spec:
  ingressClassName: nginx
  rules:
  - host: www.example.com
    http:
      paths:
      # Old version of the service.
      - path: /
        backend:
          service:
            name: old-nginx
            port:
              number: 80
        pathType: ImplementationSpecific

For clusters earlier than v1.19

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: gray-release
spec:
  rules:
  - host: www.example.com
    http:
      paths:
      # Old version of the service.
      - path: /
        backend:
          serviceName: old-nginx
          servicePort: 80

Apply the manifest:

kubectl apply -f ingress.yaml

Verify the deployment. Get the external IP address:
```
kubectl get ingress
```
Check routing access:
```
curl -H "Host: www.example.com" http://<EXTERNAL_IP>
```
Expected output:
```
old
```

Step 2: Release the new service version

Deploy the new Nginx version and configure canary routing rules.

Create the new Deployment and Service. Save the following content to nginx1.yaml.

View YAML file

apiVersion: apps/v1
kind: Deployment
metadata:
  name: new-nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      run: new-nginx
  template:
    metadata:
      labels:
        run: new-nginx
    spec:
      containers:
      - image: registry.cn-hangzhou.aliyuncs.com/acs-sample/new-nginx
        imagePullPolicy: Always
        name: new-nginx
        ports:
        - containerPort: 80
          protocol: TCP
      restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  name: new-nginx
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: new-nginx
  sessionAffinity: None
  type: NodePort

Apply the manifest:

kubectl apply -f nginx1.yaml

Configure traffic routing for the new service version by creating a canary Ingress. Three routing strategies are available — choose the one that fits your release plan. Strategy A: Route by request header Route requests to the new version only when the foo header is set to bar. All other requests go to the old version. Save the following content to ingress1.yaml. For clusters v1.19 and later:

With `foo: bar` header: 100% of traffic goes to new-nginx (controlled by canary-by-header and canary-by-header-value).
Without `foo: bar` header: 50% of traffic goes to new-nginx (controlled by canary-weight).

For clusters of v1.19 and later

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: gray-release-canary
  annotations:
    # Enable canary routing.
    nginx.ingress.kubernetes.io/canary: "true"
    # Route to new-nginx only when the foo header equals bar.
    nginx.ingress.kubernetes.io/canary-by-header: "foo"
    nginx.ingress.kubernetes.io/canary-by-header-value: "bar"
spec:
  ingressClassName: nginx
  rules:
  - host: www.example.com
    http:
      paths:
      # New version of the service.
      - path: /
        backend:
          service:
            name: new-nginx
            port:
              number: 80
        pathType: ImplementationSpecific

For clusters earlier than v1.19

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: gray-release-canary
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-header: "foo"
    nginx.ingress.kubernetes.io/canary-by-header-value: "bar"
spec:
  rules:
  - host: www.example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: new-nginx
          servicePort: 80

Apply the manifest and verify:

kubectl apply -f ingress1.yaml
kubectl get ingress

Test without the header (routed to old version):

curl -H "Host: www.example.com" http://<EXTERNAL_IP>

Expected output: old Test with foo: bar header (routed to new version):

curl -H "Host: www.example.com" -H "foo: bar" http://<EXTERNAL_IP>

Expected output: new --- Strategy B: Route by header and split remaining traffic by weight Route 100% of requests with foo=bar to the new version, and 50% of all other requests to the new version. Update ingress1.yaml with the following content. For clusters v1.19 and later:

For clusters of v1.19 and later

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: gray-release-canary
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    # Route to new-nginx when foo header equals bar.
    nginx.ingress.kubernetes.io/canary-by-header: "foo"
    nginx.ingress.kubernetes.io/canary-by-header-value: "bar"
    # For requests that don't match the header rule, route 50% to new-nginx.
    nginx.ingress.kubernetes.io/canary-weight: "50"
spec:
  ingressClassName: nginx
  rules:
  - host: www.example.com
    http:
      paths:
      - path: /
        backend:
          service:
            name: new-nginx
            port:
              number: 80
        pathType: ImplementationSpecific

For clusters earlier than v1.19

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: gray-release-canary
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-by-header: "foo"
    nginx.ingress.kubernetes.io/canary-by-header-value: "bar"
    nginx.ingress.kubernetes.io/canary-weight: "50"
spec:
  rules:
  - host: www.example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: new-nginx
          servicePort: 80

Apply the manifest:

kubectl apply -f ingress1.yaml

Expected behavior: --- Strategy C: Route by weight only Route 50% of all traffic to the new version, regardless of request headers or cookies. Update ingress1.yaml with the following content. For clusters v1.19 and later:

For clusters of v1.19 and later

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: gray-release-canary
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    # Route 50% of all traffic to new-nginx. Default total weight is 100.
    nginx.ingress.kubernetes.io/canary-weight: "50"
spec:
  ingressClassName: nginx
  rules:
  - host: www.example.com
    http:
      paths:
      - path: /
        backend:
          service:
            name: new-nginx
            port:
              number: 80
        pathType: ImplementationSpecific

For clusters earlier than v1.19

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: gray-release-canary
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "50"
spec:
  rules:
  - host: www.example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: new-nginx
          servicePort: 80

Apply the manifest:

kubectl apply -f ingress1.yaml

Verify: run the following command multiple times. Approximately 50% of responses should return new.

curl -H "Host: www.example.com" http://<EXTERNAL_IP>

Step 3: Complete the traffic cutover

Once the new version is stable and meets expectations, decommission the old version.

Update nginx.yaml to redirect the old Service to the new Deployment.

View YAML file

apiVersion: v1
kind: Service
metadata:
  name: old-nginx
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    # Point to the new service version.
    run: new-nginx
  sessionAffinity: None
  type: NodePort

Apply the update:

kubectl apply -f nginx.yaml

Verify that all traffic is now routed to the new version:
```
kubectl get ingress
curl -H "Host: www.example.com" http://<EXTERNAL_IP>
```
Expected output: new

Delete the canary Ingress:

kubectl delete ingress gray-release-canary

Delete the old Deployment and the new Service:

kubectl delete deploy old-nginx
kubectl delete svc new-nginx

The service-\* annotation method

Important

The service-* annotation is no longer available in Nginx Ingress Controller v1.12 and later. Do not use it.

Annotation reference

<table> <thead> <tr> <td><b>Annotation</b></td> <td><b>Description</b></td> </tr> </thead> <tbody> <tr> <td><code>nginx.ingress.kubernetes.io/service-match</code></td> <td> Defines routing rules that map request attributes to a service.<br/> Syntax: <pre>nginx.ingress.kubernetes.io/service-match: | <service-name>: <match-rule></pre> Supported match types: <code>header</code>, <code>cookie</code>, <code>query</code>.<br/> Match formats: <ul> <li>Regex match: <code>/regular expression/</code></li> <li>Exact match: <code>"exact value"</code></li> </ul> Examples: <pre>new-nginx: header("foo", /^bar$/) new-nginx: header("foo", "bar") new-nginx: cookie("foo", /^sticky-.+$/) new-nginx: query("foo", "bar")</pre> </td> </tr> <tr> <td><code>nginx.ingress.kubernetes.io/service-weight</code></td> <td> Sets traffic weights between the old and new service versions.<br/> Syntax: <pre>nginx.ingress.kubernetes.io/service-weight: | <new-svc-name>:<new-svc-weight>, <old-svc-name>:<old-svc-weight></pre> Example: <pre>nginx.ingress.kubernetes.io/service-weight: | new-nginx: 20, old-nginx: 60</pre> </td> </tr> </tbody> </table>

Step 1: Deploy the service

Deploy an Nginx service and expose it via Layer 7 domain access using the Nginx Ingress Controller.

Create a Deployment and a Service. Save the following content to nginx.yaml.

View YAML file

apiVersion: apps/v1
kind: Deployment
metadata:
  name: old-nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      run: old-nginx
  template:
    metadata:
      labels:
        run: old-nginx
    spec:
      containers:
      - image: registry.cn-hangzhou.aliyuncs.com/acs-sample/old-nginx
        imagePullPolicy: Always
        name: old-nginx
        ports:
        - containerPort: 80
          protocol: TCP
      restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  name: old-nginx
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: old-nginx
  sessionAffinity: None
  type: NodePort

Apply the manifest:

kubectl apply -f nginx.yaml

Create the Ingress. Save the following content to ingress.yaml.

For clusters of v1.19 and later

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: gray-release
spec:
  ingressClassName: nginx
  rules:
  - host: www.example.com
    http:
      paths:
      # Old version of the service.
      - path: /
        backend:
          service:
            name: old-nginx
            port:
              number: 80
        pathType: ImplementationSpecific

For clusters earlier than v1.19

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: gray-release
spec:
  rules:
  - host: www.example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: old-nginx
          servicePort: 80

Apply the manifest:

kubectl apply -f ingress.yaml

Verify the deployment. Get the external IP address:
```
kubectl get ingress
```
Check routing access:
```
curl -H "Host: www.example.com" http://<EXTERNAL_IP>
```
Expected output:
```
old
```

Step 2: Release the new service version

Deploy the new Nginx version and configure routing rules.

Create the new Deployment and Service. Save the following content to nginx1.yaml.

View a YAML file

apiVersion: apps/v1
kind: Deployment
metadata:
  name: new-nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      run: new-nginx
  template:
    metadata:
      labels:
        run: new-nginx
    spec:
      containers:
      - image: registry.cn-hangzhou.aliyuncs.com/acs-sample/new-nginx
        imagePullPolicy: Always
        name: new-nginx
        ports:
        - containerPort: 80
          protocol: TCP
      restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
  name: new-nginx
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 80
  selector:
    run: new-nginx
  sessionAffinity: None
  type: NodePort

Apply the manifest:

kubectl apply -f nginx1.yaml

Configure traffic routing by modifying the gray-release Ingress. Three routing strategies are available. Strategy A: Route by request header Route requests to the new version only when the foo header matches bar. Update ingress.yaml with the following content. For clusters v1.19 and later:

For clusters of v1.19 and later

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: gray-release
  annotations:
    # Route to new-nginx when the foo header matches ^bar$.
    nginx.ingress.kubernetes.io/service-match: |
      new-nginx: header("foo", /^bar$/)
spec:
  ingressClassName: nginx
  rules:
  - host: www.example.com
    http:
      paths:
      # Old version of the service.
      - path: /
        backend:
          service:
            name: old-nginx
            port:
              number: 80
        pathType: ImplementationSpecific
      # New version of the service.
      - path: /
        backend:
          service:
            name: new-nginx
            port:
              number: 80
        pathType: ImplementationSpecific

For clusters earlier than v1.19

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: gray-release
  annotations:
    nginx.ingress.kubernetes.io/service-match: |
      new-nginx: header("foo", /^bar$/)
spec:
  rules:
  - host: www.example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: old-nginx
          servicePort: 80
      - path: /
        backend:
          serviceName: new-nginx
          servicePort: 80

Apply the manifest and verify:

kubectl apply -f ingress.yaml
kubectl get ingress

Test without the header (routed to old version):

curl -H "Host: www.example.com" http://<EXTERNAL_IP>

Expected output: old Test with foo: bar header (routed to new version):

curl -H "Host: www.example.com" -H "foo: bar" http://<EXTERNAL_IP>

Expected output: new --- Strategy B: Route by header with weight-based split Route requests matching foo=bar to both versions, with 50% going to each. Update ingress.yaml with the following content. For clusters v1.19 and later:

For clusters of v1.19 and later

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: gray-release
  annotations:
    nginx.ingress.kubernetes.io/service-match: |
        new-nginx: header("foo", /^bar$/)
    # Of the matching requests, route 50% to new-nginx and 50% to old-nginx.
    nginx.ingress.kubernetes.io/service-weight: |
        new-nginx: 50, old-nginx: 50
spec:
  ingressClassName: nginx
  rules:
  - host: www.example.com
    http:
      paths:
      - path: /
        backend:
          service:
            name: old-nginx
            port:
              number: 80
        pathType: ImplementationSpecific
      - path: /
        backend:
          service:
            name: new-nginx
            port:
              number: 80
        pathType: ImplementationSpecific

For clusters earlier than v1.19

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: gray-release
  annotations:
    nginx.ingress.kubernetes.io/service-match: |
        new-nginx: header("foo", /^bar$/)
    nginx.ingress.kubernetes.io/service-weight: |
        new-nginx: 50, old-nginx: 50
spec:
  rules:
  - host: www.example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: old-nginx
          servicePort: 80
      - path: /
        backend:
          serviceName: new-nginx
          servicePort: 80

Apply the manifest:

kubectl apply -f ingress.yaml

Run the test command multiple times. Of requests with the foo: bar header, approximately 50% should return new. --- Strategy C: Route by weight only Route 50% of all traffic to the new version. Update ingress.yaml with the following content. For clusters v1.19 and later:

For clusters of v1.19 and later

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: gray-release
  annotations:
    # Route 50% of all traffic to new-nginx.
    nginx.ingress.kubernetes.io/service-weight: |
        new-nginx: 50, old-nginx: 50
spec:
  ingressClassName: nginx
  rules:
  - host: www.example.com
    http:
      paths:
      - path: /
        backend:
          service:
            name: old-nginx
            port:
              number: 80
        pathType: ImplementationSpecific
      - path: /
        backend:
          service:
            name: new-nginx
            port:
              number: 80
        pathType: ImplementationSpecific

For clusters earlier than v1.19

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: gray-release
  annotations:
    nginx.ingress.kubernetes.io/service-weight: |
        new-nginx: 50, old-nginx: 50
spec:
  rules:
  - host: www.example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: old-nginx
          servicePort: 80
      - path: /
        backend:
          serviceName: new-nginx
          servicePort: 80

Apply the manifest:

kubectl apply -f ingress.yaml

Verify: run the following command multiple times. Approximately 50% of responses should return new.

curl -H "Host: www.example.com" http://<EXTERNAL_IP>

Step 3: Complete the traffic cutover

After the new version runs stably and meets expectations, decommission the old version.

For clusters of v1.19 and later

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: gray-release
spec:
  ingressClassName: nginx
  rules:
  - host: www.example.com
    http:
      paths:
      # New version of the service.
      - path: /
        backend:
          service:
            name: new-nginx
            port:
              number: 80
        pathType: ImplementationSpecific

For clusters earlier than v1.19

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: gray-release
spec:
  rules:
  - host: www.example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: new-nginx
          servicePort: 80

Apply the update:

kubectl apply -f ingress.yaml

Verify that all traffic is routed to the new version:

kubectl get ingress
curl -H "Host: www.example.com" http://<EXTERNAL_IP>

Expected output: new

Delete the old Deployment and Service:

kubectl delete deploy <Deployment_name>
kubectl delete svc <Service_name>