Alibaba Cloud Service Mesh:Migrate an Nginx Ingress that uses a CLB instance to an ASM gateway

Get the CLB instance ID

Run the following command to retrieve the CLB instance ID:

kubectl -n kube-system get svc nginx-ingress-lb -o yaml | grep service.k8s.alibaba/loadbalancer-id

Expected output:

service.k8s.alibaba/loadbalancer-id: lb-bp1gts52ced2vgaw1ni78

Record the CLB instance ID (for example, lb-bp1gts52ced2vgaw1ni78). You need it in later steps.

Update CLB settings in the console

1. Open the Server Load Balancer console.

2. Find your CLB instance and make the following changes:

   • Disable configuration read-only mode.

   • Delete the kubernetes.do.not.delete and ack.aliyun.com tags.

   • Rename the vServer groups that start with k8s/ to the shared-<port> format. For example, rename k8s/80/nginx-ingress-lb/kube-system/c553a74e6ad13423aa839c8e5******** to shared-80.

Add annotations to the NGINX Ingress Service

Add the following annotations to the NGINX Ingress Service so that it references the CLB explicitly instead of relying on ACK-managed discovery:

metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id: "<your-clb-instance-id>"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-force-override-listeners: "false"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vgroup-port: "<your-vgroup-id-1>:80,<your-vgroup-id-2>:443"
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-weight: "100"

Replace the following placeholders with actual values:

NGINX Ingress (before)

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: helloworld
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
  rules:
    - http:
        paths:
          - backend:
              serviceName: helloworld
              servicePort: 80
            path: /helloworld(/|$)(.*)
      host: example.com

Istio VirtualService (after)

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: example-vs
spec:
  gateways:
  - istio-system/ingressgateway    # Replace with your ASM gateway name
  hosts:
  - example.com
  http:
  - name: route-helloworld
    match:
    - uri:
        prefix: /helloworld/
    - uri:
        prefix: /helloworld
    rewrite:
      uri: /
    route:
    - destination:
        host: helloworld
        port:
          number: 80

Key differences between Ingress and VirtualService

The following table summarizes the mapping between NGINX Ingress fields and their Istio VirtualService equivalents:

Common NGINX Ingress annotation equivalents

If you use advanced NGINX Ingress annotations, refer to the following table for Istio equivalents:

Cross-namespace routing

If a VirtualService and its target Service are in the same namespace, use the short service name (for example, helloworld). If they are in different namespaces, use the Fully Qualified Domain Name (FQDN) format:

<service-name>.<namespace>.svc.cluster.local

Option A: Test through a temporary CLB

Create a new CLB instance and point it to the ASM ingress gateway. Send test requests to this CLB:

curl -s -I -H "Host: example.com" http://<test-clb-ip>/helloworld/

Expected output (key headers):

HTTP/1.1 200 OK
server: istio-envoy

The server: istio-envoy header confirms that traffic is flowing through the ASM gateway.

Option B: Test from inside the cluster

Send requests directly to the ASM ingress gateway Service within the cluster:

# Get the ASM ingress gateway ClusterIP
GATEWAY_IP=$(kubectl -n istio-system get svc istio-ingressgateway -o jsonpath='{.spec.clusterIP}')

# Send a test request
curl -s -I -H "Host: example.com" http://$GATEWAY_IP/helloworld/

Confirm that the response matches what NGINX Ingress returns for the same request. The server: istio-envoy header in the response indicates that traffic is flowing through the ASM gateway.

Recommended migration schedule

Increase the ASM gateway weight incrementally and monitor at each stage:

Adjust weights

ASM gateway weight: Update the service.beta.kubernetes.io/alibaba-cloud-loadbalancer-weight annotation in the IstioGateway serviceAnnotations:

serviceAnnotations:
  service.beta.kubernetes.io/alibaba-cloud-loadbalancer-weight: "50"    # Adjust this value

NGINX Ingress weight: Update the service.beta.kubernetes.io/alibaba-cloud-loadbalancer-weight annotation on the NGINX Ingress Service. If this annotation is not configured, adjust the weight directly in the CLB console.

Rollback

If issues occur during traffic migration, set the ASM gateway weight back to "0" and the NGINX Ingress weight back to "100":

# ASM gateway: stop receiving traffic
serviceAnnotations:
  service.beta.kubernetes.io/alibaba-cloud-loadbalancer-weight: "0"

# NGINX Ingress Service: restore full traffic
metadata:
  annotations:
    service.beta.kubernetes.io/alibaba-cloud-loadbalancer-weight: "100"

All traffic returns to NGINX Ingress immediately. No DNS changes are needed because both gateways share the same CLB instance.