This topic describes how to migrate a self-managed Nginx Ingress to MSE Ingress by using the visual migration tool on the MSE console.
Prerequisites
An Nginx Ingress Controller is deployed in your Container Service cluster.
A Cloud Native Gateway is available. If you do not have one, see Create an MSE Cloud Native Gateway.
Usage notes
The migration process does not copy your Ingress configurations. Instead, it enables MSE Ingress to reuse your configurations by watching for and parsing changes to your existing Ingress resources in real time.
During the migration, both the Nginx Ingress Controller and MSE Ingress reflect any changes you make to your Ingress configurations.
After the migration is complete, do not delete any Ingress configurations that are in use. The migration only enables MSE Ingress to watch for and parse existing and future Ingress configurations.
After the migration, you must keep existing and future Ingress configurations associated with the original IngressClass. For example, if an Ingress resource previously specified
ingressClassName: nginxin its spec, it must continue to do so after the migration.
Migration methods
Details
Migration workflow
Cloud Native Gateway provides a Cloud Migration tool that guides you step-by-step through migrating your routing configurations and shifting traffic.
Step 1: Migrate routing rules
-
Log on to the MSE console. In the top navigation bar, select a region.
In the left-side navigation pane, choose Cloud Native Gateway > Migration to Cloud.
On the Migration to Cloud page, click Add Task.
In the Create Migration Configuration panel, configure the parameters.
The Cloud Native Gateway watches for all Ingress resources in the selected container cluster that are associated with the source IngressClass. It then applies their domain name and routing configurations.
ImportantMigration is not allowed if the target Cloud Native Gateway is already associated with this container cluster using a different IngressClass. Ensure the IngressClass specified here matches the one already configured for the cluster.
Parameter
Description
Cloud-native Gateway
Select the target Cloud Native Gateway for the migration. The gateway version must be 1.2.22 or later.
Container Service ACK/ACK Serverless cluster
Select the Container Service cluster that contains the Nginx Ingress to be migrated. Ensure that the Cloud Native Gateway and the container cluster are in the same VPC.
Source Ingress Class
Specify the IngressClass associated with the Ingress resources that you want to migrate.
NoteYou can specify only one IngressClass.
If you leave this parameter empty, IngressClass is ignored, and the gateway watches all Ingress resources in the cluster.
Description
Enter a description up to 64 characters long.
Click Next.
The Cloud Native Gateway now automatically watches for changes to all Ingress resources that are in the selected container cluster and associated with the source IngressClass. It then applies the domain name and routing configurations from these resources.
Assume you have an Ingress named httpbin in the source container cluster.
On the Cloud Native Gateway console, you will see that the Ingress from the source cluster is automatically synchronized to the target gateway, creating corresponding domain and routing configurations. The Routing Settings details page is displayed as follows:
Configuration wizard:
Add service source
You can use K8s, Nacos, Zookeeper, SAE, EDAS, and Function Compute (FC) as service sources. You can add an existing resource from your current VPC and then select a registered service from it.
Button: Add service source
Associate backend service
Distributes requests to a backend service. This service can be dynamically discovered from a service source or specified by a static address or domain name.
Button: Associate backend service
Bind domain name
Provides multi-domain management. By binding a domain name, you can associate protocols, certificates, and routing configurations with different domains.
Button: Bind domain name
Create route
A route contains matching rules based on domain names, paths, methods, request parameters, and request headers. When a user accesses a route, the request is forwarded to the corresponding backend service.
Button: Create route
List columns: Route name, Status, WAF protection, Source, Route condition, Associated domain name, Destination service type, Destination service, and Actions.
Step 2: Verify routes
Check the compatibility of the watched Ingress annotations:
If there are no incompatible Ingress annotations, proceed to the next step.
If you find incompatible Ingress annotations, you can submit a ticket for a solution.
ImportantYou can ignore the annotation
nginx.ingress.kubernetes.io/service-weightif its value is"". This annotation is added by default by an earlier version of the Container Service console and has no effect.During the migration, do not remove incompatible annotations that are in use. These annotations are still parsed by the Nginx Ingress Controller and affect your business traffic. You can add extended MSE annotations to the Ingress resource to achieve the same functionality in MSE Ingress. After all traffic is migrated to MSE Ingress, you can remove the incompatible annotations as needed.
Step 3: Select shifting method
Test before shifting traffic
Before you shift production traffic, perform a local test. Modify your local hosts file to resolve the business domain name to the Cloud Native Gateway SLB. Use a tool like cURL or Postman to verify that all traffic behaves as expected.
Select shifting method
Reuse source SLB
Principle: The process adds the Cloud Native Gateway node instances to the backend server group of the source SLB. During the migration, the SLB distributes business traffic to the Cloud Native Gateway based on configured weights. After the migration is complete, the SLB forwards all traffic to the Cloud Native Gateway.
The following table describes the parameters.
Parameter | Description |
ACK Cluster Namespace | Select the namespace where the Kubernetes Service associated with the Nginx Ingress SLB is located. |
ACK Cluster SLB Service | Select the name of the Kubernetes Service associated with the Nginx Ingress SLB. |
SLB ID | Click the SLB instance to confirm it is the correct target for the migration. |
Ports and Backend Servers | Select the listener port and gateway protocol (HTTP/HTTPS) of the source SLB instance. The target virtual server group appears automatically. Note Ensure you select the correct port and protocol to avoid traffic loss. |
DNS resolution
Go to your DNS provider's domain resolution service and add mappings to the Cloud Native Gateway SLB address for all domains involved in the route migration. We recommend using DNS weighted resolution to gradually shift traffic.
Step 4: Shift traffic
Reuse source SLB
Step 1: Modify SLB
After you click Modify SLB, the system automatically detaches the SLB instance from Container Service management and changes the listener's scheduling algorithm to Weighted Round Robin.
This step detaches the SLB instance from Container Service management. After detachment, the SLB instance no longer tracks Pod IP address changes in the Nginx Ingress Controller. Proceed to the next step promptly to modify the annotations and re-associate the Kubernetes Service with the SLB instance.
Step 2: Overwrite the service annotations
After you complete Step 1, go to the Container Service console. Manually delete all existing annotations from the Kubernetes Service that you selected in the Switch Traffic step. Then, copy the annotations automatically generated on the Traffic Switchover page and add them to the target Kubernetes Service. This step reconfigures the original Kubernetes Service to reuse the SLB instance. After the modification, click Precheck. If the check passes, you can proceed to the next step.
apiVersion: v1
kind: Service
metadata:
annotations:
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-id: lb-bp1lc0huzf0o1ks333yo1
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-weight: '100'
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-vgroup-port: 'rsp-bp1uybv6iy50i:80,rsp-bp10s0ndrfvdt:443'
creationTimestamp: '2023-11-29T07:14:40Z'
finalizers:
- service.k8s.alibaba/resources
labels:
app: nginx-ingress-lb
service.beta.kubernetes.io/hash: 39a51bd97b88f6eaf232da33434ce2fe44ae094b7b4d2b9bb9126eb7
service.k8s.alibaba/loadbalancer-id: lb-bp1lc0huzf0o1ks333yo1
managedFields:
- apiVersion: v1
fieldsType: FieldsV1
fieldsV1:
'f:metadata':
'f:annotations':
.: {}
'f:kubectl.kubernetes.io/last-applied-configuration': {}
'f:service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain': {}
'f:service.beta.kubernetes.io/alibaba-cloud-loadbalancer-connection-drain-timeout': {}
'f:service.beta.kubernetes.io/alibaba-cloud-loadbalancer-instance-charge-type': {}
'f:service.beta.kubernetes.io/alibaba-cloud-loadbalancer-resource-group-id': {}
'f:labels':
.: {}
'f:app': {}If application pods in the same container cluster access the Nginx Ingress gateway, you must also add the annotation service.beta.kubernetes.io/alibaba-cloud-loadbalancer-hostname: mse-ingress-migration to the Service. Before you add the annotation, make sure that IP access control is disabled for the SLB instance that is associated with the Service. This annotation forces the application pods to access the Nginx Ingress gateway through the SLB and prevents Kube Proxy from optimizing the traffic to bypass the SLB and directly access the Nginx Ingress pod.
Step 3: Weighted traffic shifting
Set the traffic weight for the Cloud Native Gateway instance. Set a value from 1 to 100 based on your business needs. We recommend an initial value between 1 and 10.
This value is the sum of the weights of all Cloud Native Gateway nodes. The SLB instance distributes traffic based on the weight ratio between the Cloud Native Gateway nodes and the Nginx Ingress nodes in the virtual server group. The total weight of the Nginx Ingress nodes defaults to 100. Therefore, if you set the Cloud Native Gateway weight to 100, it receives half of the traffic. Similarly, if you set the weight to 50, it receives one-third of the traffic.
During the migration, you can monitor gateway metrics on the Cloud Native Gateway monitoring dashboard to check the gateway's health and ensure that business metrics are stable.
If the results are as expected, you can gradually increase the weight. Wait at least 3 minutes between weight changes, as the backend applies the configuration asynchronously.
If the results are not as expected, set the weight to 0 to stop the migration.
In the Container Service console, you can lower the value of the
service.beta.kubernetes.io/alibaba-cloud-loadbalancer-weightannotation of the Kubernetes Service selected for reusing the original cluster's SLB to indirectly increase the weight of the Cloud Native Gateway. Specifically, when the value of this annotation is 0, all traffic is directed to the Cloud Native Gateway.Because SLB is a Layer 4 load balancer, it controls traffic at the connection level. The weight value cannot precisely control the distribution ratio at the request level.
If the success rate drops after shifting traffic, you can perform a fast rollback by setting this weight to 0.
If you want to remain in a migration state for an extended period and be able to adjust the traffic weights between Nginx Ingress and MSE Ingress at any time, stay at this step. Once you have verified that the traffic behaves as expected and no longer need to roll back all traffic to Nginx Ingress, you can click Complete Traffic Verification to proceed.
After you click Complete Traffic Verification, you can no longer modify the weights.
Step 4: Complete traffic switch
In the Container Service console, set the value of the service.beta.kubernetes.io/alibaba-cloud-loadbalancer-weight annotation to 0 for the K8s Service that you selected in Step 3: Select a traffic shifting method, or delete the Service resource. The Nginx Ingress Controller nodes are then automatically removed from the SLB, and all SLB traffic is switched to the Cloud Native Gateway. Click Complete Migration to finish the migration task.
DNS resolution
Go to your DNS provider's domain resolution service and add mappings to the Cloud Native Gateway SLB address for all domains involved in the route migration. We recommend using DNS weighted resolution to gradually shift traffic.
Fast rollback
If you encounter unexpected issues during traffic migration, you can use the following methods to perform a fast rollback and restore traffic to the Nginx Ingress Controller.
Reuse the source cluster SLB: Set the weight to 0. This stops the migration.
Use DNS to resolve to the Cloud Native Gateway SLB: At your DNS provider, remove the Cloud Native Gateway SLB address for all business domains.
Step 5: Complete the migration
Reuse source SLB
If you used the SLB traffic shifting method, ensure that traffic has been completely shifted for all relevant SLB instances. After you have completed the traffic shift for all SLB instances, you can delete the Kubernetes Service and the Nginx Ingress Controller as needed.
DNS resolution
If you used the DNS traffic shifting method and the IP addresses of all your business domains now resolve to the MSE gateway's SLB address list, you can delete the Kubernetes Service and the Nginx Ingress Controller as needed.
Troubleshooting
Bad request: mse.backend.gw.MIGRATE_INGRESS_CLASS_CONFLICT
This error indicates that your gateway is already associated with the Container Service cluster and has Ingress watching enabled, but the IngressClass specified for this migration does not match the original Ingress watching configuration. Ensure that the IngressClass configurations are consistent.
If you need to modify the previously configured IngressClass, choose one of the following methods based on your setup:
If you manage MSE Ingress through a MseIngressConfig, see Access Container Service and Container Compute Service by using MSE Ingress to modify it.
If you do not manage MSE Ingress through a MseIngressConfig, go to the gateway console, find the service source, and modify the Ingress watching options for the corresponding container cluster.
Bad request: mse.backend.gw.MIGRATE_SERVICE_ANNOTATION_NOT_MATCH [annotation is not expected]
This error indicates that the annotations on your Kubernetes Service have not been updated to the required values.
Bad request: mse.backend.gw.MIGRATE_SERVICE_NOT_MATCH [weight hasn't been 0 or service hasn't been deleted]
This indicates that the value of the service.beta.kubernetes.io/alibaba-cloud-loadbalancer-weight annotation for the K8s Service is not set to 0, or the K8s Service has not been deleted. In the Container Service console, set the value of the service.beta.kubernetes.io/alibaba-cloud-loadbalancer-weight annotation to 0, or delete the K8s Service.

