Container Service for Kubernetes:Deploy multiple Ingress controllers in a cluster

Usage notes

The following section describes the usage notes for configuring multiple Ingress controllers, including configuring a unique IngressClass and specifying length limits for Service names.

• Configure a unique IngressClass

  When you use multiple Ingress controllers, you must configure a unique IngressClass for each controller to ensure normal operation and prevent conflicts. You must modify the following parameters:

  • controller.ingressClassResource.name: Specify a unique name for each Ingress controller.

  • controller.ingressClassResource.controllerValue: Specify a unique identifier for each Ingress controller.

• Specify length limits for Service names

  When you deploy a Helm application named ack-ingress-nginx-v1, a Service whose name is in the following format is automatically created: <Application name>-ack-ingress-nginx-v1-controller. If the type of the LoadBalancer Service is internal, its name is in the <application name>-ack-ingress-nginx-v1-controller-internal format. Make sure that the Service name does not exceed 63 characters in length. Otherwise, resource creation may fail and the following error message is displayed:

  no service with name xxx-open-api-test-inter-ack-ingress-nginx-v1-controller-internal found in namespace open-api-test-inter: services "xxx-open-api-test-inter-ack-ingress-nginx-v1-controller-internal" not found   ,

  If this error occurs, check and shorten the name of the Helm application and redeploy the application.

Deploy a new NGINX Ingress controller

After an ACK cluster is created, an NGINX Ingress controller that has two pod replicas is automatically deployed. An Internet-facing SLB instance is also created as the frontend load balancing Service.

To deploy another independent NGINX Ingress controller in the cluster, perform the following steps:

1. Log on to the ACK console. In the left-side navigation pane, click Clusters.

2. On the Clusters page, select the resource group and region, find the cluster that you want to manage, and then click the cluster name. In the left-side navigation pane, choose Applications > Helm.

3. On the Helm page, click Deploy. In the Basic Information step, configure the parameters based on the following table.

   Parameter	Example
   Application Name	ack-ingress-nginx Note You can configure a custom application name. Take note of the length limits for Service names.
   Namespace	kube-system
   Source	The source of the application. Default value: Marketplace.
   Chart	Set the Use Scenarios parameter to All. Set the Supported Architecture parameter to amd64. Enter ack-ingress-nginx in the search box. If your cluster runs Kubernetes 1.20 or earlier, click ack-ingress-nginx. If your cluster runs Kubernetes 1.22 or later, click ack-ingress-nginx-v1.

   Click Next.

4. In the Parameters step, configure the Chart Version parameter and click OK.

   Note

   You can select the chart version 4.0.17 or later (ack-ingress-nginx-v1 1.8.0-aliyun.1 or later) only if your ACK cluster runs Kubernetes 1.22 or later. If your ACK cluster runs Kubernetes 1.20, select the chart version 4.0.16 (ack-ingress-nginx-v1 1.2.1-aliyun.1).

   The following table describes the parameters of ack-ingress-nginx-v1.

   Parameter	Description
   controller.image.repository	The image registry of ingress-nginx.
   controller.image.tag	The image version of ingress-nginx. For more information, see NGINX Ingress controller.
   controller.ingressClassResource.name	The Ingress class of the Ingress controller. The Ingress controller handles only the Ingresses that are annotated with the Ingress class. Important This parameter serves as an alternative to the controller.ingressClass parameter for ACK clusters that run a Kubernetes version earlier than 1.22. You can also specify this parameter in the kubernetes.io/ingress.class annotation. The Ingress class of each Ingress controller must be unique within a cluster. The Ingress class of the default Ingress controller in a cluster is nginx. Therefore, do not set this parameter to nginx.
   controller.ingressClassResource.controllerValue	The controller class of the Ingress controller. Important The controller class of each Ingress controller must be unique within a cluster. The controller class of the default Ingress controller in a cluster is k8s.io/ingress-nginx. Therefore, do not set this parameter to k8s.io/ingress-nginx.
   controller.replicaCount	The number of pods that are provisioned for the Ingress controller.
   controller.service.enabled	Specifies whether to use Internet-facing SLB instances and internal-facing SLB instances for load balancing.
   controller.service.external.enabled	Specifies whether to use an Internet-facing SLB instance for load balancing. If you do not want to use an Internet-facing SLB instance, set this parameter to false.
   controller.service.internal.enabled	Specifies whether to use an internal-facing SLB instance for load balancing. If you want to use an internal-facing SLB instance, set this parameter to true.
   controller.kind	The deployment mode of the Ingress controller. Valid values: Deployment and DaemonSet.
   controller.electionID	The ID that is used to update the status of Ingresses during primary Ingress controller election. Important When you deploy multiple Ingress controllers that belong to the same namespace from the marketplace of the ACK console, you must set the electionID parameter to different values for different Ingress controllers. This helps prevent conflicts during primary Ingress controller election.
   controller.metrics.enabled	Specifies whether to enable the ingress-nginx metrics.
   controller.metrics.serviceMonitor.enabled	Specifies whether to enable the ServiceMonitor, which is used to configure rules for collecting metrics. Note After you enable the ingress-nginx metrics, we recommend that you enable the ServiceMonitor. Then, the system automatically configures Managed Service for Prometheus to collect metrics.
   controller.service.internal.loadBalancerClass	Specifies whether the Service uses a Classic Load Balancer (CLB) or Network Load Balancer (NLB) instance. Valid values: "alibabacloud.com/clb": The Service uses a CLB instance. "alibabacloud.com/nlb": The Service uses an NLB instance. By default, the Service uses a CLB instance.

5. View the deployed NGINX Ingress controller.

   On the Helm page, you can view that the new NGINX Ingress controller is deployed.

Test network connectivity

In the following example, an application is created and uses the new NGINX Ingress controller to expose Internet-facing Services.

1. Deploy an NGINX application.

   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: nginx
   spec:
     replicas: 1
     selector:
       matchLabels:
         run: nginx
     template:
       metadata:
         labels:
           run: nginx
       spec:
         containers:
         - image: anolis-registry.cn-zhangjiakou.cr.aliyuncs.com/openanolis/nginx:1.14.1-8.6
           imagePullPolicy: Always
           name: nginx
           ports:
           - containerPort: 80
             protocol: TCP
         restartPolicy: Always
   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: nginx
   spec:
     ports:
     - port: 80
       protocol: TCP
       targetPort: 80
     selector:
       run: nginx
     sessionAffinity: None
     type: NodePort

2. Use the NGINX Ingress controller to provide Internet-facing Services.

   apiVersion: networking.k8s.io/v1
   kind: Ingress
   metadata:
     name: nginx
   spec:
     # Set the value to the Ingress class of the new NGINX Ingress controller. 
     ingressClassName: "<YOUR_INGRESS_CLASS>"
     rules:
     - host: foo.bar.com
       http:
         paths:
         - path: /
           backend:
             service: 
               name: nginx
               port:
                 number: 80
           pathType: ImplementationSpecific

   Note

   You must configure the kubernetes.io/spec.ingressClassName annotation.

   After you deploy the application, perform the following steps to query the Ingress IP address and the IP address of the new NGINX Ingress controller:

   • Run the following command to query the IP address of the Internet-facing SLB instance associated with the default NGINX Ingress controller:

      kubectl -n kube-system get svc nginx-ingress-lb

     Expected output:

     NAME               TYPE           CLUSTER-IP    EXTERNAL-IP    PORT(S)                      AGE
     nginx-ingress-lb   LoadBalancer   172.19.XX.XX   192.0.XX.XX   80:31429/TCP,443:32553/TCP   2d

   • Run the following command to query the IP address of the Internet-facing SLB instance associated with the new NGINX Ingress controller:

     kubectl -n <YOUR_NAMESPACE> get svc nginx-ingress-lb

     Expected output:

     NAME               TYPE           CLUSTER-IP     EXTERNAL-IP     PORT(S)                      AGE
     nginx-ingress-lb   LoadBalancer   172.19.XX.XX   198.51.XX.XX   80:30969/TCP,443:31325/TCP   39m

   • Run the following command to query the Ingress configurations:

     kubectl get ing

     Expected output:

     NAME      HOSTS         ADDRESS         PORTS     AGE
     nginx     foo.bar.com   198.51.XX.XX   80        5m

   The output shows that the Ingress IP address is the same as the IP address of the new NGINX Ingress controller.

3. Access the application by using the default NGINX Ingress controller and the new NGINX Ingress controller.

   curl -H "Host: foo.bar.com" http://192.0.XX.XX

   Expected output:

   <html>
   <head><title>404 Not Found</title></head>
   <body>
   <center><h1>404 Not Found</h1></center>
   <hr><center>nginx</center>
   </body>
   </html>

   Use the new NGINX Ingress controller to access the application.

   curl -H "Host: foo.bar.com" http://198.51.XX.XX

   Expected output:

   <!DOCTYPE html>
   <html>
   <head>
   <title>Welcome to nginx!</title>
   <style>
       body {
           width: 35em;
           margin: 0 auto;
           font-family: Tahoma, Verdana, Arial, sans-serif;
       }
   </style>
   </head>
   <body>
   <h1>Welcome to nginx!</h1>
   <p>If you see this page, the nginx web server is successfully installed and
   working. Further configuration is required.</p>
   
   <p>For online documentation and support please refer to
   <a href="http://nginx.org/">nginx.org</a>.<br/>
   Commercial support is available at
   <a href="http://nginx.com/">nginx.com</a>.</p>
   
   <p><em>Thank you for using nginx.</em></p>
   </body>
   </html>

The preceding tests show that Services exposed by using different NGINX Ingress controllers do not interfere with each other. This solution is suitable for scenarios in which some Services must be accessible to external users, whereas others allow only requests from non-Kubernetes workloads within the same VPC.