Deploy multiple Ingress controllers for traffic isolation

Updated at:
Copy as MD

Install independent NGINX Ingress controllers via Helm alongside the default Add-ons controller to create dedicated traffic entry points for different services or environments, isolate traffic with separate Internet-facing and internal-facing SLB instances, or provide different controller configurations and versions for specific applications—achieving complete fault and configuration isolation unlike a single shared controller.

Important

Controllers installed via Helm differ from the default Add-ons controller:

  • Integrated features: The Add-ons controller includes additional features such as canary release, logging and monitoring, and cluster inspection.

  • Responsibility: You manage the lifecycle of Helm-installed controllers, including upgrades, configuration, and troubleshooting.

How it works

Each controller has a unique IngressClass name. Set the spec.ingressClassName field in an Ingress to target a specific controller. Only the matching controller applies the rules, achieving traffic isolation.

Example: isolating Internet-facing and internal-facing traffic.

image

Prerequisites

Cluster runs version 1.22 or later.

Component versions for clusters on 1.20 or earlier are EOL ([Product Announcement] End of Maintenance for NGINX Ingress controller v1.2 and Earlier). Manually upgrade an ACK cluster if needed.

Deploy a new Ingress controller using Helm

  1. On the ACK Clusters page, click the name of your cluster. In the left navigation pane, click Applications > Helm.

  2. Click Deploy to install ack-ingress-nginx-v1.

    Configure these key parameters:

    Parameter

    Description

    Application name

    The name must be unique within the cluster.

    Important

    Prefixes the Service name: <Application Name>-ack-ingress-nginx-v1-controller (Internet-facing) or <Application Name>-ack-ingress-nginx-v1-controller-internal (internal-facing). Creation fails if total name exceeds 63 characters.

    Chart

    Search for and select ack-ingress-nginx-v1.

    The ack-ingress-nginx chart is no longer maintained.

    Chart Version

    • For clusters of version 1.24 or later: 4.0.22 or later.

    • For clusters of version 1.22: versions from 4.0.16 (inclusive) to 4.0.22 (exclusive).

    Parameters

    Default: deploys as a Deployment with 2 replicas, creating an Internet-facing LoadBalancer Service bound to a CLB instance.

    Adjust defaults in ack-ingress-nginx-v1 parameters.

    This example deploys an internal-facing controller. Set controller.service.external.enabled to false and controller.service.internal.enabled to true.

    On the Helm page, note the Namespace (under Basic Information) and IngressClass name and Service name (under Resource) for later use.

Verify traffic isolation

This scenario uses separate Internet-facing and internal-facing controllers to verify isolation:

Deploy a sample application and create an Ingress rule targeting only the new controller.

Step 1: Deploy a test application

  1. Create an nginx.yaml file.

    Example: NGINX Deployment and Service.
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      name: nginx
    spec:
      replicas: 1
      selector:
        matchLabels:
          run: nginx
      template:
        metadata:
          labels:
            run: nginx
        spec:
          containers:
          - image: anolis-registry.cn-zhangjiakou.cr.aliyuncs.com/openanolis/nginx:1.14.1-8.6
            imagePullPolicy: Always
            name: nginx
            ports:
            - containerPort: 80
              protocol: TCP
          restartPolicy: Always
    ---
    apiVersion: v1
    kind: Service
    metadata:
      name: nginx
    spec:
      ports:
      - port: 80
        protocol: TCP
        targetPort: 80
      selector:
        run: nginx
      sessionAffinity: None
      type: NodePort
  2. Deploy the sample application.

    kubectl apply -f nginx.yaml

Step 2: Create and bind an Ingress rule

  1. Create an ingress.yaml file.

    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      name: nginx
    spec:
      # Change the value to the IngressClass name you configured earlier (controller.ingressClassResource.name).
      ingressClassName: "<YOUR_INGRESS_CLASS>"
      rules:
      # The following domain name is for testing purposes. Replace it with your actual domain name in a production environment.
      - host: foo.bar.com
        http:
          paths:
          - path: /
            backend:
              service: 
                name: nginx
                port:
                  number: 80
            pathType: ImplementationSpecific
  2. Create the Ingress rule.

    kubectl apply -f ingress.yaml

Step 3: Test access

  1. Get the SLB IP addresses of each controller.

    • SLB IP address of the default Internet-facing controller:

      PUBLIC_IP=$(kubectl get svc -n kube-system nginx-ingress-lb -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
      echo "Public Ingress IP: $PUBLIC_IP"
    • SLB IP address of the new internal-facing controller:

      # Replace <YourNamespace> with the namespace of the new controller (for example, default).
      # Replace <YourChartName> with the application name (release name) of the new controller.
      INTERNAL_IP=$(kubectl get svc -n <YourNamespace> <YourChartName>-ack-ingress-nginx-v1-controller-internal -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
      echo "Internal Ingress IP: $INTERNAL_IP"
  2. Access the application through the internal-facing controller (expected to succeed).

    From a terminal within the VPC, send a request. A 200 status code confirms the internal-facing controller routes traffic correctly.

    # Replace with the actual internal IP address.
    curl -o /dev/null -s -w "%{http_code}\n" -H "Host: foo.bar.com" http://$INTERNAL_IP
  3. Access the application through the Internet-facing controller (expected to fail).

    Send a curl request. A 404 Not Found response confirms the Internet-facing controller ignored the Ingress rule—traffic isolation works.

    # Replace with the actual Internet-facing IP address.
    curl -H "Host: foo.bar.com" http://$PUBLIC_IP

Production deployment

  • Resource planning: Configure these parameters for high availability:

    • Configure multiple replicas: controller.replicaCount

    • Set appropriate resource requests and limits: controller.resources.requests and controller.resources.limits

    • Pod anti-affinity: Add a podAntiAffinity rule under controller.affinity to schedule Pods on different nodes.

  • Monitoring: Set controller.metrics.enabled: true and controller.metrics.serviceMonitor.enabled: true. Integrate with Prometheus and monitor metrics such as request latency, error rates (4xx/5xx), and configure alert rules.

  • Performance: For low-latency scenarios, use an NLB instance in the Service configuration:

    • Internal-facing Service: controller.service.internal.loadBalancerClass: "alibabacloud.com/nlb"

    • Internet-facing Service: controller.service.loadBalancerClass: "alibabacloud.com/nlb"

  • Version maintenance:

Appendix: Key component parameters

Parameter

Description

controller.image.repository

The image repository of the NGINX Ingress controller.

controller.image.tag

The image version of the NGINX Ingress controller.

controller.ingressClassResource.name

A unique IngressClass name within the cluster. Cannot be nginx (reserved for the default controller).

controller.ingressClassResource.controllerValue

A unique controller class value. Cannot be k8s.io/ingress-nginx (reserved for the default controller).

controller.replicaCount

The number of controller Pod replicas. Set to 2+ for high availability.

controller.service.enabled

Whether to create a LoadBalancer Service (Internet-facing or internal-facing) to expose the controller.

controller.service.external.enabled

If true, creates an Internet-facing SLB Service.

controller.service.internal.enabled

If true, creates an internal-facing SLB Service (VPC-only).

controller.kind

The deployment mode of the Ingress controller: Deployment or DaemonSet.

controller.electionID

The leader election identifier among controller replicas. Only the Leader can update Ingress resource status.

Must be unique when deploying multiple controllers in the same namespace.

controller.metrics.enabled

If true, enables a Prometheus metrics endpoint.

controller.metrics.serviceMonitor.enabled

If true, creates a ServiceMonitor for automatic Prometheus discovery.

Recommended when controller.metrics.enabled is true.

controller.service.loadBalancerClass

The SLB type for Internet-facing Services.

  • "alibabacloud.com/clb" (Default): Use a CLB instance.

  • "alibabacloud.com/nlb": Use an NLB instance.

controller.service.internal.loadBalancerClass

The SLB type for internal-facing Services.

  • "alibabacloud.com/clb" (Default): Use a CLB instance.

  • "alibabacloud.com/nlb": Use an NLB instance.