Container Service for Kubernetes:Advanced Nginx Ingress configurations

Configuration instructions

The configuration method for the Nginx Ingress Controller in Container Service for Kubernetes is fully compatible with the community version. For a full list of configurations, see NGINX Configuration.

Configure a routing service for URL redirection

When you use the Nginx Ingress Controller, Nginx forwards the full path to the backend. For example, a request to the /service1/api path through the Ingress is forwarded directly to the /service1/api path of the backend pod. If your backend Service path is /api, a path mismatch occurs, which results in a 404 status code. In this case, you can use the nginx.ingress.kubernetes.io/rewrite-target annotation to rewrite the path to the required directory.

Create an Nginx Ingress based on your cluster version.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: foo.bar.com
  namespace: default
  annotations:
    # URL redirection.
    nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
  rules:
  - host: foo.bar.com
    http:
      paths:
    # For Ingress Controller versions 0.22.0 and later, you must define the path using a regular expression and use it with a capturing group in rewrite-target.
      - path: /svc(/|$)(.*)
        backend:
          service: 
            name: web1-service
            port: 
              number: 80
        pathType: ImplementationSpecific

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: foo.bar.com
  namespace: default
  annotations:
    # URL redirection.
    nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
  rules:
  - host: foo.bar.com
    http:
      paths:
      # For Ingress Controller versions 0.22.0 and later, you must define the path using a regular expression and use it with a capturing group in rewrite-target.
      - path: /svc(/|$)(.*)
        backend:
          serviceName: web1-service
          servicePort: 80

1. Access the Nginx Service.

   1. Run the following command to obtain the ADDRESS.

      kubectl  get  ingress

      Expected output:

      NAME           CLASS   HOSTS                ADDRESS          PORTS   AGE
      foo.bar.com    nginx   foo.bar.com        172.16.XX.XX       80      46m

   2. Run the following command. Replace ADDRESS with the IP address of the Ingress.

      curl -k -H "Host: foo.bar.com"  http://<ADDRESS>/svc/foo

      Expected output:

      web1: /foo

Rewrite configuration

You can use the nginx.ingress.kubernetes.io/rewrite-target annotation for basic rewrite configurations. For more information, see Configure a routing service for URL redirection.

For more complex and advanced rewrite requirements, use the following annotations:

• nginx.ingress.kubernetes.io/server-snippet: Adds a custom configuration to the `server` block.

• nginx.ingress.kubernetes.io/configuration-snippet: Adds a custom configuration to the `location` block.

These two annotations add custom code snippets to the Nginx `server` block of the Ingress component. This provides the flexibility to customize the Nginx configuration for different scenarios.

Configuration example:

annotations:
     nginx.ingress.kubernetes.io/server-snippet: |
         rewrite ^/v4/(.*)/card/query http://foo.bar.com/v5/#!/card/query permanent;
     nginx.ingress.kubernetes.io/configuration-snippet: |
         rewrite ^/v6/(.*)/card/query http://foo.bar.com/v7/#!/card/query permanent;

Run the following command to view the Nginx configuration file in the Nginx Ingress Controller component.

kubectl exec nginx-ingress-controller-xxxxx --namespace kube-system -- cat /etc/nginx/nginx.conf   # Modify the pod name based on your environment.

The following nginx.conf file is generated from the example configuration.

# start server foo.bar.com
    server {
        server_name foo.bar.com ;
        listen 80;
        listen [::]:80;
        set $proxy_upstream_name "-";
    # server-snippet configuration.
        rewrite ^/v4/(.*)/card/query http://foo.bar.com/v5/#!/card/query permanent;
        ...
    # configuration-snippet configuration.
      rewrite ^/v6/(.*)/card/query http://foo.bar.com/v7/#!/card/query permanent;
      ...
    }
    # end server foo.bar.com

The snippet also supports global configurations. For more information, see server-snippet.

For more information about the rewrite instruction, see the official Nginx documentation.

Configure an HTTPS certificate for a routing rule

You can use the native Ingress semantics to configure an HTTPS certificate for your website.

1. Prepare your Service certificate.

   Note

   The domain name must match the configured host. Otherwise, the Nginx Ingress Controller cannot load the certificate.

   1. Run the following command to generate a certificate file tls.crt and a private key file tls.key.

      openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=foo.bar.com/O=foo.bar.com"

   2. Run the following command to create a Secret.

      This command uses the certificate and private key to create a Kubernetes Secret named `tls-test-ingress`. You must reference this Secret when you create the Ingress.

      kubectl create secret tls tls-test-ingress --key tls.key --cert tls.crt

2. Create an Ingress resource that references the Secret from the previous step in the `tls` field.

   Clusters of version 1.19 and later

   apiVersion: networking.k8s.io/v1
   kind: Ingress
   metadata:
     name: test-test-ingress
   spec:
     # Reference the TLS certificate.
     tls:
     - hosts:
       - foo.bar.com # The domain name that corresponds to the certificate. 
       secretName: tls-test-ingress
     rules:
     - host: tls-test-ingress.com
       http:
         paths:
         - path: /foo
           backend:
             service:
               name: web1-svc
               port:
                 number: 80
           pathType: ImplementationSpecific

   Clusters of versions earlier than 1.19

   apiVersion: networking.k8s.io/v1beta1
   kind: Ingress
   metadata:
     name: test-test-ingress
   spec:
     # Reference the TLS certificate.
     tls:
     - hosts:
       - foo.bar.com # The domain name that corresponds to the certificate.
       secretName: tls-test-ingress
     rules:
     - host: tls-test-ingress.com
       http:
         paths:
         - path: /foo
           backend:
             serviceName: web1-svc
             servicePort: 80

3. Configure the hosts file or use a real domain name to access the TLS service.

   You can access the web1-svc Service at https://tls-test-ingress.com/foo.

Configure HTTPS mutual authentication

To enhance security, you can configure mutual HTTPS authentication between the server and the client. The Nginx Ingress Controller supports this feature through annotations.

1. Run the following command to create a self-signed CA certificate.

   openssl req -x509 -sha256 -newkey rsa:4096 -keyout ca.key -out ca.crt -days 356 -nodes -subj '/CN=Fern Cert Authority'

   Expected output:

   Generating a 4096 bit RSA private key
   .............................................................................................................++
   .....................................................................................++
   writing new private key to 'ca.key'

2. Run the following commands to create a server certificate.

   1. Run the following command to generate a certificate signing request (CSR) file for the server certificate.

      openssl req -new -newkey rsa:4096 -keyout server.key -out server.csr -nodes -subj '/CN=foo.bar.com'

      Expected output:

      Generating a 4096 bit RSA private key
      ................................................................................................................................++
      .................................................................++
      writing new private key to 'server.key'

   2. Run the following command to use the root certificate to sign the server CSR file and generate the server certificate.

      openssl x509 -req -sha256 -days 365 -in server.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out server.crt

      Expected output:

      Signature ok
      subject=/CN=foo.bar.com
      Getting CA Private Key

3. Run the following commands to create a client certificate.

   1. Generate a CSR file for the client certificate.

      openssl req -new -newkey rsa:4096 -keyout client.key -out client.csr -nodes -subj '/CN=Fern'

      Expected output:

      Generating a 4096 bit RSA private key
      .......................................................................................................................................................................................++
      ..............................................++
      writing new private key to 'client.key'
      -----

   2. Run the following command to use the root certificate to sign the client CSR file and generate the client certificate.

      openssl x509 -req -sha256 -days 365 -in client.csr -CA ca.crt -CAkey ca.key -set_serial 02 -out client.crt

      Expected output:

      Signature ok
      subject=/CN=Fern
      Getting CA Private Key

4. Run the following command to check the created certificates.

   ls

   Expected output:

   ca.crt  ca.key  client.crt  client.csr  client.key  server.crt  server.csr  server.key

5. Run the following command to create a Secret for the CA certificate.

   kubectl create secret generic ca-secret --from-file=ca.crt=ca.crt

   Expected output:

   secret/ca-secret created

6. Run the following command to create a Secret for the server certificate.

   kubectl create secret generic tls-secret --from-file=tls.crt=server.crt --from-file=tls.key=server.key

   Expected output:

   secret/tls-secret created

7. Deploy the following template to create a test Nginx Ingress.

   Clusters of version 1.19 and later

   apiVersion: networking.k8s.io/v1
   kind: Ingress
   metadata:
     annotations:
       nginx.ingress.kubernetes.io/auth-tls-verify-client: "on"
       nginx.ingress.kubernetes.io/auth-tls-secret: "default/ca-secret"
       nginx.ingress.kubernetes.io/auth-tls-verify-depth: "1"
       nginx.ingress.kubernetes.io/auth-tls-pass-certificate-to-upstream: "true"
     name: nginx-test
     namespace: default
   spec:
     rules:
     - host: foo.bar.com
       http:
         paths:
         - backend:
             service:
               name: http-svc
               port: 
                 number: 80
           path: /
           pathType: ImplementationSpecific
     tls:
     - hosts:
       - foo.bar.com
       secretName: tls-secret

   Clusters of versions earlier than 1.19

   apiVersion: networking.k8s.io/v1beta1
   kind: Ingress
   metadata:
     annotations:
       nginx.ingress.kubernetes.io/auth-tls-verify-client: "on"
       nginx.ingress.kubernetes.io/auth-tls-secret: "default/ca-secret"
       nginx.ingress.kubernetes.io/auth-tls-verify-depth: "1"
       nginx.ingress.kubernetes.io/auth-tls-pass-certificate-to-upstream: "true"
     name: nginx-test
     namespace: default
   spec:
     rules:
     - host: foo.bar.com
       http:
         paths:
         - backend:
             serviceName: http-svc
             servicePort: 80
           path: /
     tls:
     - hosts:
       - foo.bar.com
       secretName: tls-secret

   Expected output:

   ingress.networking.k8s.io/nginx-test configured

8. Run the following command to view the IP address of the Ingress.

   kubectl get ing

   The IP address of the Ingress is displayed in the `ADDRESS` column of the output.

   NAME         HOSTS                    ADDRESS         PORTS     AGE
   nginx-test   foo.bar.com              39.102.XX.XX    80, 443   4h42m

9. Run the following command to update the `hosts` file. Replace the IP address with the actual IP address of the Ingress.

   echo "39.102.XX.XX  foo.bar.com" | sudo tee -a /etc/hosts

   Verification:

   • Access without a client certificate

     curl --cacert ./ca.crt  https://foo.bar.com

     Expected output:

     <html>
     <head><title>400 No required SSL certificate was sent</title></head>
     <body>
     <center><h1>400 Bad Request</h1></center>
     <center>No required SSL certificate was sent</center>
     <hr><center>nginx/1.19.0</center>
     </body>
     </html>

   • Authenticate with a client certificate

     curl --cacert ./ca.crt --cert ./client.crt --key ./client.key https://foo.bar.com

     Expected output:

     <!DOCTYPE html>
     <html>
     <head>
     <title>Welcome to nginx!</title>
     <style>
         body {
             width: 35em;
             margin: 0 auto;
             font-family: Tahoma, Verdana, Arial, sans-serif;
         }
     </style>
     </head>
     <body>
     <h1>Welcome to nginx!</h1>
     <p>If you see this page, the nginx web server is successfully installed and
     working. Further configuration is required.</p>
     
     <p>For online documentation and support please refer to
     <a href="http://nginx.org/">nginx.org</a>.<br/>
     Commercial support is available at
     <a href="http://nginx.com/">nginx.com</a>.</p>
     
     <p>Thank you for using nginx.</p>
     </body>
     </html>

Configure HTTPS service to forward traffic to backend containers over HTTPS

By default, the Nginx Ingress Controller forwards requests to backend application containers over HTTP. If your application containers use HTTPS, you can use the nginx.ingress.kubernetes.io/backend-protocol: "HTTPS" annotation to configure the Nginx Ingress Controller to forward requests over HTTPS.

The following is an example Nginx Ingress configuration:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: backend-https
  annotations:
    # Note: You must specify that the backend service uses HTTPS.
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
spec:
  tls:
  - hosts:
    - <YOUR-HOST-NAME>
    secretName: <YOUR-SECRET-CERT-NAME>
  rules:
  - host: <YOUR-HOST-NAME>
    http:
      paths:
      - path: /
        backend:
          service:
            name: <YOUR-SERVICE-NAME>
            port: 
              number: <YOUR-SERVICE-PORT>
        pathType: ImplementationSpecific

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: backend-https
  annotations:
    # Note: You must specify that the backend service uses HTTPS.
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
spec:
  tls:
  - hosts:
    - <YOUR-HOST-NAME>
    secretName: <YOUR-SECRET-CERT-NAME>
  rules:
  - host: <YOUR-HOST-NAME>
    http:
      paths:
      - path: /
        backend:
          serviceName: <YOUR-SERVICE-NAME>
          servicePort: <YOUR-SERVICE-PORT>

Configure domain names to support regular expressions

In a Kubernetes cluster, Ingress resources do not natively support regular expressions for domain names. However, you can use the nginx.ingress.kubernetes.io/server-alias annotation to enable this functionality.

1. Deploy the following template, which uses the regular expression ~^www\.\d+\.example\.com as an example.

   Clusters of version 1.19 and later

   apiVersion: networking.k8s.io/v1
   kind: Ingress
   metadata:
     name: ingress-regex
     namespace: default
     annotations:
       nginx.ingress.kubernetes.io/server-alias: '~^www\.\d+\.example\.com$, abc.example.com'
   spec:
     rules:
     - host: foo.bar.com
       http:
         paths:
         - path: /foo
           backend:
             service:
               name: http-svc1
               port:
                 number: 80
           pathType: ImplementationSpecific

   Clusters of versions earlier than 1.19

   apiVersion: networking.k8s.io/v1beta1
   kind: Ingress
   metadata:
     name: ingress-regex
     namespace: default
     annotations:
       nginx.ingress.kubernetes.io/server-alias: '~^www\.\d+\.example\.com$, abc.example.com'
   spec:
     rules:
     - host: foo.bar.com
       http:
         paths:
         - path: /foo
           backend:
             serviceName: http-svc1
             servicePort: 80

2. View the configuration of the Nginx Ingress Controller.

   1. Run the following command to view the pods where the Nginx Ingress Controller Service is deployed.

      kubectl get pods -n kube-system | grep nginx-ingress-controller

      Expected output:

      nginx-ingress-controller-77cd987c4c-c****         1/1     Running   0          1h
      nginx-ingress-controller-77cd987c4c-x****         1/1     Running   0          1h

   2. Run the following command to view the configuration of the Nginx Ingress Controller. The effective configuration is in the `Server_Name` field.

      kubectl exec -n kube-system nginx-ingress-controller-77cd987c4c-c**** cat /etc/nginx/nginx.conf | grep -C3 "foo.bar.com"

      Expected output:

        # start server foo.bar.com
        server {
      --
        server {
          server_name foo.bar.com abc.example.com ~^www\.\d+\.example\.com$ ;
          listen 80  ;
          listen 443  ssl http2 ;
      --
      --
          }
        }
        # end server foo.bar.com

3. Run the following command to obtain the IP address of the Ingress.

   kubectl get ing

   Expected output:

   NAME            HOSTS         ADDRESS          PORTS     AGE
   ingress-regex   foo.bar.com   101.37.XX.XX     80        11s

4. Run the following commands to test Service access with different rules.

   Replace IP_ADDRESS with the IP address that you obtained in the previous step.

   • Run the following command to access the Service with Host: foo.bar.com.

     curl -H "Host: foo.bar.com" <IP_ADDRESS>/foo

     Expected output:

     /foo

   • Run the following command to access the Service with Host: www.123.example.com.

     curl -H "Host: www.123.example.com" <IP_ADDRESS>/foo

     Expected output:

     /foo

   • Run the following command to access the Service with Host: www.321.example.com.

     curl -H "Host: www.321.example.com" <IP_ADDRESS>/foo

     Expected output:

     /foo

Configure domain names to support wildcards

Nginx Ingress resources in a Kubernetes cluster support wildcard domain names, such as *.ingress-regex.com.

1. Deploy the following template to create an Nginx Ingress.

   Clusters of version 1.19 and later

   apiVersion: networking.k8s.io/v1
   kind: Ingress
   metadata:
     name: ingress-regex
     namespace: default
   spec:
     rules:
     - host: *.ingress-regex.com
       http:
         paths:
         - path: /foo
           backend:
             service:
               name: http-svc1
               port:
                 number: 80
           pathType: ImplementationSpecific

   Clusters of versions earlier than 1.19

   apiVersion: networking.k8s.io/v1beta1
   kind: Ingress
   metadata:
     name: ingress-regex
     namespace: default
   spec:
     rules:
     - host: *.ingress-regex.com
       http:
         paths:
         - path: /foo
           backend:
             serviceName: http-svc1
             servicePort: 80

2. Run the following command to view the configuration of the Nginx Ingress Controller. The effective configuration is in the `Server_Name` field.

   kubectl exec -n kube-system <nginx-ingress-pod-name> cat /etc/nginx/nginx.conf | grep -C3 "*.ingress-regex.com"

   Note

   Replace nginx-ingress-pod-name with the name of your nginx-ingress pod.

   Expected output:

   # start server *.ingress-regex.com
     server {
       server_name *.ingress-regex.com ;
       listen 80;
       listen [::]:80;
   ...
     }
     # end server *.ingress-regex.com

   Expected output in newer versions of the Nginx Ingress Controller:

   ## start server *.ingress-regex.com
     server {
       server_name ~^(?<subdomain>[\w-]+)\.ingress-regex\.com$ ;
       listen 80;
       listen [::]:80;
   ...
     }
     ## end server *.ingress-regex.com

3. Run the following command to obtain the IP address of the Ingress.

   kubectl get ing

   Expected output:

   NAME            HOSTS                 ADDRESS           PORTS     AGE
   ingress-regex   *.ingress-regex.com   101.37.XX.XX      80        11s

4. Run the following commands to test Service access with different rules.

   Replace IP_ADDRESS with the IP address that you obtained in the previous step.

   • Run the following command to access the Service with Host: abc.ingress-regex.com.

     curl -H "Host: abc.ingress-regex.com" <IP_ADDRESS>/foo

     Expected output:

     /foo

   • Run the following command to access the Service with Host: 123.ingress-regex.com.

     curl -H "Host: 123.ingress-regex.com" <IP_ADDRESS>/foo

     Expected output:

     /foo

   • Run the following command to access the Service with Host: a1b1.ingress-regex.com.

     curl -H "Host: a1b1.ingress-regex.com" <IP_ADDRESS>/foo

     Expected output:

     /foo

Implement phased releases using annotations

Use cert-manager to request a free HTTPS certificate

cert-manager is an open source certificate management tool that provides and automatically renews HTTPS certificates in a cluster. The following example shows how to use cert-manager to request a free certificate and enable automatic renewal.

1. Run the following command to deploy cert-manager.

   kubectl apply -f https://github.com/cert-manager/cert-manager/releases/latest/download/cert-manager.yaml

2. Run the following command to check the pod status.

   kubectl get pods -n cert-manager

   Expected output:

   NAME                     READY   STATUS    RESTARTS   AGE
   cert-manager-1           1/1     Running   0          2m11s
   cert-manager-cainjector  1/1     Running   0          2m11s
   cert-manager-webhook     1/1     Running   0          2m10s

3. Use the following template to create a ClusterIssuer.

   apiVersion: cert-manager.io/v1
   kind: ClusterIssuer
   metadata:
     name: letsencrypt-prod-http01
   spec:
     acme:
       server: https://acme-v02.api.letsencrypt.org/directory
       email: <your_email_n***@gmail.com>  # Replace with your email address.
       privateKeySecretRef:
         name: letsencrypt-http01
       solvers:
       - http01: 
           ingress:
             class: nginx

4. Run the following command to view the ClusterIssuer.

   kubectl get clusterissuer

   Expected output:

   NAME                         READY   AGE
   letsencrypt-prod-http01      True    17s

5. Create an Nginx Ingress resource.

   Clusters of version 1.19 and later

   apiVersion: networking.k8s.io/v1
   kind: Ingress
   metadata:
     name: ingress-tls
     annotations:
       kubernetes.io/ingress.class: "nginx"
       cert-manager.io/cluster-issuer: "letsencrypt-prod-http01"
   spec:
     tls:
     - hosts:
       - <YOUR_DOMAIN_NAME>        # Replace with your domain name.
       secretName: ingress-tls   
     rules:
     - host: <YOUR_DOMAIN_NAME>    # Replace with your domain name.
       http:
         paths:
         - path: /
           backend:
             service:
               name: <YOUR_SERVICE_NAME>  # Replace with your backend service name.
               port: 
                 number: <YOUR_SERVICE_PORT>  # Replace with your service port.
           pathType: ImplementationSpecific

   Clusters of versions earlier than 1.19

   apiVersion: extensions/v1beta1
   kind: Ingress
   metadata:
     name: ingress-tls
     annotations:
       kubernetes.io/ingress.class: "nginx"
       cert-manager.io/cluster-issuer: "letsencrypt-prod-http01"
   spec:
     tls:
     - hosts:
       - <YOUR_DOMAIN_NAME>        # Replace with your domain name.
       secretName: ingress-tls   
     rules:
     - host: <YOUR_DOMAIN_NAME>    # Replace with your domain name.
       http:
         paths:
         - path: /
           backend:
             serviceName: <YOUR_SERVICE_NAME>  # Replace with your backend service name.
             servicePort: <YOUR_SERVICE_PORT>  # Replace with your service port.

   Note

   The domain name that you use to replace your_domain_name must meet the following conditions:

   • The domain name cannot exceed 64 characters.

   • Wildcard domain names are not supported.

   • It is accessible over the public network using the HTTP protocol.

6. Run the following command to view the certificate.

   kubectl get cert

   Expected output:

   NAME          READY   SECRET        AGE
   ingress-tls   True    ingress-tls   52m

   Note

   If the READY status is not True, run kubectl describe cert ingress-tls to view the certificate processing procedure.

7. Run the following command to view the Secret.

   kubectl get secret  ingress-tls

   Expected output:

   NAME          TYPE                DATA   AGE
   ingress-tls   kubernetes.io/tls   2      2m

8. Enter https://[your domain name] in a web browser to access the configured domain name.

Configure HTTP to HTTPS redirection

You can use the nginx.ingress.kubernetes.io/ssl-redirect annotation for Nginx Ingress to force HTTP traffic to be redirected to HTTPS. The following is an example:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "true" # Force HTTP traffic to be redirected to HTTPS.

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "true" # Force HTTP traffic to be redirected to HTTPS.