Container Service for Kubernetes:ALB Ingress FAQ

Issue

In real-world scenarios, you need to access ALB Ingress over both the public and private networks. But you are unsure whether ALB Ingress supports both simultaneously.

Solution

Yes. To use both public and private network access, create a public ALB instance. The instance creates an elastic IP address (EIP) in each zone. ALB uses these EIPs for public network communication. The instance also provides a private virtual IP address (VIP). Use this private VIP to access ALB over the private network. If you require only private network access, create a private ALB instance.

Issue

When you search for the ALB Ingress Controller pod in your cluster, you find no related pods in the kube-system namespace. You cannot view the ALB Ingress Controller component using standard methods.

Solution

You can see the ALB Ingress Controller pod in the kube-system namespace only in ACK dedicated clusters. ACK Basic Edition, ACK Pro Edition, and ACK Serverless fully manage the ALB Ingress Controller component. So you cannot see the pod in those clusters. For instructions on upgrading an ACK dedicated cluster to an ACK managed cluster Pro Edition, see Hot migrate an ACK dedicated cluster to an ACK managed cluster Pro Edition.

Issue

You want to attach backend pods to ALB by IP address. But by default, the Service does not automatically create an IP-based server group. This limits traffic distribution to backend services.

Solution

Add the annotation alb.ingress.kubernetes.io/server-group-type: Ip to the Service. This creates an IP-based server group for the Service. Then you can register backend pods with ALB by IP address.

apiVersion: v1
kind: Service
metadata:
  annotations:
    alb.ingress.kubernetes.io/server-group-type: Ip
  name: tea-svc
spec:
  ports:
    - port: 80
      targetPort: 80
      protocol: TCP
  selector:
    app: tea
  type: ClusterIP

Root cause

After you manually enable WAF protection for your ALB instance in the WAF console, the ALB Ingress Controller detects during reconciliation that the actual edition of the ALB instance does not match the edition field declared in AlbConfig. If AlbConfig does not explicitly specify edition: StandardWithWaf, the Controller reverts the instance to the default Standard Edition. This downgrades WAF to Standard Edition.

Solution

To keep WAF Enhanced Edition, explicitly set the edition field to StandardWithWaf in AlbConfig when you create a new ALB instance or upgrade or downgrade an existing one.

Root cause

• 503 (Service Temporarily Unavailable) error

  • No matching routing rule: The request path does not match the routing rule configured in the Ingress.

  • No healthy backend pods: All pods associated with the service are not ready or do not exist. This results in an empty endpoint object.

• 502 (Bad Gateway) error

  An HTTP or HTTPS listener receives a client connection request. If the ALB instance cannot forward the request to a pod or receive a response from a pod, the instance sends an HTTP 502 Bad Gateway status code to the client.

• 404 (Not Found) error

  This usually means the request matched a routing rule defined in the Ingress, but does not match the actual service URL provided by the application in the pod.

• 400 (Bad Request) error

  This can occur for various reasons, such as an HTTP request being sent to an HTTPS listener.

For more information about HTTP error codes, see ALB status codes.

Solution

1. Check the Ingress status. Run the kubectl describe ingress <ingress-name> -n <namespace> command and inspect the Events section for error messages. For example, if you see an event similar to listener does not exist in alb, you must create the listener required for the Ingress resource in the AlbConfig.

   ...
   Events:
     Type     Reason                  Age     From     Message
     ----     ------                  ----    ----     -------
     Warning  FailedBuildModel        ****    ingress  listener is not exist in alb, port: 443, protocol: HTTPS
     Warning  FailedBuildModel        ****    ingress  listener not found for (443/HTTPS), with ingresses 1
   ...

2. Check the backend endpoints. Run the kubectl get endpoints <service-name> -n <namespace> command to confirm that the ENDPOINTS field contains at least one healthy pod IP address and port. If the field is empty, verify that the service's selector matches the labels on your pods, and ensure the pods are in the Running state.

3. Check the pod status and logs. First, run the kubectl get pod -l <app=your-app> -n <namespace> command to check the status of your application pods. Then, run kubectl logs <pod-name> -n <namespace> to inspect the logs of a specific pod for any startup failures or request processing errors.

4. Test network connectivity. From within another pod or directly from a node, use curl to access the backend service's ClusterIP or a direct pod IP. This verifies that the service is reachable from within the cluster.

Root cause

• The ALB instance is not listening on port 443. You have configured TLS encryption for the Ingress, but the corresponding HTTPS:443 listener has not been created.

• The certificate is configured incorrectly. The secret type is not kubernetes.io/tls or IngressTLS, or the content of tls.crt and tls.key in the data field is incorrect or mismatched.

• The certificate update has not taken effect. You updated the certificate in the Alibaba Cloud Certificate Management Service, but you did not update the certificate ID specified in the AlbConfig, or the automatic certificate discovery reconciliation was not triggered. As a result, the ALB instance still references the old certificate.

Solution

1. Check the listener port. Run the kubectl describe albconfig <alb-name> -n <namespace> command to check whether the spec.listeners.port: 443 and spec.listeners.protocol: HTTPS configurations are missing.

2. Check the Ingress configuration. Check whether the Ingress configuration is missing the alb.ingress.kubernetes.io/listen-ports: [{"HTTP": 80}, {"HTTPS": 443}] annotation. This annotation associates the Ingress with the HTTP and HTTPS listeners.

3. Check the secret configuration. Check the secretName field of spec.tls in the Ingress configuration to confirm that the correct secret is referenced. Run the kubectl get secret <secret-name> -n <namespace> -o yaml command to confirm the secret type and data integrity.

Issue

After you create new ALB Ingress rules, they do not take effect as expected. Traffic does not forward to the intended backend services.

Root cause

ALB instances maintain routing rules in serial order. If multiple ALB Ingress resources use the same ALB instance, a misconfiguration in one ALB Ingress prevents all other ALB Ingress changes from taking effect.

Solution

If your ALB Ingress does not take effect, a previous ALB Ingress may be misconfigured. Fix the misconfigured ALB Ingress first. Then your new ALB Ingress will take effect.

Issue

After you change ALB Ingress configuration or associate it with AlbConfig, no anomalous activity appears. But the changes do not take effect.

Solution

This may happen if the binding between IngressClass and AlbConfig is incorrect. Check the parameters field in your IngressClass. For step-by-step instructions, see Associate AlbConfig with Ingress using IngressClass.

Issue

ALB Ingress forwarding rules are deleted right after creation. This returns 503 status codes and stops traffic distribution.

Solution

Check whether all Ingress resources linked to the forwarding rules have the canary: true annotation. Canary releases require a baseline version to route traffic. So you do not need to add canary: true to the baseline Ingress. For instructions on using ALB Ingress for phased releases, see Implement phased releases with ALB Ingress.

Canary releases support only two Ingress resources and have limited conditions. We recommend custom forwarding rules for richer traffic routing options. For instructions, see Customize ALB Ingress forwarding rules.

Issue

Requests to ports other than port 80 fail to connect. AlbConfig reports “listener is not exist in alb, port: xxx”. So those ports are not listening or forwarding traffic.

Solution

The default AlbConfig includes only a listener for port 80. To listen on other ports, add listener configurations for those ports in AlbConfig. For instructions, see Create listeners.

Issue

I configured HTTP and HTTPS listeners in AlbConfig. But I cannot access either port. Traffic is not being listened to or forwarded.

Solution

Confirm that you added the alb.ingress.kubernetes.io/listen-ports annotation to the Ingress resource. This annotation tells ALB Ingress to listen on both HTTP (port 80) and HTTPS (port 443). For example:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: https-ingress
  annotations:
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80},{"HTTPS": 443}]' # Add this annotation to make ALB Ingress work with multiple listeners.
spec:
  #...

Why do manual configuration changes in the ALB console get lost? Why are rules deleted and access logs disabled?

Why does an error message appear during the tuning process: Specified parameter array contains too many items, up to 15 items, Certificates is not valid?

After I configure an ALB instance in the console, why does running kubectl apply to update AlbConfig ACL settings delete some listeners?

Issue

In Kubernetes, server reconciliation takes too long when pods attached to a Service scale. This affects real-time elasticity. Investigation shows that reconciliation time increases with the number of associated Ingress resources.

Solution

To improve server tuning efficiency, you can apply the following optimization measures:

• Limit Ingress count: Attach no more than 30 Ingress resources to one Service.

• Combine Ingress rules: If you have many Ingress resources, attach multiple Services to one Ingress. Define multiple forwarding rules in that Ingress to improve server reconciliation performance.

Issue

When using the Flannel plugin with Local-mode Services, traffic distribution across nodes is uneven. Some nodes carry higher loads than others. How do I auto-assign node weights based on pod count per node to achieve balanced traffic distribution?

Solution

In Flannel clusters, when a Service is in Local mode, node weights are calculated as shown in the diagram below. In this example, application pods (app=nginx) run on three ECS instances. Service A exposes them externally.