Diagnose and troubleshoot anomalous events for ALB Ingress - Container Service for Kubernetes

ALB Ingress Controller is an ingress controller for Kubernetes clusters that routes external traffic to internal services. When you use an ALB Ingress to access a service, the ALB Ingress Controller monitors changes to the endpoint resources associated with the service. The controller then syncs the status of backend nodes to the corresponding backend server group and applies these changes to the Application Load Balancer (ALB) instance. This topic describes how to diagnose and resolve anomalous events that you might encounter when you use an ALB Ingress.

How it works

When you access a service through an ALB Ingress, the ALB Ingress Controller monitors changes to various resources and syncs the changes to the associated ALB instance. This process can fail due to various limits or configuration errors. The following figure shows the logical relationships between different resources and the synchronization process.

Step 1: Check for anomalous events

Use the console: In the left-side navigation pane, choose Network > Ingresses. Select the target namespace, click the name of the target ingress to open its details page, and then click the Events tab.

Use kubectl:

kubectl describe ingress <ingress-name> -n <namespace>

Expected output:

...
Events:
  Type     Reason                  Age                     From     Message
  ----     ------                  ----                    ----     -------
  Warning  FailedBuildModel        2m28s (x10 over 6m43s)  ingress  listener is not exist in alb, port: 443, protocol: HTTPS
  Normal   Sync                    19s (x32 over 11d)      ingress  Scheduled for sync
  Normal   SuccessfullyReconciled  4s (x20 over 11d)       ingress  Successfully reconciled

Check the event information related to the ingress in the Description column or the Message field.

Scheduled for sync: indicates that the event has started.
Successfully reconciled: indicates that the reconciliation is complete.
For Warning messages, see Step 2 to troubleshoot the issue.
If no anomalous events are reported but your changes do not take effect, see What do I do if changes to an ALB Ingress do not take effect but no anomalous events are reported?

Step 2: Analyze and resolve anomalous events

The following table describes common anomalous events, their causes, and solutions.

To prevent anomalous events caused by an old version of the ALB Ingress Controller component that is incompatible with new features, you can refer to the Release notes and upgrade the component to the target version before you perform a diagnosis.

Message	Cause	Solution
listener is not exist in alb, port: 80, protocol: HTTP	In ALB Ingress Controller v2.11.0 and later, listeners for an ingress are only associated, not automatically created. This error occurs if you use a listener in an ingress without first creating it in the corresponding AlbConfig.	For ALB Ingress Controller v2.11.0 or later, create the required listener for the ingress resource in the AlbConfig. For more information, see Configure an ALB listener by using an AlbConfig.
listener is not exist in alb, port: 443, protocol: HTTPS
listener not found for (80/HTTP), with ingresses 1	In ALB Ingress Controller v2.11.0 and later, this error occurs if you remove a listener from an AlbConfig that is still associated with an ingress. The error message shows the missing listener and the number of associated ingresses.	To remove a listener, you must first remove all ingresses associated with it. Important If you removed a listener by mistake, you must add it back.
none certificate found for host	TLS is enabled and certificate auto-discovery is active for a domain, but no matching certificate for that domain exists in Certificate Management Service.	Configure certificate auto-discovery: Create a certificate on the Certificate Management Service console. The ALB Ingress Controller automatically discovers a matching certificate based on the domain name configured for TLS in the ALB Ingress. For more information, see Configure certificate auto-discovery. Specify a certificate: Specify a certificate by using an AlbConfig. For more information, see Use an AlbConfig to specify a certificate.
Secret "XXX" not found	The Ingress references a TLS certificate Secret that does not exist in the specified namespace. This primarily affects HTTPS configuration. If the referenced Secret is missing, the ALB Ingress Controller cannot complete certificate configuration, and the AlbConfig configuration update may be blocked until the issue is resolved.	Ensure that the certificate Secret exists in the same namespace as the Ingress and that its name exactly matches the value specified in the `tls.secretName` field of the Ingress resource.
The param of Rules.1.RuleConditions.2.PathConfig.Values.1 is illegal	The routing rule contains an invalid path parameter.	If you use the Rewrite annotation in the ingress, change the value of the `pathType` field to `Prefix`. If you do not use the Rewrite annotation in the ingress, the `path` field might contain invalid characters. Note For non-regular expression path types, the path must start with `/`. It can contain letters, digits, and the following special characters: dollar signs ($), plus signs (+), forward slashes (/), ampersands (&), tildes (~), at signs (@), underscores (_), hyphens (-), periods (.), and colons (:). Asterisks (*) and question marks (?) are also supported as wildcards.
The param of ServerGroupName is illegal	The name of the ALB backend server group has an invalid format.	Ensure the server group name uses the correct format. The controller generates the server group name in the `namespace+ServiceName+port` format. The name must be 2 to 128 characters in length, start with a letter or a Chinese character, and can contain digits, periods (.), underscores (_), and hyphens (-).
The specified resource sgp-vz2fb219vv792flx3u is in use	The ACK-managed ALB backend server group is referenced by another ALB instance.	Log on to the Application Load Balancer console. In the left-side navigation pane, choose ALB > Server Group. On the Server Group page, find the target backend server group and disassociate the ALB instance as needed.
Message: Invalid parameter. Check the parameter input.	This error often occurs if you incorrectly configure the certificate ID when specifying a certificate in an AlbConfig.	Check whether a resource ID or a numeric ID is used as the certificate ID. You must use the certificate ID specified by CertIdentifier.
Message: Failed to create SSL Certificate with name default-https-secret-1-b585e6 ({namespace}-{name}-{identity}). Error: The certificate has expired .	The secret certificate used by the ingress has expired. The certificate name in the error message consists of three parts: `{namespace}-{name}-{identity}`. `{namespace}`: The namespace where the affected Secret resides. Example: `default`. `{name}`: The name of the affected Secret. Example: `https-secret-1`. `{identity}`: The hash value of the Secret content. Example: `b585e6`.	If you want to continue using this secret certificate, find the expired Secret based on `{namespace}-{name}` in the error message and update its content with a valid certificate. The new certificate must be valid for at least 1 day after the current date. If you no longer need this secret certificate, edit the corresponding ingress resource and remove the configuration that references the expired Secret from the `tls` field. For more information about the configuration, see Configure an HTTPS certificate for encrypted communication.
failed to createSSLCertificateWithName: XXX ErrorCode: NameRepeat Message: The name is already used. Please enter another name.	The secret certificate used by the ingress has expired. Reusing the name of the expired certificate when uploading a new certificate causes an SSL certificate name conflict.
invalid server group Cookie:	Cookie not configured If you configure session persistence to use the cookie rewrite method (`sticky-session-type: "Server"`), this error occurs if you do not specify a cookie value. The following code provides an example of an incorrect configuration: `... alb.ingress.kubernetes.io/sticky-session: "true" alb.ingress.kubernetes.io/sticky-session-type: "Server" alb.ingress.kubernetes.io/cookie: "" # Not allowed to be empty.`	When you use the cookie rewrite method, you must configure a cookie value. `... alb.ingress.kubernetes.io/cookie: "test"` For more information about parameters, see Implement session persistence by using annotations.
invalid server group Cookie:	The ALB Ingress Controller version is too old. In ALB Ingress Controller versions earlier than v2.15.0-aliyun.1, server group session persistence does not support custom cookies.	Upgrade the ALB Ingress Controller component.
The quota of alb_quota_server_added_num is exceeded for resource eni-xxxx, usage 202/200	The `alb_quota_server_added_num` quota for ALB is reached. This is the maximum number of times that a backend server (by IP address) can be added to ALB backend server groups.	Increase the server group quota in Quota Center.

FAQ

ALB Ingress changes do not take effect

If the controller does not trigger reconciliation events for the AlbConfig or fails to process change events, the cause might be an incorrect binding between the IngressClass and the AlbConfig. For more information, see Use an IngressClass to associate an AlbConfig with an ingress and check that the spec.parameters parameter in the IngressClass points to the correct AlbConfig resource object name.

Why is the external endpoint (Endpoint) of an ALB Ingress empty or the address cannot be obtained, and how do I resolve it?

If the external endpoint of an ALB Ingress is empty or the address cannot be obtained, check the following common causes and take the corresponding action:

IngressClass not specified: Ensure the Ingress resource specifies the correct ingressClassName (for example, alb). If ingressClassName is not set or the value does not match the IngressClass managed by the ALB Ingress Controller, the controller will not process the Ingress and the endpoint will remain empty.
Incorrect AlbConfig association: Check whether the IngressClass is associated with the correct AlbConfig name, and verify that the AlbConfig status is normal. An incorrect association or an AlbConfig in an abnormal state prevents ALB from being provisioned. For more information, see Use an IngressClass to associate an AlbConfig with an ingress.
Insufficient vSwitch IPs: Check the number of available IP addresses in the vSwitches associated with the AlbConfig. To ensure the ALB's elastic features are fully supported, reserve at least 8 IP addresses in each vSwitch where the ALB instance is deployed. If available IPs are insufficient, replace the vSwitch or expand the vSwitch CIDR range.
Insufficient availability zones: ALB requires high availability and needs vSwitches in at least two different availability zones. If only one availability zone is configured, add vSwitches in at least one other availability zone.
Component installation option: If None was selected during component installation, no ALB instance or AlbConfig was created automatically. You must manually create AlbConfig and IngressClass and associate an AlbConfig with an Ingress using IngressClass.