The Container Service for Kubernetes (ACK) console provides the Ingress diagnostics feature to help you diagnose common Ingress issues. This topic describes the Ingress diagnostic items and provides suggestions on how to fix these issues.
The diagnostic items of Ingress diagnostics include the Ingress, startup parameters, Ingress pod error log, and Ingress controller Server Load Balancer (SLB) instance.
When you use the diagnostics feature, ACK runs a data collection program on each node in the cluster and collects diagnostic results. The collected information includes the system version, status of the loads, Docker, and kubelet, and key error information in system logs. ACK does not collect business information or sensitive data.
Ingress diagnostic items
The diagnostic items may vary based on the cluster configuration. The actual diagnostic items on the diagnostic page shall prevail.
Category | Description |
Check the availability of Ingress resources. | |
Check the startup parameters of the Ingress component, the Service that is associated with the Ingress, and the Ingress pods. | |
Check the status of the SLB instance used by the Ingress controller, maximum number of connections, QPS, and health status. |
Ingress
Diagnostic item | Description | How to fix |
Ingress check | Check whether the specified Ingress exists. | Check whether an Ingress rule is created for the specified URL. If the URL is valid, check the Ingress rules. For example, check whether a regular expression is specified as the path or whether the use-regex annotation is used. |
base-url-scheme check | Check whether the | Check the version of the Ingress controller, and then delete the annotation or use other annotations. |
grpc-backend check | Check whether the | Check the version of the Ingress controller, and then delete the annotation or use other annotations. |
mirror-uri check | Check whether the | Check the version of the Ingress controller, and then delete the annotation or use other annotations. |
secure-backends check | Check whether the | Check the version of the Ingress controller, and then delete the annotation or use other annotations. |
session-cookie-hash check | Check whether the | Check the version of the Ingress controller, and then delete the annotation or use other annotations. |
nginx.com/nginx.org check | Check whether the Ingress uses annotations that start with | Use the annotations for the open source version of the NGINX Ingress controller. For more information about Ingresses, see NGINX Ingress management or NGINX Ingress Controller on the official site. |
Status of Canary | To use the Canary feature, you must specify | To enable the Canary feature for an Ingress, add the |
Addon
Diagnostic item | Description | How to fix |
Percentage of Ingress pods in the Ready state | Check the percentage of the pods that are in the Ready state compared to the pods created by the Ingress Deployment. If the value is lower than 100, some Ingress pods fail to start up or fail to pass the health check. | Check the error log to locate the faulty pods and fix the issue. For more information about Ingress troubleshooting, see NGINX Ingress controller troubleshooting. |
Ingress IP address check | Check whether the Ingress controller allocates an IP address to the Ingress. | If no IP address is allocated, check whether the Ingress controller exists in the IngressClass of the Ingress and whether the Ingress controller works as normal. Fix the issue based on the diagnostic result. |
Leader pod check | Check whether a leader pod is elected. If no leader pod is elected, the pod startup time may be too short or the permissions of the Ingress controller are not properly configured. | Check whether the Ingress pods generate error logs and fix the issue shown in the error logs. For more information about Ingress troubleshooting, see NGINX Ingress controller troubleshooting. |
Annotations used by NGINX Ingresses | The open source version of the NGINX Ingress controller uses annotations that start with | Use the annotations supported by the open source version of the NGINX Ingress controller. For more information about annotations, see Annotations. |
Use of capture groups with rewrite-target in NGINX Ingresses | Check whether the rewrite-target annotation in NGINX Ingresses is used together with capture groups. In Ingress controller 0.22.0 and later versions, you need to explicitly specify capture groups when you use the rewrite-target annotation. Otherwise, traffic forwarding errors occur. | Properly configure Ingresses. For more information about Ingress configurations, see Advanced NGINX Ingress configurations. |
NGINX Ingress canary rules | Check whether no more than two Services are specified in service-match or service-weight. You can specify service-match or service-weight to distribute traffic between only two Services. If you specify more than two Services, the excessive Services are ignored. As a result, traffic forwarding may fail to meet your requirement. | Adjust the number of Services. |
Ingress name | Display the names of the matching Ingress rules. | None. |
Pod error logs | Check whether the Ingress controller pods generate error logs. When error logs are generated, the Ingress controller may not run as expected. | Locate the cause and fix the issue based on the error logs. For more information about Ingress troubleshooting, see NGINX Ingress controller troubleshooting. |
Use of capture groups in the paths of rewrite-target | In Ingress controller 0.22.0 and later versions, if you want to use the | Properly configure the Ingresses. For more information about Ingresses, see Configure routing rules to redirect traffic from specific URLs. |
Multiple targets in service-* | You can specify service-match or service-weight to distribute traffic between only two Services. | Specify only two Services in service-weight or service-match. For more information, see Use the NGINX Ingress controller to implement canary releases and blue-green deployments. |
Ingress Service check | Check whether the Service specified in the startup parameters of the Ingress exists. If the Service is deleted, the Ingress may fail to start up. | Check the name of the deleted Service in the startup parameters of the Deployment and fix the issue. For more information, see High-risk operations related to networks and SLB instances. |
Ingress Service endpoint check | Check whether the Service associated with the Ingress has an endpoint. If the Service does not have an endpoint, the SLB instance cannot route traffic to the Ingress controller. | Check whether the label selector of the Service functions as normal. |
Ingress Service events | Check whether the Service associated with the Ingress generates Warning or Error events. This issue may be caused by SLB configuration errors. | Check the Service events, locate the cause, and then fix the issue. For more information about Service troubleshooting, see Service errors and solutions. |
Ingress Service external traffic policy | The external traffic policy specifies how external traffic is distributed in the cluster. The default external traffic policy is Local. You can also set it to Cluster. We recommend that you use the Local mode. In Cluster mode, client IP addresses may not be preserved. Consequently, health check results may be incorrect. | The preceding precaution does not apply if your business requires the Cluster mode or you want to access the Ingress controller through the SLB instance IP address from within the cluster. |
Ingress Service external IP address | Check whether the cloud controller manager assigns an IP address to the Service associated with the Ingress. If no IP address is assigned, the Ingress may not be able to access the Internet. | Make sure that the status of the Service, the status of the cloud controller manager, and the SLB quotas are normal. Most issues are exposed through events. |
Ingress Service type (LoadBalancer) | Check whether the type of Service specified in the startup parameters of the Ingress is LoadBalancer. If the Service type is not LoadBalancer, the Ingress controller may be inaccessible over the Internet. | The preceding precaution does not apply if your business does not require Services of the LoadBalancer type. Otherwise, change the type of Service to LoadBalancer. |
--force-namespace-isolation check | Check whether the | Check the version of the Ingress controller and delete the startup parameter. |
--sort-backends check | Check whether the | Check the version of the Ingress controller and delete the startup parameter. |
SLB
Diagnostic item | Description | How to fix |
SLB instance | Check whether the SLB instance of the Ingress controller exists. | Check whether the Service specified in the Ingress Controller exists, whether the Service generates anomaly events, and whether the SLB instance exists in the SLB console. If the SLB instance is accidentally deleted, recreate the Service to fix the issue. For more information, see What do I do if I accidentally delete an SLB instance? |
Ingress controller SLB instance backend server health | Check the health status of the backend servers of the SLB instance. | If the backend servers of the SLB instance are unhealthy, check whether the Service specified in the Ingress controller generates anomaly events and whether the components are overwhelmed. For more information about component troubleshooting, see NGINX Ingress controller troubleshooting. |
SLB ID | Display the ID of the SLB instance. | None. |
Ingress controller SLB instance connections | Check whether the number of connections on the SLB instance within the previous three days exceeds 80% of the upper limit. If the number of connections reaches the upper limit, clients cannot establish connections to the SLB instance. | If the number of connections on the SLB instance has reached the upper limit, upgrade the SLB instance to avoid business interruptions. For more information, see Use an existing SLB instance. |
Ingress controller SLB instance new connection rate | Check whether the rate of new connections on the SLB instance within the previous three days exceeds 80% of the upper limit. If the rate reaches the upper limit, clients cannot establish new connections to the SLB instance within a short period of time. | If the rate of new connections on the SLB instance has reached the upper limit, upgrade the SLB instance to avoid business interruptions. For more information, see Use an existing SLB instance. |
Ingress controller SLB instance QPS | Check whether the QPS value of the SLB instance within the previous three days exceeds 80% of the upper limit. If the QPS value reaches the upper limit, clients cannot connect to the SLB instance. | If the QPS of the SLB instance has reached the upper limit, upgrade the SLB instance to avoid business interruptions. For more information, see Use an existing SLB instance. |
Host and SecretName for TLS | You need to specify both the Host and SecretName parameters when you configure Ingress TLS. | specify both the Host and SecretName parameters and make sure that the host is the same as that in the certificate. |
Ingress Controller SLB health check | Check whether the SLB instance fails to pass health checks within the previous three days. This issue occurs when the components are overwhelmed or the SLB configurations contain errors. | If the SLB instance fails to pass health checks within the previous three days, to prevent business interruptions, check whether the Service specified in the Ingress controller generates anomaly events and whether the components are overwhelmed. For more information about component troubleshooting, see NGINX Ingress controller troubleshooting. |