Configure health checks for a service to monitor backend node status. The gateway automatically takes unhealthy nodes offline or isolates them to ensure API availability. A panic threshold maintains essential service capabilities in extreme situations.
Use cases
Active health check: The gateway probes service nodes by sending TCP or HTTP requests. Unhealthy nodes are automatically taken offline and restored after recovery. This improves API availability for services with multiple replicas.
Passive health check: The gateway monitors live traffic to detect unhealthy nodes. Nodes with high failure rates are temporarily isolated and automatically restored after recovery.
Panic threshold: Prevents localized failures from cascading into a cluster-wide outage by routing traffic to all nodes when too few healthy nodes remain.
Procedure
TCP health checks are enabled by default when you create a service.
Log on to the API Gateway console.
In the left-side navigation pane, click . In the top navigation bar, select a region.
On the Instance page, click the target instance ID.
In the left-side navigation pane, click Service. Then, click the Services tab.
-
Click Health Check Configuration in the Actions column for the target service. Based on the health check type, select Enable, and complete the configuration.
Active health check
In the Configure Health Check panel, enable Active Health Check, configure the parameters, and click OK.
Parameter
Example
Description
Health check protocol
HTTP
-
TCP: Sends SYN packets to test whether a server port is responsive.
-
HTTP: Sends a browser-simulated HTTP request to check whether the server application is healthy.
Health check path
/
URI path for the health check request. Use a static page for best results.
Normal status codes
http_2xx
The HTTP status codes that indicate a successful health check.
Response timeout
2
Maximum wait time for a health check response. If no response is received, the probe fails. Unit: seconds.
Health check interval
2
Interval between consecutive health check probes. Unit: seconds.
Healthy threshold
2
Consecutive successful checks required to mark an unhealthy node as healthy.
Unhealthy threshold
2
Consecutive failed checks required to mark a healthy node as unhealthy.
Passive health check
In the Configure Health Check panel, enable Passive Health Check, configure the parameters, and click OK.
Parameter
Example
Description
Failure rate threshold
80
If a node's request failure rate reaches this threshold, the system ejects it from the service pool. Unit: %.
Detection interval
30
Interval over which the system calculates each node's failure rate. Unit: seconds.
Initial isolation duration
30
Base isolation duration after ejection. Actual duration =
k * base_ejection_time, wherekstarts at 1, increments on each ejection, and decrements after consecutive successful checks. Unit: seconds.NotePassive health check requires gateway engine version 2.1.9 or later.
Updating the passive health check configuration resets the health check status and returns all isolated nodes to service.
-
Panic threshold
The panic threshold balances availability and correctness to maintain essential service capabilities when system load is high or some nodes fail.
How it works:
-
If healthy node percentage is higher than the panic threshold, health checks work normally. Only healthy nodes receive traffic.
-
If healthy node percentage is less than or equal to the panic threshold, the system enters panic mode. Health checks are bypassed, and requests are distributed evenly to all nodes, including unhealthy or ejected ones.
Routing traffic to unhealthy nodes improves overall fault tolerance and availability.
The panic threshold defaults to 1%. When healthy nodes drop to this percentage or below, the system enters panic mode and routes requests to all nodes.
Adjust this threshold based on your business requirements and disaster recovery capabilities.
Troubleshoot health check failures
General health check failures
Perform the following steps:
-
A failed TCP health check means a connection to the node cannot be established. Verify:
-
The node exists.
-
Concurrent connections are not exceeding the node's capacity.
-
-
If an HTTP health check fails, switch to TCP to verify connectivity. If TCP succeeds, verify the health check path. Use cURL or Postman to test the endpoint.
Health check failures for new services
Perform the following steps:
-
Ensure the gateway and service instance are in the same VPC. If they are in different VPCs, connect them through Cloud Enterprise Network (CEN) or a dedicated connection.
NoteThe gateway does not support on-premises services registered through Nacos or ZooKeeper.
-
Verify that the gateway and service instance VPCs are the same or connected.
-
Verify that security group authorization is configured. For ACK services, authorize the container cluster's security group as described in Configure security group rules.
-
If the unhealthy instance uses a public IP address, verify that a public NAT gateway is enabled for the gateway's VPC.