What is an outbound connection pool?
Data is transmitted serially over an HTTP persistent connection. The sender must wait for a response to the previous request before sending the next one. An outbound connection pool provides connection resources for an API Gateway instance to access a backend service.
For example, if you purchased a dedicated instance of the api.s1.small type with an outbound connection pool size of 1,200, and your backend service takes one second to respond to each request, the instance supports a maximum of 1,200 outbound requests per second (RPS). If more than 1,200 requests are sent in a second, the excess requests wait in a queue because the connection pool is fully occupied. If a request fails to obtain connection resources within 500 milliseconds, the Backend service connect failed 'Connection lease request time out error is returned. To prevent such errors, configure Simple Log Service to send alerts before connection resources are used up so that you can address the resource shortage promptly.
Prerequisites
API Gateway logs are enabled. For more information, see Use Log Service to manage logs of API calls.
Configure an alert rule for the connection pool
Log Service triggers an alert when the conditions that you define in an alert rule are met.
When the outbound connection pool is exhausted, API Gateway returns the Backend service connect failed 'Connection lease request time out' error to the client. This error is recorded in the errorMessage field in the logs. You can configure an alert based on this error message. For more information, see Configure an alert monitoring rule in Log Service.
In Step 2 of the referenced topic, configure an alert rule based on the instance ID, error ID, and error message. The following query and analysis statement is an example.
instanceId: apigateway-cn-XXXXX and errorId : D504CO and '`Connection lease request time out`' In the Query and Statistics dialog box, click the Advanced Settings tab. Set Type to Logstore. For Region, select the region where the instance is located. For Project and Logstore, enter the Project and Logstore that correspond to your API Gateway logs, such as ali-apigateway-log-cn-beijing and gateway_request_log, respectively. Set Dedicated SQL to Disabled. Set the Time Range to 15 Minutes (Relative).
Resolve outbound connection pool exhaustion
-
Upgrade the API Gateway instance.
-
If you use a serverless instance, purchase a dedicated instance first, and then migrate the API group to the dedicated instance. For more information, see Migrate API groups between instances.
-
If the instance is a dedicated instance, upgrade the instance specifications in the API Gateway console. For more information, see Change instance specifications.
-
-
Check the response time of the backend service. This metric (unit: ms) is recorded in the serviceLatency field in the logs. It measures the duration from when API Gateway sends a request to the backend service until it receives the last packet of the response. Slow responses contribute to outbound connection resource exhaustion. We recommend that you also configure an alert rule for the serviceLatency value. If serviceLatency remains high, optimize the backend response time to prevent connection pool exhaustion.