An ApsaraDB for MongoDB instance that has exhausted its connections will reject new connection requests. This topic explains how to troubleshoot connection exhaustion issues.
Symptoms
The maximum number of connections allowed varies by instance type. For more information, see Instance types.
-
Your application suddenly fails to connect to the database.
-
Connecting with the mongo shell fails with the following error, even with a correctly configured whitelist:
2019-07-10T10:30:43.597+0800 E QUERY [js] Error: network error while attempting to run command 'isMaster' on host 'dds-bpxxxxxxxx.mongodb.rds.aliyuncs.com:3717' : connect@src/mongo/shell/mongo.js:328:13 @(connect):1:6 exception: connect failed -
Connecting with Data Management Service (DMS) fails with an error, even with a correctly configured whitelist:
DMS cannot establish a connection with the MongoDB server. Check the connection address, network connectivity, and whitelist settings, and then try again. Original error: Timed out after 3000 ms while waiting for a server that matches ReadPreferenceServerSelector{readPreference=primary}. Client view of cluster state is {type=UNKNOWN, servers=[{address=xxx:60276, type=UNKNOWN, state=CONNECTING, exception={com.mongodb.MongoSocketReadException: Prematurely reached end of stream}}]} For common causes of logon failures, see https://www.alibabacloud.com/help/knowledge_detail/52244.html
Check for connection exhaustion
-
Go to the Replica Set Instances or Sharded Cluster Instances page. At the top of the page, select a resource group and region, and click the ID of the target instance.
-
In the left-side navigation pane, click Monitoring Data.
-
On the Monitoring Data page, view the Connection Utilization(%) metric for the instance.
NoteIf the instance is a sharded cluster instance, select the mongos node used by your application in the upper-right corner of the page.
For information about the maximum number of connections for different instance types, see Instance types.
Solutions
To temporarily release all connections, you can restart an instance, a component, or a node. To prevent the issue from recurring, take the following actions after the restart:
Restarting an instance triggers a rolling restart of its nodes. Each node experiences a transient disconnection of about 30 seconds. This disconnection can last longer if the instance contains many collections (over 10,000). Before you restart, ensure your application has a reconnection mechanism and can handle this temporary outage.
-
Properly configure a connection pool. For more information, see Limit the number of connections from clients.
-
Analyze the source of connections. If the workload is exhausting the connections, upgrade the instance type. For more information, see Change instance configuration.