Verify that ApsaraDB for MongoDB replica set instances of different specifications can reach their maximum connections by running stress tests from an ECS instance that accesses the ApsaraDB for MongoDB instances in the same VPC.
Test environment
Create an ECS instance and an ApsaraDB for MongoDB replica set instance. Create a replica set instance and Create an ECS instance.
The following table lists the instance configurations used in this test.
|
Configuration item |
ECS Instance |
ApsaraDB for MongoDB instance that uses cloud disks |
ApsaraDB for MongoDB instance that uses local disks |
|
Region and zone |
Beijing Zone H |
Beijing Zone H |
Beijing Zone H |
|
Network type |
Virtual Private Cloud (VPC) |
VPC |
VPC |
|
Instance category |
c6e, compute-optimized instance family with enhanced performance |
General-purpose and dedicated |
General-purpose and dedicated |
|
Instance type |
ecs.c6e.2xlarge |
Three instance types tested. Test results. |
Two instance types tested. Test results. |
|
Storage type |
Enterprise SSD (ESSD) AutoPL disks |
ESSDs |
Local SSDs |
|
Image or engine version |
Alibaba Cloud Linux 3.2104 LTS 64-bit |
4.19.91-26.al7.x86_64 |
3.10.0-327.ali2017.alios7.x86_64 |
|
Kernel version |
N/A |
|
|
-
The ApsaraDB for MongoDB instance uses a three-node architecture: primary, secondary, and hidden.
-
The ECS and ApsaraDB for MongoDB instances are in the same zone, with an average RTT of 0.103 ms.
Test tool
-
YCSB (Yahoo Cloud Serving Benchmark) 0.17.0.
NoteYCSB is a Java-based database benchmarking tool. For installation and usage, see YCSB on GitHub.
-
A custom connection stress testing program. Additional information about a stress test for 96,000 connections.
Test method
Add the primary private IP address of the ECS instance to a whitelist of the ApsaraDB for MongoDB instance. For more information, see Modify a whitelist.
NoteYou can log on to the ECS console and view the Primary Private IP Address of the ECS instance in the Configuration Information section of the Instance Details page.
Connect to the ECS instance. For more information, see Create and manage an ECS instance by using the ECS console (express version).
-
Use the YCSB tool to load test data.
./bin/ycsb.sh load mongodb -s -p workload=site.ycsb.workloads.CoreWorkload -p recordcount=10000000 -p mongodb.url="mongodb://test:****@dds-bp13e84d11****.mongodb.rds.aliyuncs.com:3717/admin" -p table=test -threads 8Modify the following settings:
-
recordcount=1000000: total records loaded into the ApsaraDB for MongoDB instance. -
mongodb.url="mongodb://test:****@dds-bp13e84d11****.mongodb.rds.aliyuncs.com:3717/admin": connection string of the ApsaraDB for MongoDB instance. Account:test, database:admin.NoteLog on to the ApsaraDB for MongoDB console. On the Database Connection page, find the connection string in the Internal Connections - VPC section.
-
threads 8: number of concurrent client threads.
-
-
Run the stress test:
./bin/ycsb.sh run mongodb -s -p workload=site.ycsb.workloads.CoreWorkload -p recordcount=10000000 -p operationcount=5000000 -p readproportion=50 -p updateproportion=50 -p requestdistribution=zipfian -p mongodb.url="mongodb://test:****@dds-bp13e84d11****.mongodb.rds.aliyuncs.com:3717/admin&maxPoolSize=8000" -p table=test -threads 8000Modify the following settings:
-
recordcount=1000000: total records loaded into the ApsaraDB for MongoDB instance. -
operationcount=5000000: total read and write operations. -
insertproportion=0: insert ratio. -
readproportion=50: read ratio. -
updateproportion=50: update ratio. -
mongodb.url="mongodb://test:****@dds-bp13e84d11****.mongodb.rds.aliyuncs.com:3717/admin": connection string of the ApsaraDB for MongoDB instance. Account:test, database:admin.Note-
Log on to the ApsaraDB for MongoDB console. On the Database Connection page, find the connection string in the Internal Connections - VPC section.
-
You must specify the
maxPoolSizeparameter. Otherwise, the default value of 100 applies, which can causeMongoWaitQueueFullExceptionerrors and prevent the test from reaching the target maximum connections.
-
-
-
View monitoring data. Node monitoring (previously basic monitoring).
On the Node Monitoring tab, select the time range for the test and view the CPU Utilization, Memory Usage, QPS, Connections, and Connection Usage metrics.
Additional information about a stress test for 96,000 connections
YCSB depends on Java. The JVM heap memory is limited, so tests with more than 20,000 concurrent threads return the Cannot allocate memory error.
Increasing JAVA_OPTS does not resolve this. Use a custom stress testing program instead: it cyclically generates threads, each creating a MongoClient that performs a query and holds the connection open without releasing it.
A single stress-test client has limited ports and cannot reach 96,000 connections (required for 32-core 128 GB specs). Run the same program on multiple machines.
Query the current port range:
sysctl net.ipv4.ip_local_port_range
Example output:
net.ipv4.ip_local_port_range = 40000 65535
Expand the port range before testing:
sudo sysctl -w net.ipv4.ip_local_port_range="10240 65535"
Test results
Instance that uses ESSDs
Dedicated instance that has 4 cores and 8 GB of memory
Maximum number of connections: 8,000
|
QPS |
Connections |
Connection utilization |
CPU utilization |
Memory usage |
|
|
|
|
|
|
Minute-level granularity on the Node Monitoring tab may not display the peak connections. The actual connections reached 8,000, verifiable through granular monitoring or the connections sub-document in the serverStatus output.
Dedicated instance that has 32 cores and 128 GB of memory
Maximum number of connections: 96,000
|
QPS |
Connections |
Connection utilization |
CPU utilization |
Memory usage |
|
|
|
|
|
|
The 96,000-connection test uses a different tool than the 8,000 and 16,000 tests, so QPS, CPU, and memory metrics are not directly comparable.
General-purpose instance that has 8 cores and 32 GB of memory
Maximum number of connections: 16,000
|
QPS |
Connections |
Connection utilization |
CPU utilization |
Memory usage |
|
|
|
|
|
|
Instances that use local disks
General-purpose instance that has 16 cores and 64 GB of memory
Maximum number of connections: 32,000
|
QPS |
Connections |
Connection utilization |
CPU utilization |
Memory usage |
|
|
|
|
|
|
With minute-level granularity, some data points show gaps for the 16-core 64 GB instance because collection commands timed out at 100% CPU. The actual connections at those points remained 32,000.
Dedicated instance that has 2 cores and 16 GB of memory
Maximum number of connections: 8,000
|
QPS |
Connections |
Connection utilization |
CPU utilization |
Memory usage |
|
|
|
|
|
|
Summary
-
ApsaraDB for MongoDB replica set instances of all tested specifications and storage types reached their maximum connections.
-
After reaching the maximum, subsequent connections are rejected. Applications experience high latency or stuck requests due to connection failures.
-
More concurrent connections consume more CPU and memory. Adjust the connection count based on your business requirements.
























