The Application Real-Time Monitoring Service (ARMS) agent instruments bytecode at runtime to monitor application performance. Like other bytecode instrumentation-based monitoring solutions, the ARMS agent for Java introduces performance overhead. This report simulates scenarios with different traffic levels to measure the performance overhead of the ARMS agent for Java V5.x. Before you connect your application to Application Monitoring, refer to this report to evaluate the performance impact.
Test scenarios
The following figure shows the overall architecture.
The Java application is developed in Spring Web model-view-controller (MVC) and accesses MySQL and Redis through the Druid and Jedis frameworks based on requests sent from Alibaba Cloud Performance Testing (PTS).
When receiving
${mall-gateway}/case/api/v1/mysql/executerequests, the application accesses MySQL. Each time the application receives this request, it accesses MySQL for 1 to 4 times.When receiving
${mall-gateway}/case/api/v1/redis/executerequests, the application accesses Redis. Each time the application receives this request, it accesses Redis for 1 to 10 times.
Test environment
The stress testing source is provided by PTS.
The Java application, MySQL, and Redis are all deployed in the same Alibaba Cloud Container Service for Kubernetes (ACK) cluster. The instance type of nodes is ecs.u1-c1m2.8xlarge, and the operating system version of the nodes is Alibaba Cloud Linux 2.1903 LTS 64-bit.
Each pod of the application has 2 cores, 4 GB of memory, and two replicas. The JVM parameters are as follows:
-Xms3500m -Xmx3500m -XX:MetaspaceSize=500mThe version of the ARMS agent for Java is Aliyun JavaAgent 5.1.2.
The basic features of Application Monitoring are enabled, including metrics, traces, and quantiles. All plug-ins are also enabled. Some advanced features are disabled during the test, such as trace compression, URL convergence, data masking, and continuous profiling.
Test procedure
Perform three stress tests at 500, 1,000, and 2,000 QPS. Each stress test lasts for 1 hour. Before each stress test, warm up the Java application at 100 QPS for 3 minutes. The test result will be used as baseline performance metrics.
Install the ARMS agent for Java V5.x. Configure a sampling rate of 10% in the sampling policy, and repeat the stress tests in Step 1 to compare the differences in the CPU overhead, memory overhead, and response time of the Java application.
Install the ARMS agent for Java V5.x. Configure a sampling rate of 100% in the sampling policy, and repeat the stress tests in Step 1 to compare the differences in the CPU overhead, memory overhead, and response time of the Java application.
For more information about how to configure the sampling policy, see Trace sampling modes for agents before 3.2.8.
The basic features of ARMS Application Monitoring (such as statistical metrics, call chains, and quantiles) and all plugin features are enabled. However, some advanced features are temporarily disabled, such as trace compression, URL convergence, the data masking switch, and continuous profiling.
Baseline performance metrics
Item | CPU | Memory | Response time (Unit: milliseconds) |
500 QPS | 6.752% | 11.48% | 57.8 |
1,000 QPS | 13.112% | 11.78% | 64.1 |
2,000 QPS | 26.910% | 12.22% | 70.8 |
The CPU metric represents the percentage of CPU used by pods to the total CPU (2 cores).
The memory metric represents the percentage of memory used by pods to the total memory. Because the memory used by pods grows naturally until it reaches the requests value, the report uses the actual memory usage at the end of the stress tests.
The response time metric represents the average response time of requests. Unit: milliseconds.
Absolute performance metrics with the ARMS agent installed
Item | CPU (10% sampling rate) | Memory (10% sampling rate) | Response time in ms (10% sampling rate) | CPU (100% sampling rate) | Memory (100% sampling rate) | Response time in ms (100% sampling rate) |
500 QPS | 8.492% | 13.66% | 58.6 | 9.392% | 13.83% | 59.0 |
1,000 QPS | 16.932% | 14.07% | 65.2 | 18.072% | 14.33% | 65.6 |
2,000 QPS | 35.120% | 14.61% | 75.1 | 35.930% | 14.64% | 76.2 |
The CPU overhead does not scale linearly with the sampling rate or QPS. By default, there is an upper limit for trace collection per second.
Performance overhead of the ARMS agent for Java V5.x
Item | CPU (10% sampling rate) | Memory (10% sampling rate) | Response time in ms (10% sampling rate) | CPU (100% sampling rate) | Memory (100% sampling rate) | Response time in ms (100% sampling rate) |
500 QPS | +1.74% | +2.18% | +0.8 | +2.64% | +2.35% | +1.2 |
1,000 QPS | +3.82% | +2.29% | +1.1 | +4.96% | +2.55% | +1.5 |
2,000 QPS | +8.21% | +2.39% | +4.3 | +9.02% | +2.42% | +5.4 |
Conclusion
The additional CPU and memory overhead caused by the ARMS agent for Java V5.x is within 10%.
The ARMS agent for Java V5.x has a small impact on response time (within 10%), which increases by approximately 1.1 milliseconds at a 10% sampling rate and 1.5 milliseconds at a 100% sampling rate when QPS is 1,000.
The performance overhead at a 100% fixed sampling rate is slightly greater than that at a 10% fixed sampling rate.
These results are based on the test conditions described above, with certain advanced features disabled during the test.