The Application Real-Time Monitoring Service (ARMS) agent for Go enhances observability during compile time to manage application performance, which brings a certain amount of performance overhead. However, the ARMS team has adopted a number of techniques to reduce the performance overhead of the agent without compromising application stability. In this test report, scenarios with different traffic are simulated to test the performance overhead of the ARMS agent for Go. Before you connect your application to Application Monitoring, we recommend that you refer to this report to perform a comprehensive evaluation.
Test scenario
Flowchart
Developed in Net/HTTP, the Go application will access MySQL and Redis based on the requests sent from Alibaba Cloud Performance Testing Service (PTS). Each type of requests account for 50% of the queries per second (QPS).
Test environment
The stress testing source is provided by PTS.
The Go application, MySQL, and Redis are all deployed in the same Alibaba Cloud Container Service for Kubernetes (ACK) cluster. The instance type of nodes is ecs.c6.2xlarge, and the operating system version of the nodes is Alibaba Cloud Linux 3.2104 LTS 64 bit.
Each pod of the application has 1 core, 2 GB of memory, and two replicas.
Use an ARMS agent for Go V1.0.0.
Test procedure
Perform three stress tests at 500, 1,000, and 2,000 QPS. Each stress test lasts for 1 hour. Before each stress test, warm up the Go application at 100 QPS for 3 minutes. The test result will be used as baseline performance metrics.
Install the ARMS agent for Go. Configure a sampling rate of 10% in the sampling policy, and repeat the stress tests in Step 1 to compare the differences in the CPU overhead, memory overhead, and response time of the Go application.
Configure a sampling rate of 100% in the sampling policy, and repeat the stress tests in Step 1 to compare the differences in the CPU overhead, memory overhead, and response time of the Go application.
The basic features of Application Monitoring are enabled, including metrics, traces, and runtime monitoring. All plug-ins are also enabled.
The runtime monitoring feature increases the CPU utilization by 0.5%. If application performance is affected, you can disable runtime monitoring in the application settings.
Baseline performance metrics
Item | CPU | Memory | Response time (Unit: milliseconds) |
500 QPS | 2.42% | 0.71% | 30 |
1,000 QPS | 4.21% | 0.91% | 30 |
2,000 QPS | 8.5% | 1.41% | 30 |
The CPU metric represents the percentage of CPU used by pods to the total CPU.
The memory metric represents the percentage of memory used by pods to the total memory. Because the memory used by pods grows naturally until it reaches the requests value, the report uses the actual memory usage at the end of the stress tests.
The response time metric represents the average response time of requests. Unit: milliseconds.
Performance metrics of the ARMS agent for Go
Item | 10% sampling rate | 100% sampling rate | ||||
CPU | Memory | Response time (Unit: milliseconds) | CPU | Memory | Response time (Unit: milliseconds) | |
500 QPS | 5.15% | 1.25% | 30 | 5.25% | 1.85% | 31 |
1,000 QPS | 8.42% | 1.52% | 31 | 10.48% | 2.02% | 32 |
2,000 QPS | 16.2% | 2.5% | 31 | 18.45% | 2.63% | 32 |
Performance overhead of the ARMS agent for Go
Item | 10% sampling rate | 100% sampling rate | ||||
CPU | Memory | Response time (Unit: milliseconds) | CPU | Memory | Response time (Unit: milliseconds) | |
500 QPS | +2.73% | +0.54% | - | +2.83% | +1.14% | +1 |
1,000 QPS | +4.21% | +0.61% | +1 | +6.27% | +1.11% | +2 |
2,000 QPS | +7.7% | +1.09% | +1 | +9.95% | +1.22% | +2 |
Conclusions
The ARMS agent for Go increases the CPU overhead by no more than 10%, and barely increases the memory overhead.
The ARMS agent for Go has small impact on the response time, which increases only by 1 millisecond at 2,000 QPS.
The performance overhead at a 100% sampling rate are 2% greater than those at a 10% sampling rate.