Performance test report of the ARMS agent for Python
The Application Real-Time Monitoring Service (ARMS) agent for Python uses monkey patching to instrument applications and collect observability data. The ARMS team has adopted a number of techniques to reduce the performance overhead of the agent without compromising application stability. This report quantifies the performance overhead at different traffic levels so you can evaluate the impact before connecting your application to Application Monitoring.
Agent version tested: V1.0.0
Test scenario
Performance Testing Service (PTS) sends traffic to a Python application built on the FastAPI framework. The Python instrumentor collects traces, generates metrics, and forwards the data to the ARMS server.
Test environment
| Component | Configuration |
|---|---|
| Stress testing source | PTS |
| Deployment platform | Alibaba Cloud Container Service for Kubernetes (ACK) cluster |
| Node instance type | ecs.c6.2xlarge |
| Node OS | Alibaba Cloud Linux 3.2104 LTS 64-bit |
| Pod resources | 2 cores, 2 GB memory, 2 replicas |
| Agent version | ARMS agent for Python V1.0.0 |
Test procedure
Run baseline tests at 500, 1,000, and 2,000 QPS. Each test runs for 1 hour. Before each test, warm up the application at 100 QPS for 3 minutes.
Install the ARMS agent for Python. Set the sampling rate to a fixed 10%, then repeat the same tests. Compare CPU usage, memory usage, and response time against the baseline.
Disable data reporting, enable the plug-in, and repeat steps 1 and 2.
Baseline performance (without agent)
| QPS | CPU | Memory | Response time (ms) |
|---|---|---|---|
| 500 | 8% | 2% | 17.28 |
| 1,000 | 15% | 2% | 36.3 |
| 2,000 | 27.12% | 2% | 44.9 |
Performance with the ARMS agent
| QPS | CPU | Memory | Response time (ms) |
|---|---|---|---|
| 500 | 11.9% | 4.8% | 17.31 |
| 1,000 | 22.5% | 4.8% | 37.1 |
| 2,000 | 36.67% | 4.8% | 46.6 |
Performance overhead
The following table shows the difference between the baseline and the agent-enabled results at each traffic level:
| QPS | CPU | Memory | Response time (ms) |
|---|---|---|---|
| 500 | +3.9% | +2.8% | +0.03 |
| 1,000 | +7.5% | +2.8% | +0.80 |
| 2,000 | +9.55% | +2.8% | +1.67 |
Summary
CPU and memory overhead stays under 10%. At 2,000 QPS, the agent adds at most 9.55% CPU and 2.8% memory.
Response time impact is minimal. The maximum increase is 1.67 ms at 2,000 QPS, which is less than a 4% change relative to the baseline.
The tests use a fixed sampling rate of 10%. If your workload requires a different sampling rate, the actual overhead may vary. For details about how to configure sampling policies, see the ARMS documentation.