This topic describes concurrent users, RPS, and TPS, and their relationships.
Terms
Concurrent user: In PTS tools, it is generally called virtual user (VU), which refers to the user who perform operations in the real system.
NoteConcurrent users are different from registered users and online users. Registered users generally refer to users that exist in databases. Online users are just connected to the system. They do not apply load pressure on the server. However, concurrent users definitely apply load pressure on the server.
Transactions per second (TPS) is a very important metric to measure system performance.
NoteA high TPS value indicates high performance of your machine.
Requests per second (RPS) is suitable for capacity planning and throttling.
Response time (RT) is the time period from when the service is initiated to when the service is received by the client.
In PTS, load pressure is applied in two modes: concurrent mode and RPS mode. Traditionally, the number of concurrent users is used to measure the performance of the system (from the perspective of the client). This method is generally used to test websites (such as H5 pages); The RPS mode is designed to directly measure the TPS of the system (from the perspective of the server). The RPS is defined based on the required TPS of the tested object. The RPS mode is mainly used in dynamic APIs, such as logon and order submission.
Conversion between VUs and TPS
Formula: TPS=VU/RT (RT unit: seconds).
For example, if a VU completes a transaction within 1 second, the TPS is 1. If the RT of a service is 1 ms, a VU can complete 1000 transactions within 1 second, and the TPS is 1000. If the RT of a service is 1s, a VU can complete one transaction within 1 second. To achieve 1000 TPS, 1000 VUs are required. Either one VU or 1000 VUs can achieve 1000 TPS, but the RT is different.
How to obtain VU and TPS values
Obtain VU values
For the existing system: You can select a specified period of time within peak hours. The users using the system can be considered as online users. Select 10% of online users as concurrent users. For example, if 100,000 users are using the system within half an hour, 10,000 users can be concurrent users.
For a new system: We recommend that you ask the business department to evaluate the condition because no historical data is available for reference.
Obtain TPS values
For the existing system: You can select a specified period of time (such as 3 minutes to 10 minutes) within peak hours. Calculate the number of transactions completed per second. Multiply it by 2 to 5 to obtain the peak TPS. For example, 180,000 transactions are completed within 3 minutes, the average TPS is 1000. The peak TPS can be 2000 to 5000.
For a new system: We recommend that you ask the business department to evaluate the condition because no historical data is available for reference.
How to evaluate the system performance
For the server side, TPS is the main metric for measuring the performance, and the number of concurrent users is an auxiliary metric. If the number of concurrent users is required, the RT must be considered as a precondition. If the system load is low and the think time (which is equal to the RT) is considered in the business session, the number of concurrent users can be doubled. Therefore, the number of concurrent users is useless for measuring the system performance. Similarly, if the TPS values vary between systems, the concurrent TPS values also differ.
PTS policies
Performance tests must be standardized and orderly. For load tests, the traditional method is to gradually increase the number of users to avoid high failure rate and long response time caused by unexpected increase of users and ensure expected performance. PTS uses TPS and custom start and maximum load levels to maximize test efficiency.
Summary
The following conclusions can be drawn:
The system performance is determined by TPS and irrelevant to the number of concurrent users.
The maximum TPS is fixed (within a range), but the number of concurrent users can be adjusted.
We recommend that you use a short think time and apply load pressure as it is in the worst case.
In general, large systems (with high business volume and a large number of machines) need about 10,000 to 50,000 concurrent users in tests. Small and medium-sized systems need about 5000 concurrent users in tests.