How to use PTS to quickly initiate microservice stress testing

What is microservices?

Generally speaking, microservices architecture is an architectural pattern or style.

This article elaborates on:

1. What is a microservice architecture

2. The impact of microservice architecture on system stability and the necessity of verifying stability through performance testing

3. Pain points for users to conduct microservice pressure testing and unique advantages of PTS

4. Steps to quickly initiate microservice pressure testing using PTS on the cloud, and tips for troubleshooting and analyzing related issues after pressure testing is completed

It advocates dividing a single application into a set of small services, each running independently in its own process, and coordinating and cooperating with each other to provide ultimate value to users. Services communicate with each other using lightweight communication mechanisms (typically HTTP based RESTful APIs). Each service is built around a specific business and can be independently deployed to production environments, similar production environments, etc.

Microservices, with their high cohesion, low coupling and other characteristics, provide better fault tolerance and are more adaptable to rapid iterations such as business, bringing many conveniences to developers. However, with the development of business, the splitting of microservices has become increasingly complex and there are more and more modules, which means that the call link between services is much longer than before, and the probability of failure on the call link also increases, which poses a significant challenge to our system stability. For example:

1. In the case of a surge in single service traffic, it is necessary to respond quickly to capacity expansion;

2. When a service cannot withstand the pressure of large requests, will it affect other services it relies on;

3. The entire system has been dismantled into many microservices. When a certain service fails, is there a fault-tolerant means to keep the business running without affecting the overall application.

Verifying microservice stability through performance pressure testing

Common ways to ensure the stability of microservices

1. If an interface is called but no response is returned, we often need to set a timeout to prevent ourselves from being dragged to death by remote calls. The setting of timeout time is also meticulous. Setting it too long has a small effect and a higher risk of being dragged down. Setting it too short may also misjudge some normal requests, greatly increasing the error rate. In practical use, we can take the value of TP999 for the application over a period of time, or take the value of TP95 * 2.

2. Flow limiting refers to limiting the flow of service requests. Service providers can set a threshold for requests based on their own situation (capacity), and when this threshold is exceeded, they will discard the requests, ensuring the normal operation of their own services. The threshold setting can be considered in two aspects: QPS, which refers to the number of requests per second, and the number of concurrent threads. From a practical perspective, we often choose the latter because high QPS is often due to high processing power, which does not reflect the system's "inability to withstand heavy loads".

3. Due to the complexity of microservice invocation relationships, if a resource in the invocation link is unstable, it will ultimately lead to request accumulation. We need to limit the invocation of a resource in an unstable state (such as a timeout or an increase in the proportion of exceptions) in the invocation link, so that the request can fail quickly and avoid affecting other resources and causing cascading errors. When a resource is downgraded, calls to that resource will automatically expire within the next downgrade time window.

4. An application in the capacity expansion link may have a high CPU utilization rate or insufficient resources in the Connection pool (rpc, jdbc, Redis Connection pool, etc.), but it processes the request to get a connection very quickly, which requires horizontal expansion resources.

Verifying microservice stability through performance pressure testing

So how do we verify whether the above measures to ensure stability meet our needs?

1. Through microservice performance testing, we can obtain the distribution of indicators such as TP95 and TP999 of the system's RT under "high voltage", and design reasonable timeout times based on these indicators;

2. How high concurrent requests can it withstand without a significant surge in RT, identify the nodes where the call link requests accumulate, design reasonable current limiting and degradation circuit breaker strategies, and better improve microservice stability without affecting the user experience as much as possible.

3. Verify the effectiveness of service expansion.

Therefore, whether it is to evaluate the impact of single service launch or changes on system performance, or to accurately expand and verify the effectiveness of the expansion of services, it is essential to conduct performance testing on key microservice applications and understand local performance limits before comprehensive and formal pressure testing.

Micro service pressure testing pain points

Currently, common microservice pressure testing tools, such as JMeter and Gatling based on custom plugins, have the following unavoidable pain points:

1. For security reasons, a single microservice application will not expose the public network entrance. Therefore, it is necessary for the pressure testing tool to have the ability to connect to the VPC internal network, and the user's self built cost is relatively high.

2. Unable to simulate cross application multi interface calls.

3. The configuration of the registry address, interface name, and parameters for each service is very cumbersome.

4. Lack of intuitive call chain analysis and monitoring.

Advantages of using PTS pressure measurement microservices

PTS, as a SaaS platform with powerful distributed pressure testing capabilities, allows users to directly utilize millions of concurrent simulation capabilities and data analysis and aggregation capabilities without having to worry about building the underlying environment. It has unique advantages in the field of microservice pressure testing.

1. Safe and affordable, supporting VPC internal network pressure testing PTS supports VPC internal network pressure testing, which can quickly connect the pressure machine and the user's VPC network during pressure testing, ensuring the smooth network of internal network pressure testing. After the pressure test is completed, the network channel will also be immediately closed to ensure network security.

2. Support multi application and multi interface scenarios at will. What performance tests are required for a microservice application from development to launch? Firstly, we need to conduct performance testing on the interface of a single service, which may reveal some application logic issues. In this case, targeted performance optimization should be carried out. After optimizing the performance of a single service interface, we need to conduct multi application and multi interface scenario performance testing based on user scenarios. At this time, we may discover some interface call issues between services, and corresponding performance optimization will also be carried out; Finally, we also need to focus on verifying the scalability of our services, in order to determine the scaling model supported by each of our services.

3. Easy to use, supports direct pressure testing of EDAS/MSE applications. PTS naturally connects to EDAS/MSE applications and can directly initiate pressure testing, eliminating the hassle of configuring various service parameters, making it fast and convenient.

4. Intuitive and clear, supporting call chain analysis and monitoring. Before starting pressure testing, users can access PTS's problem diagnosis function to achieve call chain analysis and monitoring between microservice applications. For Java type services, the user side does not need to modify the business side code to complete probe access for problem diagnosis. For various abnormal information appearing in pressure testing, even if the calling relationship is very complex, users can clearly analyze the problem.

The principle of PTS microservice pressure testing

We usually use the RPC framework to implement remote calls between microservices. The RPC framework consists of three most important components, as shown in the following figure, namely the client, server, and registry.

In an RPC call process, these three components interact as follows:

After the server is started, it will publish the list of services it provides to the registry, and the client will subscribe to the service address from the registry;

The client will call the server through the local proxy module Proxy, which is responsible for converting methods, parameters, and other data into network byte streams;

The client selects one of the service addresses from the service list and sends the data to the server through the network;

After receiving the data, the server decodes it to obtain the request information;

The server calls the corresponding service based on the decoded request information, and then returns the call result to the client.

In actual pressure testing, PTS plays the role of a client and maintains a service list locally. It proactively requests the registration center every 5 seconds to update the service list, ensuring real-time performance while minimizing the load on the registration center as much as possible. The principle is shown in the figure.

Quickly initiate microservice pressure testing using PTS on the cloud

1. Create a scene. PTS currently supports three microservice frameworks: Dubbo, Spring Cloud, and gRPC. Taking Dubbo as an example, we will test EDAS applications that were previously connected. Firstly, we create a Dubbo pressure testing scenario in the PTS console's Pressure Testing Center ->Create Scenario;

2. Select an application. We choose the source of the pressure testing application as [EDAS] and the region as [Hangzhou], and choose the default microservice space;

3. Edit the scene. Select the application, interface, and method that needs to be tested in the Scenario Configuration Basic Configuration page, and set reasonable connection and response timeout times; PTS supports simultaneous pressure testing of multiple applications and interfaces, and can also use controllers and timers to achieve scene orchestration.

4. Finally, in the pressure configuration page, users only need to select the VPC intranet, security group, or switch where the microservice application is located to enable VPC intranet pressure testing. Enable your service to detect performance indicators without exposing public network entries. In addition, PTS has also launched a VPC pressure testing exclusive resource package [1], with a price of only 1/10 of the public network pressure testing.

Analysis of Common Microservice Performance Testing Issues

1. There is a partial response timeout: a) The server is busy, such as a service node with high CPU utilization

b) Network IO exceeds VM/EIP bandwidth

c) The timeout setting for backend microservices and databases is too long

2. TPS does not increase with the increase of concurrency: a) The system performance reaches the bottleneck, and the response delay increases during the continuous concurrent pressurization process (observable response interval statistics)

b) Verify if further pressurization will result in abnormal response

3. After running for a period of time, all responses timeout or checkpoint verification fails: a) High pressure causes a microservice in the system to crash

b) No response from backend database

4. TP90 has a shorter response latency and a higher TP99 latency: a) The system performance is approaching the bottleneck

b) Verify if further pressurization will result in abnormal response


This article elaborates on:

1. What is a microservice architecture

2. The impact of microservice architecture on system stability and the necessity of verifying stability through performance testing

3. Pain points for users to conduct microservice pressure testing and unique advantages of PTS

4. Steps to quickly initiate microservice pressure testing using PTS on the cloud, and tips for troubleshooting and analyzing related issues after pressure testing is completed

Related Articles

Explore More Special Offers

  1. Short Message Service(SMS) & Mail Service

    50,000 email package starts as low as USD 1.99, 120 short messages start at only USD 1.00

phone Contact Us