The microservices stability is a major concern for customers who use microservices. With the evolution from a monolithic architecture to a distributed architecture and changes in deployment methods, the dependencies among services become more and more complex, and high availability is of great importance to business systems. Microservices Engine (MSE) traffic protection is a service that uses modules such as throttling and hotspot protection to improve the high availability of applications. This topic describes the traffic protection rules and their use scenarios.
Instability scenarios
You may encounter the following instability issues in the production environment:
When traffic surges during large-scale promotion activities, orders cannot be placed due to system overloads, high loads, or system breakdowns.
Some unknown products become popular. As a result, cache breakdowns, database crashes, or resource preemption issues occur.
The caller breaks down due to an unstable third-party service, the thread pool is full, or a large number of calls wait to be processed. As a result, the entire call process is suspended.
These instability scenarios may lead to serious consequences. However, in most cases, developers do not apply high-availability protection related to traffic and dependencies. MSE traffic protection helps prevent the impact of these unstable factors on traffic and provide high-availability traffic protection to ensure service stability.
Core scenarios
The following table describes the traffic protection rules and applicable core scenarios.
Traffic protection rule | Description | Core scenario | References |
Throttling rule | You can use MSE to configure a throttling rule based on queries per second (QPS). If the QPS value exceeds a specified threshold, the system automatically rejects extra requests. | Throttling rules are suitable for scenarios in which traffic surges need to be restricted. In these scenarios, throttling rules allow the system to continue processing requests and prevent service failures. | |
Isolation rule | Isolation rules are used to control the number of concurrent ongoing calls to prevent excessive slow calls from preempting the resources that are allocated to normal calls. | Isolation rules are suitable for scenarios in which a third-party service is called and an excess number of slow calls preempt resources that are allocated to normal calls. Isolation rules ensure service availability. | |
Circuit breaking rule | Circuit breaking rules allow the system to automatically perform circuit breaking or degradation on unstable calls with weak dependencies. These rules prevent system breakdowns caused by instability factors. | Circuit breaking rules are used to prevent system breakdowns caused by instability factors, such as a slow call or service exception. For example, a circuit breaking rule is specified to disable the call of a third-party service that has a high response time (RT). | |
Hotspot parameter protection rule | Hotspot parameter protection rules are used for the system to automatically identify hotspot parameters and control the access frequency or concurrency of each hotspot value. Hotspot parameter protection rules can effectively prevent resources that are allocated to normal calls from being preempted due to overly frequent access to specific parameters. | Hotspot parameter protection rules are suitable for scenarios in which the most frequently accessed data in specific hotspot data needs to be controlled. For example, you can use hotspot parameter protection rules to control the IDs of the most frequently purchased products over a period of time. This prevents a large number of requests from being sent to a database when cache breakdown occurs due to frequent access to some products. |
Throttling rules
Scenarios
Traffic is random and unpredictable. Traffic surges may occur at a specific point in time, such as 00:00 on November 11 (Double 11). However, the capacity of the system is always limited. If the peak traffic exceeds the capacity of the system, the following consequences may occur, which results in system breakdowns: (1) A large number of requests wait to be processed. (2) The request processing is slow. (3) The CPU utilization is high or system overload occurs. To address these issues, you must throttle traffic surges. Traffic throttling enables the system to process requests as many as possible without causing system breakdowns. Traffic throttling is a common method that is suitable for scenarios such as traffic bursts.
Rule description
In scenarios in which web portals or service providers are available, you must prevent service providers from being overwhelmed by traffic surges. In this case, traffic is throttled based on the service capabilities of the service provider or throttled for specific service callers. You can evaluate the processing capability of core interfaces by performing stress testing in advance, and use MSE to configure QPS-based throttling rules. After QPS-based throttling rules are configured, the system automatically rejects extra requests if the QPS value exceeds a specified threshold. For more information about throttling rules, see Create a throttling rule.
Concurrency control
Scenarios
In most cases, a service calls another module such as a remote service, a database, or a third-party API. For example, when you pay for bills, an API that is provided by UnionPay may be remotely called. When you query the price of a product, database queries may be involved. However, the stability of this dependent service cannot be ensured. If the dependent service is unstable and the response time of the request increases, the response time of the method that is used to call the service also increases, and a large number of threads are used. As a result, threads in the thread pool of the service may be used up and the service becomes unavailable.
Modern microservices architectures are distributed and consist of multiple microservices. Different microservices call each other, which makes call processes become complex. The preceding issues have a great impact on service calls. In a complex call process, if a node is unstable, other nodes may also become unstable. As a result, the entire call becomes unavailable.
Rule description
MSE provides the capability to control the number of concurrent threads of interfaces or dependencies to avoid service unavailability caused by unstable factors such as slow calls.
Concurrency control is used as a lightweight isolation method to control the number of concurrent threads for specific calls. This helps prevent excessive number of slow calls from preempting resources of normal calls. For more information, see Create an isolation rule.
Hotspot parameter protection rules
Scenarios
To prevent system breakdowns caused by traffic overload, you can configure throttling rules for core interfaces in most cases. However, in some scenarios, common throttling rules cannot resolve the issue.
For example, during the peak hours of large-scale promotion activities, instantaneous page views of hot products are high. In this case, you can predict hot products in advance and perform cache prefetching for information about the hot products. This way, you can quickly obtain product information without database queries when product page views are high. However, some unknown products may become popular during large-scale promotion activities. You cannot predict this type of product, and product information is not prefetched. When the page views of these products increase, a large number of requests are sent to the cache, which causes a cache breakdown. As a result, database access is time-consuming. Resources in the resource pool of normal product requests are consumed by the abnormal requests. As a result, a system breakdown occurs.
Rule description
To address this issue, you can configure hotspot parameter protection rules. These rules allow the system to automatically identify hotspot parameters and control the access frequency or concurrency of each hotspot value based on the access control on hotspot parameters in MSE. Hotspot parameter protection rules can effectively prevent excessive requests for access to hot parameters from preempting resources that are allocated to normal calls. For more information, see Create a hot parameter throttling rule for RPC requests and Create a hot parameter throttling rule for HTTP requests.