When traffic to a microservice spikes beyond its processing capacity, uncontrolled requests can exhaust resources and cause service outages. Throttling rules enforce a per-instance queries-per-second (QPS) ceiling on specific interfaces. When the QPS exceeds your threshold, Microservices Engine (MSE) either rejects excess requests immediately or queues them at a constant rate, depending on the flow control effect you choose.
Prerequisites
Before you begin, make sure you have:
MSE Enterprise Edition activated
Microservices Governance enabled for your applications. For setup instructions, see:
How throttling works
Each throttling rule monitors a specific interface and applies a QPS threshold per application instance.
When traffic exceeds the threshold, MSE handles excess requests based on the flow control effect:
| Flow control effect | Behavior | When to use | Trade-off |
|---|---|---|---|
| Fast failure | Rejects excess requests immediately. The response content depends on the adaptation module configured in your application's system settings. | Latency-sensitive services where fast rejection is preferable to queuing | Low latency for callers, but excess requests are dropped |
| Waiting in line | Queues excess requests and processes them at a constant rate. Requests that exceed the timeout period fail. | Peak-load shifting, where you want to smooth traffic spikes and process all requests if possible | Higher latency for individual requests, but fewer dropped requests |
Create a throttling rule
Log on to the MSE console and select a region in the top navigation bar.
In the left-side navigation pane, choose Microservices Governance > Application Governance.
On the Application list page, click the resource card of the target application.
Open the throttling rule dialog using one of these paths:
From Application overview: Click Application overview in the left-side navigation pane. On the Through QPS TOP tab, find the target interface and click Flow Control in the Actions column.
From API Details: Click API Details in the left-side navigation pane. On the WEB service tab, click the Interface Flow Control tab, then click Add Throttling Rule.
From Traffic management: Click Traffic management in the left-side navigation pane. Click the Flow protection tab, then click the Interface Flow Control tab, then click Add Throttling Rule.
In the Add Throttling Protection Rule or Add Rule dialog box, configure the following parameters.
Parameter Description Interface name The interface to which the throttling rule applies. Whether to open Enables or disables the rule. The change takes effect immediately. Stand-alone QPS threshold The per-instance QPS limit that triggers the rule. Flow control effect How excess requests are handled. Select Fast failure to reject them immediately, or Waiting in line to queue them at a constant processing rate. Timeout The maximum time (in milliseconds) that a request can wait in the queue. Requests that exceed this period fail. Only applicable when Flow control effect is set to Waiting in line. Click New.
Example: Smooth traffic spikes with queuing
To process requests at a steady rate during peak hours instead of rejecting them, configure a throttling rule with queuing:
Set Stand-alone QPS threshold to
5.Set Flow control effect to Waiting in line.
Set Timeout to
5000.
With these settings, MSE processes one request every 200 milliseconds per instance and queues the rest. If a request waits longer than 5,000 milliseconds, it fails and MSE returns the default throttling response, such as a static page or predefined text.