Configure traffic protection rules and view use cases - Serverless App Engine

Traffic protection integrates the traffic protection capabilities of Microservices Engine (MSE) into Serverless App Engine (SAE) and controls the traffic that reaches an application at the API level. Use throttling, concurrency isolation, circuit breaking, and hotspot parameter protection rules to keep an application available when traffic spikes or when a downstream dependency becomes unstable. Traffic protection analyzes the traffic distribution at second-level granularity.

Prerequisites

You have deployed an application.
You have activated MSE Microservices Governance Enterprise Edition.

Note

You are charged separately when you use MSE. For more information about the billing of MSE, see Billing of MSE Microservices Governance.

Limits

The following limits apply to traffic protection:

Supported applications — Traffic protection applies only to microservices applications that are created on or after November 8, 2023.
Queueing throughput — When throttled requests are queued, do not set the QPS to a value greater than 1,000, which corresponds to a request interval of 1 millisecond.
Apache Dubbo versions — Apache Dubbo 2.7.0 to 2.7.3 do not support custom behaviors. In these versions, a throttled request returns java.lang.RuntimeException, and the content contains SentinelBlockedException and the throttling information.
Custom response types — A custom response does not support object types that contain generics with undefined types, such as Map<K, V> and List<T>.

Choose a protection rule type

Traffic protection provides five rule types, and each rule type has its own tab in the SAE console. The following table describes what each rule type controls and where you configure it.

Rule type	What it controls	Statistical dimension	Console tab
Interface-level throttling	The queries per second (QPS) of an application or a service. Traffic is blocked as soon as the QPS reaches the threshold.	QPS	Interface-level Throttling
Concurrency isolation	The number of concurrent threads of an API or its dependencies.	Concurrent threads	Concurrency Isolation
Circuit breaking	Calls to an unstable API or downstream dependency, based on the response time or the error rate.	Slow call ratio or abnormal proportion	Circuit Breaking Rules
Hotspot parameter protection (RPC)	Resource calls that carry frequently called parameters. A parameter is identified by its index position at the instrumentation point.	Number of requests or concurrent number, per hotspot parameter	Hotspot Parameter Protection (RPC)
Hotspot parameter protection (HTTP request)	Access requests to an application that provides web services, based on request properties such as the client IP address, Host, Header, and URL parameter.	Request count, per hotspot parameter	Hotspot Parameter Protection (HTTP Request)

Interface-level throttling rules and hotspot parameter protection rules also let you choose how throttled traffic is handled. Fast failure blocks excess requests immediately. Waiting in line queues excess requests so that requests pass at a steady rate, and is typically used for peak-load shifting. If you queue requests, observe the queueing throughput limit described in Limits.

Go to the traffic protection tab

In the SAE application list, select the region and the namespace at the top of the page, and then click the Application ID of the application to go to the application details page.
In the left-side navigation pane, choose Microservices Governance > Traffic Governance, and then click the Traffic Protection tab.

Configure protection rules

For the rule types whose creation flow includes the Configure Protection Behavior wizard, the last step is to select an Associated Behavior, which defines the response that a request receives after it triggers the rule. Select the default behavior if you do not need a custom response for rejected requests. For behavior parameters and creation steps, see Manage behaviors.

Important

For interface-level throttling rules, concurrency isolation rules, and hotspot parameter protection rules, Whether to open is turned off by default. These rules take effect only after you turn on this switch.

Configure interface-level throttling rules

An interface-level throttling rule monitors the queries per second (QPS) of an application or a service. When the QPS reaches the specified threshold, the rule blocks traffic immediately. This prevents momentary traffic spikes from overwhelming the application and ensures high availability (HA).

Click the Interface-level Throttling tab, and then click Create Throttling Rule.
In the Create Throttling Rule dialog box, complete the following wizards:
1. In the Select Protection Scenario wizard, select the API Type, Traffic Type, and API Name, and then click Next.
2. In the Configure Protection Rule wizard, configure the parameters that are described in the following table, and then click Next.
3. In the Configure Protection Behavior wizard, select an Associated Behavior, and then click Create.
The following table describes the parameters in the Configure Protection Rule wizard.

Configuration item	Description
Whether to open	If you turn on this switch, the rule takes effect immediately after it is created. The switch is turned off by default.
Per-instance QPS threshold	The QPS threshold of the object in the statistical dimension of the throttled API.
Flow control effect	Select how throttled traffic is handled. Fast failure: When the threshold is reached, requests are blocked immediately. The response content is determined by the adapter module settings in the system settings of the application. Waiting in line: Requests pass at a steady rate and excess requests are queued. This method is typically used for peak-load shifting. You must set a timeout period. A request fails fast when it reaches the timeout period.

The rule appears on the Interface-level Throttling tab, where you can edit or delete it.

Example: shift peak loads so that traffic passes at a steady rate

Request traffic has peaks and troughs. Throttling delays peak traffic by queueing requests and processes the requests later. This approach serves as many requests as possible and maintains the user experience. Configure the throttling rule with the following settings:

Set Stand-alone QPS threshold to 5 for the steady-rate mode.
Set Flow control effect to Waiting in line.
Set Timeout to 5 seconds.
The system processes one request every 200 milliseconds and queues excess tasks. Because the wait time is 5 seconds, a task whose estimated queueing time exceeds 5 seconds fails fast and directly returns the default throttling response, such as a text message or a static page.

Configure concurrency isolation rules

A concurrency isolation rule controls the number of concurrent threads of an API or its dependencies to maintain system stability.

Click the Concurrency Isolation tab, and then click Create Isolation Rule.
In the Create Isolation Rule dialog box, complete the following wizards:
1. In the Select Protection Scenario wizard, select the API Type, Traffic Type, and API Name, and then click Next.
2. In the Configure Protection Rule wizard, configure the parameters that are described in the following table, and then click Next.
3. In the Configure Protection Behavior wizard, select an Associated Behavior, and then click Create.
The following table describes the parameters in the Configure Protection Rule wizard.

Configuration item	Description
Whether to open	If you turn on this switch, the rule takes effect immediately after it is created. The switch is turned off by default.
Concurrency threshold	The threshold for the number of concurrent threads of the resource, which is the number of threads that are running for the resource.

The rule appears on the Concurrency Isolation tab, where you can edit or delete it.

Example: ensure sufficient resources for your application

When the response time of a request increases, the number of concurrent threads grows. After the concurrency exceeds the threshold, the system rejects excess requests until the accumulated tasks are complete and the number of concurrent threads decreases. This isolates faults and reduces instability.

For example, an SQL statement takes 20 milliseconds to run, and 20 requests per second are expected for this request. Configure the isolation rule with the following settings:

Enter the Interface Name.
Set the Concurrency threshold to 10.
After this configuration, if the SQL statement encounters a deadlock or performance issues and becomes a slow SQL statement, it occupies only 10 threads even if requests keep arriving. The continuous requests, which cannot exit within a short period of time, therefore cannot exhaust the active threads of the process. After the SQL statement returns to normal, the concurrency decreases rapidly. When the concurrency drops below the preset threshold, the system stops rejecting requests and the processing capacity of the application recovers quickly. This method adjusts capacity automatically based on the response time and isolates unstable applications.

Configure circuit breaking rules

A circuit breaking rule monitors the response time or the error rate of your application or its downstream dependencies. When the specified threshold is reached, the system immediately lowers the priority of the downstream dependency. Within the specified period of time, the system does not call the unstable resource. This prevents the application from running in an unstable state and ensures high availability. After the specified period of time elapses, the system resumes calls to the resource.

Click the Circuit Breaking Rules tab, and then click Create Circuit Breaking Rule.
In the Create Circuit Breaking Rule dialog box, complete the following wizards:
1. In the Select Protection Scenario wizard, select the API Type and API Name, and then click Next.
2. In the Configure Protection Rule wizard, configure the parameters that are described in the following table, and then click Create.
The following table describes the parameters in the Configure Protection Rule wizard.

Configuration item	Description
Statistical window duration	The length of the time window for statistics. Valid values: 1 second to 120 minutes.
Minimum number of requests	The minimum number of requests that triggers circuit breaking. If the number of requests in the current statistic time window is less than this value, the rule is not triggered even if the circuit breaking condition is met.
Threshold Type	Select Slow call ratio (%) or Abnormal proportion (%) as the threshold. If you select Slow call ratio as the threshold, you must specify the allowed Slow call RT, which is the maximum response time. A request whose response time is greater than this value is counted as a slow call. In the Circuit Breaking Ratio Threshold field, set the slow call ratio that triggers circuit breaking. After the rule is enabled, if the number of requests in a statistic time window is greater than the Minimum number of requests and the slow call ratio is greater than the threshold, subsequent requests are automatically blocked for the Circuit Breaking Duration (s). After the circuit breaking duration elapses, the circuit breaker enters the probe recovery state. If the response time of the next request is less than the specified Slow call RT, circuit breaking ends. If the response time is greater than the specified Slow call RT, circuit breaking is triggered again. If you select Abnormal proportion as the threshold, you must set the error rate that triggers circuit breaking in the Circuit Breaking Ratio Threshold field. After the rule is enabled, if the number of business errors in a statistic time window is greater than the Minimum number of requests and the error rate is greater than the threshold, subsequent requests are automatically blocked for the Circuit Breaking Duration (s).
Circuit Breaking Duration (s)	Click Show Advanced Options to configure this parameter. The duration for which circuit breaking lasts after it is triggered. After a resource enters the circuit breaking state, requests fail fast within the specified circuit breaking duration.
Circuit Breaking Policy	Click Show Advanced Options to configure this parameter. The recovery policy that the circuit breaker uses when it enters the recovery phase, which is the half-open state. Single detection recovery: After the circuit breaking duration elapses, the circuit breaker probes the next request. If the request meets expectations, which means that it is neither a slow call nor an error, circuit breaking ends. Otherwise, the circuit breaker returns to the circuit breaking phase. Progressive recovery: You must set the Number of recovery phases and the Minimum number of passes per step . After the circuit breaking duration elapses, the circuit breaker recovers progressively based on the specified number of recovery phases. If the number of requests in a phase reaches the minimum number of passes per step, a check is triggered. If none of the checked requests exceeds the threshold, the system gradually increases the ratio of requests that are allowed to pass until traffic fully recovers. If a metric in one of the steps exceeds the threshold, the circuit breaker returns to the circuit breaking phase. The request ratio T = 100/N, where N is the number of recovery phases. The ratio in the first phase is T, the ratio in the second phase is 2T, and so on, until the ratio reaches 100%. For example, if the number of recovery phases is 3 and the minimum number of passes per step is 5, requests are allowed to pass at ratios of 33%, 67%, and 100% in the three phases. When the number of requests in a phase is greater than or equal to 5, a check is performed. If the request metrics do not exceed the threshold, the circuit breaker enters the next recovery phase until traffic fully recovers.

The rule appears on the Circuit Breaking Rules tab, where you can edit or delete it.

Example: trigger circuit breaking on slow calls or on errors

Apply circuit breaking when a call to a third-party service responds too slowly and affects the current API, or when your system returns errors while it displays third-party content. Both cases use the same rule type and differ only in the Threshold Type that you select.

Configuration item	Example: slow calls	Example: errors
Interface Name	test	test
Statistical window duration	1	1
Minimum number of requests	10	10
Threshold Type	Slow call ratio	Abnormal proportion
Slow call RT	1000	Not required
Circuit Breaking Ratio Threshold	80%	80%
Circuit Breaking Duration (s)	10	10
Circuit Breaking Policy	Single detection recovery	Single detection recovery

With either configuration, the statistic time window is 1 second, the rule is not triggered unless the window contains at least 10 requests, and circuit breaking lasts 10 seconds. In the slow call example, a request that takes more than 1000 milliseconds is considered a slow request, and the slow call ratio that triggers circuit breaking is 80%. In the error example, the error rate that triggers circuit breaking is 80%.

Configure hotspot parameter protection rules (RPC)

After you configure hotspot rules for an application, the system analyzes the statistical parameters, which are the parameters that are called frequently during resource calls. Based on the hotspot rules, the system throttles the resource calls that contain hotspot parameters to protect system stability. Use this rule type for RPC resource calls, in which a hotspot parameter is identified by its index position at the instrumentation point.

Click the Hotspot Parameter Protection (RPC) tab, and then click Create Hotspot Parameter Protection Rule (RPC).
In the Create Hotspot Parameter Protection Rule (RPC) dialog box, complete the following wizards:
1. In the Select Protection Scenario wizard, select the API Name, and then click Next.
2. In the Configure Protection Rule wizard, configure the parameters that are described in the following table, and then click Next.
3. In the Configure Protection Behavior wizard, select an Associated Behavior, and then click Create.
The following table describes the parameters in the Configure Protection Rule wizard.

Configuration item	Description
Parameter position Index	The index position of the parameter that is passed in at the instrumentation point resource. This corresponds to the parameter index position in `SphU.entry(xxx,args)`. For example, in the `SphU.entry(resourceName,Entry Type.IN,1,paramA,paramB)` instrumentation point, the parameter index of `paramA` is 0 and the parameter index of `paramB` is 1.
Statistical dimension	You can measure either the number of requests or the number of concurrent threads. Number of requests: Limits the number of calls within a period of time. Concurrent number: Limits the number of concurrent calls to the resource.
Statistical cycle time	The length of the statistic time window, in seconds. For example, if the statistic time window is 10 seconds and the QPS threshold is 5, each hotspot parameter can be accessed at most 5 times within 10 seconds.
Single machine threshold	The threshold that applies to each hotspot parameter.
Flow control effect	If Statistical dimension is set to Number of requests, you can select how throttled traffic is handled. Fast failure: When the threshold is reached, requests are blocked immediately. In this mode, you can also set a Number of buffered requests, which is the number of extra requests that are allowed for traffic bursts. Waiting in line: Requests pass at a steady rate and excess requests are queued. This method is typically used for peak-load shifting. You must set a timeout period. When a request is queued, the system calculates its estimated queueing time. If the estimated queueing time exceeds the maximum timeout period, the request is rejected. For example, if Single machine threshold is set to 5, only one request passes every 200 milliseconds and excess requests are queued. If Timeout is set to 1000 milliseconds, new requests are rejected when more than 5 requests are already queued, which corresponds to a queueing time of more than 1000 milliseconds.
Whether to open	If you turn on this switch, the rule takes effect immediately after it is created. The switch is turned off by default.

The rule appears on the Hotspot Parameter Protection (RPC) tab, where you can edit or delete it.

Example: queue requests for a hotspot item during a flash sale

During flash sales and other rush-purchase events, high traffic can slow down system responses or even crash the system. To maintain system stability, you can configure a hotspot rule so that the system queues traffic for popular items after the traffic exceeds a threshold. This example queues the excess requests to an RPC API instead of rejecting them: if requests to purchase the same item exceed 100 within 1 second, the remaining requests are queued.

Enter the Interface Name.
Set Statistical dimension to Number of requests.
Set Statistical cycle time to 1 second and Single machine threshold to 100.
Set Flow control effect to Waiting in line.
Set Timeout to 30 milliseconds.
If this API is called more than 100 times within 1 second, excess requests are queued. A request that waits longer than 30 milliseconds fails immediately.

Example: limit frequent calls that consume excessive system resources

In a flash sale, customers sometimes also need to change the delivery address of an order. A large number of these change requests consumes significant database write resources. You can limit the concurrent calls that carry the same hotspot parameter so that customers make the change later.

Enter the Interface Name.
Set Statistical dimension to Concurrent number.
Set Statistical cycle time to 1 second and Single machine threshold to 100.
This configuration allows a maximum of 100 concurrent calls for each hotspot parameter. Excess requests are rejected.

Configure hotspot parameter protection rules (HTTP request)

Hotspot parameter protection rules for HTTP requests apply to applications that provide web services. These rules perform granular traffic shaping on specific parameters in access requests. You can shape traffic for resource calls based on request dimensions such as the IP address, Host, Header, and URL Param to maintain business and system stability.

Click the Hotspot Parameter Protection (HTTP Request) tab, and then click Create Hotspot Parameter Protection Rule (HTTP Request).
In the Create Hotspot Parameter Protection Rule (HTTP Request) dialog box, complete the following wizards:
1. In the Select Protection Scenario wizard, select the API Name, and then click Next.
2. In the Configure Protection Rule wizard, configure the parameters that are described in the following table, and then click Next.
3. In the Configure Protection Behavior wizard, select an Associated Behavior, and then click Create.
The following table describes the parameters in the Configure Protection Rule wizard.

Configuration item	Description
Parameter Properties	Shapes traffic based on the parameter properties of the selected API. Client IP: The IP address of the client. If a request passes through a proxy, the system first attempts to obtain the IP address from the `X-Forwarded-For` request header. If that header contains IP information, the system uses that IP address as the actual client IP address. Remote Host: The `Host` header of the client. Header: The system parses the request based on the specified HTTP header. If you enter a specific header key, the rule limits each hotspot value under that header key separately. After you select Header, you can configure a match policy for request property values. Only the request property values that match the pattern are included in statistics and throttling. URL parameter: The system parses the request based on the specified HTTP request parameter. You must enter the name of the parameter. After you select URL parameter, you can configure a match policy for request property values. Only the request property values that match the pattern are included in statistics and throttling.
Threshold Type	The default value is Request count.
Threshold	The QPS threshold of the object in the statistical dimension of the throttled API. When you set the threshold, you must select a statistical interval. The supported intervals are second, minute, hour, and day. For example, if you set the threshold to 10 and select minute as the interval, no more than 10 requests are allowed per minute.
Whether to open	If you turn on this switch, the rule takes effect immediately after it is created. The switch is turned off by default.
Throttling method	Click Show Advanced Options to configure this parameter. Fast failure: If the threshold type is QPS, throttled traffic fails fast, which means that requests are blocked immediately when the threshold is reached. Rejected requests receive the custom response that is configured in behavior management. If no custom response is configured, the default behavior applies, which returns the 429 error code and the default text message. Waiting in line: If the threshold type is QPS, throttled requests pass at a steady rate and are allowed to queue. You must set a timeout period. A request that is expected to reach the timeout period fails immediately instead of being queued. For example, if the QPS is set to 10, only one request passes every 100 milliseconds and excess requests are queued. The timeout period is the maximum queueing time. Requests that exceed the maximum queueing time are rejected. For the QPS limit that applies when requests are queued, see Limits.
Burst size	Click Show Advanced Options to configure this parameter. If Throttling method is set to Fast failure, you can set a burst size, which is the number of extra requests that are allowed for traffic bursts.
Timeout	Click Show Advanced Options to configure this parameter. If Throttling method is set to Waiting in line, you must set a timeout period, in milliseconds. For example, if the QPS is set to 5, only one request passes every 200 milliseconds and excess requests are queued. The timeout period is the maximum queueing time. Requests that exceed the maximum queueing time are rejected.

The rule appears on the Hotspot Parameter Protection (HTTP Request) tab, where you can edit or delete it.

Example: flash sale of a hotspot product

During flash sales and other rush-purchase events, high traffic can slow down system responses or even crash the system. To maintain system stability, you can configure a hotspot parameter protection rule so that the system rejects excess traffic for popular items after the traffic exceeds a threshold. This example applies to an HTTP API and rejects the excess requests instead of queueing them.

For example, to reject the excess requests after the requests for the same product exceed 100 within 1 second, configure the following parameters. This rule allows a maximum of 100 order requests per second for each individual product ID and rejects the excess order requests for that product with a custom response.

Set Parameter Properties to URL parameter.
Enter stockId for URL parameter name.
Enter 100 requests per second for Threshold.
Set Throttling method to Fast failure.

Note

In the parameter properties, select the parameter field that corresponds to the ID of the hotspot product. For example, if a stockId field in the URL parameters carries the requested product ID, you can set the parameter property to URL parameter and enter the field name of the property in the request as the parameter name.

Example: prevent malicious orders

During a promotional event, a large number of malicious order requests occupies product inventory or server resources. In this case, you can queue the requests based on their source IP addresses so that access requests pass at a steady rate. This prevents excessive requests from affecting service stability. Configure the following rule:

Set Parameter Properties to Client IP.
Keep the default value Request count for Threshold Type.
Enter 100 requests per second for Threshold.
Set Throttling method to Waiting in line.
Enter 30 milliseconds for Timeout.
Turn on Whether to open.
With this rule, requests from each source IP address pass at a rate of one request every 10 milliseconds (1 second/100 = 10 milliseconds), and excess requests are queued. A queued request that waits longer than 30 milliseconds fails immediately.

Manage behaviors

A web behavior returns a custom response after a rule is triggered on a web instrumentation point resource. For example, a web API can return the message Blocked by Sentinel after it triggers a throttling rule.

Create a behavior

Click the Behavior Management tab, and then click Create Behavior.
In the Create Behavior dialog box, configure the parameters that are described in the following tables, and then click Create.
The following table describes the parameters that apply when Resource Type is set to Web.

Parameter	Description	Example value
Behavior Name	The name of the behavior. The name can be up to 128 characters in length and must be unique within the application.	Test behavior
Resource Type	Select Web.	Web
**Web throttling handlerWeb Throttling Policy	Defines how the system behaves after access to a web API triggers a rule. Custom response: You must set the HTTP status code, the format of the returned content, and the returned content. The web API returns the custom content after access to it triggers a rule.	Custom response
HTTP Status Code	The default value is 429. This parameter is required when Web throttling handler is set to Custom response.	429
Returned Content-Type	Set the format of the returned content to Plain text or JSON.	JSON string
Returned HTTP Text	Enter the content that is returned after access to the web API triggers a rule. This parameter is required when Web throttling handler is set to Custom response.	`{"message": "blocked oops"}`

The following table describes the parameters that apply when Resource Type is set to RPC.

Parameter	Description	Example value
Behavior Name	The name of the behavior. The name can be up to 128 characters in length and must be unique within the application.	Test behavior
Resource Type	Select RPC. For the Apache Dubbo versions that do not support custom behaviors, see Limits.	Rpc
RPC throttling handler	Defines how the system behaves after access to an RPC API triggers a rule. Custom response: Returns a custom result. You must set the HTTP status code, the format of the returned content, and the returned content. The RPC API returns the custom content after access to it triggers a rule. Custom exception: Throws a custom exception. You must set the exception class name and the exception text. The system returns the specified exception information after access to the RPC API triggers a rule.	Custom response/Custom exception
Custom response class name	Required when RPC throttling handler is set to Custom response. Enter the path of the class name. For the object types that a custom response does not support, see Limits.	`com.alibaba.demo.OrderService:getOrder(long)`
Custom response content (JSON)	Required when RPC throttling handler is set to Custom response. Enter the object content that is returned when access to the RPC API triggers a rule.	`{"id": "123", "name": "test"}`
Exception class name	Required when RPC throttling handler is set to Custom exception. Enter the path of the exception class name.	`java.lang.RuntimeException`
Exception message	Required when RPC throttling handler is set to Custom exception. Enter the text of the custom exception that is thrown after access to the RPC API triggers a rule.	`"Operation failed"`

The behavior that you create appears on the Behavior Management tab, where you can modify or delete it.

Associate a behavior with an existing rule

A behavior is associated with a rule in the Configure Protection Behavior wizard when you create the rule. To change the behavior of a rule that already exists, edit the rule.

Click the Traffic Protection tab, and then click the tab of the rule type.
Find the rule and click Edit in the Actions column.
Complete the Select Protection Scenario and Configure Protection Rule wizards.
In the Configure Protection Behavior wizard, select the behavior from the Associated Behavior drop-down list. Alternatively, click Create Behavior to create a behavior and associate it with the rule.
Click Save.
Before you associate a behavior, note the following points:

Default behavior — If you do not need a custom fallback behavior for throttled requests, select the default behavior. The API type of the default behavior is empty.
Existing binding — When you create a rule, if a behavior is already bound to the API, the new behavior overwrites the existing behavior of the API.
API type — When you select a behavior, the default behavior is bound if you do not select an API type. After you select an API type and bind a behavior of the corresponding type, you cannot modify the type.