Microservices Engine:Configure a throttling policy

The throttling feature on the MSE console has been upgraded to include new modes, which are available only for gateway versions 1.1.0 and later. The legacy throttling feature is still available. Therefore, select the procedure that matches your gateway version.

• If your gateway version is 1.1.0 or later, see the Configure a throttling policy (new) section.

• If your gateway version is 1.0.0, see the Configure a throttling policy (legacy) section.

Configure a throttling policy (new)

1. Log on to the MSE console. In the top navigation bar, select a region.

2. In the left-side navigation pane, choose Cloud-native Gateway > Gateways. On the Gateways page, click the ID of the gateway.

3. In the left-side navigation pane, click Routes. Then, click the Routes tab.

4. Find the route that you want to manage and click Policies in the Actions column.

5. On the Policies tab, click Throttling.

   For gateway versions 1.2.25 and later

   • Throttling rules

     Throttling rules monitor the queries per second (QPS) of a route. When the QPS reaches the configured threshold, the gateway immediately intercepts traffic. This protects backend services from being overwhelmed by sudden traffic spikes and ensures high availability.

     1. In the Throttling section, click the throttling rules tab.

     2. In the Throttling Rules section, configure the following parameters.

        Parameter	Description
        Gateway QPS threshold	The maximum queries per second (QPS) for the entire gateway.
        Web fallback behavior	The action to perform when a request is throttled. Options include Return Specified Content and Go to Specified Page.
        If you set Web Fallback Behavior to Go to Specified Page:
        Redirect URL	The URL to which throttled requests are redirected.
        If you set Web Fallback Behavior to Return Specified Content:
        HTTP status code	The HTTP status code to return. The default value is 429.
        Returned content type	The content type of the response. Options are Plain Text and JSON.
        HTTP response body	The body of the custom response to return.
        Enable	Enables the rule when turned on.

     3. Click New or Save, and then click OK in the confirmation dialog box.

   • Concurrency rules

     Concurrency rules monitor the total number of requests being processed by the gateway. When the number of concurrent requests reaches the configured threshold, the gateway immediately intercepts new traffic. You can set this value to the maximum concurrent processing capacity of your backend service to ensure its availability under high concurrency.

     1. In the Throttling section, click the concurrency rules tab.

     2. On the concurrency rules tab, configure the following parameters.

        Parameter	Description
        Gateway concurrency threshold	The maximum number of concurrent requests for the entire gateway.
        Web fallback behavior	The action to perform when a request is throttled. Options include Return Specified Content and Go to Specified Page.
        If you set Web Fallback Behavior to Go to Specified Page:
        Redirect URL	The URL to which throttled requests are redirected.
        If you set Web Fallback Behavior to Return Specified Content:
        HTTP status code	The HTTP status code to return. The default value is 429.
        Returned content type	The content type of the response. Options are Plain Text and JSON.
        HTTP response body	The body of the custom response to return.
        Enable	Enables the rule when turned on.

     3. Click New or Save, and then click OK in the confirmation dialog box.

   • Circuit breaking rules

     Circuit breaking rules monitor the response time or error ratio of a route. When a specified threshold is reached, the circuit breaker trips and stops calling the unstable resource for a specified period to protect the backend service and ensure high availability. After this period, calls to the resource are resumed.

     1. In the throttling section, click the Fuse rule tab.

     2. On the Fuse rule tab, configure the following parameters.

        Parameter	Description
        Statistical window duration	The duration of the time window for collecting statistics. Valid values: 1 second to 120 minutes.
        Minimum requests	The minimum number of requests required to trigger circuit breaking. If the number of requests within the current time window is less than this value, the rule is not triggered even if the threshold is met.
        Threshold type	Select Slow Call Ratio or Error Ratio as the threshold. If you select slow call ratio as the threshold, you must set the allowed slow call RT (maximum response time). A request is counted as a slow call if its response time is greater than this value. You must also set the slow call ratio that triggers circuit breaking. After the rule is enabled, if the number of requests within a statistical time window is greater than the minimum number of requests and the slow call ratio exceeds the threshold, requests are automatically blocked for the subsequent circuit breaking period. After the circuit breaking period ends, the circuit breaker enters the half-open state. If the response time of the next single request is less than the configured slow call RT, the circuit breaking ends. If the response time is greater than the configured slow call RT, the circuit is tripped again. If you select Error Ratio, you must set the threshold for the error ratio. When the rule is enabled, if the number of requests in a time window exceeds the minimum request count and the error ratio exceeds the threshold, the circuit breaker trips. During the subsequent circuit breaking period, all requests fail fast.
        Circuit breaking period (s)	The duration for which the circuit remains open after tripping. During this period, all calls to the resource fail fast.
        Web fallback behavior	The action to perform when a request is throttled. Options include Return Specified Content and Go to Specified Page.
        If you set Web Fallback Behavior to Go to Specified Page:
        Redirect URL	The URL to which throttled requests are redirected.
        If you set Web Fallback Behavior to Return Specified Content:
        HTTP status code	The HTTP status code to return. The default value is 429.
        Returned content type	The content type of the response. Options are Plain Text and JSON.
        HTTP response body	The body of the custom response to return.
        Enable	Enables the rule when turned on.

     3. Click New or Save, and then click OK in the confirmation dialog box.

   For gateway versions 1.1.0 to 1.2.24

   • Throttling rule

     Throttling rules monitor the queries per second (QPS) of a route. When the QPS reaches the configured threshold, the gateway immediately intercepts traffic. This prevents backend services from being overwhelmed by sudden traffic spikes and ensures high availability.

     1. On the Throttling panel, click the Throttling Rules tab, and then click Add Throttling Rule.

     2. On the Add Throttling Rule page, configure the settings.

        1. In the Configure Rule step, set the gateway QPS threshold as required and turn on the Enable switch. Then, click Next.

        2. In the Configure Throttling Behavior step, you can click New behavior to customize the HTTP response when throttling is triggered.

     3. After you configure the throttling behavior, click New to create the rule.

   • Concurrency rule

     Concurrency rules monitor the total number of requests being processed by the gateway. When the number of concurrent requests reaches the configured threshold, the gateway immediately intercepts new traffic. You can set this value to the maximum concurrent processing capacity of your backend service to ensure its availability under high concurrency.

     1. On the Throttling panel, click the concurrency rules tab, and then click Add Concurrency Rule.

     2. On the Add Concurrency Rule page, configure the settings.

        1. In the Configure Rule step, set the gateway concurrency threshold as required and turn on the Enable switch. Then, click Next.

        2. In the Configure Throttling Behavior step, you can click Add Behavior to customize the HTTP response when throttling is triggered.

     3. After you configure the throttling behavior, click New to create the rule.

   • Circuit breaking rule

     Circuit breaking rules monitor the response time or error ratio of a route. When a specified threshold is reached, the circuit breaker trips and stops calling the unstable resource for a specified period to protect the backend service and ensure high availability. After this period, calls to the resource are resumed.

     1. On the throttling panel, click the Fuse rule tab, and then click Added fusing rule.

     2. On the Add Circuit Breaking Rule page, configure the settings.

        1. In the Configure Rule step, set the parameters as required and turn on the Enable switch. Then, click Next.

           Parameter	Description
           Statistical window duration	The duration of the time window for collecting statistics. Valid values: 1 second to 120 minutes.
           Minimum requests	The minimum number of requests required to trigger circuit breaking. If the number of requests within the current time window is less than this value, the rule is not triggered even if the threshold is met.
           Threshold type	Select Slow Call Ratio or Error Ratio as the threshold. If you select the slow call ratio as the threshold, you must set an allowed slow call Response Time (RT). A request is counted as a slow call if its response time is greater than this value. You must also set the slow call ratio that will trigger circuit breaking. After the rule is enabled, if the number of requests within a statistical time window is greater than the minimum number of requests and the slow call ratio exceeds your defined threshold, subsequent requests are automatically blocked for the specified circuit breaking period. After the circuit breaking period expires, the circuit breaker enters the half-open state. If the next single request's response time is less than the configured slow call RT, the circuit breaking ends. If the response time is greater than the slow call RT, the circuit is tripped again. If you select Error Ratio, you must set the threshold for the error ratio. When the rule is enabled, if the number of requests in a time window exceeds the minimum request count and the error ratio exceeds the threshold, the circuit breaker trips. During the subsequent circuit breaking period, all requests fail fast.
           Circuit breaking period	The duration for which the circuit remains open after tripping. During this period, all calls to the resource fail fast.
           Circuit Breaking Recovery Strategy	The recovery strategy for the circuit breaker when it enters the half-open state. Single probe recovery: After the circuit breaking period, the circuit breaker allows a single request to pass through for testing. If the request is successful (not a slow call or an error), the circuit is closed. Otherwise, it trips again.

        2. In the Configure Throttling Behavior step, you can click Add Behavior to customize the HTTP response when throttling is triggered.

     3. After you configure the throttling behavior, click New to create the rule.

Configure a throttling policy (legacy)

1. Log on to the MSE console.

2. In the left-side navigation pane, choose Cloud-native Gateway > Gateways. In the top navigation bar, select a region.

3. On the Gateways page, click the ID of the gateway.

4. In the navigation pane on the left, click routes > Route Settings. Find the route that you want to manage and click policies in the Actions column.

5. In the Policies section, click the throttling tab.

   • If you have not configured a throttling policy for the route, click Configure policy.

   • If a throttling policy is already configured, click the edit icon () next to the policy.

6. In the throttling threshold rule, set the Time Window and single-machine request threshold, and then click OK.

   The following describes the throttling parameters.

   • Time window: The time period for which the request threshold is measured. The default is 1 second.

   • Single-machine request threshold: The maximum number of requests allowed for a single gateway node within the specified time window.

7. After the throttling rule is configured, turn on the Enable switch.

   • Enabled: When the throttling policy is enabled, requests for this route are limited by the configured time window and request threshold.

   • Disabled: When the throttling policy is disabled, the number of requests for this route is not limited.

Verify the result

curl -I http://121.196.XX.XX/demo/item/list     # The IP address of the ingress gateway

• Response when the throttling policy is disabled:

  HTTP/1.1 200 OK
  x-content-type-options: nosniff
  x-xss-protection: 1; mode=block
  cache-control: no-cache, no-store, max-age=0, must-revalidate
  pragma: no-cache
  expires: 0
  x-frame-options: DENY
  content-type: application/json
  content-length: 86
  date: Mon, 29 Nov 2021 08:28:00 GMT
  x-envoy-upstream-service-time: 4
  server: istio-envoy

• Response when the throttling policy is enabled and triggered (HTTP status code 429 is returned):

  HTTP/1.1 429 Too Many Requests
  x-local-rate-limit: true
  content-length: 18
  content-type: text/plain
  date: Mon, 29 Nov 2021 08:28:01 GMT
  server: istio-envoy