Use the throttling plug-in to throttle APIs - API Gateway

This topic describes the configurations of throttling plug-ins and provides examples for some common scenarios.

1. Overview

A throttling plug-in limits the rate of API calls. You can throttle traffic based on the API, App (the AccessKey used for access), user (the owner of the app), or custom parameters.
A throttling plug-in supports two configuration templates:
- Parameter-based throttling configuration: This template is used to configure throttling based on custom parameters.
- Basic throttling configuration: This template is compatible with the throttling feature in the console.
The throttling feature is now part of the plug-in system. The existing throttling interface and console are still available. Throttling policies and throttling plug-ins are the same type of plug-in. If you bind a throttling plug-in to an API, the existing throttling policy becomes invalid.
When you create or modify throttling policies in the original throttling interface or console, modifications are synchronized to the plug-in system. However, the modifications that you make in the plug-in system cannot be synchronized to the throttling interface or console.

2. Basic throttling configurations

2.1 Throttling capability

2.1.1 Throttling thresholds

API traffic limit: The maximum number of calls allowed per unit of time for an API that is bound to the policy. The unit of time can be a second, minute, hour, or day. For example, 5,000 calls/minute.
APP traffic limit: The maximum number of calls allowed per unit of time from each app to any API that is bound to the policy. For example, 50,000 calls/hour.
User traffic limit: The maximum number of calls allowed per unit of time from each Alibaba Cloud account to any API that is bound to the policy. An Alibaba Cloud account can have multiple apps, and this limit applies to the total traffic from all apps that belong to the account. For example, 500,000 calls/day.

Note

You can specify all the preceding thresholds at the same time in one throttling plug-in. The user-level throttling threshold cannot be greater than the API-level throttling threshold, and the application-level throttling threshold cannot be greater than the user-level throttling threshold.

You can also add special applications or users to a throttling policy. This way, you can set a special throttling threshold for each of the applications or users. The special throttling threshold cannot be greater than the API-level throttling threshold.

2.1.2 Time units and throttling algorithms

The API Gateway throttling plug-in supports second (SECOND), minute (MINUTE), hour (HOUR), and day (DAY) as time units. The fixed time window algorithm is used for the minute, hour, and day units. The token bucket algorithm is used by default for the second unit. If the token bucket algorithm does not meet your needs, you can configure the fixed time window algorithm instead. The two algorithms are described as follows:

Token bucket algorithm: This is the default algorithm for second-level throttling. The token bucket algorithm has two concepts: a token bucket and a waiting queue. The API Gateway engine periodically adds tokens to the token bucket. When a client request arrives at API Gateway, it first attempts to retrieve a token from the bucket. If a token is retrieved, the request passes. If a token is not retrieved, the request enters a queue. The next batch of tokens is preferentially assigned to requests in the queue. If the queue is full, subsequent requests receive a 429 error. To prevent requests from entering the queue, you can configure blockingMode: QUICK_RETURN. When this is configured, if a request fails to obtain a token, the system immediately returns a 429 error.
Fixed time window algorithm: the algorithm that is used when minute, hour, or day is used as the time unit. This algorithm limits the total number of requests that are accepted within a unit of time. For example, a limit of 1,000 requests per minute specifies that the 429 code is returned for requests beyond the limit. The request count is reset at the beginning of the next minute.

2.2 Configure a basic throttling plug-in

You can configure your plug-in in JSON or YAML format. Both formats use the same schema. You can use a yaml to json conversion tool to convert between formats. The following code shows a template in YAML format.

---
unit: SECOND         # The unit of time. Valid values: SECOND, MINUTE, HOUR, and DAY.
apiDefault: 1000     # The default API-level throttling threshold.
controlMode: FIX_WINDOW  # Specifies that the fixed time window algorithm is used when the time unit is second.
blockingMode: QUICK_RETURN     # Specifies that 429 is returned for requests that fail to obtain a token when the token bucket algorithm is used.
userDefault: 30      # Optional. The default user-level throttling threshold. If you set this threshold to 0, user-level throttling is not performed. The user-level throttling threshold cannot be greater than the API-level throttling threshold.
appDefault: 30       # Optional. The default application-level throttling threshold. If you set this threshold to 0, application-level throttling is not performed. The application-level throttling threshold cannot be greater than the user-level throttling threshold.
specials:            # Optional. The special throttling settings. You can set throttling thresholds for special applications or users in a throttling policy.
  - type: "APP"      # The special throttling type. The value APP specifies that throttling is performed for special applications based on their AppKeys.
    policies:
    - key: 10123123  # The application ID. You can obtain the ID of an application from the application details page. To go to this page, choose Call API > Apps in the navigation pane on the left of the API Gateway console and click the name of the application in the application list.
      value: 10      # The special throttling threshold for the app. This threshold cannot be greater than the user-level throttling threshold in the throttling policy.
    - key: 10123123  # The application ID.
      value: 10      # The special throttling threshold for the app. This threshold cannot be greater than the user-level throttling threshold in the throttling policy.
  - type: "USER"     # The special throttling type. The value USER specifies that throttling is performed for special Alibaba Cloud accounts.
    policies:
    - key: 123455    # The ID of an Alibaba Cloud account. You can move the pointer over the profile picture in the upper-right corner of the Alibaba Cloud console to obtain the ID.
      value: 100     # The special throttling threshold for the Alibaba Cloud account. This threshold cannot be greater than the API-level throttling threshold in the throttling policy.

2.3 Use a plug-in dataset together with a basic throttling plug-in

2.3.1 Create a plug-in dataset

Log on to the API Gateway console. In the navigation pane on the left, click API Management > Plugin Management. Then, click the Plugin Datasets tab.
In the upper-right corner, click Create Dataset. In the dialog box that appears, enter a Name for the dataset. Set Type to TRAFFIC_CONTROL_POLICY. Click OK to create the dataset.
Go to the dataset that you created. In the upper-right corner, click Create Dataset Entry. On the page that appears, configure the key and value for the throttling plug-in. The key is the AppId or Alibaba Cloud account ID. The value is the corresponding throttling value.

2.3.2 Example on the use of a plug-in dataset for a throttling plug-in

Basic throttling plug-ins support plug-in datasets. You can use a plug-in dataset to configure special throttling thresholds for specific applications or accounts. The following code shows an example:

---
unit: SECOND         # The unit of time. Valid values: SECOND, MINUTE, HOUR, and DAY.
apiDefault: 1000     # The default API-level throttling threshold.
userDefault: 30      # Optional. The default user-level throttling threshold. If you set this threshold to 0, user-level throttling is not performed. The user-level throttling threshold cannot be greater than the API-level throttling threshold.
appDefault: 30       # Optional. The default application-level throttling threshold. If you set this threshold to 0, application-level throttling is not performed. The application-level throttling threshold cannot be greater than the user-level throttling threshold.
specials:            # Optional. The special throttling settings. You can set throttling thresholds for special applications or users in a throttling policy.
  - type: "APP"      # The special throttling type. The value APP specifies that throttling is performed for special applications based on their AppKeys.
    policyDatasetId: 87 b65008e92541938XXXXXXXX6eda5 # The ID of the plug-in dataset.
    policies:
    - key: 10123123  # The application ID. You can obtain the ID of an application from the application details page. To go to this page, choose Call API > Apps in the navigation pane on the left of the API Gateway console and click the name of the application in the application list.
      value: 10      # The special throttling threshold for the app. This threshold cannot be greater than the user-level throttling threshold in the throttling policy.
    - key: 10123123  # The application ID.
      value: 10      # The special throttling threshold for the app. This threshold cannot be greater than the user-level throttling threshold in the throttling policy.
  - type: "USER"     # The special throttling type. The value USER specifies that throttling is performed for special Alibaba Cloud accounts.
    policyDatasetId: 87 b65008eXXXXXXXXXXXXa236eda5 # The ID of the plug-in dataset.
    policies:
    - key: 123455    # The ID of an Alibaba Cloud account. You can move the pointer over the profile picture in the upper-right corner of the Alibaba Cloud console to obtain the ID.
      value: 100     # The special throttling threshold for the Alibaba Cloud account. This threshold cannot be greater than the API-level throttling threshold in the throttling policy.

Important

Plug-in datasets are supported only by basic throttling plug-ins. Parameter-based throttling plug-ins do not support plug-in datasets.

3. Parameter-based throttling configurations

Parameter-based throttling lets you control traffic based on the request parameters of callers and conditional executions. The following parameter-based throttling configurations are supported:

Supports throttling at the second, minute, hour, or day level.
Allows you to set conditions based on request parameters and system parameters to execute different throttling rules.
Allows you to set up throttling using a single parameter or a combination of multiple parameters.
Allows you to set the scope of throttling to an API or a plug-in.

3.1 Quick start with parameter-based throttling

Consider a scenario where you want to apply the following throttling rules for each client IP address. If a user authenticates with a key that has an AppId of 10001, set the throttle to 100 requests/second. For all other cases, set the throttle to 10 requests/second.

For this scenario, configure the plug-in as follows. This example uses yaml to configure the plug-in.

---
scope: "PLUGIN"
#
# This throttling depends on two system parameters:
# 1. The AppId used for the user's signature, obtained from the system parameter CaAppId.
# 2. The user's source ClientIP, obtained from the system parameter CaClientIp.
parameters:
  AppId: "System: CaAppId"
  ClientIP: "System: CaClientIp"
rules:
  # The first throttling policy. It takes effect when the `AppId` is `10001`. It throttles each ClientIP to `100/second`.
  - name: "Vip"
    condition: "$AppId = 10001"
    byParameters: "ClientIP"
    limit: 100
    period: SECOND
  # The second throttling policy is named `PerClientIP`. It throttles each ClientIP to `10/second`.
  - name: "PerClientIP"
    byParameters: "ClientIP"
    bypassEmptyValue: true   # When a rule in `rules` does not have a condition and uses the byParameters parameter as the condition, this rule does not take effect if the parameter is empty or an empty value is passed. The default throttling policy is used instead.
    limit: 10
    period: SECOND

3.2 Configure a parameter-based throttling plug-in

You can use yaml or an equivalent json format to configure the plug-in metadata.

---
scope: "PLUGIN"                # The scope of the throttling plug-in. Valid values: PLUGIN and API.
blockingMode: QUICK_RETURN     # Specifies that requests without tokens do not queue up. Instead, the 429 code is returned. For more information, see the following field description.
controlMode: FIX_WINDOW  # Specifies that the fixed time window algorithm is used when the time unit is second.
defaultLimit: 100              # The default throttling value, if a default value is specified.
defaultPeriod: SECOND          # The default unit of time for throttling.
defaultRetryAfterBySecond: 60  # Returns the Retry-After header, which gives advice on how long to wait before the next request.
defaultErrorMessage: "Throttled by 100/SECOND"
parameters:                    # The parameters that can be used for throttling.
  clientIp: "System:CaClientIp"
  userId: "Token:userId"
rules:
  - name: "ByClientIp"
    byParameters: "clientIp"
    condition: "$clientIp !in_cidr '61.7.XX.XX/24'"
    limit: 10
    period: MINUTE
    retryAfterBySecond: 60    # Returns the Retry-After header, which gives advice on how long to wait before the next request.
    errorMessage: "Throttled by 10/MINUTE from ${clientIp}"
  - name: "A maximum of 10 requests per minute are allowed for each user, except the administrator."
    byParameters: "clientIp"
    condition: "$userId !like 'admin%'"
    limit: 10
    period: MINUTE
    retryAfterBySecond: 60    # Returns the Retry-After header, which gives advice on how long to wait before the next request.
  - name: "A maximum of 10 requests per minute are allowed for each client IP address."
    byParameters: "clientIp"
    condition: "$clientIp in_cidr '67.0.XX.XX/8'"
    limit: 10
    period: MINUTE
  - name: "A maximum of 15 requests per minute are allowed for each user."
    condition: "$userId !like 'admin%'"
    limit: 15
    period: MINUTE
    byParameters: "clientIp"

Fields:

scope (Required): The scope of the throttling plug-in. Valid values are API and PLUGIN. If multiple APIs are bound to the same plug-in, the value of scope affects the scope of the throttling policy. For example, if a policy is set to 10 calls/second:
- If the value is API, the throttling policy takes effect for each API separately. In this example, the limit for each API is 10 calls/second.
- If the value is PLUGIN, all APIs bound to this plug-in share this limit. If the plug-in in this example is bound to a group of APIs, the total traffic limit for the group is 10 calls/second.
parameters (Required): A list of parameters involved in throttling. For more information, see the description in Use parameters and conditional expressions.
rules (Optional): A list of throttling policies. If defaultLimit and period are not set, this field cannot be empty. Each throttling policy contains the following fields:
- name (Required): The name of the throttling policy. The value must be a valid string that matches [A-Za-z0-9_-]+ and must be unique within the same plug-in.
- byParameters (Required): The throttling parameters. If you use a combination of multiple parameters, separate them with a comma (,). For example, ClientIP means that throttling is performed separately for each value of ClientIP. UserId,Action means that throttling is performed separately for the combined values of these two parameters.
- bypassEmptyValue (Optional): If set to true, this rule does not take effect if the `byParameters` parameter is empty or an empty value is passed. This applies only when the rule does not have a `condition` and uses `byParameters` as the condition. In this case, the default throttling policy is used instead.
- condition (Optional): If a condition is set, this throttling policy is executed only if the condition is met.
- limit (Required): The throttling value. This must be a positive integer. A value of -1 means no throttling is applied if this condition is met.
- period (Required): The throttling period. Valid values: SECOND, MINUTE, HOUR, or DAY.
- errorMessage (Optional): A custom error message. You can define it as a template. Parameters defined in parameters can be referenced using the ${Name} format.
- retryAfterBySecond (Optional): Returns the Retry-After header, which suggests how long to wait before making the next request.
defaultLimit (Optional): The default throttling value. This must be a positive integer.
defaultPeriod (Optional): The throttling period. Valid values: SECOND, MINUTE, HOUR, or DAY.
defaultErrorMessage (Optional): A custom error message. When a custom error message is configured, the returned X-Ca-Error-Message header uses the custom message. This message cannot use parameters.
defaultRetryAfterBySecond (Optional): The default returned Retry-After header, which suggests how long to wait before making the next request.
blockingMode (Optional): API Gateway uses the standard token bucket algorithm to implement throttling. The following list describes the field values and the algorithm.
- QUEUE (Default): The token bucket algorithm has two concepts: a token bucket and a waiting queue. The API Gateway engine periodically adds tokens to the token bucket. When a client request arrives at API Gateway, it first attempts to retrieve a token from the bucket. If a token is retrieved, the request passes. If a token is not retrieved, the request enters a queue. The next batch of tokens is preferentially assigned to requests in the queue. If the queue is full, subsequent requests receive a 429 error.
- QUICK_RETURN: If you use this value, requests do not wait for tokens in a queue. Instead, the 429 code is returned.
controlMode: Specifies the throttling algorithm to use when the time unit is `SECOND`. The default value is TOKEN_BUCKET (token bucket algorithm). You can set it to FIX_WINDOW (fixed time window algorithm).

3.3 Parameters supported by a throttling plug-in

The following table describes the parameters that are supported by a throttling plug-in.

Location Name	Scope	Description
Method	Request	The HTTP request method (uppercase), such as `GET` or `POST`.
Path	Request	The full HTTP request path, such as `/path/to/query`.
Header	Request	Use `Header:{Name}` to get the first value of the HTTP header named `{Name}`.
Query	Request	Use `Query:{Name}` to get the first value of the parameter named `{Name}` in the query string.
Form	Request	Use `Form:{Name}` to get the first value of the parameter named `{Name}` in the request form.
Host	Request	Use `Host:{Name}` to get the matched wildcard domain name template parameters.
Parameter	Request	Use `Parameter:{Name}` to get the first value of the custom API parameter named `Name`.
System	Request	Use `System:{Name}` to get the value of the system parameter named `{Name}`.
Token	Request	In `jwt` and `oauth2` authorization scenarios, use `Token:{Name}` to get the value of the claim named `{Name}` in the token.

3.4 Execution rules

API Gateway performs parameter-based throttling using the following rules:

The plug-in uses the parameters configuration to retrieve the parameter values from the request context.
All rules for which the condition evaluates to true or for which no condition is configured are executed.
If multiple matching rules in the list have the same byParameters configuration, only the first rule in the configuration order is executed. The other rules do not take effect.

4. Examples

4.1 Basic throttling

Basic throttling supports API-level, AppKey-based, and user-level throttling.

---
unit: SECOND           # The default unit of time for throttling. Valid values: SECOND, MINUTE, HOUR, and DAY.
apiDefault: 50         # The default API-level throttling threshold.
defaultRetryAfterBySecond: 60  # By default, the Retry-After header is returned, which provides advice on how long to wait before the next request.
appDefault: 20         # Optional. The default application-level throttling threshold. This threshold cannot be greater than the user-level throttling threshold.
userDefault: 30        # Optional. The default user-level throttling threshold. This threshold cannot be greater than the API-level throttling threshold.
specials:              # Optional. The special throttling settings. You can set throttling thresholds for special applications or users in a throttling policy.
  - type: "APP"        # The special throttling type. The value APP indicates that throttling is performed for special applications based on their AppKeys.
    policies:
      - key: 10001     # The application ID. You can obtain the ID of an application from the application details page. To go to this page, choose Call API > Apps in the navigation pane on the left of the API Gateway console and click the name of the application in the application list.
        value: 3       # The special throttling threshold for the application. This threshold cannot be greater than the user-level throttling threshold.
      - key: 10003
        value: 40
  - type: "USER"       # The special throttling type. The value USER indicates that throttling is performed for special Alibaba Cloud accounts.
    policies:
      - key: 102       # The ID of an Alibaba Cloud account. You can move the pointer over the profile picture in the upper-right corner of the Alibaba Cloud Management Console to obtain the ID.
        value: 10      # The special throttling threshold for the Alibaba Cloud account. This threshold cannot be greater than the API-level throttling threshold.
      - key: 233
        value: 35

4.2 Parameter-based throttling

This example shows how to configure a rate limiting policy.

Each source IP is allowed 100 calls/minute.
If the client IP is in the 58.66.XX.XX/24 range, access is not restricted.
For client IPs in the 63.0.XX.XX and 73.0.XX.XX/24 ranges, access is limited to 5 calls/day.

---
scope: API             # The throttling scope. Valid values: API and PLUGIN.
parameters:            # The parameters that are used for throttling. Throttling is performed only based on the ClientIP parameter. You can obtain the values of this parameter from the CaClientIp system parameter.
  ClientIp: "System:CaClientIp"
rules:
  - name: whitelist    # The whitelist policy. Throttling is not performed for client IP addresses that meet the condition specified in this policy.
    condition: "$ClientIp in_cidr '58.66.XX.XX/24'"
    limit: -1          # The value -1 indicates that throttling is not performed.
  - name: banList      # The special throttling policy. If a client IP address meets the condition specified in this policy, a maximum of five API calls per day are allowed for the client IP address.
    condition: "$ClientIp in_cidr '63.0.XX.XX' or $ClientIp in_cidr '73.0.XX.XX/24'"  
    byParameters: "ClientIp"
    limit: 5
    period: DAY
  - name: 100perIp     # The default throttling policy. A maximum of 100 API calls per minute are allowed for each client IP address.
    byParameters: "ClientIp"
    limit: 100
    period: MINUTE     # The unit of time. Valid values: SECOND, MINUTE, HOUR, and DAY.

4.3 Anti-CC attack configurations

This section provides an example on how to configure a throttling plug-in to prevent challenge collapsar (CC) attacks.

Each source IP is allowed 3 calls/second.
If a source IP exceeds the 3 calls/second limit, access is blocked for 10 seconds.

---
scope: API             # The throttling scope. Valid values: API and PLUGIN.
defaultLimit: 3000        # The default throttling threshold.
defaultPeriod: SECOND     # The default unit of time for throttling.
defaultRetryAfterBySecond: 60  # By default, the Retry-After header is returned, which provides advice on how long to wait before the next request.
parameters:        # The parameters that are used for throttling. Throttling is performed only based on the ClientIP parameter. You can obtain the values of this parameter from the CaClientIp system parameter.
  clientIp: "system:CaClientIp"
rules:
  -name: "Each client IP address can initiate three API calls per second. If this threshold is exceeded, the client IP address is blocked for 10 seconds."
    byParameters: "clientIp"
    limit: 3
    period: SECOND
    blockingPeriodBySecond: 10     # Only effective for dedicated instances.

5. Error codes

Error codes	HTTP status code	Error message	Description
T429ID	429	Throttled by INNER DOMAIN Flow Control, ${Domain} is a test domain, only 1000 requests per day	When you use a default second-level domain name for access, the limit is 1,000 calls/day (100 calls/day for regions outside China and in China (Hong Kong)). Bind a custom domain name to remove this limit.
T429IN	429	Throttled by INSTANCE Flow Control	The throttling limit for the current instance is triggered.
T429GR	429	Throttled by GROUP Flow Control	The throttling limit for the current group is triggered.
T429PA	429	Throttled by API Flow Control	The default API throttling on the plug-in is triggered.
T429PR	429	Throttled by PLUGIN Flow Control	A special throttling rule in the plug-in is triggered.

6. Limits

A maximum of 16 parameters can be defined.
A single expression cannot exceed 512 characters.
The metadata of a single plug-in cannot exceed 50 KB in size.
Each plug-in can have a maximum of 16 rules.
Each rule can have a maximum of three parameters in byParameters.
If throttling parameters have many unique values, for example, when you use day-level throttling based on source IPs, the system may release some throttling records to manage memory usage. For a serverless instance, a parameter-based throttling plug-in allows a maximum of 1,000 distinct parameter values. For a dedicated instance, the plug-in allows a maximum of 100,000 distinct parameter values.