API Gateway:Throttling plug-in

1. Overview

• A throttling plug-in is used to control traffic based on APIs, applications, application owners, or another custom parameter. You can control the traffic of applications based on the AppKeys of the applications.

• A throttling plug-in supports two configuration templates:

  • Configuration template for parameter-based throttling: This template is used to configure custom parameter-based throttling.

  • Configuration template for basic throttling: This template is compatible with the throttling feature in the API Gateway console.

• The throttling feature is integrated into the plug-in system. The original throttling interface and console are still in use. Throttling policies and throttling plug-ins belong to the same plug-in type. If you bind a throttling plug-in to an API, the throttling policies that are bound to this API become invalid.

• When you create or modify throttling policies in the original throttling interface or console, modifications are synchronized to the plug-in system. However, the modifications that you make in the plug-in system cannot be synchronized to the throttling interface or console.

2. Basic throttling configurations

2.1 Throttling capability

2.1.1 Throttling thresholds

• API-level throttling threshold: the maximum number of times that an API can be called within a specific unit of time. This unit of time can be second, minute, hour, or day. For example, you can set this threshold to 5,000 times per minute.

• Application-level throttling threshold: the maximum number of times that each application can call an API within a specific unit of time. For example, you can set this threshold to 50,000 times per hour.

• User-level throttling threshold: the maximum number of times that each Alibaba Cloud account can call an API within a specific unit of time. An Alibaba Cloud account may have multiple applications. This throttling threshold applies to the total number of API calls that all the applications within this account can initiate. For example, you can set this threshold to 500,000 times per day.

You can also add special applications or users to a throttling policy. This way, you can set a special throttling threshold for each of the applications or users. The special throttling threshold cannot be greater than the API-level throttling threshold.

2.1.2 Time units and throttling algorithms

The throttling plug-in supports second, minute, hour, and day as the time units for throttling. When minute, hour, or day is used as the time unit, the fixed time window algorithm is used. When second is used as the time unit, you can change the default token bucket algorithm to the fixed time window algorithm based on your business requirements. The following items describe the algorithms:

• Token bucket algorithm: the default algorithm that is used when second is used as the time unit. This algorithm involves two concepts: token bucket and waiting queue. API Gateway periodically issues tokens to the token bucket. Requests must obtain tokens from the bucket before they can be processed by API Gateway. If they fail to obtain a token, they wait in a queue to be preferentially allocated a token when tokens become available. When the queue is full, the 429 code is returned for subsequent requests. If you do not want the requests of your callers to wait in the queue, you can configure the blockingMode: QUICK_RETURN parameter, in which case the system directly returns 429 for requests that fail to obtain a token without making them wait in a queue.

• Fixed time window algorithm: the algorithm that is used when minute, hour, or day is used as the time unit. This algorithm limits the total number of requests that are accepted within a unit of time. For example, a limit of 1,000 requests per minute specifies that the 429 code is returned for requests beyond the limit. The request count is reset at the beginning of the next minute.

2.2 Configure a basic throttling plug-in

You can configure a throttling plug-in in JSON or YAML format. The two formats have the same schema and can be converted to each other by using a conversion tool. The following code describes a plug-in configuration template in YAML format:

---
unit: SECOND         # The unit of time. Valid values: SECOND, MINUTE, HOUR, and DAY.
apiDefault: 1000     # The default API-level throttling threshold.
controlMode: FIX_WINDOW  # Specifies that the fixed time window algorithm is used when the time unit is second.
blockingMode: QUICK_RETURN     # Specifies that 429 is returned for requests that fail to obtain a token when the token bucket algorithm is used.
userDefault: 30      # Optional. The default user-level throttling threshold. If you set this threshold to 0, user-level throttling is not performed. The user-level throttling threshold cannot be greater than the API-level throttling threshold.
appDefault: 30       # Optional. The default application-level throttling threshold. If you set this threshold to 0, application-level throttling is not performed. The application-level throttling threshold cannot be greater than the user-level throttling threshold.
specials:            # Optional. The special throttling settings. You can set throttling thresholds for special applications or users in a throttling policy.
  - type: "APP"      # The special throttling type. The value APP specifies that throttling is performed for special applications based on their AppKeys.
    policies:
    - key: 10123123  # The application ID. You can obtain the ID of an application from the application details page. To go to this page, choose Call API > Apps in the left-side navigation pane of the API Gateway console and click the name of the application in the application list.
      value: 10      # The special throttling threshold for the application. This threshold cannot be greater than the user-level throttling threshold in the throttling policy.
    - key: 10123123  # The application ID.
      value: 10      # The special throttling threshold for the application. This threshold cannot be greater than the user-level throttling threshold in the throttling policy.
  - type: "USER"     # The special throttling type. The value USER specifies that throttling is performed for special Alibaba Cloud accounts.
    policies:
    - key: 123455    # The ID of an Alibaba Cloud account. You can move the pointer over the profile picture in the upper-right corner of the Alibaba Cloud console to obtain the ID.
      value: 100     # The special throttling threshold for the Alibaba Cloud account. This threshold cannot be greater than the API-level throttling threshold in the throttling policy.

2.3 Use a plug-in dataset together with a basic throttling plug-in

2.3.1 Create a plug-in dataset

1. Log on to the API Gateway console. In the left-side navigation pane, choose Manage APIs > Plug-ins. Then, click the Custom Dataset tab.

2. Click Create Dataset in the upper-right corner. In the dialog box that appears, specify a name for the dataset, select TRAFFIC_CONTROL_POLICY for the dataset type, and then click Confirm.

3. On the Custom Dataset tab, click the created dataset. On the dataset details page, click Create Dataset Entry in the upper-right corner. In the Create Dataset Entry dialog box that appears, specify the Key, Value, and Validity Period parameters. For the Key parameter, you must enter the ID of the application or Alibaba Cloud account. For the Value parameter, you must enter the throttling threshold.

2.3.2 Example on the use of a plug-in dataset for a throttling plug-in

Basic throttling plug-ins support plug-in datasets. You can use a plug-in dataset to configure special throttling thresholds for specific applications or accounts. Example:

---
unit: SECOND         # The unit of time. Valid values: SECOND, MINUTE, HOUR, and DAY.
apiDefault: 1000     # The default API-level throttling threshold.
userDefault: 30      # Optional. The default user-level throttling threshold. If you set this threshold to 0, user-level throttling is not performed. The user-level throttling threshold cannot be greater than the API-level throttling threshold.
appDefault: 30       # Optional. The default application-level throttling threshold. If you set this threshold to 0, application-level throttling is not performed. The application-level throttling threshold cannot be greater than the user-level throttling threshold.
specials:            # Optional. The special throttling settings. You can set throttling thresholds for special applications or users in a throttling policy.
  - type: "APP"      # The special throttling type. The value APP specifies that throttling is performed for special applications based on their AppKeys.
    policyDatasetId: 87 b65008e92541938XXXXXXXX6eda5 # The ID of the plug-in dataset.
    policies:
    - key: 10123123  # The application ID. You can obtain the ID of an application from the application details page. To go to this page, choose Call APIs > Apps in the left-side navigation pane of the API Gateway console and click the name of the application in the application list.
      value: 10      # The special throttling threshold for the application. This threshold cannot be greater than the user-level throttling threshold in the throttling policy.
    - key: 10123123  # The application ID.
      value: 10      # The special throttling threshold for the application. This threshold cannot be greater than the user-level throttling threshold in the throttling policy.
  - type: "USER"     # The special throttling type. The value USER specifies that throttling is performed for special Alibaba Cloud accounts.
    policyDatasetId: 87 b65008eXXXXXXXXXXXXa236eda5 # The ID of the plug-in dataset.
    policies:
    - key: 123455    # The ID of an Alibaba Cloud account. You can move the pointer over the profile picture in the upper-right corner of the Alibaba Cloud console to obtain the ID.
      value: 100     # The special throttling threshold for the Alibaba Cloud account. This threshold cannot be greater than the API-level throttling threshold in the throttling policy.

3. Parameter-based throttling configurations

Parameter-based throttling allows you to control traffic based on the request parameters of callers and conditional executions. The following parameter-based throttling configurations are supported:

• Throttling at the second, minute, hour, or day level is supported.

• You can use request and system parameters to set conditions and implement different throttling thresholds.

• You can configure throttling by using a single parameter or a combination of multiple parameters.

• You can set the throttling scope to API or PLUGIN.

3.1 Quick start with parameter-based throttling

This section uses a scenario where the following throttling policies apply as an example: the traffic is throttled to 100 requests per second for an application whose ID is 10001 and to 10 requests per second for other applications.

For this scenario, a throttling plug-in in YAML format is configured.

---
scope: "PLUGIN"
#
# Throttling depends on two system parameters.
# 1. The AppId value in the signature. The application ID can be obtained from the CaAppId system parameter.
# 2. The client IP address of the caller, which can be obtained from the CaClientIp system parameter.
parameters:
  AppId: "System: CaAppId"
  ClientIP: "System: CaClientIp"
rules:
  # The first throttling policy. This policy applies to the application whose ID is 10001. The throttling threshold for each client IP address is 100 requests per second.
  - name: "Vip"
    condition: "$AppId = 10001"
    byParameters: "ClientIP"
    value: 100
    period: SECOND
  # The second throttling policy named PerClientIP. The throttling threshold for each client IP address is 10 requests per second.
  - name: "PerClientIP"
    byParameters: "ClientIP"
    bypassEmptyValue: true   # If the condition parameter is left empty in the rules section, the system throttles based on the value of the byParameters parameter. If this parameter is also left empty, the corresponding policy does not take effect and the system throttles based on the default throttling policy.
    value: 10
    period: SECOND

3.2 Configure a parameter-based throttling plug-in

The YAML or equivalent JSON format is used to configure the metadata of the throttling plug-in.

---
scope: "PLUGIN"                # The scope of the throttling plug-in. Valid values: PLUGIN and API.
blockingMode: QUICK_RETURN     # Specifies that requests without tokens do not queue up. Instead, the 429 code is returned. For more information, see the following field description.
controlMode: FIX_WINDOW  # Specifies that the fixed time window algorithm is used when the time unit is second.
defaultLimit: 100              # The default throttling value, if a default value is specified.
defaultPeriod: SECOND          # The default unit of time for throttling.
defaultRetryAfterBySecond: 60  # Returns the Retry-After header, which gives advice on how long to wait before the next request.
defaultErrorMessage: "Throttled by 100/SECOND"
parameters:                    # The parameters that can be used for throttling.
  clientIp: "System:CaClientIp"
  userId: "Token:userId"
rules:
  - name: "ByClientIp"
    byParameters: "clientIp"
    condition: "$clientIp !in_cidr '61.7.XX.XX/24'"
    limit: 10
    period: MINUTE
    retryAfterBySecond: 60    # Returns the Retry-After header, which gives advice on how long to wait before the next request.
    errorMessage: "Throttled by 10/MINUTE from ${clientIp}"
  - name: "A maximum of 10 requests per minute are allowed for each user, except the administrator."
    byParameters: "clientIp"
    condition: "$userId !like 'admin%'"
    limit: 10
    period: MINUTE
    retryAfterBySecond: 60    # Returns the Retry-After header, which gives advice on how long to wait before the next request.
  - name: "A maximum of 10 requests per minute are allowed for each client IP address."
    byParameters: "clientIp"
    condition: "$clientIp in_cidr '67.0.XX.XX/8'"
    limit: 10
    period: MINUTE
  - name: "A maximum of 15 requests per minute are allowed for each user."
    condition: "$userId !like 'admin%'"
    limit: 15
    period: MINUTE
    byParameters: "clientIp"

Fields:

• scope: required. The scope of the throttling plug-in. Valid values: API and PLUGIN. If a throttling plug-in is bound to multiple APIs, the setting of the scope field affects the scope of throttling rules bound to the APIs. For example, a throttling policy is 10 requests per second.

  • If the value of this field is API, the throttling threshold for each API is 10 requests per second.

  • If the value of this field is PLUGIN, the total throttling threshold for all the APIs is 10 requests per second.

• parameters: required. The parameters that are used for throttling. For more information, see Use parameters and conditional expressions.

• rules: optional. The throttling policies. If defaultLimit and defaultPeriod are not specified, the rules section must be specified. Each throttling policy contains the following fields:

  • name: required. The name of the throttling policy. The value of this field must comply with the [A-Za-z0-9_-]+ regular expression. The name of each throttling policy must be unique in a throttling plug-in.

  • byParameters: required. The parameter that is used for throttling. If multiple parameters are specified, separate them with commas (,). If the value of this field is clientIp, throttling is performed based on the value of the clientIp parameter. If the value of this field is userId,action, throttling is performed based on the combined values of the two parameters.

  • bypassEmptyValue: optional. If the condition parameter is left empty in the rules section, the system throttles based on the value of the byParameters parameter. If this parameter is also left empty, the corresponding policy does not take effect and the system throttles based on the default throttling policy.

  • condition: optional. If you specify this field, the related throttling policy is used only when the condition indicated by this field is met.

  • limit: required. The throttling threshold. The value must be a positive integer. If you set this field to -1, throttling is not performed even if the specified condition is met.

  • period: required. The unit of time for throttling. Valid values: SECOND, MINUTE, HOUR, and DAY.

  • errorMessage: optional. The error message. You can use a template to customize error messages. For parameters specified in the parameters section, you can define error messages for them by using the ${Name} format.

  • retryAfterBySecond: optional. Returns the Retry-After header, which provides recommendations on how long to wait before the next request.

• defaultLimit: optional. The default throttling threshold. The value must be a positive integer.

• defaultPeriod: optional. The unit of time for throttling. Valid values: SECOND, MINUTE, HOUR, and DAY.

• defaultErrorMessage: optional. The custom error message. If this field is specified, the returned X-Ca-Error-Message header contains the custom error message. Do not specify any parameter in the message.

• defaultRetryAfterBySecond: optional. Returns the Retry-After header, which provides recommendations on how long to wait before the next request.

• blockingMode: API Gateway uses the standard token bucket algorithm to throttle requests. The following items describe the field values and the algorithm:

  • QUEUE: the default value. The token bucket algorithm involves two concepts: token bucket and waiting queue. API Gateway periodically sends tokens to the token bucket. Requests must obtain tokens before they can be accepted by API Gateway. If they fail to obtain a token, they wait in a queue to be preferentially allocated a token when tokens become abundant. If the queue is full, the 429 code is returned for subsequent requests.

  • QUICK_RETURN: If you use this value, requests do not wait for tokens in a queue. Instead, the 429 code is returned.

• controlMode: If the time unit is second, you can use this parameter to specify the throttling algorithm. The default value is TOKEN_BUCKET, which indicates the token bucket algorithm. You can set this parameter to FIX_WINDOW, which indicates the fixed time window algorithm.

3.3 Parameters supported by a throttling plug-in

The following table describes the parameters that are supported by a throttling plug-in.

3.4 Execution rules

API Gateway performs parameter-based throttling by using the following rules:

• Plug-ins use the parameters field to obtain parameters from the context of a request.

• If all specified conditions are met or no conditions are specified in a throttling policy, the policy is executed.

• If multiple matched policies have the same byParameters setting, only the policy that comes first in the throttling plug-in takes effect.

4. Examples

4.1 Basic throttling

You can perform basic throttling for APIs, applications, or users. Throttling for applications is based on their AppKeys.

---
unit: SECOND           # The default unit of time for throttling. Valid values: SECOND, MINUTE, HOUR, and DAY.
apiDefault: 50         # The default API-level throttling threshold.
defaultRetryAfterBySecond: 60  # By default, the Retry-After header is returned, which provides advice on how long to wait before the next request.
appDefault: 20         # Optional. The default application-level throttling threshold. This threshold cannot be greater than the user-level throttling threshold.
userDefault: 30        # Optional. The default user-level throttling threshold. This threshold cannot be greater than the API-level throttling threshold.
specials:              # Optional. The special throttling settings. You can set throttling thresholds for special applications or users in a throttling policy.
  - type: "APP"        # The special throttling type. The value APP indicates that throttling is performed for special applications based on their AppKeys.
    policies:
      - key: 10001     # The application ID. You can obtain the ID of an application from the application details page. To go to this page, choose Call API > Apps in the left-side navigation pane of the API Gateway console and click the name of the application in the application list.
        value: 3       # The special throttling threshold for the application. This threshold cannot be greater than the user-level throttling threshold.
      - key: 10003
        value: 40
  - type: "USER"       # The special throttling type. The value USER indicates that throttling is performed for special Alibaba Cloud accounts.
    policies:
      - key: 102       # The ID of an Alibaba Cloud account. You can move the pointer over the profile picture in the upper-right corner of the Alibaba Cloud Management Console to obtain the ID.
        value: 10      # The special throttling threshold for the Alibaba Cloud account. This threshold cannot be greater than the API-level throttling threshold.
      - key: 233
        value: 35

4.2 Parameter-based throttling

This section provides a client IP address-based throttling plug-in. In this example, the following throttling policies are used:

• A maximum of 100 API calls per minute are allowed for each client IP address.

• If a client IP address falls in the 58.66.XX.XX/24 CIDR block, throttling is not performed for this client IP address.

• If a client IP address falls in the 63.0.XX.XX or 73.0.XX.XX/24 CIDR block, a maximum of 5 API calls per day are allowed for this client IP address.

---
scope: API             # The throttling scope. Valid values: API and PLUGIN.
parameters:            # The parameters that are used for throttling. Throttling is performed only based on the ClientIP parameter. You can obtain the values of this parameter from the CaClientIp system parameter.
  ClientIp: "System:CaClientIp"
rules:
  - name: whitelist    # The whitelist policy. Throttling is not performed for client IP addresses that meet the condition specified in this policy.
    condition: "$ClientIp in_cidr '58.66.XX.XX/24'"
    limit: -1          # The value -1 indicates that throttling is not performed.
  - name: banList      # The special throttling policy. If a client IP address meets the condition specified in this policy, a maximum of five API calls per day are allowed for the client IP address.
    condition: "$ClientIp in_cidr '63.0.XX.XX' or $ClientIp in_cidr '73.0.XX.XX/24'"  
    byParameters: "ClientIp"
    limit: 5
    period: DAY
  - name: 100perIp     # The default throttling policy. A maximum of 100 API calls per minute are allowed for each client IP address.
    byParameters: "ClientIp"
    limit: 100
    period: MINUTE     # The unit of time. Valid values: SECOND, MINUTE, HOUR, and DAY.

4.3 Anti-CC attack configurations

This section provides an example on how to configure a throttling plug-in to prevent challenge collapsar (CC) attacks.

• A maximum of 3 API calls per second are allowed for each client IP address.

• When the threshold of 3 calls per second is exceeded, access from the client IP address is blocked for 10 seconds.

---
scope: API             # The throttling scope. Valid values: API and PLUGIN.
defaultLimit: 3000        # The default throttling threshold.
defaultPeriod: SECOND     # The default unit of time for throttling.
defaultRetryAfterBySecond: 60  # By default, the Retry-After header is returned, which provides advice on how long to wait before the next request.
parameters:        # The parameters that are used for throttling. Throttling is performed only based on the ClientIP parameter. You can obtain the values of this parameter from the CaClientIp system parameter.
  clientIp: "system:CaClientIp"
rules:
  -name: "Each client IP address can initiate three API calls per second. If this threshold is exceeded, the client IP address is blocked for 10 seconds."
    byParameters: "clientIp"
    limit: 3
    period: SECOND
    blockingPeriodBySecond: 10     # Only effective for dedicated instances.

5. Error codes

6. Limits

• A maximum of 16 parameters can be specified in a throttling plug-in.

• Each conditional expression can contain a maximum of 512 characters.

• The metadata of a single plug-in cannot exceed 50 KB in size.

• A maximum of 16 throttling policies can be defined in a throttling plug-in.

• A maximum of three parameters can be specified in the byParameters field of each throttling policy.

• If day-level throttling is specified for client IP addresses and the system stores excessive throttling records, some records are released. For a shared instance, a throttling plug-in can control traffic based on a maximum of 1,000 parameters. For a dedicated instance, a throttling plug-in can control traffic based on a maximum of 100,000 parameters.