API Gateway:Circuit breaker plug-ins (for dedicated instances only)

Limits

• Dedicated instances only.

• Maximum 512 characters per expression.

• Maximum plug-in configuration size: 50 KB.

1. Overview

By default, if 1,000 timeouts occur at an API backend within 30 seconds, the circuit breaker trips and enters the open state for 90 seconds. During this period, all requests return Status=503 with error code X-Ca-Error-Code=D503CB. After 90 seconds, the circuit breaker enters the half-open state and allows a small number of requests through. If the backend recovers, the circuit breaker closes and requests resume normally.

On a dedicated instance, the circuit breaker plugin lets you customize the following circuit breaker settings:

• Trip condition: the circuit breaker trips when backend timeouts or specified errors exceed a threshold within a time window.

• Time window for evaluating trip conditions.

• Open duration after the circuit breaker trips.

• Fallback backend for requests while the circuit breaker is open.

2. Configurations

Circuit breaker plug-in configurations apply only to APIs on dedicated instances. On serverless instances, the default circuit breaker configuration is used even if a plug-in is bound to the API.

2.1 Configuring degradation policies based on backend timeouts

Configure a degradation policy based on backend timeouts. A request counts as a timeout if the backend does not respond within the API-defined timeout period.

timeoutThreshold: 15         # The threshold of the number of occurrences of timeout at the backend.
windowInSeconds: 30          # The time window during which the number of occurrences of timeout at the backend is checked by the circuit breaker to determine whether to trip.
openTimeoutSeconds: 15       # The period of time during which the circuit breaker stays open after it trips.
downgradeBackend:            # The backend to which API requests are directed when the circuit breaker is open.
  type: mock
  statusCode: 418

Fields:

• timeoutThreshold: the number of backend timeouts that triggers the circuit breaker. Maximum value: 5000. A value that is too small causes frequent trips.

• windowInSeconds: the evaluation time window. Valid values: 10 to 90. Unit: seconds.

• openTimeoutSeconds: the duration the circuit breaker stays open. Valid values: 15 to 300. Unit: seconds.

• downgradeBackend: optional. The fallback backend used while the circuit breaker is open.

2.2 Configuring a degradation policy based on backend response time

Configure a degradation policy based on backend response time, measured from when the gateway sends a request to when it receives a response.

errorThreshold: 10         # The threshold of the number of occurrences of long response at the backend.
windowInSeconds: 60          # The time window during which the number of occurrences of long response at the backend is checked by the circuit breaker to determine whether to trip.
openTimeoutSeconds: 120        # The period of time during which the circuit breaker stays open after it trips.
errorCondition: "$LatencyMilliSeconds > 500"     # The conditional expression that is used to determine whether the backend response is counted as a long response. In this example, if the backend response time exceeds 500 ms, the response is considered as a long response.
downgradeBackend:               # The backend to which API requests are directed when the circuit breaker is open.
  type: mock
  statusCode: 403

Fields:

• errorThreshold: the number of slow responses that triggers the circuit breaker.

• windowInSeconds: the evaluation time window. Valid values: 10 to 90. Unit: seconds.

• openTimeoutSeconds: the duration the circuit breaker stays open. Valid values: 15 to 300. Unit: seconds.

• errorCondition: the expression that defines a slow response. Available variables: $LatencyMilliSeconds (milliseconds) and $LatencySeconds (seconds).

• downgradeBackend: optional. The fallback backend used while the circuit breaker is open.

2.3 Configure a degradation policy based on backend errors

Configure a degradation policy based on backend error codes.

errorCondition: "$StatusCode == 503"  # The conditional expression that specifies the error whose number of occurrences is checked by the circuit breaker to determine whether to trip.
errorThreshold: 1000                  # The threshold of the number of occurrences of the specified error.
windowInSeconds: 30                   # The time window during which the number of occurrences of the specified error at the backend is checked by the circuit breaker to determine whether to trip.
openTimeoutSeconds: 15                # The period of time during which the circuit breaker stays open after it trips.
downgradeBackend:                     # The backend to which API requests are directed when the circuit breaker is open.
  type: "HTTP"
  address: "http://api.foo.com"
  path: "/system-busy.json"
  method: GET

• errorCondition: the error expression. Available variables: $StatusCode (response code) and $LatencySeconds (latency in seconds).

  • For example, the expression $StatusCode = 503 or $StatusCode = 504 evaluates to true if the status code of the backend response is 503 or 504.

  • For example, $LatencySeconds > 30 indicates that the timeout exceeds 30 seconds.

• errorThreshold: the number of matching errors that triggers the circuit breaker.

• windowsInSeconds: the evaluation time window. Valid values: 10 to 90. Unit: seconds.

• openTimeoutSeconds: the duration the circuit breaker stays open. Valid values: 15 to 300. Unit: seconds.

• downgradeBackend: optional. The fallback backend used while the circuit breaker is open.

2.4 Accurate status control

API Gateway runs on multiple cluster nodes, each maintaining its own circuit breaker state. This can cause status inaccuracy across nodes. To enable globally consistent circuit breaker status, add the useGlobalState field to your plug-in configuration:

---
timeoutThreshold: 15 # The threshold of the number of occurrences of timeout at the backend.
windowInSeconds: 30 # The time window during which the number of occurrences of timeout at the backend is checked by the circuit breaker to determine whether to trip.
openTimeoutSeconds: 15 # The period of time during which the circuit breaker stays open after it trips.
useGlobalState: true # Accurate status control is enabled.
downgradeBackend: # The backend to which API requests are directed when the circuit breaker is open.
 type: mock
 statusCode: 302
 body: |
 <result>
 <errorCode>I's a teapot</errorCode>
 </result>

The default value of useGlobalState is false. Setting it to true enables accurate status at the cost of minor performance overhead, which does not affect the guaranteed QPS and SLA of your instance.

2.5 Configure a degradation policy by percentage

The circuit breaker trips when any of the following four conditions is met. All conditions have equal priority.

• errorThreshold: the number of backend errors (matching errorCondition) that triggers the circuit breaker.

• timeoutThreshold: the number of backend timeouts that triggers the circuit breaker.

• errorThresholdByPercent: the error rate threshold, evaluated against the error rate of the previous time window.

• timeoutThresholdByPercent: the timeout rate threshold as a percentage of total requests in a time window.

Example:

---
windowInSeconds: 3  # The time window during which the circuit breaker determines whether to trip. Valid values: 10 to 90. Unit: seconds.
openTimeoutSeconds: 3
errorThreshold: 90  # The threshold of the number of occurrences of the specified error.
timeoutThreshold: 90   # The threshold of the number of occurrences of timeout.
errorThresholdByPercent: 20    # The threshold of the percentage of requests in which the specified error occurs to the total number of requests.
timeoutThresholdByPercent: 20   # The threshold of the percentage of requests in which timeout occurs to the total number of requests.
errorCondition: "$StatusCode = 500"   # The error condition.
downgradeBackend:
  type: mock
  statusCode: 418
  body: |
    <result>
      <errorCode>I's a teapot</errorCode>
    </result>

2.6 Throttle requests when the circuit breaker trips

When the circuit breaker trips, a temporary throttling configuration applies to all traffic while the circuit breaker is open or half-open:

---
windowInSeconds: 1             # The time window in which the circuit breaker checks the number of occurrences of timeout at the backend.
openTimeoutSeconds: 15          # The period of time during which the circuit breaker stays open after it trips.
errorThreshold: 3
errorCondition: "$LatencyMilliSeconds > 1"
downgradeTrafficLimit:               # The backend to which API requests are directed when the circuit breaker is open.
  limit: 2
  period: MINUTE

3. Fallback backend configuration

Use downgradeBackend to specify a fallback backend when the circuit breaker is open. The backend configuration must match the API specification format used in API Gateway (Import Swagger files to create APIs with API Gateway extensions). The following backend types are supported:

• HTTP backend

---
backend:
  type: HTTP
  address: "http://10.10.100.2:8000"
  path: "/users/{userId}"
  method: GET
  timeout: 7000        

• HTTP backend (VPC)

---
backend:
  type: HTTP-VPC
  vpcAccessName: vpcAccess1
  path: "/users/{userId}"
  method: GET
  timeout: 10000        

• Function Compute

---
backend:
  type: FC
  fcRegion: cn-shanghai
  serviceName: fcService
  functionName: fcFunction
  arn: "acs:ram::111111111:role/aliyunapigatewayaccessingfcrole"        

• MOCK

---
backend:
  type: MOCK
  mockResult: "mock result sample"
  mockStatusCode: 200
  mockHeaders:
    - name: Content-Type
      value: text-plain
    - name: Content-Language
      value: zhCN

4. Error codes