Alibaba Cloud SDK:Advanced backoff mechanism based on the throttling policy

Enable the backoff policy based on the throttling policy

By default, the backoff policy based on the throttling policy is disabled. You can use one of the following methods to enable the backoff policy based on the throttling policy:

1. Configure a retry policy.

RetryPolicy retryPolicy = RetryPolicy.builder()
                .maxNumberOfRetries(3) // The maximum number of retries.
                .maxDelayTimeMillis(20 * 1000) // The maximum retry interval. Unit: milliseconds. If the interval is exceeded, no retries are performed.
                //.retryConditions(retryConditions) // The policy that is used to trigger retries.
                .enableAliyunThrottlingControl(true) // Enable the backoff policy based on the throttling policy. 
                //.throttlingConditions(throttlingConditions) // The policy that is used to restrict retries.
                .build();

request.setSysRetryPolicy(retryPolicy);

2. Use the default retry policy.

// The default policy. The request parameter is enableAliyunThrottlingControl. The value true specifies that the backoff policy based on the throttling policy is enabled. The value false specifies that the backoff policy based on the throttling policy is disabled.
RetryPolicy retryPolicy = RetryPolicy.defaultRetryPolicy( 
true 
);

request.setSysRetryPolicy(retryPolicy);

Benefits of enabling the backoff policy based on the throttling policy

1. Unnecessary retries are reduced. This reduces system consumption.

2. Backoff algorithms can be used to calculate more accurate backoff time to improve the success rate of retries.

Backoff policy: Calculate backoff time based on the throttling policy

After the backoff policy based on the throttling policy is enabled, backoff time is calculated in the following logic:

1. If no throttling is triggered, EqualJitter is used to calculate a retry interval.

2. If throttling is triggered, the retry interval is a larger value between the remaining time of the current throttling cycle and the calculation result of EqualJitter. If the retry interval is greater than the maximum retry interval, an exception is thrown, and the remaining time of the current throttling cycle is stored in the cache pool. This way, the remaining time of the current throttling cycle can be used by the policy to disable API requests during throttling.

Policy of disabling API requests during throttling

After the backoff policy based on the throttling policy is enabled, if throttling is triggered, the server returns the remaining quota information about throttling in the header. The server returns information such as the remaining number of API calls and the remaining time of the current throttling cycle to the client in the "User" and "API + User" dimensions. If API calls are throttled, the remaining number of API calls is 0.

Limit: the maximum throttling threshold. The value is of the INT type.
Time: the time span of throttling. The value is of the LONG type. Unit: milliseconds.
TimeLeft: the remaining time of the current throttling cycle. The value is of the LONG type. Unit: milliseconds.
Reset: the start time of the next throttling cycle. The value is of the LONG type and is a timestamp in milliseconds

The SDK manages the remaining time of the current throttling cycle. During the backoff time that is calculated based on the remaining time of the current throttling cycle, call initiations or retires are limited. If the backoff time is greater than the maximum retry interval, an error is thrown. If the backoff time is not greater than the maximum retry interval, the system waits until the call can be initiated or retried.

Enable throttling debugging

After the backoff policy based on the throttling policy is enabled, if throttling is triggered, the server returns the remaining quota information about throttling in the header. You can also request to return the quota information in a call by setting X-RateLimit-Mode to debug in the request header.

request.putHeadParameter("X-RateLimit-Mode", "debug");