ApsaraMQ for RocketMQ provides the consumption retry feature. If a consumer fails to consume a message, ApsaraMQ for RocketMQ redelivers the message to the consumer based on the consumption retry policy. This topic describes the consumption retry policies for messages that are sent over HTTP and TCP in ApsaraMQ for RocketMQ.

Usage notes

  • The ID of a message remains unchanged regardless of the number of retries.
  • ApsaraMQ for RocketMQ redelivers messages only in clustering consumption mode. If a consumer fails to consume a message in broadcasting consumption mode, ApsaraMQ for RocketMQ does not redeliver the message to the consumer. In this case, the consumer continues consuming new messages.

Overview

If a consumption failure or consumption timeout occurs for a message, ApsaraMQ for RocketMQ redelivers the message to the consumer after the specified retry interval is elapsed. If the message consumption still fails after the maximum number of retries is reached, ApsaraMQ for RocketMQ delivers the message to a dead-letter queue. For information about dead-letter queues, see Dead-letter queues. You can consume messages in dead-letter queues to restore your business.

The following items describe the major behaviors for consumption retry:
  • Retry interval: the interval from the point in time when the consumption failure or timeout of a message occurs to the point in time when the next consumption of the message starts.
  • Maximum number of retries: the maximum number of times that ApsaraMQ for RocketMQ can redeliver a message to the consumer after the message fails to be consumed.

The maximum number of retries for messages that are sent over TCP is different from the maximum number of retries for messages that are sent over HTTP. For more information, see Retry policies for messages sent over TCP and Retry policies for the messages that are sent over HTTP.

Retry policies for messages sent over TCP

Retry status

The following figure shows how the status of a message changes when a consumer consumes the message.

The status of a message that is sent over TCP
  • Ready

    The message is waiting to be consumed on the ApsaraMQ for RocketMQ broker.

  • Inflight

    The message is obtained and is being consumed by the consumer. The consumption result is not returned.

  • WaitingRetry

    If a message fails to be consumed or the consumption of the message times out, the consumption retry logic is triggered. If the maximum number of retries is not reached, the status of the message changes to Ready after the retry interval is elapsed. Failed messages that are in the Ready state can be consumed again. You can increase the interval between retries to prevent frequent retries.

  • Commit

    The message is consumed. After the consumer returns a successful response, the consumption is complete.

  • DLQ

    A mechanism that is used to ensure the implementation of consumption logic. If the feature for retaining dead-letter messages is enabled, a message that fails to be consumed after the maximum number of retries is reached is sent to the dead-letter topic. You can consume messages in the dead-letter topic to restore your business. For more information, see Dead-letter queues.

Figure-Interval between retries

The preceding figure shows a sample retry process. In the figure, the message remains in the Ready state for 5 seconds and requires 6 seconds to be consumed.

Each time the message is retried, the status of the message changes from Ready to Inflight, and then changes to WaitingRetry. A retry interval refers to the interval from the point in time when the consumption failure or timeout of a message occurs to the point in time when the next consumption of the message starts. The interval between two consecutive consumptions includes the retry interval, the consumption duration, and the period of time for which the message remains in the Ready state. Example:

  • The first time that a message is delivered for consumption, the message enters the Ready state at the 0th second.
  • The message is pulled at the 5th second. At the 6th second, a consumption error occurs, and a message that indicates the message failed to be consumed is returned by the client.
  • The message cannot be immediately retried because a 10-second retry interval is specified.
  • At the 21st second, the message re-enters the Ready state.
  • After 5 seconds, the client starts to consume the message again.

The consumption interval is calculated as 21 seconds based on the following formula: Consumption interval = Consumption duration + Retry interval + Duration in the Ready state = 6 + 10 + 5 = 21.

Retry intervals and number of retries

ProtocolMessage typeRetry intervalMaximum number of retries
TCPOrdered messageThe suspendTimeMillis parameter is used to specify the retry interval for ordered messages. Valid values: 10 to 30000. Unit: milliseconds. Default value: 1000. The default value indicates that failed ordered messages are redelivered at an interval of 1 second. The MaxReconsumeTimes parameter is used to specify the maximum number of retries for ordered messages. No upper limit is imposed on the value of this parameter. If you leave this parameter empty, the maximum number of retries is Integer.MAX.
Unordered messageThe retry interval for unordered messages varies based on the number of retries. The retry interval ranges from 10 seconds to 2 hours. You cannot specify a custom retry interval for unordered messages.
  • If the maximum number of retries for a message does not exceed 16, the message is redelivered at a predefined interval. For more information, see Retry intervals for unordered messages that are sent over TCP.
  • If the maximum number of retries for a message exceeds 16, the message is redelivered at a predefined interval for the first 16 retries, and at a 2-hour interval for the subsequent retries.
The MaxReconsumeTimes parameter is used to specify the maximum number of retries for unordered messages. Default value: 16. No upper limit is imposed on the value of this parameter. We recommend that you use the default value.
Table 1. Retry intervals for unordered messages that are sent over TCP
NumberIntervalNumberInterval
110 seconds97 minutes
230 seconds108 minutes
31 minute119 minutes
42 minutes1210 minutes
53 minutes1320 minutes
64 minutes1430 minutes
75 minutes151 hour
86 minutes162 hours

Configuration methods

Important The following examples use the messages that are sent over TCP to describe the configuration methods of retry policies.
  • Enable the message retry feature

    If you want ApsaraMQ for RocketMQ to redeliver a message that fails to be consumed in clustering consumption mode, use one of the following methods to implement the MessageListener method:

    • Method 1: Return Action.ReconsumeLater. We recommend that you use this method.
    • Method 2: Return null.
    • Method 3: Throw an exception.

    Sample code

    public class MessageListenerImpl implements MessageListener {
    
        @Override
        public Action consume(Message message, ConsumeContext context) {
            // If the consumption logic throws an exception, the message is retried. 
            doConsumeMessage(message);
            // Method 1: Return Action.ReconsumeLater and retry the message. 
            return Action.ReconsumeLater;
            // Method 2: Return null and retry the message. 
            return null;
            // Method 3: Throw an exception and retry the message. 
            throw new RuntimeException("Consumer Message exception");
        }
    }
  • Disable the message retry feature

    If you do not want ApsaraMQ for RocketMQ to redeliver a message that fails to be consumed in clustering consumption mode, configure the message consumption code to capture all exceptions that are thrown by the consumption logic and return Action.CommitMessage. This way, ApsaraMQ for RocketMQ does not redeliver the message.

    Sample code

    public class MessageListenerImpl implements MessageListener {
    
        @Override
        public Action consume(Message message, ConsumeContext context) {
            try {
                doConsumeMessage(message);
            } catch (Throwable e) {
                // Capture all exceptions that are thrown by the consumption logic and return Action.CommitMessage.
                return Action.CommitMessage;
            }
            // The message is processed as expected and Action.CommitMessage is returned.
            return Action.CommitMessage;
        }
    }
  • Specify a custom retry interval and maximum number of retries
    Note If you want to specify custom values for the log configurations of your ApsaraMQ for RocketMQ client, update your TCP client SDK for Java to version 1.2.2 or later. For more information, see Release notes.

    ApsaraMQ for RocketMQ allows you to specify custom retry intervals and maximum number of retries when you start a consumer. You cannot specify custom retry intervals for unordered messages. For more information about the retry intervals for unordered messages, see Retry intervals for unordered messages that are sent over TCP.

    The following sample code provides an example on how to specify a custom interval and maximum number of retries:

    Properties properties = new Properties();
    // Set the maximum number of retries allowed for messages in the specified consumer group to 20. The value is a string. 
    properties.put(PropertyKeyConst.MaxReconsumeTimes,"20");
    // Set the retry interval for messages in the specified consumer group to 3,000 milliseconds. The value is a string. 
    properties.put(PropertyKeyConst.suspendTimeMillis,"3000");
    Consumer consumer = ONSFactory.createConsumer(properties);
    Important The most recent configuration takes effect for the consumers in the same group. The configuration for the consumer that is started at the most recent point in time overwrites the configurations for consumers that are started at a previous point in time. Make sure that all consumers in a consumer group use the same configurations for the retry interval and the maximum number of retries. If all consumers in a consumer group do not use the same configurations for the retry interval and the maximum number of retries, the configurations for the consumers overwrite each other.
  • Query the number of retries

    After a consumer receives a message, you can use the following method to query the number of retries. In most cases, you do not need to query the retry interval.

    public class MessageListenerImpl implements MessageListener {
    
        @Override
        public Action consume(Message message, ConsumeContext context) {
            // Query the number of retries. 
            System.out.println(message.getReconsumeTimes());
            return Action.CommitMessage;
        }
    }

Retry policies for the messages that are sent over HTTP

ProtocolMessage typeRetry intervalMaximum number of retriesConfiguration
HTTPOrdered message1 minute288The message retry method is predefined by the system and cannot be modified.
Unordered message5 minutes288The configurations are predefined by the system and cannot be modified.