ApsaraMQ for RocketMQ:Message retry

Usage notes

• The ID of a message remains unchanged regardless of the number of retries.
• ApsaraMQ for RocketMQ redelivers messages only in clustering consumption mode. If a consumer fails to consume a message in broadcasting consumption mode, ApsaraMQ for RocketMQ does not redeliver the message to the consumer. In this case, the consumer continues consuming new messages.

Overview

If a consumption failure or consumption timeout occurs for a message, ApsaraMQ for RocketMQ redelivers the message to the consumer after the specified retry interval is elapsed. If the message consumption still fails after the maximum number of retries is reached, ApsaraMQ for RocketMQ delivers the message to a dead-letter queue. For information about dead-letter queues, see Dead-letter queues. You can consume messages in dead-letter queues to restore your business.

The maximum number of retries for messages that are sent over TCP is different from the maximum number of retries for messages that are sent over HTTP. For more information, see Retry policies for messages sent over TCP and Retry policies for the messages that are sent over HTTP.

Retry policies for messages sent over TCP

Retry status

The following figure shows how the status of a message changes when a consumer consumes the message.

• Ready

  The message is waiting to be consumed on the ApsaraMQ for RocketMQ broker.

• Inflight

  The message is obtained and is being consumed by the consumer. The consumption result is not returned.

• WaitingRetry

  If a message fails to be consumed or the consumption of the message times out, the consumption retry logic is triggered. If the maximum number of retries is not reached, the status of the message changes to Ready after the retry interval is elapsed. Failed messages that are in the Ready state can be consumed again. You can increase the interval between retries to prevent frequent retries.

• Commit

  The message is consumed. After the consumer returns a successful response, the consumption is complete.

• DLQ

  A mechanism that is used to ensure the implementation of consumption logic. If the feature for retaining dead-letter messages is enabled, a message that fails to be consumed after the maximum number of retries is reached is sent to the dead-letter topic. You can consume messages in the dead-letter topic to restore your business. For more information, see Dead-letter queues.

The preceding figure shows a sample retry process. In the figure, the message remains in the Ready state for 5 seconds and requires 6 seconds to be consumed.

Each time the message is retried, the status of the message changes from Ready to Inflight, and then changes to WaitingRetry. A retry interval refers to the interval from the point in time when the consumption failure or timeout of a message occurs to the point in time when the next consumption of the message starts. The interval between two consecutive consumptions includes the retry interval, the consumption duration, and the period of time for which the message remains in the Ready state. Example:

• The first time that a message is delivered for consumption, the message enters the Ready state at the 0th second.
• The message is pulled at the 5th second. At the 6th second, a consumption error occurs, and a message that indicates the message failed to be consumed is returned by the client.
• The message cannot be immediately retried because a 10-second retry interval is specified.
• At the 21st second, the message re-enters the Ready state.
• After 5 seconds, the client starts to consume the message again.

The consumption interval is calculated as 21 seconds based on the following formula: Consumption interval = Consumption duration + Retry interval + Duration in the Ready state = 6 + 10 + 5 = 21.

Retry intervals and number of retries

Configuration methods

• Enable the message retry feature

  If you want ApsaraMQ for RocketMQ to redeliver a message that fails to be consumed in clustering consumption mode, use one of the following methods to implement the MessageListener method:

  • Method 1: Return Action.ReconsumeLater. We recommend that you use this method.
  • Method 2: Return null.
  • Method 3: Throw an exception.

  Sample code

  public class MessageListenerImpl implements MessageListener {
  
      @Override
      public Action consume(Message message, ConsumeContext context) {
          // If the consumption logic throws an exception, the message is retried. 
          doConsumeMessage(message);
          // Method 1: Return Action.ReconsumeLater and retry the message. 
          return Action.ReconsumeLater;
          // Method 2: Return null and retry the message. 
          return null;
          // Method 3: Throw an exception and retry the message. 
          throw new RuntimeException("Consumer Message exception");
      }
  }

• Disable the message retry feature

  If you do not want ApsaraMQ for RocketMQ to redeliver a message that fails to be consumed in clustering consumption mode, configure the message consumption code to capture all exceptions that are thrown by the consumption logic and return Action.CommitMessage. This way, ApsaraMQ for RocketMQ does not redeliver the message.

  Sample code

  public class MessageListenerImpl implements MessageListener {
  
      @Override
      public Action consume(Message message, ConsumeContext context) {
          try {
              doConsumeMessage(message);
          } catch (Throwable e) {
              // Capture all exceptions that are thrown by the consumption logic and return Action.CommitMessage.
              return Action.CommitMessage;
          }
          // The message is processed as expected and Action.CommitMessage is returned.
          return Action.CommitMessage;
      }
  }

• Specify a custom retry interval and maximum number of retries
  Note If you want to specify custom values for the log configurations of your ApsaraMQ for RocketMQ client, update your TCP client SDK for Java to version 1.2.2 or later. For more information, see Release notes.

  ApsaraMQ for RocketMQ allows you to specify custom retry intervals and maximum number of retries when you start a consumer. You cannot specify custom retry intervals for unordered messages. For more information about the retry intervals for unordered messages, see Retry intervals for unordered messages that are sent over TCP.

  The following sample code provides an example on how to specify a custom interval and maximum number of retries:

  Properties properties = new Properties();
  // Set the maximum number of retries allowed for messages in the specified consumer group to 20. The value is a string. 
  properties.put(PropertyKeyConst.MaxReconsumeTimes,"20");
  // Set the retry interval for messages in the specified consumer group to 3,000 milliseconds. The value is a string. 
  properties.put(PropertyKeyConst.suspendTimeMillis,"3000");
  Consumer consumer = ONSFactory.createConsumer(properties);

  Important The most recent configuration takes effect for the consumers in the same group. The configuration for the consumer that is started at the most recent point in time overwrites the configurations for consumers that are started at a previous point in time. Make sure that all consumers in a consumer group use the same configurations for the retry interval and the maximum number of retries. If all consumers in a consumer group do not use the same configurations for the retry interval and the maximum number of retries, the configurations for the consumers overwrite each other.
• Query the number of retries

  After a consumer receives a message, you can use the following method to query the number of retries. In most cases, you do not need to query the retry interval.

  public class MessageListenerImpl implements MessageListener {
  
      @Override
      public Action consume(Message message, ConsumeContext context) {
          // Query the number of retries. 
          System.out.println(message.getReconsumeTimes());
          return Action.CommitMessage;
      }
  }

Retry policies for the messages that are sent over HTTP