edit-icon download-icon

Consumption idempotence

Last Updated: Sep 26, 2018

It is recommended that MQ consumers perform an idempotent process for messages according to the unique key defined for a specific business after the message has been received.

The Necessity for Consumption Idempotence

In Internet applications, especially in the case of poor networks, MQ messages might be duplicated. This could be caused by the following two reasons:

  • Duplicate when sending messages

    When a network disconnection occurs or the client goes down after a message is sent to the server and is persisted, the server fails to reply to the client. At this time, if MQ producer realizes that the message failed to be sent and resends the message, MQ consumers will receive two messages subsequently with the same content and the same message IDs.

  • Duplicate when delivering messages

    A message is delivered to the consumer and the related task is completed, but a network disconnection occurs when the client sends a reply to the server. In order to ensure that the message is consumed at least once, the MQ server will retry delivering the previously processed message after the network recovers. The MQ consumer will subsequently receive two messages with the same content and the same Message ID.

Suggested Solution

Because there may be conflicts caused by the repetition of Message IDs, a real secure idempotent processing should not be done based on message IDs. The best way is to use the unique identifier of related business tasks as the key basis for idempotent processing. The unique identifier of the business tasks can be set using the message key:

  1. Message message = new Message();
  2. message.setKey("ORDERID_100");
  3. SendResult sendResult = producer.send(message);

When a consumer receives the message, it can perform idempotent processing according to the message key:

  1. consumer.subscribe("ons_test", "*", new MessageListener() {
  2. public Action consume(Message message, ConsumeContext context) {
  3. String key = message.getKey()
  4. // Use the unique identifier of the business task as the key basis for idempotent processing.
  5. }
  6. });
Thank you! We've received your feedback.