All Products
Search
Document Center

Message idempotence

Last Updated: Mar 28, 2019

After receiving messages, an RocketMQ consumer must perform idempotent processing on messages according to the unique key defined for a specific business.

Necessity for consumption idempotence

In Internet applications, especially in the case of poor networks, RocketMQ messages may be duplicated. The duplicates can be simply summarized in the following two cases:

  • Duplicate when sending messages

When a network disconnection occurs or the client breaks down after a message is sent to the broker and is persisted, the broker fails to reply to the client. At this time, if the producer realizes that the message failed to be sent and resends the message, consumers will receive two messages subsequently with the same content and message ID.

  • Duplicate when delivering messages

A message is delivered to the consumer and the related task is completed, but a network disconnection occurs when the client sends a reply to the broker. To ensure that the message is consumed at least once, the RocketMQ broker will retry delivering the previously processed message after the network recovers. The consumer will subsequently receive two messages with the same content and the same message ID.

Suggested solution

As there may be conflicts caused by duplicate message IDs, a secure idempotent processing should not be based on message IDs. The best way is to use the unique business identifier as the key basis for idempotent processing. The unique business identifier can be set using the message key:

  1. Message message = new Message();
  2. message.setKey("ORDERID_100");
  3. SendResult sendResult = producer.send(message);

When receiving the message, the subscriber can perform idempotent processing according to the message key:

  1. consumer.subscribe("ons_test", "*", new MessageListener() {
  2. public Action consume(Message message, ConsumeContext context) {
  3. String key = message.getKey()
  4. //Use the unique business identifier as the key basis for idempotent processing.
  5. }
  6. });