Message Queue for Apache RocketMQ supports distributed transactional messages that are applicable to scenarios in which the final consistency of data is required. This topic describes the concepts, benefits, scenarios, interaction process, and precautions of Message Queue for Apache RocketMQ transactional messages.

Concepts

  • Transactional message: Message Queue for Apache RocketMQ provides a distributed transaction processing feature similar to X/Open XA to ensure transaction consistency in Message Queue for Apache RocketMQ.
  • Half message: A half message is a message that cannot be delivered temporarily. If a message is sent to the Message Queue for Apache RocketMQ broker, but the broker does not receive the second acknowledgment (ACK) of the message from the producer, the message is marked as "temporarily undeliverable". A message in this state is called a half message.
  • Message status check: The second ACK for a transactional message may be lost if the network connection is unavailable or the corresponding producer is restarted. If the Message Queue for Apache RocketMQ broker finds that a message remains as a half message for a long period of time, the broker sends a request to the producer to check whether the final status of the message is Commit or Rollback.

Benefits

Message Queue for Apache RocketMQ supports distributed transactional messages that can realize decoupling between applications and ensure the final consistency of data. In addition, traditional large transactions can be split into smaller transactions. This not only improves efficiency, but also ensures the high availability of the core system because messages are not rolled back if an application is not running as expected. If an application remains unable to receive messages, you can supplement or correct data for the application for troubleshooting, without the need to roll back all messages.

Scenarios

When you place orders by using the shopping cart on Taobao, the shopping cart system and the trading system are involved. The final consistency of data between the two systems is ensured based on distributed transaction processing with asynchronous message delivery. In such a scenario, the trading system is crucial because it determines whether an order can be properly placed. The shopping cart system can subscribe to only the topics related to orders from Message Queue for Apache RocketMQ and execute corresponding transactions to ensure that its final data is consistent with that of the trading system.

Interaction process

The following figure shows the interaction process of transactional messages. Interaction process of transactional messages

The procedure for sending a transactional message includes the following steps:

  1. A producer sends a half message to the Message Queue for Apache RocketMQ broker.
  2. The Message Queue for Apache RocketMQ broker persists the message and sends an ACK to the producer to confirm that the message is sent. At this time, the message is a half message.
  3. The producer executes a local transaction.
  4. The producer sends a second ACK to the Message Queue for Apache RocketMQ broker to submit the execution result of the local transaction. The execution result may be Commit or Rollback. If the broker receives Commit, the broker marks the half message as deliverable, and the consumer can receive the half message. If the broker receives Rollback, the broker deletes the half message, and the consumer cannot receive the half message.

The procedure for checking the status of a transactional message includes the following steps:

  1. If the network connection is unavailable or the corresponding producer is restarted, the Message Queue for Apache RocketMQ broker may not receive the ACK in Step 4. After a specific period of time, the Message Queue for Apache RocketMQ broker sends a request to a producer in the producer cluster to query the status of the half message.
  2. After the producer receives the request, the producer checks the final status of the local transaction that corresponds to the half message.
  3. The producer sends another ACK to the Message Queue for Apache RocketMQ broker based on the final status of the local transaction. The broker processes the half message by following Step 4.

Usage notes

  1. Transactional messages cannot share group IDs with messages of other types. Unlike messages of other types, transactional messages provide the mechanism of message status check. The Message Queue for Apache RocketMQ broker can query the message status on clients by group ID.
  2. You must specify the implementation class of the LocalTransactionChecker method when you create a producer of transactional messages by calling the ONSFactory.createTransactionProducer method. This way, the status of the transactional messages can be checked if exceptions occur.
  3. When a transactional message is sent to execute the local transaction, one of the following three states is returned in the execute method:
    • TransactionStatus.CommitTransaction: The transaction is committed. The consumer can consume the message.
    • TransactionStatus.RollbackTransaction: The transaction is rolled back. The message is discarded and cannot be consumed.
    • TransactionStatus.Unknow: The transaction is in an unknown state. The Message Queue for Apache RocketMQ broker is expected to send a request to the producer to check the message status after a fixed period of time.
  4. You can use the following code to speicfy the earliest point in time for the initial message status check for each message:
    Message message = new Message();
    // Specify the earliest point in time for the initial message status check in the message property. The value is in seconds. For example, you can use the following setting to specify that the initial message status check is performed after 120 to 125 seconds elapse. 
    message.putUserProperties(PropertyKeyConst.CheckImmunityTimeInSeconds,"120");
    // The preceding setting is used to specify the earliest point in time for the initial message status check. A latency of 0 to 5 seconds may occur in actual message processing. If the transaction is still not committed after the initial message status check, a subsequent message status check is performed every 5 seconds. 

References

The sample code in different programming languages for sending and subscribing to transactional messages is provided in the following topics: