Ordered messages are consumed in the same order in which they are sent. The earliest message that is sent is consumed as the first message, and the message that is sent next is consumed later. In most cases, ordered messages are used in scenarios that require a strict message order. For example, financial and securities industries, and e-commerce business. This topic describes the terms, usage scenarios, working mechanism, and usage notes of ordered messages in ApsaraMQ for RocketMQ.
What are ordered messages?
Ordered messages are a type of messages provided by ApsaraMQ for RocketMQ. Messages of this type are published and consumed in a first in, first out (FIFO) order in a topic. The earliest published message is consumed as the first message.
Ordered messages are classified into partitionally ordered messages and globally ordered messages. For information about partitionally ordered messages, see Partitionally ordered messages. For information about globally ordered messages, see Globally ordered messages.
Partitionally ordered messages
All messages in a specific topic are partitioned based on a sharding key. The messages in each partition are published and consumed in a FIFO order. FIFO order is suitable for only messages in the same partition. No limits are imposed on the publishing and consumption of messages across partitions.
Partitionally ordered messages are used in scenarios in which high performance is required. Messages in each partition must be published and consumed in a FIFO order.
- If a verification code is required for user registration and the user ID is used as the sharding key, the messages that are sent by the same user are published and consumed in a FIFO order.
- In an e-commerce scenario, if the order ID is used as the sharding key, messages for the creation, payment, and refund of the same order are published and consumed in a FIFO order.
All internal e-commerce systems in Alibaba Group use partitionally ordered messages to ensure stable performance and that online orders are processed in sequence.
Globally ordered messages
All messages in a topic are published and consumed in a FIFO order.
Globally ordered messages are used in scenarios in which performance requirements are not high but all messages are required to be published and consumed in a FIFO order.
In a securities processing scenario in which the topic is RMB to USD exchange, the transaction that is initiated by the first bidder must be processed as the first transaction when the bid prices of multiple transactions are the same. In this case, the system can publish and consume messages in the topic in a global FIFO order.
How do ordered messages work?Globally ordered messages work in the same manner as partitionally ordered messages. The following section describes how to ensure that messages are sent and consumed in the same order in ApsaraMQ for RocketMQ. In this example, partitionally ordered messages are used.
- Message sending
In the preceding figure, messages for Orders A and B are produced in the following sequence: A1, B1, A2, A3, B2, and B3. The messages for the same order must be sent and consumed in the same sequence in which the messages are produced. For example, the messages for Order A are sent and consumed in the following sequence: A1, A2, and A3. Normal messages for Order A may be sent to different queues in polling mode. The original order of the normal messages cannot be retained in the queues. Ordered messages that use the same sharding key such as the same order ID are sequentially routed to the same queue in ApsaraMQ for RocketMQ.
The ApsaraMQ for RocketMQ broker uses the time order in which messages are produced by the same producer to determine the message order. The ApsaraMQ for RocketMQ broker cannot determine the order of the messages that are sent in parallel by different producers or threads.
- Message storage
Each logical partition in a topic of ordered messages corresponds to a physical partition, as shown in the preceding figure. When messages are sent to the logical partitions in order, the messages are stored in the corresponding physical partitions in the same order.
- Message consumption
ApsaraMQ for RocketMQ delivers messages to a consumer in the order in which the messages are stored. Then, the consumers consume the messages from the earliest received message.
When consumers consume messages, messages that contain the same sharding key are consumed by using a single thread. This helps ensure the consistency between the message consumption order and the message storage order, and the consistency between the message consumption order and the message publishing order.
- Each group ID corresponds to a topic type. A group ID cannot be used to send or receive both ordered messages and unordered messages.
- We recommend that you prevent the queues that contain globally ordered messages from being congested. You can run multiple instances at the same time to ensure business continuity. If the active instance fails, the system performs a failover to switch to another instance to handle the business. This helps prevent service interruptions for your business. Only one instance is active each time.
- A ApsaraMQ for RocketMQ broker determines the order in which messages are produced based on the order in which the sender uses a single producer and a single thread to concurrently send messages. If the sender uses multiple producers or multiple threads to send messages in parallel, the message order is determined based on the order in which the messages are received by the ApsaraMQ for RocketMQ broker. This order may be different from the order in which the messages are sent.
- Can a single message be an ordered message, a scheduled message, and a transactional message?
No, a single message cannot be an ordered message, a scheduled message, and a transactional message. Ordered messages, scheduled messages, and transactional messages are different and mutually exclusive message types.
- What are the regions where I can use ordered messages?
Ordered messages are supported in all Alibaba Cloud regions and Finance Cloud regions where ApsaraMQ for RocketMQ is available.
- Why is the performance of globally ordered messages not high?
Globally ordered messages are processed in a FIFO order. If a message is not consumed, the next message remains in the topic queue and cannot be processed. To improve the transactions per second (TPS) of globally ordered messages, upgrade your instance configuration and make sure that the applications on the message client require a shorter period of time to process local business logic.
- What transmission modes do ordered messages support?
Ordered messages support only the reliable synchronous transmission mode that ensures strict message order. Asynchronous transmission is not supported.
- Do ordered messages support clustering consumption and broadcasting consumption?
Ordered messages support clustering consumption but do not support broadcasting consumption.
Sample code for using TCP client SDKs
Sample code for HTTP client SDKs
- Java: Send and subscribe to ordered messages
- Go: Send and subscribe to ordered messages
- Python: Send and subscribe to ordered messages
- Node.js: Send and subscribe to ordered messages
- PHP: Send and subscribe to ordered messages
- C#: Send and subscribe to ordered messages
- C++: Send and subscribe to ordered messages