Ordered messages ensure that the order in which messages are consumed is the same as the order in which the messages are sent. The earliest message that is sent is consumed as the first message and the message that is sent after the previous message is consumed later. In most cases, the messages are used in scenarios that have strict requirements for the order of message instructions. For example, the messages are used in financial securities and e-commerce business. This topic describes the concepts, scenarios, working mechanism, and usage notes of ordered messages 1.0 in Message Queue for Apache RocketMQ.

What are ordered messages?

In Message Queue for Apache RocketMQ, ordered messages are messages that are sent and consumed in a strict order. Messages in a topic are published and consumed in the first in, first out (FIFO) order. The earliest published message is consumed as the first message.

Ordered messages are classified into partitionally ordered messages and globally ordered messages. For more information about partitionally ordered messages, see Partitionally ordered messages. For more information about globally ordered messages, see Globally ordered messages.

Partitionally ordered message

All messages in a specific topic are partitioned based on the Sharding Key. The messages in each partition are published and consumed in the FIFO order. Only the messages in the same partition are published and consumed in a strict order. Messages across partitions do not need to be consumed in order.

  • Terms
    • Sharding Key: the key field used in ordered messages to identify partitions in a topic. This field is different from the key of a normal message. Message Queue for Apache RocketMQ routes the messages that contain the same Sharding Key to a partition. The earliest published message in the partition is consumed as the first message.
    • Partition: topic-level partitions. Each topic contains one or more partitions. Messages in the same topic are distributed to these partitions. Logical partitions mentioned in this topic are topic-level partitions. For more information, see Terms.
    • Physical partition: the physical unit used to store messages. Each physical partition is created on a specified node of a server.
  • Scenarios

    Partitionally ordered messages are used in scenarios in which high performance is required. Messages in each partition must be published and consumed in the FIFO order.

  • Examples
    • If a verification code is required for user registration and the user ID is used as the Sharding Key, the messages sent by the same user are published and consumed in the FIFO order.
    • In e-commerce scenarios, the order ID is used as the Sharding Key. Messages for the creation, payment, and refund of the same order are published and consumed in the FIFO order.

All internal e-commerce systems in Alibaba Group use partitionally ordered messages to ensure the order and high performance of your business.

Globally ordered messages

All messages in a topic are published and consumed in the FIFO order.

  • Scenarios

    Globally ordered messages are used in scenarios in which the performance requirements are not high and all messages must be published and consumed in the FIFO order.

  • Examples

    When securities are processed, the topic is RMB to USD exchange transactions. If the bid prices of multiple transactions are the same, the earliest transaction is processed as the first transaction. In this scenario, the system must publish and consume messages in the topic in the global FIFO order.

Note Globally ordered messages can be used as partitionally ordered messages. Globally ordered messages are stored only in one partition of a topic. This way, globally ordered messages work in the same way as partitionally ordered messages. Partitionally ordered messages can be processed faster in parallel compared with globally ordered messages. This is because multiple partitions are used to store partitionally ordered messages.

How do ordered messages work?

Globally ordered messages work in the same way as partitionally ordered messages. The following section describes how to make sure that messages are sent and consumed in the same order in Message Queue for Apache RocketMQ. Partitionally ordered messages are used in the example. Ordered messages 1.0
In Message Queue for Apache RocketMQ, you can perform the following steps to make sure that messages are sent and consumed in the same order.
  • Message sending

    In the preceding figure, messages for Orders A and B are produced in the sequence of A1, B1, A2, A3, B2, and B3. The messages for the same order must be sent and consumed in the same sequence as the sequence in which these messages are produced. For example, the messages for Order A are sent and consumed in the sequence of A1, A2, and A3. Normal messages for Order A may be sent to different queues in polling mode. The original order of the normal messages cannot be retained in the queues. Ordered messages that use the same Sharding Key such as the same order ID are sequentially routed to the same partition in Message Queue for Apache RocketMQ.

    The Message Queue for Apache RocketMQ broker determines that the order in which messages are generated is the same as the time sequence of the messages that are sent by the same producer. The Message Queue for Apache RocketMQ broker cannot determine the sequence of messages that are concurrently generated by different producers and threads.

  • Message storage

    In the preceding figure of ordered messages 1.0, each logical partition corresponds to a physical partition. When messages are sent to the logical partitions in order, the messages are stored in the corresponding physical partitions in the same order.

  • Message consumption

    Message Queue for Apache RocketMQ delivers messages to a consumer in the order in which messages are stored. Then, the consumer consumes the messages from the earliest received message.

    The consumer uses a single thread to consume messages of the same Sharding Key. This ensures that the earliest stored message such as the earliest published message is consumed as the first message.

In ordered messages 1.0, a logical partition corresponds to only one physical partition and the preset number of partitions cannot be changed. If the business traffic grows, a physical node may not meet the capacity requirements. If you add physical nodes, the message routing policy and sequence are affected. If you split your business, additional reconstruction costs are incurred. Ordered messages 1.0 are suitable for scenarios in which the business scale is stable, the number of messages slightly fluctuates, and additional nodes are not required.

If you want to add nodes to meet traffic requirements, we recommend that you use ordered messages 2.0 that support unlimited scale-out and has no negative impact on your business. Ordered messages 2.0 prevent messages from being unevenly distributed among partitions. When a physical node fails, the business can be switched from the failed node to another node in seconds and the business is not affected. For more information, see Ordered messages 2.0.

Precautions

Before you use ordered messages 1.0, take note of the following items:

  • Each group ID corresponds to a topic type. A group ID cannot be used to send and receive ordered messages and unordered messages.
  • We recommend that you prevent queues that contain globally ordered messages from being congested. You can run multiple instances at the same time to ensure business continuity. If the active instance fails, the system performs a failover on another instance to handle the business. This ensures that your business remains uninterrupted. Only one instance is active at a time.
  • The Message Queue for Apache RocketMQ broker determines the order in which messages are generated based on the order in which the sender uses a single producer to concurrently send messages in a single thread. If the sender uses multiple producers or multiple threads to concurrently send messages, the message order is determined based on the order in which the messages are received by the Message Queue for Apache RocketMQ broker. This order may be different from the sending order on the business side.

FAQ

  • Can a single message be an ordered message, a scheduled message, and a transactional message at the same time?

    No, ordered messages, scheduled messages, and transactional messages are different and mutually exclusive message types.

  • In which regions can I use ordered messages?

    Ordered messages are supported in all Alibaba Cloud regions and Finance Cloud regions in which Message Queue for Apache RocketMQ is available.

  • Why is the performance of globally ordered messages not high?

    Globally ordered messages are processed in the FIFO order. If a message is not consumed, the next message remains in the topic queue and cannot be processed. To improve the transactions per second (TPS) of globally ordered messages, upgrade your instance configuration and make sure that the application on the message client processes local business logic in a timely manner.

  • What transmission modes do ordered messages support?

    Ordered messages support only the reliable synchronous transmission mode that ensures strict message order. Asynchronous transmission is not supported.

  • Do ordered messages support clustering consumption and broadcasting consumption?

    Ordered messages support clustering consumption but do not support broadcasting consumption.

TCP SDK sample code

For information about the TCP sample code, see the following documents:

HTTP client SDK sample code