Implement sequential processing with ordered messages - ApsaraMQ for RocketMQ

Ordered messages let consumers process messages in exactly the order they were sent. ApsaraMQ for RocketMQ uses message groups to scope ordering: messages within the same group are always delivered sequentially, while messages across different groups are processed independently.

Use cases

Use ordered messages when downstream systems must process events in the exact sequence they occurred upstream:

Trade matchmaking -- In securities trading, when multiple bids share the same price, the first-bid-first-trade rule applies. The order processing system must handle bids in the exact sequence they were placed.
Incremental data synchronization -- When replicating database changes (inserts, updates, deletes) through a message queue to a search system, operations must be replayed in the original order. Out-of-order replay produces an inconsistent state.

How ordering works

End-to-end message ordering has two parts: production order (send side) and consumption order (receive side). Both must be satisfied for strict first in, first out (FIFO) processing.

Production order

To guarantee production order, meet all three conditions:

Same message group -- Set the same message group for all messages that must be ordered relative to each other. Messages in different groups have no ordering relationship.
Single producer -- Send all related messages from a single producer. Even with the same message group, messages from different producers in different systems have no deterministic order.
Serial sending -- Send messages sequentially from a single thread. The producer client supports multi-threaded access, but messages sent in parallel from different threads have no deterministic order.

When these conditions are met, messages with the same message group are stored in the same queue in send order.

Storage behavior:

Messages from the same group are stored sequentially in the same queue.
Messages from different groups can coexist in the same queue, but their relative order is not guaranteed.

In the diagram above, Message Group 1 (G1-M1, G1-M2, G1-M3) and Message Group 4 (G4-M1, G4-M2) share Queue 1. ApsaraMQ for RocketMQ guarantees order within each group but not between groups.

Consumption order

To guarantee consumption order, two mechanisms work together:

Delivery order -- The SDK and server protocol deliver messages in storage order. Follow the receive-process-acknowledge pattern strictly. Asynchronous processing can break the ordering guarantee.
Important
With PushConsumer, messages are delivered one at a time in storage order. With SimpleConsumer, multiple messages may arrive in a single pull -- your application must process them sequentially and acknowledge each one before calling receive again for the same group. For details, see Consumer types.
Limited retries -- When an ordered message fails after the maximum number of retries, it is skipped to unblock subsequent messages. Choose a retry count that balances reliability with the risk of blocking the entire group. Retries within a message group:
- Do not disrupt the ordering guarantee.
- Do not affect messages in other message groups.
- Do block subsequent messages in the same group until the current message is resolved.
Important
While a failed ordered message is being retried, subsequent messages in the same group are blocked. They are delivered only after the current message succeeds or exhausts its retries.

Production and consumption order combinations

Strict FIFO requires both production and consumption order. However, not every consumer needs ordered delivery. Mix and match based on throughput and ordering requirements:

Production order	Consumption order	Result
Message group set; serial sending	Ordered	Strict FIFO within each message group
Message group set; serial sending	Concurrent	Best-effort chronological order; higher throughput
No message group; unordered sending	Ordered	Strict ordering at queue level (matches storage order, not send order)
No message group; unordered sending	Concurrent	Best-effort chronological order

Message lifecycle

An ordered message moves through five states:

Initialized -- The producer builds the message and prepares to send it.
Ready -- The message arrives at the broker and becomes visible to consumers.
Inflight -- A consumer retrieves the message and processes it. If the broker receives no acknowledgment within the timeout, it retries delivery. For details, see Consumption retry.
Acked -- The consumer commits the result. By default, ApsaraMQ for RocketMQ retains all messages. The broker marks the message as consumed but does not delete it immediately.
Deleted -- After the retention period expires or storage runs low, the broker deletes the oldest messages on a rolling basis. Before deletion, messages can be re-consumed. For details, see Message storage and cleanup.

Important

A retried message is treated as a new message. The original message's lifecycle ends.
While an ordered message is being retried, subsequent messages in the same group are blocked until the current message succeeds.

Limits

Ordered messages can only be sent to topics with MessageType set to FIFO. The message type must match the topic type.

Important

If a consumer group is set to ordered delivery, all messages consumed by that group are billed as ordered messages, regardless of their actual type. If strict ordering is not required, set the group to concurrent delivery to reduce costs.

Prerequisites

Before you begin, ensure that you have:

A topic with MessageType set to FIFO, created in the ApsaraMQ for RocketMQ console
A consumer group configured for ordered delivery (required only if you need ordered consumption)
The RocketMQ 5.x gRPC SDK installed in your project

Important

Ordered messages can only be sent to topics with MessageType set to FIFO. A consumer group that is not set to ordered delivery mode delivers messages concurrently. Verify both the topic type and the consumer group delivery mode before you write any code.

Optimize consumption concurrency

With the RocketMQ 5.x gRPC SDK, PushConsumer can dispatch messages from the same MessageQueue to different threads based on their message group. The more distinct your message group values, the greater the throughput gain.

Supported SDK versions:

SDK	Minimum version
Java	5.0.8
C++	5.0.3
Other SDKs	Not supported

Send and consume ordered messages

Ordered messages require a message group on every send call. Design message groups with the finest granularity your business allows -- for example, use an order ID or user ID. This enables per-entity ordering while maximizing parallelism across entities.

All examples below use Java. For complete SDK samples in other languages, see RocketMQ 5.x gRPC SDK.

Sample code

Send ordered messages

import org.apache.rocketmq.client.apis.*;
import org.apache.rocketmq.client.apis.message.Message;
import org.apache.rocketmq.client.apis.producer.Producer;
import org.apache.rocketmq.client.apis.producer.SendReceipt;


public class ProducerExample {
    public static void main(String[] args) throws ClientException {
        // Instance endpoint. Get it from the Endpoints tab on the instance details page.
        // Use the VPC endpoint for access from ECS over the internal network.
        // Use the public endpoint for access from a local machine or on-premises data center.
        String endpoints = "<your-endpoint>";
        // Topic name. Create the topic in the console first.
        String topic = "<your-fifo-topic>";
        ClientServiceProvider provider = ClientServiceProvider.loadService();
        ClientConfigurationBuilder builder = ClientConfiguration.newBuilder().setEndpoints(endpoints);
        // For public network access, set the instance username and password.
        // Get them from the Intelligent Authentication tab on the Access Control page.
        // For internal network access from ECS, skip this -- the server authenticates via VPC.
        // For Serverless instances, set credentials for public access.
        // If authentication-free internal access is enabled, skip this for internal access.
        // builder.setCredentialProvider(new StaticSessionCredentialsProvider("<instance-username>", "<instance-password>"));
        ClientConfiguration configuration = builder.build();
        Producer producer = provider.newProducerBuilder()
                .setTopics(topic)
                .setClientConfiguration(configuration)
                .build();
        // Build an ordered message with a message group.
        Message message = provider.newMessageBuilder()
                .setTopic(topic)
                .setKeys("messageKey")         // Message key for lookup
                .setTag("messageTag")          // Tag for consumer-side filtering
                .setMessageGroup("fifoGroup001") // Message group -- keep values discrete to avoid hot spots
                .setBody("messageBody".getBytes())
                .build();
        try {
            SendReceipt sendReceipt = producer.send(message);
            System.out.println(sendReceipt.getMessageId());
        } catch (ClientException e) {
            e.printStackTrace();
        }
    }
}

Replace the following placeholders with your actual values:

Placeholder	Description	Example
`<your-endpoint>`	Instance endpoint from the console	`rmq-cn-xxx.cn-hangzhou.rmq.aliyuncs.com:8080`
`<your-fifo-topic>`	FIFO topic name	`order-events`
`<instance-username>`	Instance username (public access only)	`LTAI5tXxx`
`<instance-password>`	Instance password (public access only)	`xXxXxXx`

Consume with PushConsumer

PushConsumer delivers ordered messages one at a time in storage order. Set the consumer group to ordered delivery mode in the console before starting the consumer. Otherwise, messages are delivered concurrently.

import org.apache.rocketmq.client.apis.*;
import org.apache.rocketmq.client.apis.consumer.ConsumeResult;
import org.apache.rocketmq.client.apis.consumer.FilterExpression;
import org.apache.rocketmq.client.apis.consumer.FilterExpressionType;
import org.apache.rocketmq.client.apis.consumer.PushConsumer;
import org.apache.rocketmq.shaded.org.slf4j.Logger;
import org.apache.rocketmq.shaded.org.slf4j.LoggerFactory;

import java.io.IOException;
import java.util.Collections;

public class PushConsumerExample {
    private static final Logger LOGGER = LoggerFactory.getLogger(PushConsumerExample.class);

    private PushConsumerExample() {
    }

    public static void main(String[] args) throws ClientException, IOException, InterruptedException {
        // Instance endpoint. Get it from the Endpoints tab on the instance details page.
        String endpoints = "<your-endpoint>";
        // Topic and consumer group. Create both in the console first.
        String topic = "<your-fifo-topic>";
        String consumerGroup = "<your-consumer-group>";
        final ClientServiceProvider provider = ClientServiceProvider.loadService();
        ClientConfigurationBuilder builder = ClientConfiguration.newBuilder().setEndpoints(endpoints);
        // For public network access, set credentials.
        // builder.setCredentialProvider(new StaticSessionCredentialsProvider("<instance-username>", "<instance-password>"));
        ClientConfiguration clientConfiguration = builder.build();
        // Subscribe to all tags.
        String tag = "*";
        FilterExpression filterExpression = new FilterExpression(tag, FilterExpressionType.TAG);
        PushConsumer pushConsumer = provider.newPushConsumerBuilder()
                .setClientConfiguration(clientConfiguration)
                .setConsumerGroup(consumerGroup)
                .setSubscriptionExpressions(Collections.singletonMap(topic, filterExpression))
                .setMessageListener(messageView -> {
                    // Process the message and return the result.
                    System.out.println("Consume Message: " + messageView);
                    return ConsumeResult.SUCCESS;
                })
                .build();
        Thread.sleep(Long.MAX_VALUE);
        // Close the consumer when no longer needed.
        // pushConsumer.close();
    }
}

Consume with SimpleConsumer

SimpleConsumer pulls messages in batches. Process messages within each batch sequentially and acknowledge each one individually.

Important

For messages in the same message group, if a preceding message has not been acknowledged, calling receive again does not return subsequent messages from that group. This locking mechanism prevents out-of-order processing. Messages from other message groups are not affected and can still be received concurrently.

import org.apache.rocketmq.client.apis.ClientConfiguration;
import org.apache.rocketmq.client.apis.ClientConfigurationBuilder;
import org.apache.rocketmq.client.apis.ClientException;
import org.apache.rocketmq.client.apis.ClientServiceProvider;
import org.apache.rocketmq.client.apis.consumer.FilterExpression;
import org.apache.rocketmq.client.apis.consumer.FilterExpressionType;
import org.apache.rocketmq.client.apis.consumer.SimpleConsumer;
import org.apache.rocketmq.client.apis.message.MessageView;
import org.apache.rocketmq.shaded.org.slf4j.Logger;
import org.apache.rocketmq.shaded.org.slf4j.LoggerFactory;

import java.io.IOException;
import java.time.Duration;
import java.util.Collections;
import java.util.List;

public class SimpleConsumerExample {
    private static final Logger LOGGER = LoggerFactory.getLogger(SimpleConsumerExample.class);

    private SimpleConsumerExample() {
    }

    public static void main(String[] args) throws ClientException, IOException {
        // Instance endpoint. Get it from the Endpoints tab on the instance details page.
        String endpoints = "<your-endpoint>";
        // Topic and consumer group. Create both in the console first.
        String topic = "<your-fifo-topic>";
        String consumerGroup = "<your-consumer-group>";
        final ClientServiceProvider provider = ClientServiceProvider.loadService();
        ClientConfigurationBuilder builder = ClientConfiguration.newBuilder().setEndpoints(endpoints);
        // For public network access, set credentials.
        // builder.setCredentialProvider(new StaticSessionCredentialsProvider("<instance-username>", "<instance-password>"));
        ClientConfiguration clientConfiguration = builder.build();

        Duration awaitDuration = Duration.ofSeconds(10);
        // Subscribe to all tags.
        String tag = "*";
        FilterExpression filterExpression = new FilterExpression(tag, FilterExpressionType.TAG);
        SimpleConsumer consumer = provider.newSimpleConsumerBuilder()
                .setClientConfiguration(clientConfiguration)
                .setConsumerGroup(consumerGroup)
                .setAwaitDuration(awaitDuration)                    // Long polling timeout
                .setSubscriptionExpressions(Collections.singletonMap(topic, filterExpression))
                .build();
        int maxMessageNum = 16;
        Duration invisibleDuration = Duration.ofSeconds(10);       // Invisibility window for processing
        // Poll for messages in a loop. Use multiple threads for higher throughput.
        while (true) {
            final List<MessageView> messageViewList = consumer.receive(maxMessageNum, invisibleDuration);
            messageViewList.forEach(messageView -> {
                System.out.println(messageView);
                // Acknowledge each message after processing.
                try {
                    consumer.ack(messageView);
                } catch (ClientException e) {
                    e.printStackTrace();
                }
            });
        }
        // Close the consumer when no longer needed.
        // consumer.close();
    }
}

Troubleshoot consumption retries

Ordered consumption retries for PushConsumer happen on the client side -- the server does not log retry details. If a message trace shows a failed delivery result, check the consumer client logs.

For the client log path, see Log configuration.

Search for these keywords in the client logs:

Message listener raised an exception while consuming messages
Failed to consume fifo message finally, run out of attempt times

Best practices

Process messages serially, not in batches

Consume one message at a time. Batch consumption can break ordering.

Example: Messages are sent in order 1 -> 2 -> 3 -> 4. During batch consumption, messages 2 and 3 are processed together and fail. On retry, both 2 and 3 are redelivered -- but message 2 may be processed again after message 3 already succeeded on another attempt, resulting in out-of-order consumption.

Distribute message groups to avoid hot spots

ApsaraMQ for RocketMQ uses the message group value to determine which server-side queue stores each message. All messages in the same group are routed to the same queue. Concentrating too many messages in a few groups overloads those queues, creating storage hot spots and limiting scalability.

Use fine-grained keys as message groups -- for example, order IDs or user IDs. The more distinct your message group values, the more evenly messages are distributed across queues. This keeps messages for the same entity in order while spreading load across queues for different entities.