ApsaraMQ for MQTT:Shared subscriptions

Introduction

MQTT 3.3.1 and 5.0 introduced the shared subscription feature to enable load balancing by distributing messages among multiple clients. In the standard MQTT protocol, every client that subscribes to the same topic receives a copy of each message. Shared subscriptions change this behavior. Messages are distributed among a group of clients, and each message is sent to only one client in the group.

Limits

• This feature is available only for Professional and Platinum Edition instances. To enable this feature, you can submit a ticket.

• Shared subscriptions support only Quality of Service (QoS) 1 semantics, which means at-least-once delivery. Duplicate messages may be pushed in scenarios such as slow client consumption, client reconnections, server upgrades, and server breakdowns.

• The number of consumer groups cannot exceed 5,000.

Scenarios

• Load balancing: In large-scale distributed systems, shared subscriptions provide load balancing for clients, especially with high message volumes.

• System redundancy and reliability: Running multiple client instances improves system redundancy and reliability. If one client fails, other clients can take over.

• High availability services: Shared subscriptions provide high availability. Multiple processing nodes ensure service stability and continuity.

Example

Consider a messaging system with multiple backend worker nodes. These nodes process new order messages that are published to the `orders/new` topic. Using shared subscriptions, the orders are distributed evenly among all online worker nodes.

The worker node clients subscribe as follows:

SUBSCRIBE
Topic Filter: $share/group1/orders/new

Here, $share/group1/ is the shared subscription identifier, `group1` is the consumer group name, and `orders/new` is the topic that the clients subscribe to.

When a new order message is published to `orders/new`, only one client in the consumer group receives and processes the order:

PUBLISH
Topic: orders/new
Payload: { "order_id": "xyz", "product": "ABC" }

The MQTT broker uses an algorithm, such as polling, random, or hashing, to distribute messages evenly among the clients in the `group1` consumer group. This achieves load balancing.

Shared subscriptions make MQTT ideal for enterprise applications that handle high traffic and require high availability and reliability. By distributing the message payload across multiple subscribers, this feature prevents a single subscriber from becoming a bottleneck, which improves system extensibility and robustness.