ApsaraMQ for RocketMQ:Clustering consumption and broadcasting consumption

Background information

ApsaraMQ for RocketMQ is a messaging system that is based on the message publish/subscribe pattern. In ApsaraMQ for RocketMQ, a consumer subscribes to a topic to obtain and consume messages.

In most cases, consumer applications are clustered and deployed in a distributed system. When you subscribe to a topic for a consumer, you can specify the clustering consumption mode or the broadcasting consumption mode for the consumer based on your business requirements. Consumers in the same group consume messages based on the consumption mode that you specified.

Consumers that use the same group ID belong to the same group. These consumers must use the same consumption logic and the same tags. For more information, see Subscription consistency.

Clustering consumption mode

Basic concept

Scenarios

This mode is suitable for scenarios in which consumer instances are deployed in a group and each message needs to be processed only once. The following figure shows an example.

Broadcasting consumption mode

Basic concept

Broadcasting consumption: In this mode, ApsaraMQ for RocketMQ broadcasts each message to all consumers in a group to ensure that the message is consumed by each consumer at least once.

Scenarios

This mode is suitable for scenarios in which consumer instances are deployed in a group and each message needs to be processed by each consumer in the group. The following figure shows an example on how to consume messages in broadcasting consumption mode.

Feature differences between the clustering consumption mode and the broadcasting consumption mode

Feature	Clustering consumption	Broadcasting consumption
TCP client SDKs
HTTP client SDKs
Ordered messages
Consumer offset resetting
Message retries
Accumulated message query and message accumulation alerting
Subscription relationship query
Consumption progress	The consumption process is maintained by brokers. The service reliability is higher in clustering consumption mode than that in broadcasting consumption mode. After a consumer is restarted, the consumer continues to consume messages based on the last consumption progress. The message retry mechanisms are maintained on brokers. For more information, see Message retry.	The consumption process is maintained by consumers. The probability that a message is repeatedly consumed in broadcasting consumption mode is higher than that in clustering consumption mode. Each time a consumer is restarted, the consumer starts consumption from the latest message.

Precautions

Clustering consumption

• In clustering consumption mode, each message is delivered to only one consumer in a group for processing. If a message needs to be processed by each consumer in the group, use the broadcasting consumption mode.
• In clustering consumption mode, a message may be routed to different consumers during message retries.

Broadcasting consumption

• In broadcasting consumption mode, ApsaraMQ for RocketMQ ensures that a message is delivered to each consumer at least once for consumption, but does not resend messages that fail to be consumed. You need to manually handle consumption failures.
• In broadcasting consumption mode, a consumer starts message consumption from the latest message each time the consumer is restarted. The consumer automatically skips the messages that were sent to the ApsaraMQ for RocketMQ broker after the consumer was stopped. Therefore, exercise caution when you use this mode.
• In broadcasting consumption mode, each message is repeatedly processed by a large number of consumers. In most cases, we recommend that you use the clustering consumption mode.

Specify a consumption mode

When you use a client SDK to subscribe to messages, you can configure the relevant parameter in the SDK to specify a consumption mode.

TCP client SDKs

• Clustering consumption

  // Set the MessageModel parameter to CLUSTERING. By default, the clustering consumption mode is used. 
  properties.put(PropertyKeyConst.MessageModel, PropertyValueConst.CLUSTERING);

• Broadcasting consumption

  // Set the MessageModel parameter to BROADCASTING. 
  properties.put(PropertyKeyConst.MessageModel, PropertyValueConst.BROADCASTING); 

HTTP client SDKs