This topic introduces concepts of Message Queue for Apache RocketMQ clustering consumption and broadcasting consumption, their application scenarios, and precautions for use.

Message Queue for Apache RocketMQ is a messaging system that is based on the message publish/subscribe pattern. In Message Queue for Apache RocketMQ, consumers subscribe to a topic to obtain and consume messages. Consumer applications are deployed in clusters in a distributed system. Therefore, Message Queue for Apache RocketMQ defines the following terms:

  • Cluster: Consumers using the same group ID belong to the same cluster. These consumers must have the same consumption logic, including the same tags. For more information, see Subscription consistency.
  • Clustering consumption: In this pattern, a message only needs to be processed by any consumer in the cluster for Message Queue for Apache RocketMQ.
  • Broadcasting consumption: In this mode, Message Queue for Apache RocketMQ broadcasts each message to all clients registered in the cluster to ensure that the message is consumed by each consumer at least once.

Clustering consumption pattern

  • Scenarios
    This pattern is applicable to the scenario where consumer instances are deployed in a cluster and each message needs to be processed only once. The consumption progress is maintained on the Message Queue for Apache RocketMQ broker, ensuring high reliability. The following figure shows a consumption example.Clustering consumption
  • Precautions
    • In clustering consumption pattern, each message is delivered to only one machine in the cluster for processing. If a message needs to be processed by each machine in the cluster, use the broadcasting consumption pattern instead.
    • In clustering consumption pattern, a message may be routed to different machines during consumption retries.

Broadcasting consumption pattern

  • Scenarios
    This pattern is applicable to the scenario where consumer instances are deployed in a cluster and each message needs to be processed by each consumer in the cluster. The following figure shows a consumption example.Broadcasting consumption
  • Precautions
    • Ordered messages are not supported in broadcasting consumption pattern.
    • Consumption offsets cannot be reset in broadcasting consumption pattern.
    • Each message needs to be processed by multiple machines with the same subscription logic.
    • The consumption progress is maintained at the client. The ratio of consumption repetition is higher than that of the clustering consumption pattern.
    • In broadcasting consumption pattern, Message Queue for Apache RocketMQ ensures that each message is consumed by each client at least once, but does not resend any messages that fail to be consumed. Therefore, your customer needs to pay attention to consumption failures.
    • In broadcasting consumption pattern, the consumption starts from the latest message every time the client is restarted, but skips the messages sent to the Message Queue for Apache RocketMQ broker when the client is stopped. For this reason, use this pattern with caution.
    • In broadcasting consumption pattern, each message is repeatedly processed by many clients. Therefore, we recommend that you use the clustering consumption pattern whenever possible.
    • In broadcasting consumption mode, the Message Queue for Apache RocketMQ broker does not maintain the consumption progress, so you cannot query message accumulation, set message accumulation alerts, or query subscriptions in the Message Queue for Apache RocketMQ console.

References

For more information about how to configure the clustering consumption pattern and broadcasting consumption pattern, see the following documents: