Message Queue for Apache RocketMQ[******] uses different message load balancing policies for the producer and consumer. For the consumer, the message load balancing policy has certain impact on message accumulation.
Message load balancing policy for the producer
Message Queue for Apache RocketMQ The round-robin scheduling mechanism is used for the producer. To be specific, messages from the producer are sent to queues in round-robin mode, as shown in the following figure.
In the preceding figure, the numbers on the arrow lines indicate the sequence. The producer sends the first message to Queue 0, the second message to Queue 1, and follows the same rule for other messages. The producer sends the eighth message to Queue 7 and the ninth message to Queue 0 again, implementing round-robin scheduling.
Message load balancing policy for the consumer
Message Queue for Apache RocketMQ[******] includes nodes such as brokers and name servers. A broker node reports the route information of a topic to a name server.
Assume that Message Queue for Apache RocketMQ includes only one broker node. When messages are sent from the producer to topics of Message Queue for Apache RocketMQ, the messages under these topics are balanced to the eight queues (logical concepts) by default.Message Queue for Apache RocketMQ Then, the broker node equally distributes these queues to consumers in a consumer cluster that has the same group ID.
Therefore, each consumer may possibly process the following numbers of queues:
- If the number of consumers is greater than the number of queues, the additional consumers will not process messages from any queues. For example, in the following figure, consumer 9 and consumer 10 do not process any messages from the eight queues.
- If the number of consumers is equal to the number of queues, each consumer will process messages from one queue, as shown in the following figure.
- If the number of consumers is smaller than the number of queues, each consumer will process messages on multiple queues, as shown in the following figure.
A consumer may not process messages in a queue promptly due to various reasons such as limitations of the hardware or system, remote procedure calls (RPCs), or Java garbage collection (GC). In addition, Message Queue for Apache RocketMQ maintains message loads at the granularity of queues. Therefore, the messages on the entire queue may be accumulated.