In addition to asynchronous decoupling, Message Queue for Apache RocketMQ can also block data peaks from the frontend to keep stability of the backend system. This requires that Message Queue for Apache RocketMQ be capable of accumulating messages. Message Queue for Apache RocketMQ supports the accumulation of billions of messages and does not significantly decrease performance due to message accumulation.

Problem description

On the Consumer Status page of the Message Queue for Apache RocketMQ console, the value of Real-time Message Accumulation for a group ID exceeds the expected value. In addition, the performance decreases significantly.

Solution

You can take the following measures:

  1. Log on to the Message Queue for Apache RocketMQ console, obtain the host IP address of the consumer instance where messages are accumulated by View the consumer status, and log on to the host or container.
  2. Run either of the following commands to view the process ID:
    • ps -ef |grep java
    • jps -lm
  3. Run the following command to view the stack information:
    jstack -l pid > /tmp/pid.jstack
  4. Run the following command to view the ConsumeMessageThread information and focus on the thread status and stack:
    cat /tmp/pid.jstack|grep ConsumeMessageThread -A 10 --color

    The following figure shows the command output.

For more information about the thread status, see official Java documentation.

References

If the performance decrease caused by message accumulation persists, open a ticket. Attach the following information with the ticket:

  • heap.bin file

    You can obtain this file by running the jmap -dump:format=b,file=heap.bin [pid] command and then run the gzip heap.bin command to generate a compressed package. [pid] indicates the process ID you found in the first step.

  • The local ons.log file of the consumer client where messages are accumulated
  • Version of the consumer client