This topic introduces the basic principles, scenarios, and use cases of RocketMQ message tracing.
Note: RocketMQ supports message tracing over TCP currently.
Definition: Message tracing shows the complete links of a message from its issuance by the producer to its consumption by a consumer, and consists of the time, location, and other information on every node.
Principles:In RocketMQ, a complete link of a message consists of the producer, service provider, and consumer. Each role adds relevant information to the trace when it processes the message. The information is aggregated to reflect the current status of a message, providing robust data support for troubleshooting in production environments.
Message trace data
|Producer information||Consumer information||Service provider information|
|Production client information||Consumption client information||The message topic.|
|Sending time||Delivery time, number of delivery retries||Message storage region|
|Sent or not||Consumed or not||Message key|
|Sending duration||Consumption duration||Message tag|
Message trace query rules:
|Message type||Time that can be queried||Query description|
|Normal message||After the message is sent||Sending trace is generated once the message is sent. If the message is not consumed yet, Not Consumed appears. Once it is consumed, the delivery and consumption information appears.|
|Ordered message||After the message is sent||Sending trace is generated once the message is sent. If the message is not consumed yet, Not Consumed appears. Once it is consumed, the delivery and consumption information appears.|
|Scheduled and delayed messages||After the current system time reaches the specified message consumption time||If the current system time does not reach the specified message consumption time, the trace can be queried while the message cannot.|
|Transactional message||After the message is sent||Sending trace is generated once the message is sent. Before the transaction is committed, the trace can be queried while the message cannot.|
You can use the message tracing tool for troubleshooting when messages are not sent and received as expected in the production environment. Query the trace of a message by its properties (message ID, message key, or topic) to check the sending and receiving status of the message, helping diagnose the problem.
Assume that you find out that you have not received a message as expected based on the log information. Perform the following steps to troubleshoot the issue with message tracing:
Collect the information about the message that you suspect went wrong, including the message ID, message key, topic, and approximate sending time.
Log on to the RocketMQ console, and create a query task with the collected information to query the message trace.
Check the results and analyze for the cause. If the trace shows that the message has not been consumed, you can go to the Consumers page to check whether there are any messages accumulated.
If the message has been consumed, locate the corresponding client and the message delivery time in the consumption information, and log on to the client to view the log.