If you cannot consume a message as expected or issues such as message loss and message delay occur in a message when you use ApsaraMQ for Kafka, you can use the message query feature to query the message to troubleshoot the issue. If you know the ID of the partition in which the message is stored and the offset of the message, you can query the message by offset. If you do not know the offset of the message but know the time range in which the message was sent, you can query the message by time.
Usage notes
In the ApsaraMQ for Kafka console, up to 1 KB of content for each queried message is displayed. If a message is larger than 1 KB in size, the system automatically truncates the excess content of the message. If you want to view the complete content of a message, download the message. The total size of messages that you download at a time cannot exceed 10 MB.
Up to 10 queried messages can be displayed in the ApsaraMQ for Kafka console, and the total size of the displayed messages cannot exceed 10 MB.
If the total size of three queried messages exceeds 10 MB, only two messages are displayed in the ApsaraMQ for Kafka console.
If the total size of the 10 messages is less than 10 MB, you can view up to 10 messages in the ApsaraMQ for Kafka console. The number of messages that are actually displayed in the ApsaraMQ for Kafka console is subject to the actual consumption data of the consumer.
Query results are related to the following message cleanup policies of ApsaraMQ for Kafka:
If the disk usage of an instance is lower than 75%, messages whose retention period expires are deleted at 04:00 every day.
If the disk usage of an instance is equal to or higher than 75% but lower than 85%, messages whose retention period expires are regularly deleted to keep the disk usage below 75%.
If the disk usage of an instance is equal to or higher than 85% but lower than 90%, the earliest stored messages on the ApsaraMQ for Kafka broker are deleted, regardless of whether their retention period expires.
If the disk usage of an instance is equal to or higher than 90%, no more messages can be sent to the instance. This ensures service stability.
ImportantWhen messages are deleted, at least one storage file is retained by ApsaraMQ for Kafka. As a result, when you query messages, the query results may contain messages whose retention period has expired.
Query messages by offset
Log on to the ApsaraMQ for Kafka console.
In the Resource Distribution section of the Overview page, select the region where the ApsaraMQ for Kafka instance that you want to manage resides.
On the Instances page, click the name of the instance that you want to manage.
In the left-side navigation pane, click Message Query.
On the Message Query page, select Search by offset from the Search Method drop-down list.
Select a topic from the Topic drop-down list and a partition from the Partition drop-down list, enter an offset value in the Offset field, and then click Search.
Messages whose offset values are equal to or greater than the specified offset value are displayed. For example, if the specified partition and offset are both 5, the query results are messages that start from Offset 5 in Partition 5.
Table 1. Parameters in message query results Parameter
Description
Partition
The partition from which the message is retrieved.
Offset
The message offset.
Key
The message key. The key is forcefully transformed into a string.
Value
The message value, which is also known as message content. The message value is forcefully transformed into a string.
Created At
The timestamp that is recorded by the producer when the message was sent or the value of the timestamp field that you specified for the
ProducerRecordobject.NoteIf a value is specified for the timestamp field, the specified value is displayed.
If no value is specified for the timestamp field, the system time when the message was sent is displayed.
A value in the 1970/x/x x:x:x format indicates that the timestamp field is specified as 0 or an invalid value.
You cannot specify the timestamp field on ApsaraMQ for Kafka clients of version 0.9 or earlier.
Actions
Click Download Key to download the message key.
Click Download Value to download the message content.
Query messages by time
You can query messages in all partitions based on a specific point in time. If you do not know the offset of the message that you want to query but know the time range in which the message was sent, you can specify a point in time in the time range to query messages that were sent at and after the specified point in time.
On the Message Query page, select Search by Point in Time from the Search Method drop-down list.
Select a topic from the Topic drop-down list and a partition from the Partition drop-down list, specify a point in time in the Point in Time box, and then click Search.
Messages that were sent at and after the specified point in time are displayed. For information about parameters that are included in a query result, see Parameters in message query results.
If you select All Partitions, messages in all partitions are displayed.
If you specify a partition, messages in the specified partition are displayed.
References
For information about offsets, see Terms.
For information about consumer details, see View consumer details.