ApsaraMQ for Kafka:Message retrieval

Prerequisites

Before you begin, make sure that:

• A topic exists on your ApsaraMQ for Kafka instance as the data source. For more information, see Create a topic.

• Message retrieval is available in the region where your instance resides. For more information, see Supported regions.

Billing

Message retrieval is in public preview. During the preview period:

• ApsaraMQ for Kafka does not charge for the message retrieval feature.

• Auto-created Tablestore instances and tables are also free of charge.

Limits

Step 1: Enable message retrieval

1. Log on to the ApsaraMQ for Kafka console.

2. In the Resource Distribution section of the Overview page, select the region where your instance resides.

3. In the left-side navigation pane, click Instances.

4. On the Instances page, click the name of your instance.

5. In the left-side navigation pane, click Message Retrieval, then click Enable Message Retrieval.

   Note

   If the connector feature has not been enabled on this instance, a prompt appears. Click OK to enable the connector feature, then configure the parameters in the Enable Message Retrieval panel.

6. In the Enable Message Retrieval panel, configure the following parameters and click OK. Record Time options:

   • Native Time: The message creation timestamp recorded by the producer -- either the timestamp from ProducerRecord or the time when the producer sent the message.

   • Instant Time: The time when the message was synchronized to Tablestore. This may differ from the actual production time.

   Parameters for enabling the message retrieval feature

   Parameter	Description	Example
   Data Source Topic	The topic to enable message retrieval for.	test
   Consumer Offset	The offset from which to start consuming messages. Valid values: Earliest Offset and Latest Offset. Default: Latest Offset.	Latest Offset
   Record Time	The timestamp associated with each retrieved message. Valid values: Native Time and Instant Time. Default: Native Time.	Native Time

After you enable message retrieval, the topic appears on the Message Retrieval page.

Step 2: Send a test message

To verify that the message retrieval task is running, send a test message.

1. On the Message Retrieval page, find your topic and click Details in the Actions column.

2. In the upper-right corner of the Task Details page, click Test.

3. In the Send Message panel, configure the following parameters and click OK:

   • Message Key: Enter a message key. Example: demo.

   • Message Content: Enter the message body. Example: {"key": "test"}.

   • Send to Specified Partition:

     • To send to a specific partition, select Yes and enter the partition ID. Example: 0. For more information about partition IDs, see View partition status.

     • To let ApsaraMQ for Kafka assign a partition automatically, select No.

Step 3: Retrieve messages

1. On the Message Retrieval page, find your topic and click Search in the Actions column.

2. In the Search panel, select a search condition from the dropdown, click Add Search Condition, enter the value, and click OK. You can combine one or more of the following search conditions: Keyword search rules for Key and Value: For more information about search patterns, see Tokenization, Match phrase query, and Wildcard query.

   • Phrase matching (default): Enter a phrase to match. For example, if the message key is "ApsaraMQ for Kafka is a distributed, high-throughput, and scalable message queue service", search for distributed or combine Alibaba Cloud and distributed.

   • Wildcard matching: Use * for any string and ? for any single character. Letters are case-insensitive. For example, search for AliKafkaTe* to match AliKafkaTest001qaz. Strings containing wildcards must not exceed 20 characters. Avoid starting a search keyword with a wildcard character to improve retrieval efficiency.

   Note

   Use the unique business identifier as the message key when producing messages. This enables exact matching during retrieval.

   Search parameters

   Search condition	Description
   Partition	The partition to search.
   Offset Range	The offset range to search within.
   Time Range	The time range to search within. Select from the last three days or specify a custom range, accurate to the minute.
   Key	A keyword to match against message keys.
   Value	A keyword to match against message values.

   

3. On the Topic Search page, view the retrieval results.

   Note

   Up to 10 messages are displayed per search. Specify precise search conditions for accurate results. Each message displays up to 1 KB of content in the console. Download the message to view the full content.

   Parameters in the retrieval result

   Column	Description
   Partition	The partition from which the message was retrieved.
   Offset	The message offset within the partition.
   Key	The message key, displayed as a string.
   Value	The message value (content), displayed as a string.
   Created At	The message timestamp. If you specified a value for the timestamp field, that value is displayed. Otherwise, the local system time when the message was sent is displayed. A value in the format 1970/x/x x:x:x indicates the timestamp field was set to 0 or an invalid value.
   Actions	Click Download Key to download the message key, or Download Value to download the full message content.

   

Manage retrieval tasks

View task details

After you enable message retrieval for a topic, a retrieval task is created automatically. The Task Details page shows the auto-created topics, consumer groups, and Tablestore instance and table names. From this page, navigate directly to the Tablestore table details page.

On the Message Retrieval page, find your topic and click Details in the Actions column.

View consumption progress

Monitor the consumption progress of active groups across partitions to identify message consumption and accumulation.

On the Message Retrieval page, find your topic and click Consumption Progress in the Actions column. The Consumer Details page displays the consumption status per partition.