Resetting a consumer offset changes the current consumption position of a subscriber. If a consumer fails or consumes incorrect data, you can reset the consumer offset to roll back to a previous position and restart consumption. Alternatively, you can move the consumption position to the latest offset to temporarily skip accumulated messages.
Background information
ApsaraMQ for RocketMQ uses consumer offsets to manage message consumption progress. After a consumer processes a message, the message is not immediately deleted from the queue. ApsaraMQ for RocketMQ maintains a consumption record for each group. This record, known as the consumer offset, specifies the offset of the latest message that consumers in the group have consumed for a specific topic.
When a consumer client goes offline and then comes back online, it resumes processing messages based on the consumption progress saved on the server-side.
If consumption is abnormal or the current consumer offset is incorrect, you can adjust the consumption progress by resetting the consumer offset.
ApsaraMQ for RocketMQ supports resetting consumer offsets in the following ways:
Start Consumption from Latest Offset
Consumers that process messages in a specific topic skip all accumulated messages and start consumption from the latest message sent after the reset operation.
Start Consumption from Offset Corresponding to Specified Point in Time
Consumers process messages from the consumer offset that corresponds to the specified time, regardless of whether these messages were previously consumed.
The selectable time range is between the creation times of the earliest and latest messages stored in the topic.
When you reset to a consumer offset at a specific point in time, the server-side automatically finds the closest matching consumer offset.
Scenarios
Clear message accumulation: If the performance of a downstream consumption system is insufficient or the consumption speed is slower than the production speed, many messages can accumulate. If these accumulated messages can be discarded, you can reset the consumer offset to a specified position. This lets you bypass the accumulated messages and reduce the processing pressure on the downstream system.
Backtrack for corrective processing: If the business logic for consumption is abnormal and messages are processed incorrectly, you may need to re-consume these messages. To do this, you can reset the consumer offset to a historical position. This enables consumption backtracking.
Usage notes
Resetting a consumer offset applies to the consumer offset of a specified group for a target topic. It does not affect the consumption progress of other groups subscribed to the same topic.
Resetting consumer offsets is not supported in broadcasting consumption mode.
ApsaraMQ for RocketMQ supports resetting consumer offsets only for groups that use the TCP protocol, not for groups that use the HTTP protocol.
The reset consumer offset feature resets only messages that are visible to consumers. It cannot reset scheduled messages or messages that are awaiting retry.
You can reset consumer offsets only when consumers are online.
Procedure
Log on to the ApsaraMQ for RocketMQ console. In the left-side navigation pane, click Instances.
In the top navigation bar, select a region, such as China (Hangzhou). On the Instances page, click the name of the instance that you want to manage.
In the navigation pane on the left, click Groups. Then, click the TCP tab.
Find the group ID for which you want to reset the consumer offset. In the Actions column, click More and select Reset Consumer Offset from the drop-down menu.
In the Reset Consumer Offset panel, select the target topic and reset method, and then click OK.
WarningIf you select Start Consumption from Latest Offset, all accumulated messages for the group ID in the specified topic are cleared. This operation takes effect in approximately 2 to 3 minutes. Do not repeat the operation. During this period, all consumers in the application pause consumption for 2 to 3 minutes. Exercise caution if your business is latency-sensitive.
Start Consumption from Latest Offset: If you select this option, consumers in the group skip all accumulated messages in the specified topic and start consumption from the latest message.
Start Consumption from Offset Corresponding to Specified Point in Time: If you select this option, a time selection control is displayed. You can specify a point in time. Consumers start to consume messages that are sent after this time, regardless of whether the messages were previously consumed.
In the Prompt dialog box, confirm the information and click OK.
FAQ
What do I do if I fail to reset a consumer offset using both methods?
Confirm that you are using clustering consumption. The broadcasting consumption mode does not support resetting consumer offsets.
Confirm that the consumer is online. You can reset consumer offsets only when consumers are online.
Check the SDK version. This issue can occur if the SDK version is outdated. We recommend that you upgrade the SDK to the latest version. For example, the version of the TCP client SDK for Java must be 1.8.0.Final or later. For more information, see Version Guide.
Why are some messages not reset after I reset the consumer offset to skip accumulated messages?
The consumer offset reset feature does not apply to messages that are in the retry state. Therefore, a small number of messages that are awaiting retry may still be delivered after a reset.
What is the earliest message consumers can re-consume by resetting the consumer offset?
When you reset an offset to a specified point in time, the earliest time that you can select corresponds to when the earliest message in the topic was stored. ApsaraMQ for RocketMQ 4.x series instances store messages for a maximum of 3 days. Messages older than 3 days are deleted from the topic.
Therefore, consumers can re-consume messages that are up to 3 days old by resetting the consumer offset.
References
In addition to the console, ApsaraMQ for RocketMQ lets you reset consumer offsets by calling API operations.