Problem description
When you use a Sarama Go client, the following issues may occur:
After you add a partition for a topic, the client may not detect the new partition or consume messages in the partition. You must restart the client to enable the client to consume messages from the new partition.
Many protocols of Sarama Go clients do not comply with the standards of the Apache Kafka community. When exceptions occur in a broker, the following issues may occur:
The OutOfRange mechanism may be triggered on the client. If the consumer offset reset policy is set to
Oldest(earliest)on the client, messages are reconsumed from the earliest offset.The client may remain in the Rebalance state.
Solution
We recommend that you replace your Sarama Go client with a Confluent Go client at the earliest opportunity.
For a demo of the Confluent Go client, see kafka-confluent-go-demo.
If you cannot replace the client immediately, consider these workarounds:
For production environments, set the consumer offset reset policy to
Newest(latest). For debugging environments or other scenarios where receiving a large number of duplicate messages is acceptable, set the policy toOldest(earliest).If a consumer offset reset causes a significant message backlog, reset the consumer offset to a specific point in time in the ApsaraMQ for Kafka console. This way, you do not need to modify your code or change the consumer group. For more information, see Reset consumer offsets.