Migrate consumer group metadata between ApsaraMQ for Kafka instances - ApsaraMQ for Kafka

This topic describes how to use the metadata migration tool provided by ApsaraMQ for Kafka to migrate consumer group metadata from one ApsaraMQ for Kafka instance to another ApsaraMQ for Kafka instance.

Prerequisites

Before you migrate consumer group metadata between ApsaraMQ for Kafka instances, make sure that the following operations are complete:

Java Development Kit (JDK) 8 is downloaded. For more information, see Java SE Development Kit 8 Downloads.
The Java Archive (JAR) file of the migration tool is downloaded. To download the JAR file, click Java Archive file (JAR) of the migration tool Downloads.

Important

After the migration is complete, the consumer group in the source ApsaraMQ for Kafka instance is not deleted. Instead, a consumer group with the same configurations is created in the destination ApsaraMQ for Kafka instance.
Only the consumer group configurations are migrated. Topics and consumer offsets are not migrated.

Procedure

Start the command-line tool.
Run the cd command to switch to the directory where the migration tool is located.

Run the following command to check the consumer group metadata to be migrated:

java -jar kafka-migration.jar ConsumerGroupMigrationFromAliyun --sourceAk <yoursourceAccessKeyId> --sourceSk <yoursourceAccessKeySecret> --sourceRegionId <yoursourceRegionId> --sourceInstanceId <yoursourceInstanceId> --destAk <yourdestAccessKeyId> --destSk <yourdestAccessKeySecret> --destRegionId <yourdestRegionId> --destInstanceId <yourdestInstanceId>

Parameter	Description
sourceAk	The AccessKey ID of the Alibaba Cloud account to which the source ApsaraMQ for Kafka instance belongs.
sourceSk	The AccessKey secret of the Alibaba Cloud account to which the source ApsaraMQ for Kafka instance belongs.
sourceRegionId	The ID of the region where the source ApsaraMQ for Kafka instance is located.
sourceInstanceId	The ID of the source ApsaraMQ for Kafka instance.
destAk	The AccessKey ID of the Alibaba Cloud account to which the destination ApsaraMQ for Kafka instance belongs.
destSk	The AccessKey secret of the Alibaba Cloud account to which the destination ApsaraMQ for Kafka instance belongs.
destRegionId	The ID of the region where the destination ApsaraMQ for Kafka instance is located.
destInstanceId	The ID of the destination ApsaraMQ for Kafka instance.

The following code provides an example of the consumer group metadata to be checked:

10:54:26 INFO - cmd=ConsumerGroupMigrationFromAliyun, request=null, response={"code":200,"requestId":"9793DADB-55A5-4D4E-9E9C-D4DA8B35370C","success":true,"consumerList":[{"instanceId":"alikafka_post-cn-0pp1h0uv6***","regionId":"cn-hangzhou","consumerId":"Demo","tags":[{"value":"","key":"migration"}]}],"message":"operation success."}
10:54:26 INFO - Will create consumer groups:[Demo]

Run the following command to commit the consumer group metadata to be migrated:
java -jar kafka-migration.jar ConsumerGroupMigrationFromAliyun --sourceAk LTAI4FwQ5aK1mFYCspJ1**** --sourceSk wvDxjjRQ1tHPiL0oj7Y2Z7WDNk**** --sourceRegionId cn-hangzhou --sourceInstanceId alikafka_post-cn-0pp1h0uv**** --destAk LTAI4FwQ5aK1mFYCspJ1**** --destSk wvDxjjRQ1tHPiL0oj7Y2Z**** --destRegionId cn-hangzhou --destInstanceId alikafka_pre-cn-v0h1cng0**** --commit
Parameter
Description
commit
Commits the consumer group metadata to be migrated.
The following code provides an example of the output generated after the consumer group metadata to be migrated is committed:
```
10:54:40 INFO - cmd=ConsumerGroupMigrationFromAliyun, request=null, response={"code":200,"requestId":"49E53B79-3C2C-4BCF-8BC8-07B0BB14B52A","success":true,"consumerList":[{"instanceId":"alikafka_post-cn-0pp1h0uv6***","regionId":"cn-hangzhou","consumerId":"Demo","tags":[{"value":"","key":"migration"}]}],"message":"operation success."}
10:54:41 INFO - cmd=ConsumerGroupMigrationFromAliyun, request=null, response={"code":200,"requestId":"5AEEFB13-2A6B-4265-97CB-902CFA483339","success":true,"message":"operation success"}
10:54:41 INFO - ConsumerCreate success, consumer group=Demo
```
Check whether the group is migrated.
1. Log on to the ApsaraMQ for Kafka console.
2. In the Resource Distribution section of the Overview page, select the region where your instance is deployed.
3. On the Instances page, click the name of the instance that you want to manage.
4. In the left-side navigation pane, click Groups.
5. On the Groups page, check whether the group is displayed.

Parameter	Description
commit	Commits the consumer group metadata to be migrated.