This topic describes how to use the metadata migration tool provided by ApsaraMQ for Kafka to migrate topic metadata from one ApsaraMQ for Kafka instance to another ApsaraMQ for Kafka instance.
Prerequisites
Before you migrate topic metadata between Message Queue for Apache Kafka instances, make sure that the following operations are complete:- Java Development Kit (JDK) 8 is downloaded and installed. For more information, see Java Downloads.
- The migration tool is downloaded and installed. If the migration tool is not downloaded, click kafka-migration-assessment.jar to download the migration tool.
- A ApsaraMQ for Kafka instance is purchased and deployed.
- For information about how to purchase an instance and deploy the instance in a virtual private cloud (VPC), see Purchase and deploy an instance that allows access from a VPC.
- For information about how to purchase an instance that can be accessed over the Internet and a VPC and deploy the instance, see Purchase and deploy an instance that allows access from the Internet and a VPC.
Important
- After the migration is complete, the relevant topic in the source ApsaraMQ for Kafka instance is not deleted. Instead, a new topic with the same configuration is created in the destination ApsaraMQ for Kafka instance.
- Only the topic configuration is migrated. The messages in the source topic are not migrated.
Procedure
- Start the command-line tool.
- Run the cd command to switch to the directory where the migration tool is located.
- Run the following command to check the topic metadata to be migrated:
java -jar kafka-migration.jar TopicMigrationFromAliyun --sourceAk <yoursourceAccessKeyId> --sourceSk <yoursourceAccessKeySecret> --sourceRegionId <yoursourceRegionId> --sourceInstanceId <yoursourceInstanceId> --destAk <yourdestAccessKeyId> --destSk <yourdestAccessKeySecret> --destRegionId <yourdestRegionId> --destInstanceId <yourdestInstanceId>
Parameter Description sourceAk The AccessKey ID of the Alibaba Cloud account to which the source ApsaraMQ for Kafka instance belongs. sourceSk The AccessKey secret of the Alibaba Cloud account to which the source ApsaraMQ for Kafka instance belongs. sourceRegionId The ID of the region where the source ApsaraMQ for Kafka instance is located. sourceInstanceId The ID of the source ApsaraMQ for Kafka instance. destAk The AccessKey ID of the Alibaba Cloud account to which the destination ApsaraMQ for Kafka instance belongs. destSk The AccessKey secret of the Alibaba Cloud account to which the destination ApsaraMQ for Kafka instance belongs. destRegionId The ID of the region where the destination ApsaraMQ for Kafka instance is located. destInstanceId The ID of the destination ApsaraMQ for Kafka instance. The following code provides an example of the topic metadata to be checked:15:13:12 INFO - cmd=TopicMigrationFromAliyun, request=null, response={"total":4,"code":200,"requestId":"1CBAB340-2146-43A3-8470-84D77DB8B43E","success":true,"pageSize":10000,"currentPage":1,"message":"operation success.","topicList":[{"instanceId":"alikafka_pre-cn-0pp1cng20***","localTopic":false,"createTime":1578558314000,"regionId":"cn-hangzhou","statusName":"Running","topic":"agdagasdg","remark":"agdadgdasg","partitionNum":12,"compactTopic":false,"status":0,"tags":[]},{"instanceId":"alikafka_pre-cn-0pp1cng20***","localTopic":false,"createTime":1578558294000,"regionId":"cn-hangzhou","statusName":"Running","topic":"135215","remark":"1315215","partitionNum":12,"compactTopic":false,"status":0,"tags":[]},{"instanceId":"alikafka_pre-cn-0pp1cng20***","localTopic":false,"createTime":1578558214000,"regionId":"cn-hangzhou","statusName":"Running","topic":"1332","remark":"13414","partitionNum":12,"compactTopic":false,"status":0,"tags":[]},{"instanceId":"alikafka_pre-cn-0pp1cng20***","localTopic":false,"createTime":1578558141000,"regionId":"cn-hangzhou","statusName":"Running","topic":"aete","remark":"est","partitionNum":12,"compactTopic":false,"status":0,"tags":[]}]} 15:13:12 INFO - Will create topic:agdagasdg, isCompactTopic:false, partition number:12 15:13:12 INFO - Will create topic:135215, isCompactTopic:false, partition number:12 15:13:12 INFO - Will create topic:1332, isCompactTopic:false, partition number:12 15:13:12 INFO - Will create topic:aete, isCompactTopic:false, partition number:12
- Run the following command to commit the topic metadata to be migrated:
java -jar kafka-migration.jar TopicMigrationFromAliyun --sourceAk <yoursourceAccessKeyId> --sourceSk <yoursourceAccessKeySecret> --sourceRegionId <yoursourceRegionId> --sourceInstanceId <yoursourceInstanceId> --destAk <yourdestAccessKeyId> --destSk <yourdestAccessKeySecret> --destRegionId <yourdestRegionId> --destInstanceId <yourdestInstanceId> --commit
Parameter Description commit Commits the topic metadata to be migrated. The following code provides an example of the output generated after the topic metadata to be migrated is committed:16:38:30 INFO - cmd=TopicMigrationFromAliyun, request=null, response={"code":200,"requestId":"A0CA4D70-46D4-45CF-B9E0-B117610A26DB","success":true,"message":"operation success"} 16:38:30 INFO - TopicCreate success, topic=agdagasdg, partition number=12, isCompactTopic=false 16:38:36 INFO - cmd=TopicMigrationFromAliyun, request=null, response={"code":200,"requestId":"05E88C75-64B6-4C87-B962-A63D906FD993","success":true,"message":"operation success"} 16:38:36 INFO - TopicCreate success, topic=135215, partition number=12, isCompactTopic=false 16:38:42 INFO - cmd=TopicMigrationFromAliyun, request=null, response={"code":200,"requestId":"9D54F6DB-6FA0-4F6D-B19A-09109F70BDDA","success":true,"message":"operation success"} 16:38:42 INFO - TopicCreate success, topic=1332, partition number=12, isCompactTopic=false 16:38:49 INFO - cmd=TopicMigrationFromAliyun, request=null, response={"code":200,"requestId":"6C265013-D15E-49AE-BE55-BF7657ADA1B7","success":true,"message":"operation success"} 16:38:49 INFO - TopicCreate success, topic=aete, partition number=12, isCompactTopic=false
- Check whether the topic is migrated.
- Log on to the ApsaraMQ for Kafka console.
- In the Resource Distribution section of the Overview page, select the region where your instance is deployed.
- On the Instances page, click the name of the instance that you want to manage.
- In the left-side navigation pane, click Topics.
- On the Topics page, check whether the topic is displayed.