Migrate topic metadata from a user-created Kafka cluster to Message Queue for Apache Kafka - User guide| Alibaba Cloud Documentation Center

This topic describes how to use a metadata migration tool provided by Message Queue for Apache Kafka to migrate topic metadata from a user-created Kafka cluster to a Message Queue for Apache Kafka instance.

Prerequisites

The following operations are completed:

Notice

After the migration, the corresponding topic metadata in the source user-created Kafka cluster is not deleted. Instead, a new topic with the same configuration is created in the destination Message Queue for Apache Kafka instance.
Only the topic configuration is migrated, whereas the messages in the topic are not migrated.

Procedure

Start the command-line tool.
Run the cd command to switch to the directory where the migration tool is located.

Run the following command to confirm the topic metadata to be migrated:

java -jar kafka-migration.jar TopicMigrationFromZk --sourceZkConnect 192.168.XX.XX
                           --destAk <yourdestAccessKeyId> --destSk <yourdestAccessKeySecret> --destRegionId <yourdestRegionId>
                           --destInstanceId <yourdestInstanceId>


Parameter	Description
sourceZkConnect	The IP address of the source user-created Kafka cluster that uses ZooKeeper.
destAk	The AccessKey ID of the Alibaba Cloud account to which the destination Message Queue for Apache Kafka instance belongs.
destSk	The AccessKey secret of the Alibaba Cloud account to which the destination Message Queue for Apache Kafka instance belongs.
destRegionId	The ID of the region where the destination Message Queue for Apache Kafka instance is located.
destInstanceId	The ID of the destination Message Queue for Apache Kafka instance.

The following code provides an example of the output to be confirmed:

13:40:08 INFO - Begin to migrate topics:[test]
13:40:08 INFO - Total topic number:1
13:40:08 INFO - Will create topic:test, isCompactTopic:false, partition number:1

Run the following command to commit the topic metadata to be migrated:

java -jar kafka-migration.jar TopicMigrationFromZk --sourceZkConnect 192.168.XX.XX
                           --destAk <yourAccessKeyId> --destSk <yourAccessKeySecret> --destRegionId <yourRegionID>
                           --destInstanceId <yourInstanceId> --commit


Parameter	Description
commit	Commits the topic metadata to be migrated.

The following code provides an example of the output generated after the migration is committed:

13:51:12 INFO - Begin to migrate topics:[test]
13:51:12 INFO - Total topic number:1
13:51:13 INFO - cmd=TopicMigrationFromZk, request=null, response={"code":200,"requestId":"7F76C7D7-AAB5-4E29-B49B-CD6F1E0F508B","success":true,"message":"operation success"}
13:51:13 INFO - TopicCreate success, topic=test, partition number=1, isCompactTopic=false

Check whether the topic metadata is migrated.
1. Log on to the Message Queue for Apache Kafka console.
2. In the top navigation bar, select the region where the destination instance is located.
3. In the left-side navigation pane, click Topics.
4. On the Topics page, click the destination instance.
  The migrated topic appears on the Topics page.