This topic describes how to use a metadata migration tool provided by Message Queue for Apache Kafka to migrate topic metadata from a user-created Kafka cluster to a Message Queue for Apache Kafka instance.
Prerequisites
The following operations are completed:
Notice
- After the migration, the corresponding topic metadata in the source user-created Kafka
cluster is not deleted. Instead, a new topic with the same configuration is created
in the destination Message Queue for Apache Kafka instance.
- Only the topic configuration is migrated, whereas the messages in the topic are not
migrated.
Procedure
- Start the command-line tool.
- Run the cd command to switch to the directory where the migration tool is located.
- Run the following command to confirm the topic metadata to be migrated:
java -jar kafka-migration.jar TopicMigrationFromZk --sourceZkConnect 192.168.XX.XX
--destAk <yourdestAccessKeyId> --destSk <yourdestAccessKeySecret> --destRegionId <yourdestRegionId>
--destInstanceId <yourdestInstanceId>
Parameter |
Description |
sourceZkConnect |
The IP address of the source user-created Kafka cluster that uses ZooKeeper. |
destAk |
The AccessKey ID of the Alibaba Cloud account to which the destination Message Queue for Apache Kafka instance belongs.
|
destSk |
The AccessKey secret of the Alibaba Cloud account to which the destination Message Queue for Apache Kafka instance belongs.
|
destRegionId |
The ID of the region where the destination Message Queue for Apache Kafka instance is located.
|
destInstanceId |
The ID of the destination Message Queue for Apache Kafka instance.
|
The following code provides an example of the output to be confirmed:
13:40:08 INFO - Begin to migrate topics:[test]
13:40:08 INFO - Total topic number:1
13:40:08 INFO - Will create topic:test, isCompactTopic:false, partition number:1
- Run the following command to commit the topic metadata to be migrated:
java -jar kafka-migration.jar TopicMigrationFromZk --sourceZkConnect 192.168.XX.XX
--destAk <yourAccessKeyId> --destSk <yourAccessKeySecret> --destRegionId <yourRegionID>
--destInstanceId <yourInstanceId> --commit
Parameter |
Description |
commit |
Commits the topic metadata to be migrated. |
The following code provides an example of the output generated after the migration
is committed:
13:51:12 INFO - Begin to migrate topics:[test]
13:51:12 INFO - Total topic number:1
13:51:13 INFO - cmd=TopicMigrationFromZk, request=null, response={"code":200,"requestId":"7F76C7D7-AAB5-4E29-B49B-CD6F1E0F508B","success":true,"message":"operation success"}
13:51:13 INFO - TopicCreate success, topic=test, partition number=1, isCompactTopic=false
- Check whether the topic metadata is migrated.
- Log on to the Message Queue for Apache Kafka console.
- In the top navigation bar, select the region where the destination instance is located.
- In the left-side navigation pane, click Topics.
- On the Topics page, click the destination instance.
The migrated topic appears on the Topics page.