This topic describes how to use the metadata migration tool provided by Message Queue for Apache Kafka to migrate topic metadata from a self-managed Apache Kafka cluster to a Message Queue for Apache Kafka instance.

Prerequisites

Before you migrate topic metadata from a self-managed Apache Kafka cluster to Message Queue for Apache Kafka, make sure that the following operations are complete:
  • Java Development Kit (JDK) 8 is downloaded. For more information, see Java SE Development Kit 8 Downloads.
  • The Java Archive (JAR) file of the migration tool is downloaded. To download the JAR file, click here.
Notice
  • After the migration is complete, the source topic in the self-managed Apache Kafka cluster is not deleted. Instead, a new topic with the same configuration is created in the destination Message Queue for Apache Kafka instance.
  • Only the topic configuration is migrated. The messages in the source topic are not migrated.

Procedure

  1. Start the command-line tool.
  2. Run the cd command to switch to the directory where the migration tool is located.
  3. Run the following command to check the topic metadata to be migrated:
    java -jar kafka-migration.jar TopicMigrationFromZk --sourceZkConnect 192.168.XX.XX --destAk <yourdestAccessKeyId> --destSk <yourdestAccessKeySecret> --destRegionId <yourdestRegionId> --destInstanceId <yourdestInstanceId>
    Parameter Description
    sourceZkConnect The IP address of the source self-managed Apache Kafka cluster that uses ZooKeeper.
    destAk The AccessKey ID of the Alibaba Cloud account to which the destination Message Queue for Apache Kafka instance belongs.
    destSk The AccessKey secret of the Alibaba Cloud account to which the destination Message Queue for Apache Kafka instance belongs.
    destRegionId The ID of the region where the destination Message Queue for Apache Kafka instance is located.
    destInstanceId The ID of the destination Message Queue for Apache Kafka instance.

    The following code provides an example of the topic metadata to be checked:

    13:40:08 INFO - Begin to migrate topics:[test]
13:40:08 INFO - Total topic number:1
13:40:08 INFO - Will create topic:test, isCompactTopic:false, partition number:1
  4. Run the following command to commit the topic metadata to be migrated:
    java -jar kafka-migration.jar TopicMigrationFromZk --sourceZkConnect 192.168.XX.XX --destAk <yourAccessKeyId> --destSk <yourAccessKeySecret> --destRegionId <yourRegionID> --destInstanceId <yourInstanceId> --commit
    Parameter Description
    commit Commits the topic metadata to be migrated.
    The following code provides an example of the output generated after the topic metadata to be migrated is committed: 
    13:51:12 INFO - Begin to migrate topics:[test]
13:51:12 INFO - Total topic number:1
13:51:13 INFO - cmd=TopicMigrationFromZk, request=null, response={"code":200,"requestId":"7F76C7D7-AAB5-4E29-B49B-CD6F1E0F508B","success":true,"message":"operation success"}
13:51:13 INFO - TopicCreate success, topic=test, partition number=1, isCompactTopic=false
  5. Check whether the required topic is migrated.
    1. Log on to the Message Queue for Apache Kafka console.
    2. In the Resource Distribution section of the Overview page, select the region where your instance resides.
    3. On the Instances page, click the name of the instance that you want to manage.
    4. In the left-side navigation pane, click Topics.
    5. On the Topics page, check whether the required topic is displayed.