:Migrate metadata using a file

Step 1: Export metadata

Use the metadata export tool to export the metadata file from your self-managed Kafka cluster.

1. Download the metadata export tool: kafka-migration-assessment.jar.

2. Upload the migration tool to your self-managed Kafka cluster.

3. In the directory that contains the tool, run the following command to grant execute permissions to the JAR file.

   chmod 777 kafka-migration-assessment.jar

4. Run the following command to export the metadata.

   java -jar kafka-migration-assessment.jar MigrationFromZk  \ 
   --sourceZkConnect <host:port>  \ 
   --sourceBootstrapServers <host:port>  \ 
   --targetDirectory ../xxx/  \ 
   --fileName metadata.json  \ 
   --commit

   Parameter	Description	Example
   sourceZkConnect	The IP address and port number of ZooKeeper for your self-managed Kafka cluster. If you do not specify this parameter, the tool automatically obtains the value.	192.168.XX.XX:2181
   sourceBootstrapServers	The IP address and port number of your self-managed Kafka cluster. If you do not specify this parameter, the tool automatically obtains the value.	192.168.XX.XX:9092
   targetDirectory	The directory to store the exported metadata file. If you do not specify this parameter, it defaults to the current directory.	../home/
   fileName	The name of the exported metadata file. If you do not specify this parameter, the file name defaults to kafka-metadata-export.json.	metadata.json
   commit	Submits the task for execution.	commit
   installTsar	Specifies whether to automatically install Tsar. By default, Tsar is not automatically installed. Tsar helps you obtain more accurate information about the current machine specifications, recent storage usage, traffic, and Kafka cluster configuration. However, automatic installation is time-consuming and may fail due to differences in the environment.	None
   evaluate	Specifies whether to collect and evaluate the current machine specifications, recent storage usage, traffic, and Kafka cluster configurations during migration. This helps you select a more suitable Message Queue for Apache Kafka instance. The default value is true. If you do not require an evaluation, set this parameter to false.	None

   A successful metadata export generates a JSON file in the specified directory.

   Find the exported metadata file in the specified directory. Download and save the file to your local machine.

Step 2: Create a metadata import task

Create a task

1. Log on to the ApsaraMQ for Kafka console.

2. In the Resource Distribution section of the Overview page, select the region where the ApsaraMQ for Kafka instance that you want to manage resides.

3. In the left-side navigation pane, click Migration.

4. On the Migration page, click the Metadata Import tab, and then click Create Task.

5. In the Create Metadata Import Task panel, configure the following parameters and click Create.

   1. On the Create Task tab, set Task Name and Destination Instance, and upload the metadata file that you obtained in Step 1: Export metadata.

   2. On the Edit Topic tab, you can perform the following actions on the topics to be migrated:

      • Add topic: Click Add topic and specify the topic Name, Description, Partitions, and Message Type.

      • Edit topic: In the Actions column, click Modify and update the topic Description, Partitions, Message Type, and Log Cleanup Policy.

      • Delete topic: In the Actions column, click Delete, and then click OK in the Note dialog box.

   3. On the Edit Group tab, you can perform the following actions:

      • Add group: Click Add group and specify the Group ID and Description.

      • Edit group: In the Actions column, click Modify and update the group Description.

      • Delete group: In the Actions column, click Delete, and then click OK in the Note dialog box.

View migration progress

After the task is created, on the Migration page, go to the Metadata Import tab to see the Topic Migration Progress and Group Migration Progress. In the Actions column, click Details to open the Details dialog box, which shows the progress for each item. The table in the group section shows the name and migration status (such as Not started) for each group. The table in the topic section shows the name, partition number, and migration status (such as Completed) for each topic.

The metadata migration takes about 30 minutes. After the migration is complete, you can start using the migrated topics and groups.

Step 3: Verify the migration result

1. On the Instances page, click the name of the instance that you want to manage.

2. View the resource list.

   • In the left-side navigation pane, click Topics. On the Topics page, verify that the created topics are in the topic list.

   • In the left-side navigation pane, click Groups. On the Groups page, verify that the created groups are in the group list.

Step 4: Delete the migration task

On the Metadata Import tab of the Migration page, find the task that you want to delete and click Delete in the Actions column.

What to do next

After migrating your self-managed Kafka cluster, check if you need to update the client endpoints. For more information, see View endpoints.