All Products
Search

This Product

    ApsaraMQ for Kafka:Migrate self-managed Apache Kafka clusters to ApsaraMQ for Kafka instances

    Document Center

    ApsaraMQ for Kafka:Migrate self-managed Apache Kafka clusters to ApsaraMQ for Kafka instances

    Last Updated:Jun 10, 2026

    Migrate your self-managed Apache Kafka cluster to an ApsaraMQ for Kafka instance for lower costs, better scalability, and higher reliability. ApsaraMQ for Kafka is fully managed, so you can focus on business development instead of cluster operations.

    Usage notes

    • Migration does not delete source topics or groups from the self-managed cluster. Topics and groups with the same configurations are created in the destination ApsaraMQ for Kafka instance.

    • Message data and offset migration is optional.

    Migration solutions

    During migration, original consumers must drain all queued messages in the self-managed cluster. You can migrate producer and consumer servers one by one without affecting your applications. For information about the benefits of migrating a self-managed Apache Kafka cluster to an ApsaraMQ for Kafka instance, see Benefits.

    The following figure shows the ApsaraMQ for Kafka migration process.

    image

    ApsaraMQ for Kafka supports three migration solutions:

    Item

    Cloud migration

    Metadata import

    Tool-assisted migration

    Migration method

    Create a migration task in the ApsaraMQ for Kafka console. Recommended for most scenarios.

    Export data as a JSON file by using kafka-migration-assessment.jar, then create a metadata import task in the ApsaraMQ for Kafka console.

    Note

    Use this method if your self-managed cluster has no Internet access.

    Migrate by using kafka-migration-assessment.jar or MirrorMaker.

    Migration content

    • Topics

    • Groups

    • Message data and offset information

    • Access control lists (ACLs) attached to topics

      Create SASL users in the ApsaraMQ for Kafka console before migration.

    • Topics

    • Groups

    • kafka-migration-assessment.jar

      • Topics

      • Groups

    • MirrorMaker

      • Message data and offset information

    Migration process

    1. Obtain the public endpoint of the self-managed Apache Kafka cluster.

    2. Create a cloud migration task in the ApsaraMQ for Kafka console.

    3. Start consumer groups on the ApsaraMQ for Kafka instance.

    4. Start producers on the ApsaraMQ for Kafka instance, stop the self-managed cluster producers, and then use the original consumer groups to drain queued messages from the self-managed cluster.

    5. After all queued messages are consumed, stop the consumer groups and decommission the self-managed cluster.

    1. Use a migration tool to migrate data from the self-managed cluster to the ApsaraMQ for Kafka instance.

    2. Start consumer groups on the ApsaraMQ for Kafka instance.

    3. Start producers on the ApsaraMQ for Kafka instance, stop the self-managed cluster producers, and then use the original consumer groups to drain queued messages from the self-managed cluster.

    4. After all queued messages are consumed, stop the consumer groups and decommission the self-managed cluster.

    References