This topic describes the advantages, principle, and procedure of migrating a user-created Kafka cluster to a Message Queue for Apache Kafka instance.

Advantages

For more information about the advantages of migrating a user-created Kafka cluster to a Message Queue for Apache Kafka instance, see Benefits.

How it works

To migrate an existing cluster for message queues, you only need to consume all messages in the existing cluster. The producers and consumers are deployed in clusters and can be migrated one by one. The migration is not perceived by upper-layer services.

Procedure

Perform the following operations to migrate a user-created Kafka cluster to a Message Queue for Apache Kafka instance:

  1. Evaluate the specifications of the user-created Kafka cluster to determine the edition of the Message Queue for Apache Kafka instance you need to purchase.
    For more information, see Evaluate specifications.
  2. Purchase a Message Queue for Apache Kafka instance and deploy it based on the recommendation.
    dg_migrate_2
  3. Migrate the topics of the user-created Kafka cluster to the Message Queue for Apache Kafka instance.
  4. Migrate the consumer groups of the user-created Kafka cluster to the Message Queue for Apache Kafka instance.
  5. Optional:Migrate the data of the user-created Kafka cluster to the Message Queue for Apache Kafka instance.
    Notice For message queues, after the data of a cluster is consumed, the data will not be used except for backup. Therefore, we recommend that you do not migrate data except that you must back up data of the user-created Kafka cluster to the Message Queue for Apache Kafka instance.
    For more information, see Migrate data to the cloud. dg_migrate_5
  6. Enable a new consumer group for the Message Queue for Apache Kafka instance to consume messages of the Message Queue for Apache Kafka instance.
    dg_migrate_6
  7. Enable a new producer for the Message Queue for Apache Kafka instance, deprecate the old producer, and enable the old consumer group to consume messages of the user-created Kafka cluster.
    dg_migrate_7
  8. After all messages of the user-created Kafka cluster are consumed by the old consumer group, deprecate the old consumer group and the user-created Kafka cluster.
    dg_migrate_8