This topic describes the benefits, principle, and procedure of migrating a self-managed Apache Kafka cluster to Message Queue for Apache Kafka.

Benefits

For more information about the benefits of migrating a self-managed Apache Kafka cluster to Message Queue for Apache Kafka, see Benefits.

How it works

To migrate a self-managed Apache Kafka cluster to Message Queue for Apache Kafka, you need only to consume all messages in the original Apache Kafka cluster. Producers and consumer groups send and consume messages based on Apache Kafka clusters. The clusters can be migrated one by one, and the migration is not perceived by upper-layer services.

Procedure

Perform the following operations to migrate a self-managed Apache Kafka cluster to Message Queue for Apache Kafka:

  1. Specify the specifications of the self-managed Apache Kafka cluster to obtain the recommended specifications for a Message Queue for Apache Kafka instance.
    For more information, see Evaluate specifications.
  2. Purchase a Message Queue for Apache Kafka instance and deploy it based on the recommendation.
    Message Queue for Apache Kafka_Migration_2
  3. Migrate the topics of the self-managed Apache Kafka cluster to the Message Queue for Apache Kafka instance.
  4. Migrate the consumer groups of the self-managed Apache Kafka cluster to the Message Queue for Apache Kafka instance.
  5. Optional:Migrate the messages of the self-managed Apache Kafka cluster to the Message Queue for Apache Kafka instance.
    Notice In scenarios that involve message queues, after the messages of a cluster is consumed, the messages will not be used except for backups. Therefore, we recommend that you do not migrate messages except that you must back up the messages of a self-managed Apache Kafka cluster to a Message Queue for Apache Kafka instance.
    For more information, see Migrate data to the cloud. Message Queue for Apache Kafka_Migration_5
  6. Create and enable a consumer group for the Message Queue for Apache Kafka instance to consume new messages.
    Message Queue for Apache Kafka_Migration_6
  7. Enable a new producer for the Message Queue for Apache Kafka instance, deprecate the old producer, and enable the old consumer group to consume messages of the self-managed Apache Kafka cluster.
    Message Queue for Apache Kafka_Migration_7
  8. After all messages of the self-managed Apache Kafka cluster are consumed by the old consumer group, deprecate the old consumer group and the self-managed Apache Kafka cluster.
    Message Queue for Apache Kafka_Migration_8