To solve pain points in open source RabbitMQ clusters, such as message accumulation and split-brain issues, and achieve high concurrency, distributed storage, and flexible scaling, you can migrate open source RabbitMQ clusters to ApsaraMQ for RabbitMQ instances. This topic describes the prerequisites, procedure, and usage notes for this type of migration.
Usage notes
ApsaraMQ for RabbitMQ is an advanced message service that is developed based on highly available distributed storage. This service strictly follows the AMQP 0-9-1 protocol but is not simply a managed version of open source RabbitMQ. ApsaraMQ for RabbitMQ uses an original architecture, which prevents memory leaks and broker failures caused by message accumulation and resolves split-brain issues in distributed systems. ApsaraMQ for RabbitMQ provides high scalability and supports the pay-as-you-go billing method. This enhances service elasticity and reduces costs.
Compared with open source RabbitMQ, ApsaraMQ for RabbitMQ uses different implementation methods for specific features. Before you migrate an open source RabbitMQ cluster to an ApsaraMQ for RabbitMQ instance, you must evaluate technology capabilities and cost-effectiveness to select an instance type that meets your business requirements.
Evaluate migration
Evaluate technology
ApsaraMQ for RabbitMQ and open source RabbitMQ differ in terms of features, service and performance, exchanges, and queues. Before you migrate an open source RabbitMQ cluster to an ApsaraMQ for RabbitMQ instance, check the differences and evaluate whether the features provided by ApsaraMQ for RabbitMQ meet your expectations. For information about the differences between ApsaraMQ for RabbitMQ and open source RabbitMQ, see Comparison between ApsaraMQ for RabbitMQ and open source RabbitMQ.
ApsaraMQ for RabbitMQ imposes limits on clusters, API operations, and string characters. When you use ApsaraMQ for RabbitMQ, you cannot exceed the limits. Before you migrate an open source RabbitMQ cluster to an ApsaraMQ for RabbitMQ instance, evaluate whether the limits meet your business requirements. For more information, see Instance editions.
Evaluate costs
To meet your business requirements, ApsaraMQ for RabbitMQ provides the following instance editions: Serverless Edition, Professional Edition, Enterprise Edition, and Enterprise Platinum Edition. For information about the differences among the editions, see Instance editions.
Compared with other instance editions provided by ApsaraMQ for RabbitMQ, Serverless Edition instances have higher scalability and lower costs. The billable items of an ApsaraMQ for RabbitMQ Serverless Edition instance include the number of API calls initiated to receive and deliver messages, the number of queues, message storage space, outbound Internet traffic, and message traces. The last two billable items are optional. For information about the billing rules, see Billing of Serverless Edition instances.
You can evaluate the number of API calls initiated to receive and deliver messages, the number of queues, and the number of messages in an open source RabbitMQ cluster by using the following methods:
Number of queues: Go to the Overview tab in the open source RabbitMQ console and view the number of queues and exchanges in the Global counts section.
Number of API calls initiated to send and receive messages and number of messages
Method 1: Go to the Overview tab in the open source RabbitMQ console, view the information in the Message rates section, and calculate the total value of the Publish, Deliver(manual ack), Deliver (auto ack), and Redelivered metrics.
Method 2: Check the queries per second (QPS) and the number of inbound and outbound messages of the cluster on the Prometheus or Grafana dashboard.
Migrate an open source RabbitMQ cluster to an ApsaraMQ for RabbitMQ instance
Migrate metadata
Metadata migration is the process in which the metadata of an open source RabbitMQ cluster is exported and then imported to an ApsaraMQ for RabbitMQ instance. After the metadata is imported, ApsaraMQ for RabbitMQ creates vhosts, queues, exchanges, and bindings on the ApsaraMQ for RabbitMQ instance based on the imported metadata. Then, the metadata of the open source RabbitMQ cluster is migrated. For more information, see Migrate the metadata of an open source RabbitMQ cluster to an ApsaraMQ for RabbitMQ instance.
Create a username and password and grant permissions
When you access an ApsaraMQ for RabbitMQ broker from an open source RabbitMQ client, you must specify the username and password for authentication. You can access the ApsaraMQ for RabbitMQ broker only after the authentication is passed. Usernames and passwords in open source RabbitMQ are specified by users. Usernames and passwords in ApsaraMQ for RabbitMQ can be generated by using the AccessKey pairs provided by Resource Access Management (RAM). For more information, see Manage static usernames and passwords.
Migrate message data
Use the Shovel plug-in to migrate message data
If you use the Shovel plug-in to migrate message data, messages in the open source cluster are consumed by using Shovel and then produced to the ApsaraMQ for RabbitMQ instance. After the migration, messages in the open source cluster are cleared.
Start the Shovel plug-in in the open source RabbitMQ cluster.
Create a Shovel task. Specify the endpoint and queue of the open source RabbitMQ cluster as the source. Specify the endpoint and queue of the ApsaraMQ for RabbitMQ instance as the destination. For information about how to view endpoints, see Manage instances.
Modify the endpoint, username, and password of the consumer of the open source RabbitMQ queue and connect to the ApsaraMQ for RabbitMQ instance for consumption.
Modify the endpoint, username, and password of the producer of the open source RabbitMQ queue and send messages to the ApsaraMQ for RabbitMQ instance.
Disable the Shovel task when no message is accumulated in the open source RabbitMQ cluster.
Use the dual-read mode to migrate message data
If you use the dual-read mode to migrate message data, you need to only consume messages in the open source RabbitMQ cluster. Messages are produced and consumed in open source RabbitMQ clusters. You can migrate the clusters one by one without affecting upper-layer services.
Modify the endpoint, username, and password of several consumer nodes in the application and connect to the ApsaraMQ for RabbitMQ instance for consumption. The remaining consumer nodes remain connected to the open source cluster for consumption.
Modify the endpoint, username, and password of the producer of the open source RabbitMQ queue and send messages to the ApsaraMQ for RabbitMQ instance.
Connect the remaining consumer nodes to the ApsaraMQ for RabbitMQ instance for consumption when no message is accumulated in the open source RabbitMQ cluster.
References
For more information about the Shovel plug-in, see Shovel Plugin.