1. Scenarios
This solution is for migrating a self-managed RabbitMQ cluster to ApsaraMQ for RabbitMQ.
Migration type: Metadata migration.
2. Advantages of ApsaraMQ for RabbitMQ over self-managed RabbitMQ
Item | ApsaraMQ for RabbitMQ | Self-managed RabbitMQ |
Maximum cluster TPS | Unlimited. ApsaraMQ for RabbitMQ uses a distributed, masterless cluster architecture that supports horizontal scaling. | Limited by machine performance. Scaling out is achieved by upgrading device specifications, which has its own limits. |
Maximum TPS for a single queue | Unlimited. ApsaraMQ for RabbitMQ supports horizontal scaling for single queues, with no performance or capacity limits related to concurrency. | Limited. The maximum performance of a single queue is limited by the performance of a single node. |
Connections | Unlimited. The number of supported connections scales with the cluster. An increase in connections does not affect performance. | Limited. The number of connections is limited by the performance of a single machine and cannot be scaled out. |
Scheduled messages | Second-level precision, high performance, and ready to use out of the box. | Complex to use. |
Service availability and data reliability |
| Achieved using mirrored or quorum queues. This can easily lead to split-brain issues. |
Message accumulation capacity | Maintains high performance even with a large accumulation of messages. The cluster service is not affected. | A large accumulation of messages can easily cause memory issues, which may lead to service downtime. |
Elasticity |
| The maximum concurrent capacity of the cluster is limited by a single machine. Scaling out requires upgrading device specifications or splitting the cluster. |
Inspection System | Automatically discovers and fixes issues such as deadlocks and downtime. | None. |
3. Migration tools
1. ApsaraMQ for RabbitMQ console migration tool
The ApsaraMQ for RabbitMQ console migration tool lets you import metadata files from an open source RabbitMQ instance by creating a migration task. The tool supports ALL and Vhost import modes and creates the corresponding Vhosts, Queues, Exchanges, and Bindings in the target instance.
The migration tool supports only metadata migration and does not support message data migration.
4. Migration plan
1. Migration workflow
Instance creation: Create an ApsaraMQ for RabbitMQ instance with the required specifications in Alibaba Cloud.
Data migration:
Metadata migration: Use the ApsaraMQ for RabbitMQ console migration tool to migrate metadata.
Data validation:
Metadata validation: Check the number of vhosts and queues before and after migration to verify metadata consistency.
2. Data migration plan
2.1. Migrate metadata using the ApsaraMQ for RabbitMQ console migration tool
How it works
Metadata migration involves exporting metadata from an open source RabbitMQ cluster and importing it into an ApsaraMQ for RabbitMQ instance. ApsaraMQ for RabbitMQ creates the corresponding Vhosts, Queues, Exchanges, and Bindings in the target instance based on the imported metadata.
Prerequisites
The RabbitMQ management plugin must be enabled on the self-managed source instance.
Create an ApsaraMQ for RabbitMQ instance and a Vhost to serve as the migration target.
Risks and notes
Because of differences in permission control mechanisms between self-managed RabbitMQ and ApsaraMQ for RabbitMQ, the following metadata types are not supported for import: rabbit_version, users, permissions, parameters, global_parameters, and policies. They are automatically ignored during import.
3. Data validation plan
3.1. Metadata validation
Check the number of vhosts and queues before and after migration to verify metadata consistency.
5. Migration procedure
1. Migrate metadata using the ApsaraMQ for RabbitMQ console migration tool
1.1. Export metadata from open source RabbitMQ
Open the open source RabbitMQ console in a browser. The console URL is http://{RabbitMQ IP address}:1567.
On the logon page, enter your username in the Username text box and your password in the Password text box. Then, click Login.
On the Overview tab, click Export definitions. From the Virtual host drop-down list, select All or a specific Vhost, and then click Download broker definitions. The options in the Virtual host list are described as follows:
All: Exports the metadata of all Vhosts.
Vhost name: Exports the metadata of a specific Vhost.
Save the downloaded metadata file to your computer.
1.2. Import metadata into ApsaraMQ for RabbitMQ
Log on to the ApsaraMQ for RabbitMQ console. In the navigation pane on the left, click Cloud Migration.
In the top menu bar of the Cloud Migration page, select a region. Then, in the upper-left corner of the page, click Create Task.
In the Create Task panel, set the parameters and click OK.
Parameter | Description | Example |
Instance | The name of the target instance to which you want to migrate the metadata. | amqp-cn-7mz2cjgk**** |
Import Mode |
| Vhost |
Vhost | The Vhost in the ApsaraMQ for RabbitMQ instance to which you want to migrate the metadata. This parameter is displayed when you set Import Mode to Vhost. | test-vhost**** |
Metadata | The metadata file to migrate. Click Select File, select the local metadata file, and then click Open. Note The metadata file cannot exceed 20 MB. | rabbit_mq-amqp-load-test011122063**** |
After you create the task, you can view the details of the migrated metadata and the reasons for any task failures.
View details of successfully imported metadata
On the Cloud Migration page, find the target task and click the instance name in the Target Instance column.
In the navigation pane on the left, click Vhost List. Find the target Vhost and click Details in the Actions column to view the Vhost details. For more information, see Vhost management.
View details of a failed task
On the Cloud Migration page, find the target task and click Details in the Actions column. Alternatively, find the target instance and click the number in the Synced Metadata Count column.
On the Migration Details page, click the Vhost, Exchange, Queue, or Binding tab to view the failure reason.
6. Data validation procedure
Metadata validation
In the ApsaraMQ for RabbitMQ console, check the number of vhosts and queues. Compare these numbers with the corresponding numbers in the source RabbitMQ console to verify that the metadata is consistent.
7. Migration notes
Because of differences in permission control mechanisms between self-managed RabbitMQ and ApsaraMQ for RabbitMQ, the following metadata types are not supported for import: rabbit_version, users, permissions, parameters, global_parameters, and policies. They are automatically ignored during import.
Queue names, Exchange names, and Bindings in self-managed RabbitMQ can contain special characters. ApsaraMQ for RabbitMQ instances do not support special characters such as spaces, $, #, or %.
ApsaraMQ for RabbitMQ instances do not support plugins. Before migrating from a self-managed RabbitMQ, evaluate the capabilities of your installed plugins and find alternative solutions if necessary. For example, self-managed RabbitMQ supports delayed messages through a plugin. ApsaraMQ for RabbitMQ instances support delayed messages as a native feature, not through a plugin.
In self-managed RabbitMQ, the default maximum message delay is 30 minutes, after which the message is deleted. In an ApsaraMQ for RabbitMQ instance, the default is 5 minutes. This time varies by instance specification. Before migration, ensure the maximum delay in your on-premises environment matches the delay configured for the ApsaraMQ for RabbitMQ instance.
ApsaraMQ for RabbitMQ instances support message redelivery after a failure or timeout. After migrating, check whether this feature triggers duplicate message deliveries to prevent a surge in messages.
User passwords for ApsaraMQ for RabbitMQ instances can only be regenerated from an AccessKey pair (AK/SK). You cannot set them to match existing user passwords.