Instructions

Prerequisites

You must have an ApsaraMQ for RocketMQ instance. The following versions are supported:

ApsaraMQ for RocketMQ 4.0 series instances.

Note
The Standard Edition of 4.0 series instances supports only one-way synchronization.
ApsaraMQ for RocketMQ 5.0 series instances.

Examples in this guide use ApsaraMQ for RocketMQ 5.0 series instances as source and destination clusters.

Specification recommendations

Message synchronization consumes additional TPS resources after you configure topic mappings. Ensure both source and destination ApsaraMQ for RocketMQ instances have sufficient peak sending and receiving TPS capacity.

The following table provides specification recommendations:

For example, before synchronization, an ApsaraMQ for RocketMQ instance has a peak message sending TPS of 10,000 messages/s and a peak message receiving TPS of 10,000 messages/s.

Recommended TPS configurations for the source and destination instances:

Task type	Source cluster			Destination cluster
Task type	Peak TPS limit (messages/s)	Peak sending TPS (messages/s)	Peak receiving TPS (messages/s)	Peak TPS limit (messages/s)	Peak sending TPS (messages/s)	Peak receiving TPS (messages/s)
one-way synchronization	30,000	10,000	20,000	20,000	10,000	10,000
two-way synchronization	40,000	20,000	20,000	40,000	20,000	20,000

Note

You can view the peak messaging TPS limit of an instance on the Instance Details page. For more information about the TPS limits of different instance specifications, see Specification limits.
You can adjust the peak message receiving TPS of the destination cluster based on your business processing requirements.

Create a Global Replicator task

Note

When you use Global Replicator for the first time, the Alibaba Cloud Service Authorization - RocketMQ Global Replicator dialog box appears. Clicking Confirm automatically creates the service-linked role AliyunServiceRoleForRMQDisasterRecovery, which enables the Global Replicator feature in ApsaraMQ for RocketMQ. For more information, see service-linked roles.

Step 1: Create a task

Log on to the ApsaraMQ for RocketMQ console.
In the left-side navigation pane, click Global Replicator and then click Create Task.

On the Create Task page, configure the required parameters, and then click Create.

At the top of the page, set Task name to rocket-test, select One-way Backup: The source cluster synchronizes data to the destination cluster for Task type, and enter a Task description as required. Configure the source and destination clusters in the left and right panes. An arrow between the panes indicates the synchronization direction.

The following tables describe how to configure different types of source and destination clusters.

ApsaraMQ for RocketMQ

Parameter	Description	Example
Version	Select the version of the ApsaraMQ for RocketMQ instance. 4.0 series instance 5.0 series instance	5.0 series instance
Region	Select the region of the ApsaraMQ for RocketMQ instance.	China (Hangzhou)
Instance	Select the ApsaraMQ for RocketMQ instance.	rmq-cn-778***
Authentication mode	No Configuration Required ACL-based authentication	ACL-based authentication
Username	Select a username.	yS8x01****
Security group ID	The security group of the ECS instance that hosts the ApsaraMQ for RocketMQ instance. This parameter is required when Version is set to a 5.0 series instance. Inbound rule: No requirements. Outbound rule: Policy: Allow Protocol: Custom TCP Destination: The CIDR block must include all CIDR blocks of the cluster's vSwitches. The port range must include 8080 and 8081.	sg-2ze4jlbqy2s40pc4****
Filter condition system configuration	(Optional) You can add custom attributes (user properties) to messages. Consumers can then filter messages based on a subscription policy that uses SQL-92 syntax. Key: The key of the user property. Source cluster value: The value of the user property that is added to messages from the source cluster. Destination cluster value: The value of the user property that is added to messages from the destination cluster. The following code provides an example of SQL-92 subscription filtering: `String sqlStr = "{Key} IS NOT NULL AND {Key} = {Source/Destination Cluster Value}"; consumer.subscribe(MqConfig.TOPIC, MessageSelector.bySql(sqlStr));`	--

Apache RocketMQ

Parameter	Description	Example
Access point	The format for an access point is `{domain name/IP}:{port number}`.	XX.XX.XX.XX:8080
Authentication mode	No Configuration Required ACL-based authentication	ACL-based authentication
Username	Enter the required username.	yS8x01****
Password	Enter the required password.	F17R4to****
Network type	Public network VPC	Public network
Region	This parameter is required if you set Network type to VPC. Select the region where the Apache RocketMQ cluster is deployed.	China (Hangzhou)
VPC	This parameter is required if you set Network type to VPC. Select the VPC where the Apache RocketMQ cluster is deployed.	vpc-bp17fapfdj0dwzjkd****
vSwitch	This parameter is required if you set Network type to VPC. Select the vSwitch where the Apache RocketMQ cluster is deployed.	vsw-bp1gbjhj53hdjdkg****
Security group ID	This parameter is required if you set Network type to VPC. Select the security group where the Apache RocketMQ cluster is deployed. Inbound rule: No requirements. Outbound rule: Policy: Allow Protocol: Custom TCP Destination: The CIDR block must include the CIDR blocks of all Apache RocketMQ nodes. The port range must include the service ports of all Apache RocketMQ nodes. The default ports are 9876, 10911, and 10909. If you have modified the NameServer or broker ports, you must specify the updated ports.	sg-2ze4jlbqy2s40pc4****
Filter condition system configuration	(Optional) You can add custom attributes (user properties) to messages. Consumers can then filter messages based on a subscription policy that uses SQL-92 syntax. Key: The key of the user property. Source cluster value: The value of the user property that is added to messages from the source cluster. Destination cluster value: The value of the user property that is added to messages from the destination cluster. The following code provides an example of SQL-92 subscription filtering: `String sqlStr = "{Key} IS NOT NULL AND {Key} = {Source/Destination Cluster Value}"; consumer.subscribe(MqConfig.TOPIC, MessageSelector.bySql(sqlStr));`	--

Step 2: Configure topic mappings

On the task details page, click Message Synchronization in the left-side navigation pane.
Click Batch Mapping or Add Mapping to configure topic mappings for synchronization in the dialog box.
- Batch Mapping: Configure and select the required topic mappings, and then click OK.
- Add Mapping: Configure the required topic mapping, and then click OK.
Note
- If the source cluster is an Apache RocketMQ cluster, you must specify a group for message synchronization. Do not use a group that is already in use for business operations.
- Message synchronization starts only after you configure the topic mappings.

Step 3 (Optional): Enable consumer progress synchronization

Consumer progress synchronization replicates the current consumption progress of a group for a topic to the destination cluster. For more information, see Consumer progress synchronization.

Consumer client configuration

In this example, a two-way synchronization task connects a source cluster (an ApsaraMQ for RocketMQ instance in the China (Shanghai) region) with a destination cluster (an ApsaraMQ for RocketMQ instance in the China (Hangzhou) region). The filter condition Key is Region, with Source cluster value set to Shanghai and Destination cluster value set to Hangzhou.

To ensure a consumer client in the China (Shanghai) region consumes only locally produced data, filter messages by using a custom attribute. For more information, see Event patterns.

Subscribe to messages and match them based on a single custom attribute.

String topic = "topic";
// Subscribe only to messages replicated from other regions.
FilterExpression filterExpression = new FilterExpression("Region IS NOT NULL", FilterExpressionType.SQL92);
// The Region key in the user property is NULL for locally produced messages because the local producer application does not set this key.
simpleConsumer.subscribe(topic, filterExpression);

Subscribe to all messages in the topic without filtering.

String topic = "topic";
// Subscribe to all messages.
FilterExpression filterExpression = new FilterExpression("True", FilterExpressionType.SQL92);
simpleConsumer.subscribe(topic, filterExpression);

FAQ

Can I synchronize data among three instances?

Yes. A Global Replicator task is configured between two instances. To synchronize data among three instances, you must configure three separate tasks, one for each pair of instances. This ensures that each instance has the complete dataset.

Can I use tag-based filtering for instances in a two-way synchronization setup?

You can use either tag-based filtering or SQL filtering, but not both at the same time. Because Global Replicator adds custom properties to track message origin, you must use SQL filtering to distinguish between local and replicated messages. Therefore, we recommend using SQL filtering for all instances, including those that would otherwise use tag-based filtering. For more information, see Message filtering.

Can I change an existing one-way synchronization task to a two-way task?

If both the source and destination clusters are ApsaraMQ for RocketMQ instances, you can change the task type without interrupting the existing data flow. If one of the clusters is an Apache RocketMQ cluster, you cannot change a one-way synchronization task to a two-way synchronization task. You must delete the task and create a new one.

How is Global Replicator billed?

Global Replicator uses EventBridge as its synchronization channel. For EventBridge pricing, see the linked documentation. Synchronization consumes read resources on the ApsaraMQ for RocketMQ source instance and write resources on the destination instance. Ensure your ApsaraMQ for RocketMQ instances have sufficient computing specifications to handle this load.

I configured ACL-based authentication for Global Replicator. Why are synchronized messages not received?

After you configure a synchronization task in Global Replicator, you must specify the topics to synchronize. The system creates a non-deletable system group for each topic to synchronize data. This group is deleted only when the synchronization task is deleted. If you enable ACL-based authentication, you must grant the selected ACL user read and write permissions on the topics and read permissions on the groups. On the Access Control page of the instance, select the user and set Resource name to a wildcard character (*) to grant full permissions to the account. This simplifies the configuration.

If I pause and then resume a task, will synchronization continue from where it left off?

Yes. After resumption, synchronization continues from the message following the latest acknowledged offset. Ensure the pause duration does not exceed the message retention period of the ApsaraMQ for RocketMQ instance.