ApsaraMQ for Kafka supports synchronizing message data from a source Kafka instance to a target Kafka instance through the data synchronization feature. You can create synchronization tasks in same-account or cross-account scenarios, and select Topic, running resources, task concurrency, compression type, and message offset reset strategy as needed. The data synchronization capability is designed for long-term data transfer between two instances, such as data distribution, data backup, and disaster recovery.
Usage limits
This capability is currently available through a support ticket. If you need it, please submit a support ticket.
Billing
Data synchronization is billed based on CU. For details, see CU Postpaid.
If cross-account/VPC data synchronization is involved, you must additionally pay VPC data transfer fees. For details, see VPC data transfer postpaid.
Prerequisites
Before creating a cross-account data synchronization task, make sure that the source instance or target instance account has completed authorization, see Authorization management. After authorization, the authorized account can select the corresponding instance as the source or target when creating synchronization tasks.
Create a data synchronization task
Log on to the ApsaraMQ for Kafka console.
In the left-side navigation pane, select Data Synchronization.
On the Data Synchronization page, click Create Task.
On the Create Synchronization Task page, complete the configuration as instructed.
Step 1: Configure basic information
Parameter | Description |
Task name | Enter the name of the data synchronization task. We recommend using a name that can identify the business, source instance, and target instance for subsequent O&M and troubleshooting. |
After configuration, click Next.
Step 2: Configure the source service
Parameter | Description |
Cross-account instance | Select whether the source instance belongs to another Alibaba Cloud account. If you select cross-account instance, the source instance account must complete authorization in authorization management beforehand. |
Source cluster instance | Select the source Kafka instance to read data from. |
Sync all Topics | Select whether to synchronize all Topics from the source instance. If you select No, select or enter the Topics to be synchronized as prompted on the page. |
CU | Select the compute resource specification for the synchronization task. A higher CU typically provides stronger synchronization capability. For details, refer to the available specifications on the page. |
Number of tasks | Select the number of data synchronization tasks. |
Compression type | Select the message compression type for writing to the target. Choose based on business compatibility and compression benefits. |
Message offset reset strategy | Select the message offset reset strategy for the task. |
After configuration, click Next.
Step 3: Configure the target service
Parameter | Description |
Target cluster | Select the target Kafka instance to receive synchronized data. If the target instance belongs to another account, the target instance account must complete authorization in authorization management beforehand. |
After confirming the configuration, click Create Synchronization Task.
Next steps
After the task is created, you can view the task status, synchronization progress, and running information on the Data Synchronization page. We recommend confirming that the Topic, message writing status, and consumption chain in the target instance meet your expectations before officially switching traffic.