Data Transmission Service:Synchronize data from an RDS for MySQL instance to an ApsaraMQ for RocketMQ instance

Source database

• Tables being synchronized must have a primary key or a UNIQUE constraint. Without one, duplicate records may appear in the destination.

• At the table level with object name mapping, a single task supports up to 1,000 tables. To synchronize more tables, split them across multiple tasks or switch to database-level synchronization.

• The binlog_row_image parameter must be set to full. DTS requires row-level binary log images to capture the complete before and after state of each changed row. If this parameter is not set correctly, DTS reports a precheck error and the task cannot start.

  • Binary logging is enabled by default for ApsaraDB RDS for MySQL instances.

  • For self-managed MySQL databases, you must manually enable binary logging and set the binlog_format parameter to row and the binlog_row_image parameter to full.

  • For self-managed MySQL databases in a primary/primary architecture (where both databases act as primary and secondary of each other), you must also enable the log_slave_updates parameter to ensure DTS can obtain all binary logs. For more information, see Create a database account for a self-managed MySQL database and configure binary logging.

• Binary logs must be retained long enough for DTS to read them: If binary logs are purged before DTS reads them, the task fails and data loss or inconsistency may result. Issues caused by insufficient log retention are outside the DTS service-level agreement (SLA).

  • RDS for MySQL: at least 3 days (7 days recommended)

  • Self-managed MySQL: at least 7 days

• Do not perform DDL operations on source tables during full data synchronization. DTS queries the source database during this phase, which creates metadata locks that can block DDL operations.

• Data changes not recorded in binary logs—such as recovery from physical backups or cascade operations—are not captured and not delivered to the destination topic. If this occurs and your business permits, you can remove the database or table from the synchronization objects and then add it back. For more information, see Modify synchronization objects.

• Do not use pt-online-schema-change or similar tools to perform online DDL operations on synchronization objects in the source database during synchronization. Doing so causes the task to fail.

• MySQL 8.0.23 and later: if synchronized tables contain invisible columns, data in those columns may be lost because DTS cannot read them. To make a column visible, run:

  ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE;

  Tables without an explicit primary key automatically get an invisible generated primary key—make that visible too. See Invisible columns and Generated invisible primary keys.

Destination instance

• Serverless ApsaraMQ for RocketMQ instances cannot be used as a destination.

• Synchronization writes only to specific topics—not to the entire instance.

• The destination instance supports a maximum message body size of 4 MB. If a record exceeds 4 MB and filtering is disabled, the synchronization task fails.

• For RocketMQ V4.x: if the destination instance is in a different region from the source, DTS accesses it through a public endpoint, which incurs data transfer costs. Enable public access on the destination instance before starting the task. Check the access status on the Basic Information tab of the instance's details page in the ApsaraMQ for RocketMQ console. For pricing, see Public bandwidth pricing.

• Upgrading or downgrading the instance class of the destination instance changes how messages are delivered.

• Do not write data to the destination topic from sources other than DTS during synchronization. Concurrent writes from other sources can cause data inconsistency.

Encryption features

• Always-Encrypted (EncDB) on the source: full data synchronization is not supported.

• Transparent Data Encryption (TDE) on the source: both full and incremental data synchronization are supported.

Performance

During full data synchronization, DTS reads from the source database and writes to the destination instance, which increases load on both. Run the initial synchronization during off-peak hours—for example, when CPU utilization on both the source and destination is below 30%.

Concurrent inserts during full synchronization cause table fragmentation in the destination. After full synchronization completes, table storage in the destination is larger than in the source.

Recovery

If a DTS instance fails, DTS support attempts recovery within 8 hours. Recovery may involve restarting the instance or adjusting DTS instance parameters (not database parameters). For the list of adjustable parameters, see Modify instance parameters.

Step 1: Open the data synchronization page

DTS console

1. Log on to the DTS console.

2. In the left-side navigation pane, click Data Synchronization.

3. In the upper-left corner, select the region where your synchronization task will reside.

DMS console

The exact steps vary depending on the mode and layout of the DMS console. See Simple mode and Customize the layout and style of the DMS console.

1. Log on to the DMS console.

2. In the top navigation bar, move the pointer over Data + AI and choose DTS (DTS) > Data Synchronization.

3. From the drop-down list to the right of Data Synchronization Tasks, select the region where the synchronization instance will reside.

Step 2: Configure source and destination databases

Click Create Task, then configure the following parameters.

Click Test Connectivity and Proceed at the bottom of the page.

DTS server CIDR blocks must be allowed in the security settings of both the source and destination. For details, see Add DTS server IP addresses to a whitelist.

Step 3: Configure synchronization objects

In the Configure Objects step, set the following options.

Click Next: Advanced Settings.

Step 4: Run the precheck and purchase the instance

1. Click Next: Save Task Settings and Precheck.

   To preview the API parameters that correspond to this configuration, move the pointer over Next: Save Task Settings and Precheck and click Preview OpenAPI parameters.

2. Wait for the precheck to complete.

   • If an item fails, click View Details to see the cause, fix the issue, then click Precheck Again.

   • If an alert appears on an item that can be safely ignored, click Confirm Alert Details, then Ignore, then OK, then Precheck Again.

3. After the Success Rate reaches 100%, click Next: Purchase Instance.

4. On the purchase page, configure the following parameters.

   Parameter	Description
   Billing Method	Subscription: pay upfront for a fixed period—more cost-effective for long-term use. Pay-as-you-go: billed hourly—suitable for short-term or evaluation use. Release the instance when it is no longer needed to stop charges.
   Resource Group Settings	The resource group for the synchronization instance. Default: default resource group. See What is Resource Management?
   Instance Class	The synchronization throughput class. Select based on your expected data volume. See Instance classes of data synchronization instances.
   Subscription Duration	Available for the subscription billing method only. Options: 1–9 months, 1 year, 2 years, 3 years, or 5 years.

5. Read and accept the Data Transmission Service (Pay-as-you-go) Service Terms.

6. Click Buy and Start, then click OK in the confirmation dialog.

The task appears in the task list. You can monitor its progress there.

For self-managed MySQL sources

• If a primary/secondary failover occurs during synchronization, the task fails.

• Synchronization latency is calculated as the time difference between the current timestamp and the timestamp of the last record delivered to the destination. If no DML operations run on the source for an extended period, the displayed latency may be inaccurate. Run a DML operation on the source to refresh the latency display. Alternatively, configure a heartbeat table that DTS updates every second.

• DTS periodically runs CREATE DATABASE IF NOT EXISTS \test\`` on the source to advance the binary log offset.

• For Amazon Aurora MySQL or other cluster-mode MySQL sources, the domain name or IP address configured in the task must always resolve to the read/write (RW) node. If it does not, the task may not run correctly.

For RDS for MySQL sources

• ApsaraDB RDS for MySQL instances that do not record transaction logs—such as read-only RDS for MySQL 5.6 instances—cannot be used as a source.

• DTS periodically runs CREATE DATABASE IF NOT EXISTS \test\`` on the source to advance the binary log offset.