Data Transmission Service:Synchronize data from an RDS MySQL database to Lindorm

Source database limits

• Tables to be synchronized must have primary keys with unique fields. Without primary keys, duplicate data may appear in the destination.

• When synchronizing at the table level with name mapping, a single task supports a maximum of 1,000 tables. Exceeding this limit causes an error after task submission. Split the tables across multiple tasks, or configure the task to synchronize the entire database instead.

• Do not run DDL operations on the source database during the full data synchronization phase — this causes the task to fail. DTS queries add a metadata lock that may also block DDL operations on the source during this phase.

• Data changes from operations not recorded in binary logs — such as physical backup recovery and cascade operations — are not synchronized to the destination. If this occurs and your business allows it, remove the affected database or table from the sync objects and add it back. See Modify synchronization objects.

• For MySQL 8.0.23 and later, invisible columns cause data loss because DTS cannot read them. Make invisible columns visible before starting synchronization:

  • Run ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE; for each invisible column. See Invisible Columns.

  • Tables without a primary key auto-generate an invisible primary key — make it visible as well. See Generated Invisible Primary Keys.

Other limits

Special cases

Self-managed MySQL source

• A primary/secondary failover in the source database during synchronization causes the task to fail.

• Latency is calculated from the timestamp difference between the current time and the last record synchronized to the destination. If no DML operations are performed on the source for an extended period, the displayed latency may be inaccurate. To refresh latency, perform a DML operation on the source. Alternatively, create a heartbeat table — DTS writes to it every second when you synchronize the entire database.

• DTS periodically runs CREATE DATABASE IF NOT EXISTS \test\`` on the source to advance the binary log offset.

• For Amazon Aurora MySQL or other cluster-mode MySQL instances, the domain name or IP address configured for the task must always resolve to the read/write (RW) node. Otherwise, the task may not run correctly.

ApsaraDB RDS for MySQL source

• Read-only instances of ApsaraDB RDS for MySQL 5.6 cannot be used as a source, because they do not record transaction logs.

• DTS periodically runs CREATE DATABASE IF NOT EXISTS \test\`` on the source to advance the binary log offset.

Step 1: Open the Data Synchronization page

Use either the DTS console or the DMS console.

Step 2: Configure source and destination databases

1. Click Create Task.

2. Configure the source and destination databases using the parameters in the following table.

1. Click Test Connectivity and Proceed.

   Note DTS server IP addresses must be added to the security settings of both the source and destination databases. This can be done automatically or manually. See Add DTS server IP addresses to a whitelist. If the source or destination is a self-managed database with an access method other than Alibaba Cloud Instance, click Test Connectivity in the CIDR Blocks of DTS Servers dialog box.

Step 3: Configure synchronization objects

In the Configure Objects step, set the following parameters.

Parameter	Description
Synchronization Types	Incremental Data Synchronization is selected by default. Optionally also select Full Data Synchronization. Schema Synchronization is not available. When full data synchronization is enabled, DTS synchronizes historical data from the source to the destination before starting incremental synchronization.
Processing Mode of Conflicting Tables	Keep the default value.
Capitalization of Object Names in Destination Instance	Controls the case of database names, table names, and column names in the destination. The default is DTS default policy. Select a different option if you need the capitalization to match the source or destination. See Specify the capitalization of object names in the destination instance.
Source Objects	Select the databases or tables to synchronize, then click the icon to move them to Selected Objects.
Selected Objects	If the names of databases, tables, or columns in the destination Lindorm instance differ from those in the source RDS MySQL instance, use object name mapping. See Map table and column names. Note Renaming an object with name mapping may cause dependent objects to fail synchronization. To filter rows, right-click a table in Selected Objects and specify WHERE conditions. See Specify filter conditions. To select specific SQL operations for incremental synchronization, right-click an object and choose the operations to include.

Step 4: Configure advanced settings

Click Next: Advanced Settings and configure the following parameters.

Step 5: Run the precheck

Click Next: Save Task Settings and Precheck.

DTS runs a precheck before starting the task. If any items fail:

• Click View Details next to the failed item, analyze the cause, fix the issue, and click Precheck Again.

• If an alert item cannot be ignored, click View Details next to the failed item, troubleshoot the issue, and run a precheck again.

• If an alert item can be ignored, click Confirm Alert Details, then click Ignore in the details dialog, click OK, and then click Precheck Again. Ignoring alert items may result in data inconsistency.

Step 6: Purchase the instance

1. Wait for the Success Rate to reach 100%, then click Next: Purchase Instance.

2. On the purchase page, configure the following parameters.

1. Read and select Data Transmission Service (Pay-as-you-go) Service Terms.

2. Click Buy and Start, then click OK in the confirmation dialog.

The task appears in the task list. Monitor its progress from there.