Use Data Transmission Service (DTS) to continuously synchronize data between two AnalyticDB for MySQL 3.0 clusters. This supports disaster recovery, multi-region reads, and large-scale data migrations.
Synchronization runs in three phases: schema synchronization, full data synchronization, and incremental data synchronization driven by change data capture (CDC). Schema synchronization and full data synchronization are free. Incremental data synchronization is billed by the hour. For pricing details, see Billing overview.
Prerequisites
Before you begin, make sure that:
The destination AnalyticDB for MySQL 3.0 cluster is created. Its storage capacity must exceed the total data size on the source cluster.
The source cluster runs kernel version 3.2.1.0 or later. If not, upgrade the kernel version first.
CDC is enabled on the source cluster, and binary logging is enabled for each table to synchronize.
Each table to synchronize has a custom primary key with unique values. Tables whose primary key is the auto-generated
__adb_auto_id__cannot be synchronized by DTS.The source database account has read permissions on the tables to synchronize. The destination database account has read and write permissions on the destination database. See Create a database account for instructions.
XUANWU_V2 tables do not support binary logging. DTS cannot synchronize or incrementally migrate XUANWU_V2 tables between AnalyticDB for MySQL clusters.
Limitations
Setup requirements
A single DTS synchronization instance handles only one table. To synchronize multiple tables, create one instance per table.
Do not perform DDL operations that change the database or table schema during schema synchronization or full data synchronization. The task will fail.
The destination database must have a custom primary key, or you must configure a Primary Key Column in the Configurations for Databases, Tables, and Columns step. Otherwise, the data synchronization may fail.
The server that hosts the source database must have sufficient outbound bandwidth. Otherwise, the data synchronization speed is affected.
CDC and binary logging
If a synchronization task is paused for more than one day, the binary logs on the source cluster may expire, preventing the task from resuming. Keep tasks running continuously, or pause them for less than 24 hours.
While the task is running, do not disable binary logging on the source cluster. If binary logging is disabled, the task fails and cannot be recovered — you must create a new task.
Data validation is not supported.
Synchronization objects cannot be modified after the task is configured.
Runtime behavior
During full data synchronization, DTS runs concurrent INSERT operations, which causes table fragmentation in the destination database. The storage space used by the destination tables may be larger than the source after the initial synchronization.
During full data synchronization and incremental data synchronization, DTS temporarily disables foreign key constraint checks and cascade operations at the session level. If you run cascade UPDATE or DELETE operations on the source during synchronization, data inconsistency may occur.
DTS does not synchronize foreign keys from source to destination during schema synchronization.
If the destination cluster's node disk usage exceeds 80%, DTS may experience delays. Estimate the required storage in advance to ensure sufficient capacity.
If the destination cluster is being backed up while the DTS task is running, the task fails.
Before you synchronize data, evaluate the performance of the source and destination databases. Perform data synchronization during off-peak hours. For example, synchronize data when the CPU load of both the source and destination databases is below 30%.
If a DTS instance fails, the DTS team will attempt recovery within 8 hours. During recovery, the instance may be restarted and its parameters adjusted.
Parameter adjustments affect only the DTS instance, not the database. For adjustable parameters, see Modify instance parameters.
Supported SQL operations
| Operation type | SQL statements |
|---|---|
| DML | INSERT, UPDATE, DELETE |
| DDL | ADD COLUMN, DROP COLUMN |
When data is written to an AnalyticDB for MySQL 3.0 cluster, UPDATE statements are automatically converted to REPLACE INTO statements. If the primary key is updated, the statement is converted to a DELETE statement followed by an INSERT statement.
Create a synchronization task
Step 1: Open the Data Synchronization page
Use one of the following consoles:
DTS console
Log on to the DTS console.DTS console
In the left-side navigation pane, click Data Synchronization.
In the upper-left corner, select the region where the synchronization task will run.
DMS console
The actual steps may vary depending on your DMS console mode and layout. See Simple mode and Customize the layout and style of the DMS console for details.
Log on to the DMS console.DMS console
In the top navigation bar, move the pointer over Data + AI and choose DTS (DTS) > Data Synchronization.
From the drop-down list to the right of Data Synchronization Tasks, select the region where the synchronization instance resides.
Step 2: Configure source and destination databases
Click Create Task.
Configure the source and destination databases using the parameters below.
| Category | Parameter | Description |
|---|---|---|
| (None) | Task Name | A descriptive name for the task. DTS generates a name automatically; you don't need to make it unique. |
| Source Database | Select Existing Connection | If the source cluster is already registered with DTS, select it from the list — DTS fills in the remaining parameters automatically. Otherwise, configure the parameters below manually. For registration instructions, see Manage database connections. |
| Database Type | Select AnalyticDB for MySQL 3.0. | |
| Access Method | Select Alibaba Cloud Instance. | |
| Instance Region | Select the region where the source cluster resides. | |
| Replicate Data Across Alibaba Cloud Accounts | Select No if the source and destination clusters belong to the same Alibaba Cloud account. | |
| Instance ID | Select the source cluster ID. | |
| Database Account | Enter the source database account. | |
| Database Password | Enter the account password. | |
| Destination Database | Select Existing Connection | If the destination cluster is already registered with DTS, select it from the list. Otherwise, configure the parameters manually. |
| Database Type | Select AnalyticDB for MySQL 3.0. | |
| Access Method | Select Alibaba Cloud Instance. | |
| Instance Region | Select the region where the destination cluster resides. | |
| Instance ID | Select the destination cluster ID. | |
| Database Account | Enter the destination database account. | |
| Database Password | Enter the account password. |
Click Test Connectivity and Proceed.
DTS server CIDR blocks must be added to the security settings of both clusters. DTS attempts to add them automatically. If automatic addition fails, add them manually. See Add DTS server IP addresses to a whitelist for instructions.
Step 3: Select synchronization objects
In the Configure Objects step, configure the following settings.
| Configuration | Description |
|---|---|
| Synchronization Types | By default, Incremental Data Synchronization is selected. You must also select Schema Synchronization and Full Data Synchronization. After the precheck, DTS copies historical data to the destination cluster as the baseline for subsequent incremental sync. |
| DDL and DML Operations to Be Synchronized | Select the instance-level operations to include in incremental synchronization. To configure per-table operations, right-click a table in Selected Objects and select the desired operations. |
| Processing Mode of Conflicting Tables | Precheck and Report Errors (default): the precheck fails if source and destination tables share the same name. Use object name mapping to rename conflicting objects before proceeding. Ignore Errors and Proceed: skips the precheck for identical table names. If source and destination have the same schema and a record shares a primary key or unique key value, DTS skips the record during full sync and overwrites it during incremental sync. If schemas differ, initialization may fail. |
| Source Objects | In Source Objects, click the table to synchronize, then click the right-arrow icon to move it to Selected Objects. Select only one table per synchronization instance. |
| Selected Objects | To rename an object in the destination or map it to a different object, right-click it in Selected Objects. See Map object names for details. To remove an object, click it and then click the remove icon. To add WHERE conditions to filter rows, right-click the table. See Specify filter conditions. |
If you rename an object using object name mapping, other objects that depend on it may fail to synchronize.
Click Next: Advanced Settings and configure the following options.
| Configuration | Description |
|---|---|
| Dedicated Cluster for Task Scheduling | By default, DTS schedules the task to the shared cluster. For improved stability, purchase a dedicated cluster. See What is a DTS dedicated cluster. |
| Retry Time for Failed Connections | How long DTS retries if the source or destination database connection fails. Valid values: 10–1440 minutes. Default: 720 minutes. Set to at least 30 minutes. If the connection is restored within this window, DTS resumes the task; otherwise, the task fails. |
| Retry Time for Other Issues | How long DTS retries if a DDL or DML operation fails. Valid values: 1–1440 minutes. Default: 10 minutes. Set to at least 10 minutes, and keep this value lower than Retry Time for Failed Connections. |
| Enable Throttling for Full Data Synchronization | Throttle full synchronization to reduce load on the destination database server. Configure Queries per second (QPS) to the source database, RPS of Full Data Migration, and Data migration speed for full migration (MB/s). Available only when Full Data Synchronization is selected. |
| Enable Throttling for Incremental Data Synchronization | Throttle incremental synchronization. Configure RPS of Incremental Data Synchronization and Data synchronization speed for incremental synchronization (MB/s). |
| Environment Tag | Optional. Tag the instance to identify its environment (for example, production or staging). |
| Configure ETL | Optional. Enable the extract, transform, and load (ETL) feature to transform data during synchronization. Select Yesalert notification settings to enter data processing statements in the code editor. See Configure ETL in a data migration or data synchronization task. |
| Monitoring and Alerting | Optional. Configure alerts so that designated contacts are notified if the task fails or synchronization latency exceeds a threshold. See the "Configure monitoring and alerting when you create a DTS task" section for details. |
If multiple synchronization tasks share the same source or destination database with different retry time ranges configured, the shortest range applies.
DTS charges for the instance during connection retry periods. Set the retry window based on your business requirements, and release the DTS instance promptly after the source and destination instances are released.
Click Next Step: Data Verification to configure data verification. See Configure a data verification task for details.
Optional: Click Next: Configure Database and Table Fields to configure the Type, Primary Key Column, Distribution Key, and partition key settings (Partition Key, Partitioning Rules, and Partition Lifecycle) for destination tables.
This step is available only when Schema Synchronization is selected. Set Definition Status to All to view and edit all tables. For Primary Key Column, you can select multiple columns to create a composite primary key. If you create a composite primary key, also select one or more columns as the Distribution Key and Partition Key. See CREATE TABLE.
Step 4: Run the precheck and purchase the instance
Click Next: Save Task Settings and Precheck. To preview the API parameters for this task, move the pointer over Next: Save Task Settings and Precheck and click Preview OpenAPI parameters before proceeding.
The task cannot start until it passes the precheck. If the precheck fails, click View Details next to each failed item, resolve the issues, then click Precheck Again. If an alert item can be ignored, click Confirm Alert Details, then Ignore, then OK, then Precheck Again. Ignoring alerts may result in data inconsistency.
When the Success Rate reaches 100%, click Next: Purchase Instance.
On the purchase page, configure the following parameters.
| Parameter | Description |
|---|---|
| Billing Method | Subscription: pay upfront for a fixed term (1–9 months, or 1, 2, 3, or 5 years). More cost-effective for long-term use. Pay-as-you-go: billed hourly. Release the instance when no longer needed to avoid ongoing charges. |
| Resource Group Settings | The resource group for the synchronization instance. Defaults to the default resource group. See What is Resource Management?. |
| Instance Class | Determines the synchronization speed. Select based on your data volume and latency requirements. See Instance classes of data synchronization instances. |
| Subscription Duration | Available when Subscription is selected. Choose the subscription term and the number of instances to create. |
Read and accept the Data Transmission Service (Pay-as-you-go) Service Terms.
Click Buy and Start, then click OK in the dialog box.
The task appears in the task list. Monitor the synchronization progress from there.
What's next
To monitor task health and latency, see Configure monitoring and alerting.
To stop or release the task when synchronization is no longer needed, see the task management section in the DTS console.
To synchronize additional tables, create a separate synchronization task for each table.