Data Transmission Service:Configure a data synchronization task

Source database requirements

• Tables to be synchronized must have PRIMARY KEY or UNIQUE constraints, and all fields must be unique. Otherwise, the destination database may contain duplicate records.

• If you select tables (not an entire database) and need to rename tables or columns: the limit is 1,000 tables per synchronization task. For more than 1,000 tables, split them across multiple tasks or synchronize the entire database instead.

• Binary logging requirements:

  • Binary logging must be enabled, and binlog_row_image must be set to full. For details, see Modify the parameters of an ApsaraDB RDS for MySQL instance.

    Important

    For self-managed MySQL databases: - Set binlog_format to row and binlog_row_image to full. - For dual-primary cluster deployments, set log_slave_updates to ON to make sure DTS can obtain all binary logs. For more information, see Create an account for a self-managed MySQL database and configure binary logging.

  • For incremental synchronization only: retain binary logs for at least 24 hours.

  • For both full and incremental synchronization: retain binary logs for at least seven days. After full data synchronization completes, you can set the retention period to more than 24 hours.

  If binary log retention requirements are not met, DTS may fail to obtain the required logs, causing task failure, data inconsistency, or data loss. In such cases, the DTS Service Level Agreement (SLA) does not guarantee service reliability or performance. For more information about binary log files, see View and delete the binary log files of an ApsaraDB RDS for MySQL instance.

Schema synchronization behavior

• DTS synchronizes foreign keys from the source database to the destination database during schema synchronization.

• During full and incremental synchronization, DTS temporarily disables foreign key constraint checks and cascade operations at the session level. Cascade update or delete operations on the source during synchronization may cause data inconsistency.

Other limitations

• The destination database version must be the same as or later than the source database version. An older destination version may cause compatibility issues.

• Run synchronization during off-peak hours. Initial full data synchronization uses read and write resources on both instances and increases server load.

• After initial full data synchronization, the destination tablespace will be larger than the source due to fragmentation from concurrent INSERT operations.

• If synchronizing selected tables (not the entire database), do not use tools such as pt-online-schema-change for DDL operations during synchronization — the task will fail. If you need online DDL on source tables, use Data Management (DMS). For more information, see Perform lock-free DDL operations.

• Write to the destination database only through DTS during synchronization. Writing through other tools can cause data inconsistency or loss.

Notes for self-managed MySQL databases

• A primary/secondary switchover on the source database while the task is running causes the task to fail.

• DTS calculates synchronization latency based on the timestamp of the latest synchronized data in the destination and the current timestamp in the source. If no DML operations are performed on the source for an extended period, the displayed latency may be inaccurate. To update the latency reading, perform a DML operation on the source.

  If you synchronize an entire database, create a heartbeat table that is updated every second to keep the latency reading accurate.

• DTS periodically executes CREATE DATABASE IF NOT EXISTS 'test' in the source database to advance the binary log file position.

Step 1: Navigate to the dedicated cluster

1. Go to the Dedicated Cluster page.

2. In the top navigation bar, select the region where your DTS dedicated cluster resides.

3. In the Actions column of the DTS dedicated cluster, choose Configure Task > Configure Data Synchronization Task.

Step 2: Configure source and destination databases

General

Source Database

Destination Database

Step 3: Test connectivity

Click Test Connectivity and Proceed.

DTS automatically handles network access based on your database type:

• Alibaba Cloud database instances (such as ApsaraDB RDS for MySQL or ApsaraDB for MongoDB): DTS adds its server CIDR blocks to the instance whitelist automatically.

• Self-managed databases on Elastic Compute Service (ECS): DTS adds its server CIDR blocks to the ECS security group rules. Make sure the ECS instance can reach the database.

• On-premises or third-party cloud databases: Manually add the DTS server CIDR blocks to the database whitelist. For more information, see Add the CIDR blocks of DTS servers to the security settings of on-premises databases.

Step 4: Select objects and configure synchronization settings

Choose a synchronization mode

DTS supports three synchronization modes. Select the combination that fits your use case:

• Schema synchronization: Synchronizes table schemas (DDL structure) from source to destination.

• Full data synchronization: Copies all existing data from source to destination. This is the baseline for subsequent incremental synchronization.

• Incremental data synchronization (selected by default): Continuously replicates ongoing data changes after the initial full sync.

For this example, select all three: Schema Synchronization, Full Data Synchronization, and Incremental Data Synchronization.

Synchronization parameters

Select objects to synchronize

In the Source Objects section, select the objects to synchronize and click the rightwards arrow icon to move them to the Selected Objects section.

You can select columns, tables, or entire databases. Selecting tables or columns means DTS does not synchronize views, triggers, or stored procedures to the destination.

In the Selected Objects section, you can:

• Rename a single object: right-click the object. For more information, see Map the name of a single object.

• Rename multiple objects at once: click Batch Edit in the upper-right corner. For more information, see Map multiple object names at a time.

• Filter specific SQL operations per object: right-click the object and select the SQL operations to synchronize.

• Filter data with WHERE conditions: right-click the object and specify the conditions. For more information, see Use SQL conditions to filter data.

Renaming an object with object name mapping may cause other objects dependent on it to fail synchronization.

Step 5: Configure advanced settings

Click Next: Advanced Settings and configure the following parameters.

Data verification

For information on enabling data verification, see Enable data verification.

Advanced settings

Step 6: Run the precheck

Click Next: Save Task Settings and Precheck.

DTS runs a precheck before the task can start. The task launches only after all precheck items pass.

• If a precheck item fails: click View Details, troubleshoot the issue, and click Precheck Again.

• If an alert is triggered:

  • If the alert cannot be ignored: click View Details, resolve the issue, and run the precheck again.

  • If the alert can be ignored: click Confirm Alert Details, then Ignore, then OK, and then Precheck Again.

Ignoring an alert item may result in data inconsistency and exposes your business to potential risks.

Step 7: Start the task

1. Wait until the precheck success rate reaches 100%, then click Next: Select DTS Instance Type.

2. In the New Instance Class section, configure the instance class. Set a minimum of 1 DTS unit (DU) up to the number of remaining available DUs.

3. Read and select the checkbox to agree to the Data Transmission Service (Pay-as-you-go) Service Terms.

4. Click Start Task, then click OK.

To monitor task progress, go to the cluster details page and click Cluster Task List in the left-side navigation pane.