Synchronize RDS MySQL to PolarDB MySQL | DTS - Data Transmission Service

PolarDB is a next-generation, cloud-native, distributed relational database developed by Alibaba Cloud. It is fully compatible with MySQL and offers ease of use, high performance, high reliability, and high availability. You can use DTS to synchronize data from ApsaraDB RDS for MySQL to PolarDB for MySQL.

Prerequisites

Create a PolarDB for MySQL cluster. For more information, see Custom Purchase.

Usage notes

During full data initialization, DTS consumes read and write resources from the source and destination databases, increasing their load. If your databases have poor performance, low specifications, or high business traffic (for example, the source database has many slow SQL queries or tables without primary keys, or the destination database experiences deadlocks), the database load may increase and even cause service interruptions. Before you synchronize data, evaluate the performance of the source and destination instances. We recommend synchronizing data during off-peak hours, such as when the CPU utilization of both instances is below 30%.
During data synchronization, do not use tools such as gh-ost or pt-online-schema-change to perform online DDL changes on the synchronization objects in the source database. Otherwise, the synchronization task fails.
During initial full data synchronization, concurrent INSERT operations cause table fragmentation in the destination cluster. After the initial full data synchronization is complete, the tablespace of the destination cluster is larger than that of the source database.
If a table in the source database has no primary key or unique constraint, and its records are not unique across all fields, duplicate data may occur in the destination database.

Billing

Synchronization type	Pricing
Schema synchronization and full data synchronization	Free of charge.
Incremental data synchronization	Charged. For more information, see Billing overview.

Supported SQL operations

Operation type	SQL statement
DML	INSERT, UPDATE, DELETE, and REPLACE
DDL	ALTER TABLE and ALTER VIEW CREATE FUNCTION, CREATE INDEX, CREATE PROCEDURE, CREATE TABLE, and CREATE VIEW DROP INDEX and DROP TABLE RENAME TABLE Important RENAME TABLE operations may cause data inconsistency between the source and destination databases. For example, if you select a table as the object to be synchronized and rename the table during data synchronization, the data of this table is not synchronized to the destination database. To prevent this situation, you can select the database to which this table belongs as the object to be synchronized when you configure the data synchronization task. Make sure that the databases to which the table belongs before and after the RENAME TABLE operation are added to the objects to be synchronized. TRUNCATE TABLE

Supported synchronization topologies

One-way one-to-one synchronization
One-way one-to-many synchronization
One-way cascade synchronization
One-way many-to-one synchronization

Each topology has specific usage notes. Synchronization topologies.

Limitations

Incompatibility with triggers

If you select an entire database as the synchronization object and it contains a trigger that updates a table within that database, data inconsistency may occur between the source and destination databases. For a solution, see Configure a data synchronization or migration task for a source database that contains a trigger.
Limits on RENAME TABLE operations

The RENAME TABLE operation may cause data inconsistency. For example, if you synchronize only a single table, and that table is renamed on the source instance during synchronization, its data will not be synchronized to the destination database. To prevent this, select the entire database containing the table as a synchronization object. If the table is renamed to a different database, that database must also be synchronized.

Procedure

Purchase a data synchronization task. For more information, see Purchase process.

Note
When you purchase the task, set Source Database to MySQL, Destination Database to PolarDB, and Synchronization Topology to One-way Synchronization.
Log on to the DTS console.

Note
If you are automatically redirected to the Data Management (DMS) console, you can click the icon in the lower-right corner and then click to return to the classic DTS console.
In the left-side navigation pane, click Data Synchronization.
At the top of the Synchronization Tasks page, select the region where your destination instance is located.
Find the data synchronization task that you purchased and click Configure Task.

Configure the source and destination instances.

Category	Parameter	Description
N/A	Synchronization Task Name	DTS generates a default name. Specify a descriptive name. Uniqueness is not required.
Source Instance Details	Instance Type	Select RDS Instance.
	Instance Region	The region of the source instance selected during DTS instance purchase. This value cannot be changed.
	Database Account	Enter the database account of the source ApsaraDB RDS for MySQL instance. Note The account must have REPLICATION CLIENT, REPLICATION SLAVE, SHOW VIEW, and SELECT permissions on all objects to synchronize. For source RDS for MySQL instances running MySQL 5.5 or MySQL 5.6, the Database Account and Database Password parameters are not required.
	Database Password	Enter the password of the database account.
	Connection Method	Select Non-encrypted or SSL-encrypted. If you select SSL-encrypted, enable SSL encryption for the source ApsaraDB RDS for MySQL instance first. Set SSL encryption. Important Currently, you can set the Encryption only in regions in the Chinese mainland and China (Hong Kong).
Destination Instance Details	Instance Type	Set to PolarDB. This value cannot be changed.
	Instance Region	The region of the destination instance selected during DTS instance purchase. This value cannot be changed.
	PolarDB Instance ID	Select the cluster ID of the destination PolarDB for MySQL cluster.
	Database Account	Enter the database account for the PolarDB for MySQL cluster. Note The account must have ALL permissions on the objects to synchronize.
	Database Password	Enter the password for the database account.

In the lower-right corner of the page, click Set Whitelist and Next.

If the source or destination database is an Alibaba Cloud database instance, such as an ApsaraDB RDS for MySQL or ApsaraDB for MongoDB instance, DTS automatically adds the CIDR blocks of DTS servers to the IP address whitelist of the instance. If the source or destination database is a self-managed database hosted on an Elastic Compute Service (ECS) instance, DTS automatically adds the CIDR blocks of DTS servers to the security group rules of the ECS instance, and you must make sure that the ECS instance can access the database. If the self-managed database is hosted on multiple ECS instances, you must manually add the CIDR blocks of DTS servers to the security group rules of each ECS instance. If the source or destination database is a self-managed database that is deployed in a data center or provided by a third-party cloud service provider, you must manually add the CIDR blocks of DTS servers to the IP address whitelist of the database to allow DTS to access the database. For more information, see Whitelist DTS server IP addresses.

Warning
Adding the public IP address blocks of the DTS service, either automatically or manually, may pose security risks. Using this product, you acknowledge that you understand and accept the potential security risks and that you must implement basic security measures. These measures include, but are not limited to, strengthening password security, limiting the ports open to each CIDR block, using authentication for internal API calls, and regularly checking and restricting unnecessary CIDR blocks. Alternatively, you can connect through a private network using a leased line, VPN Gateway, or Smart Access Gateway.

Configure the processing mode for existing target tables and select the objects to synchronize.

Parameter	Description
Processing mode for existing target tables	Precheck and Report Errors: DTS checks for tables in the destination database with the same names as the source tables. If no conflicts are found, the precheck passes. If a name conflict exists, the precheck fails and the task stops. Note If you cannot delete or rename the conflicting tables in the destination database, you can Set object names in the destination instance to avoid name conflicts. Ignore Errors and Proceed: Skips the check for tables with conflicting names in the destination database. Warning Selecting Ignore Errors and Proceed may cause data inconsistency and other risks. For example: If the table schemas are identical, existing records in the destination table that share a primary key with source records are not changed during initial synchronization. However, they will be overwritten by changes from the source during incremental synchronization. If the table schemas are different, the initial synchronization may fail to transfer some columns or may fail entirely.
Select synchronization objects	In the Source Objects box, click the objects that you want to synchronize, and then click the icon to move them to the Selected Objects box. You can select databases and tables as synchronization objects. Note If you select an entire database, all schema changes for objects within that database are synchronized to the destination database. By default, object names are not changed. To rename objects in the destination cluster, use the object name mapping feature. For more information, see Set object names in the destination instance.
Map or rename objects	Change the names of synchronized objects in the destination instance. For more information, see Map databases, tables, and columns.
Replicate temporary tables from DMS online DDL operations	If you use Data Management (DMS) to perform online DDL changes on the source database, you can choose whether to synchronize the temporary tables generated by the DDL changes. Yes: Synchronizes the temporary tables generated by online DDL changes. Note If a large amount of temporary table data is generated by online DDL changes, the data synchronization task may be delayed. No: Does not synchronize the temporary tables generated by online DDL changes. Only the original DDL operations from the source database are synchronized. Note This option causes tables in the destination database to be locked.
Connection retry period	If DTS cannot connect to the source or destination instance, it retries for 720 minutes (12 hours) by default. You can also specify a custom retry duration. If DTS reconnects to the source or destination instance within the specified duration, the synchronization task automatically resumes. Otherwise, the task fails. Note You are billed for task run time during connection retries. Customize the retry duration based on your business needs, or release the DTS instance as soon as the source and destination instances are released.

After you complete the preceding configurations, click Next in the lower-right corner of the page.
Configure advanced settings for initial synchronization.

Note
The initial synchronization types include initial schema synchronization and initial full data synchronization. If you select both Initial Schema Synchronization and Initial Full Data Synchronization, DTS synchronizes the schema and existing data of the selected objects from the source database to the destination database before starting incremental data synchronization.
After you complete the preceding configurations, click Precheck and Start in the lower-right corner of the page.
Note
- DTS performs a precheck before the synchronization task starts. The task can begin only after it passes this check.
- If the precheck fails, click the icon next to the failed item to view details.
  
  Fix the issues based on the cause and run the precheck again.
  
  If you do not need to fix an alert item, you can click Ignore or Ignore and Re-precheck to skip the alert item and run the precheck again.
After the Precheck dialog box displays Precheck Passed, close the Precheck dialog box. The synchronization task starts automatically.
Wait for the task to finish initialization and enter the Synchronizing state.

You can view the status of the data synchronization task on the Data Synchronization page.