Data Transmission Service:Synchronize data from an RDS MariaDB instance to an RDS PostgreSQL instance

Source database

• Tables to be synchronized must have PRIMARY KEY or UNIQUE constraints, with all fields unique. Otherwise, duplicate records may appear in the destination database.

• If you select tables (not an entire database) and need to rename tables or columns in the destination, a single task can synchronize up to 1,000 tables. For more than 1,000 tables, configure multiple tasks or synchronize the entire database instead.

• Binary logging is enabled by default on ApsaraDB RDS for MariaDB. The following binary log settings are required:

  Parameter	Required value	Notes
  binlog_row_image	full	Must be set before starting the task. If not set to full, the precheck fails. See Modify parameters.
  Binary log retention period	At least 24 hours (incremental sync only) or at least 7 days (full + incremental sync)	After full data synchronization completes, you can set the retention period to more than 24 hours. Insufficient retention may cause the task to fail or result in data inconsistency or data loss. Retention periods that do not meet these requirements are outside the DTS service level agreement (SLA).

Other limitations

• The destination database version must be the same as or later than the source database version. Earlier destination versions may cause compatibility issues.

• Run the task during off-peak hours. During initial full data synchronization, DTS uses read and write resources on both databases, which may increase database load.

• After full data synchronization, the tablespace of the destination database is larger than that of the source database due to fragmentation from concurrent INSERT operations.

• During schema synchronization, DTS synchronizes foreign keys from the source database to the destination database.

• During full data synchronization and incremental data synchronization, DTS temporarily disables foreign key constraint checks and cascade operations at the session level. If cascade UPDATE or DELETE operations are performed on the source database during synchronization, data inconsistency may occur.

• If you select tables (not an entire database), do not use tools such as pt-online-schema-change for online DDL operations during synchronization. Use Data Management (DMS) instead. For more information, see Perform lock-free DDL operations.

• Write data to the destination database through DTS only during synchronization. If other tools write to the destination database concurrently, do not use DMS for online DDL operations, as this may cause data loss.

• If a TIMESTAMP field has the value 0 in the source database, DTS converts it to null in the destination database.

Limitations for self-managed MariaDB sources

These limitations apply only if the source is a self-managed MariaDB database, not an ApsaraDB RDS for MariaDB instance.

• If a primary/secondary switchover occurs on the source database while the task is running, the task fails.

• DTS calculates synchronization latency based on the timestamp of the latest synchronized record in the destination and the current timestamp in the source. If no DML operations are performed on the source for an extended period, the reported latency may be inaccurate. To refresh the latency, perform a DML operation on the source. If you synchronize an entire database, create a heartbeat table — DTS updates it every second.

• DTS periodically runs CREATE DATABASE IF NOT EXISTS \test\`` on the source to advance the binary log file position.

Step 1: Open the Data Synchronization Tasks page

1. Log on to the Data Management (DMS) console.

2. In the top navigation bar, click Data + AI.

3. In the left-side navigation pane, choose DTS (DTS) > Data Synchronization.

Step 2: Configure source and destination databases

1. On the right side of Data Synchronization Tasks, select the region where the synchronization instance will reside.

   Note

   If you use the new DTS console, select the region in the top navigation bar.

2. Click Create Task. In the Create Task wizard, configure the parameters below.

Source database

Destination database

Step 3: Test connectivity

Click Test Connectivity and Proceed.

If the source or destination database is an Alibaba Cloud database instance, DTS automatically adds its server CIDR blocks to the whitelist of the instance. If the source or destination database is a self-managed database hosted on an Elastic Compute Service (ECS) instance, DTS automatically adds the CIDR blocks of DTS servers to the security group rules of the ECS instance, and you must ensure that the ECS instance can access the database. For self-managed databases in data centers or hosted by third-party cloud providers, manually add the DTS server CIDR blocks to the database whitelist. For more information, see Add the CIDR blocks of DTS servers.

Step 4: Select objects and synchronization types

Configure the parameters below.

Parameter	Description
Synchronization Types	By default, Incremental Data Synchronization is selected. Also select Schema Synchronization and Full Data Synchronization. DTS synchronizes historical data first, which is the baseline for incremental synchronization.
Processing Mode of Conflicting Tables	Precheck and Report Errors: Checks for tables with identical names in the source and destination. If identical table names exist, the precheck fails and the task cannot start. To resolve name conflicts without deleting destination tables, use the object name mapping feature. For more information, see Map object names. Ignore Errors and Proceed: Skips the precheck for identical table names. If schemas match and a record has the same primary key as an existing record in the destination, DTS skips the record during full data synchronization (keeping the existing record) and overwrites it during incremental synchronization. If schemas differ, some columns may fail to initialize or synchronization may fail. Use with caution.
Source Objects	Select objects from the Source Objects section and click to move them to Selected Objects. You can select columns, tables, or databases. If you select tables or columns, DTS does not synchronize views, triggers, or stored procedures.
Selected Objects	To rename a single object, right-click it in the Selected Objects section. To rename multiple objects at once, click Batch Edit in the upper-right corner. For more information, see Map object names.

Step 5: Configure advanced settings

Click Next: Advanced Settings and configure the parameters below.

Step 6: Save settings and run the precheck

Click Next: Save Task Settings and Precheck.

DTS runs a precheck before the task can start. If any item fails:

• Click View Details next to the failed item, fix the issue, then click Precheck Again.

• If an item triggers an alert that can be safely ignored, click Confirm Alert Details, then click Ignore in the View Details dialog box, and confirm. Then click Precheck Again. Ignoring alerts may result in data inconsistency.

Step 7: Purchase the instance and start the task

1. Wait until the Success Rate reaches 100%, then click Next: Purchase Instance.

2. On the buy page, configure the following parameters.

   Parameter	Description
   Billing Method	Subscription: Pay upfront for a fixed period. More cost-effective for long-term use. Subscription duration options: 1–9 months, 1 year, 2 years, 3 years, or 5 years. Pay-as-you-go: Billed hourly. Suitable for short-term use. Release the instance when no longer needed to stop charges.
   Resource Group Settings	The resource group for the synchronization instance. Defaults to the default resource group. For more information, see What is Resource Management?
   Instance Class	DTS offers instance classes with different synchronization speeds. Select a class based on your workload. For more information, see Instance classes of data synchronization instances.

3. Read and select Data Transmission Service (Pay-as-you-go) Service Terms.

4. Click Buy and Start. In the confirmation dialog box, click OK.

The task appears in the task list. Monitor its progress there.