Use Data Transmission Service (DTS) to continuously sync incremental data from a PolarDB-X 1.0 instance to an ApsaraDB RDS for MySQL instance in real time. This guide covers prerequisites, limitations, required permissions, and the full configuration procedure.
In this guide, you will:
-
Check prerequisites and limitations before starting
-
Create a synchronization task and configure source and destination databases
-
Select objects, set advanced options, and run a precheck
-
Purchase the synchronization instance and start the task
Prerequisites
Before you begin, make sure you have:
-
A PolarDB-X 1.0 instance with ApsaraDB RDS for MySQL as the storage type (both custom and purchased instances are supported; PolarDB for MySQL cannot be used as the storage type). See Create a PolarDB-X 1.0 instance and Create a databaseCreate a database.
-
A destination ApsaraDB RDS for MySQL instance. See Create an ApsaraDB RDS for MySQL instance.
-
Binary logs configured on the ApsaraDB RDS for MySQL instances attached to your PolarDB-X 1.0 instance:PolarDB-X 1.0
-
Binary logging enabled, with
binlog_row_imageset tofull -
Binary log retention of at least 24 hours for incremental-only sync, or at least 7 days for full + incremental sync
-
Limitations
Review these limitations before configuring the task.
Source database requirements:
-
Tables must have a PRIMARY KEY or UNIQUE constraint with all fields unique. Tables with only a UNIQUE constraint do not support schema synchronization — use tables with a PRIMARY KEY constraint. Tables with secondary indexes cannot be synchronized.
-
When syncing more than 5,000 tables with object name editing (such as renaming tables or columns), a request error occurs. Split the work across multiple tasks or sync the entire database instead.
-
PolarDB-X 1.0 supports only horizontal splitting for storage resources. Vertical splitting is not supported.PolarDB-X 1.0
-
Read-only instances at the PolarDB-X 1.0 compute layer are not supported.PolarDB-X 1.0
Restricted operations during synchronization:
-
Do not change the network type of the PolarDB-X 1.0 instance. If you must change it, update the network connection information in the synchronization task immediately after.PolarDB-X 1.0
-
Do not scale the source instance or any attached ApsaraDB RDS for MySQL instance, change physical database distribution, migrate hot tables, change shard keys, or run online DDL operations. These actions cause task failures or data inconsistency.
General limitations:
-
DTS creates a subtask for each ApsaraDB RDS for MySQL instance attached to the PolarDB-X 1.0 source. Monitor subtask status in PolarDB-X 1.0Task Topology.
-
During schema synchronization, DTS syncs foreign keys to the destination. During full and incremental sync, DTS temporarily disables foreign key constraint checks and cascade operations at the session level. Cascade update and delete operations on the source during this time may cause data inconsistency.
-
Do not use gh-ost or pt-online-schema-change to run DDL on objects being synchronized.
-
Write data to the destination database only through DTS during synchronization. Using other tools alongside DTS to write to the destination — including DMS online DDL operations — can cause data loss.
-
After full data synchronization completes, the destination tablespace is larger than the source due to fragmentation from concurrent INSERT operations.
Billing
| Synchronization type | Cost |
|---|---|
| Schema synchronization and full data synchronization | Free |
| Incremental data synchronization | Charged. See Billing overview. |
Supported synchronization topologies
-
One-way one-to-one
-
One-way one-to-many
-
One-way cascade
-
One-way many-to-one
For details, see Synchronization topologies.
SQL operations that can be synchronized
| Operation type | SQL statements |
|---|---|
| DML | INSERT, UPDATE, DELETE |
Required permissions
| Database | Required permission |
|---|---|
| Source PolarDB-X 1.0 instance | Read permissions on the objects to be synchronized. See Manage accounts. |
| Destination ApsaraDB RDS for MySQL instance | Read and write permissions on the destination objects. See Create an account and Modify account permissions. |
Create a synchronization task
Before you synchronize data, evaluate the impact of data synchronization on the performance of the source and destination databases. Schedule the task during off-peak hours. Initial full data synchronization uses read and write resources from both source and destination databases, which increases load on the database servers.
Step 1: Go to the Data Synchronization Tasks page
-
Log on to the Data Management (DMS) console.
-
In the top navigation bar, click Data + AI.
-
In the left-side navigation pane, choose DTS (DTS) > Data Synchronization.
Console navigation may vary by mode and layout. See Simple mode or Customize the layout and style of the DMS console for details. Alternatively, go directly to the Data Synchronization Tasks page in the new DTS console.
Step 2: Select a region
On the right side of Data Synchronization Tasks, select the region where the synchronization instance will reside.
In the new DTS console, select the region in the top navigation bar.
Step 3: Configure source and destination databases
Click Create Task. In the Create Data Synchronization Task wizard, configure the following parameters.
After configuring source and destination databases, read the Limits displayed on the page before proceeding.
Source Database
| Parameter | Value |
|---|---|
| Task Name | A descriptive name for the task. DTS generates a name automatically — rename it to make the task easy to identify. Task names do not need to be unique. |
| Database Type | Select PolarDB-X 1.0. |
| Connection Type | Select Alibaba Cloud Instance. |
| Instance Region | The region of the source PolarDB-X 1.0 instance. |
| Instance ID | The ID of the source PolarDB-X 1.0 instance. |
| Database Account | The database account for the source instance. See Required permissions. |
| Database Password | The password for the database account. |
Destination Database
| Parameter | Value |
|---|---|
| Database Type | Select MySQL. |
| Connection Type | Select Alibaba Cloud Instance. |
| Instance Region | The region of the destination ApsaraDB RDS for MySQL instance. |
| RDS Instance ID | The ID of the destination ApsaraDB RDS for MySQL instance. |
| Database Account | The database account for the destination instance. See Required permissions. |
| Database Password | The password for the database account. |
| Encryption | Select Non-encrypted or SSL-encrypted. To use SSL encryption, enable it on the ApsaraDB RDS for MySQL instance first. See Use a cloud certificate to enable SSL encryption. |
Step 4: Test connectivity
Click Test Connectivity and Proceed.
DTS automatically adds its server CIDR blocks to the whitelist of Alibaba Cloud database instances (such as ApsaraDB RDS for MySQL or ApsaraDB for MongoDB). For self-managed databases on Elastic Compute Service (ECS) instances, DTS automatically adds its CIDR blocks to the ECS security group rules — make sure the ECS instance can reach the database. For databases deployed across multiple ECS instances, manually add the CIDR blocks to each instance's security group rules. For on-premises databases or databases from third-party cloud providers, manually add the CIDR blocks to the database whitelist. See CIDR blocks of DTS servers.
Adding DTS server CIDR blocks to whitelists or security groups carries security risks. Before using DTS, take preventive measures: use strong credentials, limit exposed ports, authenticate API calls, audit whitelist and security group rules regularly, and remove unauthorized CIDR blocks. For higher security, connect databases to DTS using Express Connect, VPN Gateway, or Smart Access Gateway.
Step 5: Select objects and configure synchronization settings
| Parameter | Description |
|---|---|
| Synchronization Types | Incremental Data Synchronization is selected by default. Also select Schema Synchronization and Full Data Synchronization. DTS syncs historical data first to establish a baseline for incremental synchronization. |
| Processing Mode of Conflicting Tables | Precheck and Report Errors (default): fails the precheck if the source and destination have tables with identical names. Fix the conflicts before proceeding. If tables in the destination cannot be deleted or renamed, use object name mapping to rename the destination tables. Ignore Errors and Proceed: skips the precheck for name conflicts. During full sync, existing records in the destination are retained when primary or unique key values match. During incremental sync, existing records are overwritten. If schemas differ, initialization may partially fail. Use this option with caution. |
| Capitalization of object names in destination instance | Controls capitalization of database names, table names, and column names in the destination. Defaults to DTS default policy. See Specify the capitalization of object names in the destination instance. |
| Source Objects | Select objects and click the arrow icon to move them to Selected Objects. Select tables as the objects to sync. If you select an entire database, DTS does not sync table creation or deletion changes. |
| Selected Objects | To rename a single object, right-click it. To rename multiple objects, click Batch Edit. See Map object names. To filter data with WHERE conditions, right-click an object and specify conditions. See Specify filter conditions. To select specific SQL operations per object, right-click the object and choose the operations to sync. |
Step 6: Configure advanced settings
Click Next: Advanced Settings.
| Parameter | Description |
|---|---|
| Monitoring and Alerting | Enable to receive notifications when the task fails or synchronization latency exceeds a threshold. Set the alert threshold and notification contacts. See Configure monitoring and alerting. |
| Retry Time for Failed Connections | How long DTS retries failed connections after the task starts. Range: 10–1440 minutes. Default: 720 minutes. Set this to at least 30 minutes. If DTS reconnects within this window, the task resumes automatically; otherwise it fails. If multiple tasks share the same source or destination database, the shortest retry window applies. While retrying, the DTS instance continues to accrue charges. |
| Configure ETL | Enable to transform data during synchronization using the built-in extract, transform, and load (ETL) feature. See What is ETL? and Configure ETL. |
Step 7: Run the precheck
Click Next: Save Task Settings and Precheck.
To preview the OpenAPI parameters for this task, hover over the button and click Preview OpenAPI parameters before clicking through.
DTS runs a precheck before starting the task. Wait for all items to pass.
-
If an item fails, click View Details, fix the issue, then click Precheck Again.
-
If an item shows an alert that can be safely ignored, click Confirm Alert Details, then Ignore, confirm with OK, and click Precheck Again. Ignoring alert items may cause data inconsistency.
Step 8: Purchase the synchronization instance
Wait until Success Rate reaches 100%, then click Next: Purchase Instance.
On the buy page, configure the following parameters.
| Parameter | Description |
|---|---|
| Billing Method | Subscription: pay upfront for a fixed term; more cost-effective for long-term use. Pay-as-you-go: billed hourly; suitable for short-term use. Release the instance when no longer needed to stop charges. |
| Resource Group Settings | The resource group for the synchronization instance. Defaults to the default resource group. See What is Resource Management? |
| Instance Class | Determines synchronization speed. Choose based on your throughput requirements. See Instance classes of data synchronization instances. |
| Subscription Duration | Available only for subscription billing. Options: 1–9 months, 1, 2, 3, or 5 years. |
Read and select Data Transmission Service (Pay-as-you-go) Service Terms, then click Buy and Start. In the confirmation dialog, click OK.
The task appears in the task list. DTS runs full data synchronization first, then switches to incremental synchronization automatically.
What's next
After the task is running, verify that synchronization is working as expected:
-
Monitor task status and subtask progress in Task Topology.
-
Check synchronization latency in the task list. For latency alerts, see Configure monitoring and alerting.
-
After full data synchronization completes, you can set the binary log retention period on the source to more than 24 hours. Make sure that you set the retention period of binary logs in accordance with the requirements described in the Prerequisites section. Otherwise, the service reliability and performance stated in the Service Level Agreement (SLA) of DTS cannot be achieved.