Use Data Transmission Service (DTS) to set up one-way synchronization between two PolarDB-X 2.0 instances. DTS replicates schema structure, historical data, and ongoing changes from the source instance to the destination, keeping both instances in sync with minimal lag.
Prerequisites
Before you begin, make sure that you have:
Created both the source and destination PolarDB-X 2.0 instances
Verified that the destination instance has more available storage space than the total data size in the source instance
Enabled binary logging in the PolarDB-X 2.0 console (see Parameter settings) and confirmed that
binlog_row_imageis set toFULL
Limitations
Source database
| Limitation | Details |
|---|---|
| Primary key or unique key required | Tables to be synchronized must have a PRIMARY KEY or UNIQUE constraint with all fields unique. Otherwise, the destination database may contain duplicate records. |
| 5,000-table limit per task | If you select tables as the synchronization objects and need to rename tables or columns in the destination, a single task supports up to 5,000 tables. Exceeding this limit causes a request error. Split the tables across multiple tasks, or synchronize the entire database in one task. |
| Binary log retention | For incremental synchronization only: retain binary logs for at least 24 hours. For full + incremental synchronization: retain binary logs for at least 7 days. After full synchronization completes, set the retention period to more than 24 hours. If DTS cannot retrieve the binary logs, the task fails and data loss may occur. |
| Uppercase table names | If a table name contains uppercase letters, only schema synchronization is supported for that table. |
| MySQL 5.7 compatibility | The source PolarDB-X 2.0 instance must be compatible with MySQL 5.7. |
| TABLEGROUP and Locality not supported | The synchronization task does not support TABLEGROUP or databases and schemas that contain a Locality attribute. |
| No DDL during schema sync or full sync | Do not run DDL statements on the source database during schema synchronization or full data synchronization. Doing so causes the task to fail. |
Other limitations
| Limitation | Details |
|---|---|
| Schema synchronization not supported | DTS does not synchronize schemas in this scenario. Create the target databases and tables in the destination instance before starting the task. |
| MySQL 5.7 compatibility | The destination PolarDB-X 2.0 instance must be compatible with MySQL 5.7. |
| Synchronize during off-peak hours | Initial full data synchronization reads from the source and writes to the destination concurrently, increasing load on both database servers. Schedule synchronization during off-peak hours. |
| Table fragmentation after full sync | Concurrent INSERT operations during full data synchronization cause table fragmentation in the destination. The used tablespace on the destination will be larger than on the source after full synchronization completes. |
| Avoid pt-online-schema-change | Do not use pt-online-schema-change for DDL operations on synchronized objects. This may cause the synchronization task to fail. |
| Write to the destination only through DTS | Writing to the destination database using tools other than DTS may cause data inconsistency. If you also use DMS to run online DDL operations, data loss in the destination may occur. |
| Task failure recovery | If a DTS task fails, DTS support attempts to restore it within 8 hours. During restoration, the task may restart and task parameters (not database parameters) may be modified. For the list of parameters that may change, see the Modify instance parameters section of the Modify the parameters of a DTS instance topic. |
DTS updates thedts_health_check.ha_health_checktable in the source database on a schedule to advance the binary log position.
During schema synchronization, DTS copies foreign keys from the source to the destination. During full and incremental data synchronization, DTS temporarily disables foreign key constraint checks and cascade operations at the session level. If you run cascade update or delete operations on the source during synchronization, data inconsistency may occur.
Billing
| Synchronization type | Cost |
|---|---|
| Schema synchronization and full data synchronization | Free |
| Incremental data synchronization | Charged. See Billing overview. |
SQL operations that can be synchronized
| Operation type | Statements |
|---|---|
| DML | INSERT, UPDATE, DELETE |
| DDL | ALTER TABLE; CREATE FUNCTION, CREATE INDEX, CREATE TABLE; DROP INDEX, DROP TABLE; RENAME TABLE; TRUNCATE TABLE |
The RENAME TABLE operation may cause data inconsistency. If you select a table as a synchronization object and rename it during synchronization, changes to that table stop being replicated. To avoid this, select the database (not the table) as the synchronization object, and make sure that both the pre-rename and post-rename databases are included in the selected objects.
Required permissions
| Database | Schema synchronization | Full data synchronization | Incremental data synchronization |
|---|---|---|---|
| Source PolarDB-X 2.0 | SELECT | SELECT | SELECT (on synchronized objects), REPLICATION SLAVE, REPLICATION CLIENT |
| Destination PolarDB-X 2.0 | Read and write | — | — |
What each permission is used for:
| Permission | Used for | Required during |
|---|---|---|
| SELECT | Reads rows from synchronized tables. | Schema synchronization, full data synchronization, incremental data synchronization |
| REPLICATION SLAVE | Connects to the source instance and reads the binary log stream. | Incremental data synchronization |
| REPLICATION CLIENT | Runs SHOW MASTER STATUS, SHOW SLAVE STATUS, and SHOW BINARY LOGS to monitor replication state. | Incremental data synchronization (throughout) |
For instructions on granting these permissions, see Data synchronization tools for PolarDB-X.
Configure the synchronization task
Step 1: Go to the Data Synchronization Tasks page
Log on to the Data Management (DMS) console.
In the top navigation bar, click Data + AI.
In the left-side navigation pane, choose DTS (DTS) > Data Synchronization.
The DMS console layout may vary based on your configuration. For details, see Simple mode and Customize the layout and style of the DMS console. You can also go directly to the Data Synchronization Tasks page in the new DTS console.
Step 2: Select the region
On the right side of Data Synchronization Tasks, select the region where the data synchronization instance resides.
In the new DTS console, select the region from the top navigation bar.
Step 3: Configure the source and destination databases
Click Create Task and fill in the database connection details.
After configuring the source and destination databases, review the Limits displayed on the page before proceeding. Skipping this review may cause the task to fail or result in data inconsistency.
Task information
| Parameter | Description |
|---|---|
| Task Name | A descriptive name for the DTS task. DTS generates a default name automatically. The name does not need to be unique. |
Source Database
| Parameter | Description |
|---|---|
| Select a DMS database instance | Select an existing DMS database instance, or leave this blank to configure the connection manually. If you select an existing instance, DTS fills in the remaining parameters automatically. |
| Database Type | Select PolarDB-X 2.0. |
| Connection Type | Select Alibaba Cloud Instance. |
| Instance Region | The region where the source PolarDB-X 2.0 instance resides. |
| Replicate Data Across Alibaba Cloud Accounts | Select No for single-account synchronization. |
| Instance ID | The ID of the source PolarDB-X 2.0 instance. |
| Database Account | The database account for the source instance. The account must have SELECT, REPLICATION CLIENT, and REPLICATION SLAVE permissions. |
| Database Password | The password for the source database account. |
Destination Database
| Parameter | Description |
|---|---|
| Select a DMS database instance | Select an existing DMS database instance, or leave this blank to configure the connection manually. |
| Database Type | Select PolarDB-X 2.0. |
| Connection Type | Select Alibaba Cloud Instance. |
| Instance Region | The region where the destination PolarDB-X 2.0 instance resides. |
| Instance ID | The ID of the destination PolarDB-X 2.0 instance. |
| Database Account | The database account for the destination instance. The account must have read and write permissions. |
| Database Password | The password for the destination database account. |
Step 4: Test connectivity
Click Test Connectivity and Proceed.
DTS automatically adds its server CIDR blocks to the whitelist of Alibaba Cloud database instances (such as ApsaraDB RDS for MySQL or ApsaraDB for MongoDB) and to the security group rules of ECS-hosted databases. For self-managed databases in a data center or hosted by a third-party provider, manually add the DTS server CIDR blocks to the database whitelist. For the CIDR block list, see the CIDR blocks of DTS servers section.
Adding DTS server CIDR blocks to your whitelist or security group rules introduces potential security exposure. Before proceeding, take preventive measures such as: enforcing strong credentials, limiting exposed ports, authenticating API calls, auditing whitelist entries regularly, and removing unauthorized CIDR blocks. Alternatively, connect the database to DTS over Express Connect, VPN Gateway, or Smart Access Gateway.
Step 5: Configure synchronization objects
| Parameter | Description |
|---|---|
| Synchronization Types | Select Schema Synchronization, Full Data Synchronization, and Incremental Data Synchronization. DTS first synchronizes the schema and historical data, then switches to continuous incremental replication. |
| Processing Mode of Conflicting Tables | Precheck and Report Errors (default): the precheck fails if the destination contains tables with the same names as the source. Fix the conflict before starting the task. Use object name mapping to rename destination tables if they cannot be deleted. Ignore Errors and Proceed: skips the table-name conflict check. During full synchronization, existing destination records with matching primary or unique key values are kept. During incremental synchronization, they are overwritten. Use with caution — schema mismatches may cause partial column sync or task failure. |
| Synchronization Topology | Select One-way Synchronization. |
| Source Objects | Select the objects to synchronize (columns, tables, or databases) and move them to the Selected Objects section. If you select tables or columns, DTS does not synchronize views, triggers, or stored procedures. |
| Selected Objects | To rename a single object, right-click it and follow the instructions in Map the name of a single object. To rename multiple objects at once, click Batch Edit in the upper-right corner. To filter data by SQL condition, right-click an object and specify the WHERE clause. See Specify filter conditions. |
Step 6: Configure advanced settings
Click Next: Advanced Settings and configure the following parameters.
| Parameter | Description |
|---|---|
| Dedicated Cluster for Task Scheduling | By default, DTS schedules the task to the shared cluster. For higher stability, purchase a dedicated cluster. See What is a DTS dedicated cluster. |
| Retry Time for Failed Connections | How long DTS retries after a connection failure. Valid range: 10–1440 minutes. Default: 720 minutes. Set to at least 30 minutes. If DTS reconnects within this window, the task resumes. Otherwise, the task fails. If multiple tasks share the same source or destination database with different retry windows, the shortest window applies. DTS charges for the instance during retry. |
| Retry Time for Other Issues | How long DTS retries after DDL or DML operation failures. Valid range: 1–1440 minutes. Default: 10 minutes. Set to at least 10 minutes. This value must be less than the Retry Time for Failed Connections value. |
| Enable Throttling for Full Data Migration | Limit the load on the destination during full data synchronization by configuring Queries per second (QPS) to the source database, RPS of Full Data Migration, and Data migration speed for full migration (MB/s). Available only when Full Data Synchronization is selected. |
| Enable Throttling for Incremental Data Synchronization | Limit the load on the destination during incremental synchronization by configuring RPS of Incremental Data Synchronization and Data synchronization speed for incremental synchronization (MB/s). |
| Environment Tag | An optional tag to identify the instance environment. |
| Whether to delete SQL operations on heartbeat tables of forward and reverse tasks | Controls whether DTS writes heartbeat SQL operations to the source database. Yes: heartbeat SQL is not written; a latency value may be shown for the DTS instance. No: heartbeat SQL is written; this may affect features such as physical backup and cloning of the source database. |
| Configure ETL | Enables the extract, transform, and load (ETL) feature. Yes: enter data processing statements in the code editor. See Configure ETL in a data migration or data synchronization task. No: ETL is disabled. |
| Monitoring and Alerting | Configures alerts for task failures or synchronization latency exceeding a threshold. Yes: specify the alert threshold and notification contacts. See Configure monitoring and alerting when you create a DTS task. No: alerting is disabled. |
Step 7: Run the precheck
Click Next: Save Task Settings and Precheck.
To preview the API parameters used to configure this task, hover over Next: Save Task Settings and Precheck and click Preview OpenAPI parameters.
DTS runs a precheck before starting the task. If any items fail:
Click View Details next to the failed item, resolve the issue, and click Precheck Again.
If an alert item cannot be ignored, click View Details next to the failed item, troubleshoot the issue, and click Precheck Again.
If an alert item can be ignored, click Confirm Alert Details, then click Ignore in the dialog box, click OK, and click Precheck Again. Ignoring alert items may cause data inconsistency.
Step 8: Purchase the instance
Wait until the Success Rate reaches 100%, then click Next: Purchase Instance.
On the purchase page, configure the following parameters.
| Parameter | Description |
|---|---|
| Billing Method | Subscription: pay upfront for a fixed period. More cost-effective for long-term use. Pay-as-you-go: billed hourly. Suitable for short-term use. Release the instance when no longer needed to stop charges. |
| Resource Group Settings | The resource group for the data synchronization instance. Default: default resource group. See What is Resource Management? |
| Instance Class | The synchronization throughput tier. See Instance classes of data synchronization instances. |
| Subscription Duration | Available when Subscription is selected. Options: 1–9 months, or 1, 2, 3, or 5 years. |
Step 9: Start the task
Read and select Data Transmission Service (Pay-as-you-go) Service Terms, then click Buy and Start. In the confirmation dialog box, click OK.
The task appears in the task list. DTS runs schema synchronization and full data synchronization first, then switches to incremental data synchronization automatically.