When you need to keep two ApsaraDB RDS for MariaDB instances in continuous sync — for migrations, disaster recovery, or read scaling — Data Transmission Service (DTS) handles schema synchronization, full data synchronization, and incremental data synchronization in a single task. Follow the steps below to create a synchronization task between two MariaDB instances.
Prerequisites
Before you begin, make sure that you have:
A destination ApsaraDB RDS for MariaDB instance. See Create an ApsaraDB RDS for MariaDB instance.
Available storage space on the destination instance that exceeds the total data size of the source instance
A source database account with the permissions listed in Required permissions
Billing
| Synchronization type | Cost |
|---|---|
| Schema synchronization and full data synchronization | Free |
| Incremental data synchronization | Charged. See Billing overview. |
Supported synchronization topologies
DTS supports the following topologies between MariaDB instances:
One-way one-to-one synchronization
One-way one-to-many synchronization
One-way cascade synchronization
One-way many-to-one synchronization
Two-way one-to-one synchronization
For the full topology reference, see Synchronization topologies.
Supported SQL operations
| Operation type | SQL statements |
|---|---|
| DML | INSERT, UPDATE, DELETE |
| DDL | ALTER TABLE, ALTER VIEW, CREATE FUNCTION, CREATE INDEX, CREATE PROCEDURE, CREATE TABLE, CREATE VIEW, DROP INDEX, DROP TABLE, RENAME TABLE, TRUNCATE TABLE |
The RENAME TABLE operation can cause data inconsistency. If a synchronized table is renamed during synchronization, DTS stops syncing its data. To prevent this, select the entire database — not individual tables — as the synchronization object. Make sure the database containing the table is included in the synchronization objects both before and after the rename.
Required permissions
Grant the following permissions to the database account used for each instance:
| Instance | Required permissions |
|---|---|
| Source ApsaraDB RDS for MariaDB | Read and write permissions. See Create an account and Modify or reset account permissions. |
| Destination ApsaraDB RDS for MariaDB | Read and write permissions |
If the source account was not created through the ApsaraDB RDS for MariaDB console, grant it the following explicit permissions: REPLICATION CLIENT, REPLICATION SLAVE, SHOW VIEW, and SELECT.
Limitations
Source database requirements
Tables to be synchronized must have PRIMARY KEY or UNIQUE constraints, with all fields unique. Without these constraints, the destination database may contain duplicate records.
When synchronizing specific tables (not the entire database) that require renaming tables or columns in the destination, a single task supports up to 1,000 tables. Tasks exceeding this limit return a request error. Use multiple tasks in batches or synchronize the entire database instead.
Configure the binary log parameters as follows before starting the task:
Parameter Required value Why binlog_row_imagefullBinary logging is enabled by default on ApsaraDB RDS for MariaDB. If this parameter is not set to full, the precheck fails and the task cannot start.Binary log retention — incremental sync only At least 24 hours If DTS cannot obtain binary logs, the task fails and data inconsistency or data loss may occur. The DTS SLA does not apply when retention requirements are not met. Binary log retention — full + incremental sync At least 7 days After full data synchronization completes, the retention period can be reduced to more than 24 hours.
Foreign key behavior
During schema synchronization, DTS synchronizes foreign keys from the source to the destination.
During full data synchronization and incremental data synchronization, DTS temporarily disables constraint checks and cascade operations on foreign keys at the session level. Cascade update and delete operations on the source during synchronization may cause data inconsistency.
Other limitations
The destination database version must be the same as or later than the source database version. An earlier destination version causes compatibility issues.
Run synchronization tasks during off-peak hours. Initial full data synchronization uses read and write resources on both instances, which increases server load.
Concurrent INSERT operations during initial full data synchronization cause tablespace fragmentation in the destination. After full synchronization completes, the used tablespace in the destination is larger than in the source.
Do not use tools such as pt-online-schema-change for online DDL on synchronized tables. Use Data Management (DMS) instead. See Perform lock-free DDL operations.
Write to the destination database only through DTS during synchronization. Using other tools to write to the destination can cause data inconsistency. If you must use other tools, do not use DMS for online DDL operations, as this may cause data loss in the destination.
If a DDL statement fails in the destination database, the DTS task continues running. Check failed DDL statements in the task logs. See View task logs.
Create a synchronization task
Step 1: Go to the Data Synchronization Tasks page
Log on to the Data Management (DMS) console.
In the top navigation bar, click Data + AI.
In the left-side navigation pane, choose DTS (DTS) > Data Synchronization.
The DMS console layout may vary based on your configuration. See Simple mode and Customize the layout and style of the DMS console. You can also go directly to the Data Synchronization Tasks page in the new DTS console.
Step 2: Select the region
On the right side of Data Synchronization Tasks, select the region where the synchronization instance resides.
In the new DTS console, select the region in the top navigation bar.
Step 3: Configure source and destination databases
Click Create Task. In the Create Task wizard, configure the following settings.
Read the Limits displayed on the page before proceeding. Skipping this step may result in task failures or data inconsistency.
Task settings:
| Parameter | Description |
|---|---|
| Task Name | DTS generates a name automatically. Specify a descriptive name to help identify the task. Unique names are not required. |
Source Database
| Parameter | Value |
|---|---|
| Select an existing DMS database instance | Optional. Leave blank if no DMS database instance is registered. |
| Database Type | MariaDB |
| Connection Type | Alibaba Cloud Instance |
| Instance Region | Region where the source ApsaraDB RDS for MariaDB instance resides |
| Cross-account | No (for same-account synchronization) |
| Instance ID | ID of the source ApsaraDB RDS for MariaDB instance |
| Database Account | Account with read permissions on the objects to be synchronized |
| Database Password | Password for the database account |
| Encryption | Non-encrypted |
Destination Database
| Parameter | Value |
|---|---|
| Select an existing DMS database instance | Optional. Leave blank if no DMS database instance is registered. |
| Database Type | MariaDB |
| Connection Type | Alibaba Cloud Instance |
| Instance Region | Region where the destination ApsaraDB RDS for MariaDB instance resides |
| RDS Instance ID | ID of the destination ApsaraDB RDS for MariaDB instance |
| Database Account | Account with read and write permissions on the destination database |
| Database Password | Password for the database account |
| Encryption | Non-encrypted |
Step 4: Test connectivity
Click Test Connectivity and Proceed.
DTS automatically adds its CIDR blocks to the whitelist of Alibaba Cloud database instances and to the security group rules of Elastic Compute Service (ECS) instances hosting self-managed databases. You must make sure that the ECS instance can access the database. If the database is deployed on multiple ECS instances, you must manually add the CIDR blocks of DTS servers to the security group rules of each ECS instance. For self-managed databases in a data center or hosted by a third-party cloud provider, manually add the DTS CIDR blocks to the database whitelist. See CIDR blocks of DTS servers.
Adding CIDR blocks to whitelists or security group rules creates potential security exposure. Before using DTS, take precautions such as: using strong credentials, limiting exposed ports, authenticating API calls, reviewing whitelist and security group rules regularly, and connecting through Express Connect, VPN Gateway, or Smart Access Gateway when possible.
Step 5: Configure synchronization objects
Configure the following parameters, then proceed to advanced settings.
| Parameter | Description |
|---|---|
| Synchronization Types | Select Schema Synchronization, Full Data Synchronization, and Incremental Data Synchronization. Incremental data synchronization is selected by default. Full data synchronization copies historical data from the source to the destination, which serves as the baseline for incremental synchronization. |
| Method to Migrate Triggers in Source Database | Select how to synchronize triggers. Only displayed when Schema Synchronization is selected. If no triggers need to be synchronized, skip this parameter. See Synchronize or migrate triggers from the source database. |
| Processing Mode of Conflicting Tables | Precheck and Report Errors (default): fails the precheck if the source and destination have tables with identical names. If tables in the destination cannot be deleted or renamed, use the object name mapping feature to rename them. See Map object names. Ignore Errors and Proceed: skips the conflict check. Use with caution — data inconsistency can occur. During full data synchronization, if a destination record has the same primary key or unique key as a source record, the destination record is retained (not overwritten). During incremental data synchronization, the destination record is overwritten. If schemas differ, only some columns may sync or the task may fail. |
| Source Objects | Select objects from the Source Objects section and click the arrow icon to move them to the Selected Objects section. Select columns, tables, or databases. Selecting tables or columns excludes views, triggers, and stored procedures. |
| Selected Objects | Right-click an object to rename it. To rename multiple objects, click Batch Edit in the upper-right corner. Right-click a table to select specific SQL operations to synchronize or to add WHERE conditions for data filtering. See Map object names and Use SQL conditions to filter data. Renaming an object with object name mapping may cause dependent objects to fail to synchronize. |
Step 6: Configure advanced settings
Click Next: Advanced Settings and configure the following:
| Parameter | Description |
|---|---|
| Monitoring and Alerting | Enable alerting to receive notifications when the task fails or synchronization latency exceeds a threshold. Set the alert threshold and notification recipients. See Configure monitoring and alerting when you create a DTS task. |
| Retry Time for Failed Connections | How long DTS retries failed connections after the task starts. Range: 10–1,440 minutes. Default: 720 minutes. Set to a value greater than 30 minutes. If DTS reconnects within this time, the task resumes; otherwise the task fails. DTS charges for the instance during retries. If multiple tasks share the same source or destination database, the shortest retry time takes effect. |
| Retry Time for Other Issues | How long DTS retries failed DDL or DML operations. Range: 1–1,440 minutes. Default: 10 minutes. Set to a value greater than 10 minutes. This value must be less than Retry Time for Failed Connections. |
| Enable Throttling for Full Data Migration | Limit queries per second (QPS) to the source database, RPS of full data migration, and data migration speed (MB/s) to reduce the load on the destination server. Only available when Full Data Synchronization is selected. |
| Enable Throttling for Incremental Data Synchronization | Limit RPS and data synchronization speed (MB/s) for incremental synchronization to reduce load on the destination server. |
| Environment Tag | Tag to identify the DTS instance. Optional. |
| Configure ETL | Enable the extract, transform, and load (ETL) feature to process data during synchronization. Enter data processing statements in the code editor. See What is ETL? and Configure ETL in a data migration or data synchronization task. |
Step 7: Run the precheck
Click Next: Save Task Settings and Precheck.
To preview the API parameters for this task, hover over Next: Save Task Settings and Precheck and click Preview OpenAPI parameters before proceeding.
DTS runs a precheck before the task can start. If the precheck fails, click View Details next to each failed item, resolve the issues, then rerun the precheck. If an alert item can be ignored, click Confirm Alert Details > Ignore > OK, then click Precheck Again. Ignoring alert items may result in data inconsistency.
Step 8: Wait for the precheck to pass
Wait until the Success Rate reaches 100%, then click Next: Purchase Instance.
Step 9: Purchase a synchronization instance
On the buy page, configure the following:
| Parameter | Description |
|---|---|
| Billing Method | Subscription: pay upfront for a set duration; more cost-effective for long-term use. Pay-as-you-go: billed hourly; suitable for short-term use. Release the instance when no longer needed to stop charges. |
| Resource Group Settings | Resource group for the synchronization instance. Defaults to the default resource group. See What is Resource Management? |
| Instance Class | Determines synchronization speed. See Instance classes of data synchronization instances. |
| Subscription Duration | Available only with the Subscription billing method. Options: 1–9 months, or 1, 2, 3, or 5 years. |
Step 10: Accept the service terms
Read and select Data Transmission Service (Pay-as-you-go) Service Terms.
Step 11: Start the task
Click Buy and Start. In the dialog box, click OK.
Track progress in the task list.