Use Data Transmission Service (DTS) to migrate data from an ApsaraDB RDS for MariaDB TX instance to an ApsaraDB RDS for MySQL instance. DTS supports schema migration, full data migration, and incremental data migration, allowing you to migrate data with minimal downtime.
Prerequisites
Before you begin, make sure that you have:
-
Created the destination ApsaraDB RDS for MySQL instance. For more information, see Create an ApsaraDB RDS for MySQL instance
-
Verified that the destination instance has enough free storage space to accommodate all data from the source instance
-
Granted the required permissions to the database accounts. See Permissions required for database accounts
Billing
| Migration type | Task configuration fee | Data transfer fee |
|---|---|---|
| Schema migration and full data migration | Free | Free |
| Incremental data migration | Charged | See Billing overview |
Permissions required for database accounts
| Database | Schema migration | Full data migration | Incremental data migration |
|---|---|---|---|
| Source ApsaraDB RDS for MariaDB TX instance | SELECT on the objects to be migrated | Read and write on the objects to be migrated | Read and write on the objects to be migrated |
| Destination ApsaraDB RDS for MySQL instance | Read and write on the destination database | ||
For instructions on creating accounts and granting the required permissions, see:
-
ApsaraDB RDS for MariaDB TX: Create an account and Modify or reset account permissions
-
ApsaraDB RDS for MySQL: Create an account and Modify account permissions
SQL operations supported for incremental migration
| Type | Supported statements |
|---|---|
| DML | INSERT, UPDATE, DELETE |
| DDL | ALTER TABLE, ALTER VIEW, CREATE FUNCTION, CREATE INDEX, CREATE PROCEDURE, CREATE TABLE, CREATE VIEW, DROP INDEX, DROP TABLE, RENAME TABLE, TRUNCATE TABLE |
RENAME TABLE operations can cause data inconsistency. If you rename a table during migration and the table is selected as a migration object (not the database), data from that table will not migrate. To avoid this, select the database as the migration object instead of the individual table. Make sure both the original and renamed table belong to databases included in the migration objects.
Limitations
Foreign key behavior
DTS migrates foreign keys from the source database to the destination database during schema migration. During full data migration and incremental data migration, DTS temporarily disables foreign key constraint checks and cascade operations at the session level. If you perform cascade or delete operations on the source database during migration, data inconsistency may occur.
Source database requirements
| Requirement | Details |
|---|---|
| Bandwidth | The source server must have sufficient outbound bandwidth. Insufficient bandwidth reduces migration speed. |
| Table constraints | Tables must have PRIMARY KEY or UNIQUE constraints, and all fields must be unique. Without these constraints, the destination database may contain duplicate records. |
| Table count limit | If you select tables as migration objects and need to rename them in the destination database, a single migration task supports up to 1,000 tables. For more than 1,000 tables, split the task or select the entire database as the migration object. |
| DDL operations | Do not perform DDL operations on the source database during schema migration or full data migration. |
| Write operations | If you run only full data migration (without incremental migration), do not write to the source database during migration. To ensure data consistency, select schema migration, full data migration, and incremental data migration together. |
Binlog requirements for incremental data migration
To enable incremental data migration, configure the following parameters on the source instance:
| Parameter | Required value | Why |
|---|---|---|
| Binary logging | Enabled | DTS reads changes from binary logs for incremental migration. |
binlog_format |
row |
Row-based format captures full row-level changes required by DTS. |
binlog_row_image |
full |
Full image captures before and after values for each row change. |
Set the binary log retention period based on your migration type:
-
Incremental migration only: Retain binary logs for more than 24 hours.
-
Full data migration + incremental data migration: Retain binary logs for at least 7 days. After full data migration completes, you can reduce the retention period to more than 24 hours.
If DTS cannot retrieve the required binary logs, the task fails or data loss may occur. Setting retention periods below these minimums voids the DTS service level agreement (SLA).
Other limitations
-
Timing: Evaluate the impact on source and destination database performance before starting. Run migrations during off-peak hours — full data migration consumes read and write resources on both instances.
-
Tablespace size: Concurrent INSERT operations during full data migration cause fragmentation in destination tables. After migration, the destination tablespace is typically larger than the source.
-
FLOAT and DOUBLE precision: DTS retrieves FLOAT and DOUBLE column values using
ROUND(COLUMN,PRECISION). If no precision is specified, DTS uses 38 digits for FLOAT and 308 digits for DOUBLE. Verify that these defaults meet your requirements. -
Failed task resumption: DTS retries failed migration tasks for up to 7 days. Before switching workloads to the destination database, stop or release any failed tasks. Alternatively, run the
REVOKEstatement to revoke write permissions from the DTS accounts on the destination database. Otherwise, a resumed failed task may overwrite data in the destination database. -
DDL failures in destination: If DDL statements fail in the destination database, the DTS task continues running. Check the task logs to view failed DDL statements. For more information, see View task logs.
-
Column name case sensitivity: MySQL column names are case-insensitive. If the source database has columns whose names differ only in capitalization, the migration result may not match expectations.
-
Post-migration verification: After migration completes, run
analyze table <table_name>to verify that data was written to the destination table. In some high availability (HA) switchover scenarios, data may be written only to memory in the source instance, resulting in data loss.
Pre-migration checklist
Before starting the migration task, verify the following items. Skipping any item may cause the task to fail or data to be inconsistent.
| # | Item | How to verify |
|---|---|---|
| 1 | Destination RDS for MySQL instance is created and has enough storage space | Check instance status and storage in the RDS console |
| 2 | Database accounts have the required permissions | Run SHOW GRANTS FOR '<dts_user>'@'%'; on each database |
| 3 | Source instance has sufficient outbound bandwidth | Check bandwidth usage in the monitoring console |
| 4 | Tables to be migrated have PRIMARY KEY or UNIQUE constraints | Run SHOW CREATE TABLE <table_name>; to verify |
| 5 | Binary logging is enabled (incremental migration only) | Run SHOW VARIABLES LIKE 'log_bin'; — value must be ON |
| 6 | binlog_format = row (incremental migration only) |
Run SHOW VARIABLES LIKE 'binlog_format'; |
| 7 | binlog_row_image = full (incremental migration only) |
Run SHOW VARIABLES LIKE 'binlog_row_image'; |
| 8 | Binary log retention period meets requirements (incremental migration only) | Check the RDS parameter group settings |
| 9 | No DDL operations are planned during schema/full data migration | Coordinate with your team |
| 10 | Migration is scheduled during off-peak hours | Review application traffic patterns |
Create a migration task
Step 1: Go to the Data Migration Tasks page
-
Log on to the Data Management (DMS) console.
-
In the top navigation bar, click DTS.
-
In the left-side navigation pane, choose DTS (DTS) > Data Migration.
Alternatively, go directly to the Data Migration Tasks page of the new DTS console. Console layout and available operations may vary. For more information, see Simple mode and Customize the layout and style of the DMS console.
Step 2: Select the region
From the drop-down list next to Data Migration Tasks, select the region where the migration instance resides.
In the new DTS console, select the region in the upper-left corner.
Step 3: Configure source and destination databases
Click Create Task. On the Create Task page, configure the source and destination databases.
Read the limits displayed at the top of the page before proceeding. Skipping this step may cause the task to fail or result in data inconsistency.
Source database
| Parameter | Description |
|---|---|
| Select an existing DMS database instance (optional) | Select an existing instance to auto-populate the connection parameters, or leave blank to configure manually. |
| Database Type | Select MariaDB. |
| Access Method | Select Alibaba Cloud Instance. |
| Instance Region | Select the region where the source ApsaraDB RDS for MariaDB TX instance resides. |
| Replicate Data Across Alibaba Cloud Accounts | Select No for same-account migration. |
| Instance ID | Select or enter the ID of the source ApsaraDB RDS for MariaDB TX instance. |
| Database Account | Enter the database account for the source instance. See Permissions required for database accounts. |
| Database Password | Enter the password for the database account. |
| Encryption | Select Non-encrypted for this example. |
Destination database
| Parameter | Description |
|---|---|
| Select an existing DMS database instance (optional) | Select an existing instance to auto-populate the connection parameters, or leave blank to configure manually. |
| Database Type | Select MySQL. |
| Access Method | Select Alibaba Cloud Instance. |
| Instance Region | Select the region where the destination ApsaraDB RDS for MySQL instance resides. |
| Replicate Data Across Alibaba Cloud Accounts | Select No for same-account migration. |
| RDS Instance ID | Select or enter the ID of the destination ApsaraDB RDS for MySQL instance. |
| Database Account | Enter the database account for the destination instance. See Permissions required for database accounts. |
| Database Password | Enter the password for the database account. |
| Encryption | Select Non-encrypted or SSL-encrypted. If you select SSL-encrypted, enable SSL encryption on the destination instance first. See Configure the SSL encryption feature. |
Step 4: Test connectivity
Click Test Connectivity and Proceed.
DTS automatically adds its server CIDR blocks to the IP address whitelist of Alibaba Cloud database instances, and to the security group rules of Elastic Compute Service (ECS) instances hosting self-managed databases. For self-managed databases on multiple ECS instances or on-premises databases, manually add the DTS server CIDR blocks. See Add the CIDR blocks of DTS servers to the security settings of on-premises databases.
Adding public CIDR blocks to IP address whitelists or security group rules introduces security risks. Before using DTS, take preventive measures including: strengthening account passwords, limiting exposed ports, authenticating API calls, and regularly auditing IP whitelists and security group rules. Where possible, connect to DTS using Express Connect, VPN Gateway, or Smart Access Gateway instead of public network access.
Step 5: Configure migration objects
Configure the migration types and select objects to migrate.
| Parameter | Description |
|---|---|
| Migration Types |
Select the migration types based on your needs: - Full migration only: Select Schema Migration and Full Data Migration. - Migration with service continuity: Select Schema Migration, Full Data Migration, and Incremental Data Migration. If you do not select Incremental Data Migration, do not write to the source database during migration. |
| Method to Migrate Triggers in Source Database | Select the trigger migration method based on your requirements. This parameter is available only when both Schema Migration and Incremental Data Migration are selected. See Synchronize or migrate triggers from the source database. |
| Processing Mode of Conflicting Tables |
Precheck and Report Errors (default): DTS checks for tables with identical names in both databases. The task fails the precheck if duplicates exist. Use the object name mapping feature to resolve name conflicts. See Map object names. Ignore Errors and Proceed: Skips the duplicate name check. If source and destination tables share the same schema, DTS skips records with matching primary keys. If schemas differ, only specific columns migrate or the task fails. |
| Source Objects | Select objects to migrate from the Source Objects section, then click the arrow icon to add them to Selected Objects. You can select columns, tables, or databases. Selecting tables or columns excludes views, triggers, and stored procedures from migration. |
| Selected Objects | To rename a single object, right-click it in Selected Objects. See Map the name of a single object. To rename multiple objects at once, click Batch Edit. See Map multiple object names at a time. To filter data with WHERE conditions, right-click an object and specify the conditions. See Set filter conditions. To specify DML or DDL operations to migrate for an object, right-click the object and select the operations. |
Renaming an object with the object name mapping feature may cause other objects that depend on it to fail migration.
Step 6: Configure advanced settings
Click Next: Advanced Settings.
| Parameter | Description |
|---|---|
| Select the dedicated cluster used to schedule the task | By default, DTS uses the shared cluster. To use a dedicated cluster, purchase and specify one. See What is a DTS dedicated cluster. |
| Set Alerts | No: No alerting. Yes: Alerts are sent when the task fails or migration latency exceeds the threshold. Specify the alert threshold and contacts. See Configure monitoring and alerting. |
| Retry Time for Failed Connections | The retry window for connection failures. Valid values: 10–1,440 minutes. Default: 720 minutes. Set to more than 30 minutes. If DTS reconnects within the retry window, the task resumes. Otherwise, the task fails. Multiple tasks sharing the same source or destination database use the most recently configured retry time. |
| The wait time before a retry when other issues occur in the source and destination databases | The retry window for non-connection issues such as failed DML or DDL operations. Valid values: 1–1,440 minutes. Default: 10 minutes. Set to more than 10 minutes. This value must be less than the Retry Time for Failed Connections value. |
| Enable Throttling for Full Data Migration | Limits the QPS to the source database, RPS of full data migration, and migration speed (MB/s) to reduce load on the destination. Displayed only when Full Data Migration is selected. |
| Enable Throttling for Incremental Data Migration | Limits the RPS and migration speed (MB/s) for incremental migration. Displayed only when Incremental Data Migration is selected. |
| Environment Tag | An optional tag to identify the DTS instance. |
| Configure ETL | Yes: Opens a code editor to enter data processing statements. See Configure ETL in a data migration or synchronization task and What is ETL?. No: Skips ETL configuration. |
Step 7: Run the precheck
Click Next: Save Task Settings and Precheck.
To preview the API parameters for this task configuration, hover over Next: Save Task Settings and Precheck and click Preview OpenAPI parameters.
DTS runs a precheck before the migration task can start. If items fail:
-
Click View Details next to the failed item, fix the issue based on the error message, then click Precheck Again.
-
For alert items: if the item can be ignored, click Confirm Alert Details > Ignore > OK, then click Precheck Again. Ignoring alerts may cause data inconsistency.
Step 8: Purchase a migration instance
Wait until the precheck success rate reaches 100%, then click Next: Purchase Instance.
On the Purchase Instance page, configure the following:
| Section | Parameter | Description |
|---|---|---|
| New Instance Class | Resource Group | The resource group for the migration instance. Default: default resource group. See What is Resource Management? |
| Instance Class | Select the instance class based on your required migration speed. See Specifications of data migration instances. |
Read and accept the Data Transmission Service (Pay-as-you-go) Service Terms by selecting the check box, then click Buy and Start.
The migration task starts and appears in the task list. Monitor the task progress from the task list.
What's next
After the migration task completes, verify that data was written correctly by running analyze table <table_name> on the destination instance. You can then switch your workloads to the destination ApsaraDB RDS for MySQL instance.