Use Data Transmission Service (DTS) to migrate data from an ApsaraDB RDS for MySQL instance on RDS Enterprise Edition to an ApsaraDB RDS for MySQL instance on Cluster Edition. DTS supports schema migration, full data migration, and incremental data migration, so you can choose between a brief-downtime cutover or a near-zero-downtime migration.
Prerequisites
Before you begin, make sure you have:
Created both the source (RDS Enterprise Edition) and destination (Cluster Edition) ApsaraDB RDS for MySQL instances. For details, see Create an ApsaraDB RDS for MySQL instance.
Provisioned the destination instance with more storage than the source instance currently uses. Full data migration runs concurrent INSERT statements, which causes table fragmentation and increases the storage footprint in the destination.
Choose a migration strategy
Select a combination of migration types based on your downtime tolerance:
| Goal | Migration types | Downtime |
|---|---|---|
| Minimize downtime — keep the source online during migration | Schema migration + Full data migration + Incremental data migration | Near-zero (cut over when ready) |
| Brief downtime is acceptable — simpler setup | Schema migration + Full data migration | Required during migration window |
If you run only schema migration and full data migration, do not write new data to the source database during migration. Writing to the source after full data migration begins causes data inconsistency between the source and destination. To ensure real-time data consistency, select Schema Migration, Full Data Migration, and Incremental Data Migration.
Required permissions
Grant the following permissions to the database accounts used for the migration task:
| Database | Schema migration | Full data migration | Incremental data migration |
|---|---|---|---|
| Source (RDS Enterprise Edition) | SELECT | SELECT | Read and write |
| Destination (Cluster Edition) | Read and write | Read and write | Read and write |
To create an account, see Create an account. To adjust its permissions after creation, see Modify the permissions of an account.
If you configure the account outside the ApsaraDB RDS for MySQL console, the account must have the REPLICATION CLIENT, REPLICATION SLAVE, SHOW VIEW, and SELECT permissions. To migrate account information from the source, additional permissions are required. For details, see Migrate database accounts.
Configure and start the migration task
Step 1: Open the Data Migration page
Use one of the following methods:
DTS console
Log on to the DTS console .
In the left-side navigation pane, click Data Migration.
In the upper-left corner, select the region where the migration instance will reside.
DMS console
The actual navigation may vary based on your DMS console mode and layout. For details, see Simple mode. You can also customize the layout and style of the DMS console.
Log on to the DMS console .
In the top navigation bar, go to .
From the drop-down list next to Data Migration Tasks, select the region where the migration instance will reside.
Step 2: Create a task and configure source and destination databases
Click Create Task.
Configure the source and destination databases using the parameters in the following table.
WarningAfter configuring the source and destination databases, review the Limits displayed at the top of the page before proceeding. Skipping this step may result in task failure or data inconsistency.
Category Parameter Description — Task Name DTS generates a name automatically. Specify a descriptive name to make the task easy to identify. The name does not need to be unique. Source Database Select Existing Connection If the source instance is already registered with DTS, select it from the drop-down list — DTS populates the remaining fields automatically. For details, see Manage database connections. Otherwise, fill in the fields below. In the DMS console, use the Select a DMS database instance drop-down list. Database Type Select MySQL. Access Method Select Cloud Instance. Instance Region Select the region of the source ApsaraDB RDS for MySQL instance. Cross-account Select No (same Alibaba Cloud account). RDS Instance ID Select the ID of the source instance. Database Account Enter the database account. For permission requirements, see Required permissions. Database Password Enter the password for the database account. Connection Method Select Non-encrypted or SSL-encrypted. To use SSL encryption, enable SSL on the source instance before configuring the task. For details, see Use a cloud certificate to enable SSL encryption. Destination Database Select Existing Connection If the destination instance is already registered with DTS, select it from the drop-down list — DTS populates the remaining fields automatically. For details, see Manage database connections. Otherwise, fill in the fields below. In the DMS console, use the Select a DMS database instance drop-down list. Database Type Select MySQL. Access Method Select Cloud Instance. Instance Region Select the region of the destination ApsaraDB RDS for MySQL instance. Cross-account Select No (same Alibaba Cloud account). RDS Instance ID Select the ID of the destination instance. Database Account Enter the database account. For permission requirements, see Required permissions. Database Password Enter the password for the database account. Connection Method Select Non-encrypted or SSL-encrypted. To use SSL encryption, enable SSL on the destination instance before configuring the task. For details, see Use a cloud certificate to enable SSL encryption. Click Test Connectivity and Proceed.
NoteDTS must be able to reach both the source and destination databases. If you have security group or whitelist restrictions, add the CIDR blocks of DTS servers to the allowed list. For details, see Add DTS server IP addresses to a whitelist.
Step 3: Select objects and configure migration settings
On the Configure Objects page, configure the following settings:
Setting Description Migration Types Select the migration types based on your strategy. See Choose a migration strategy. NoteIf you do not select Schema Migration, create the target database and tables in the destination before the task runs and enable object name mapping in Selected Objects.
Method to Migrate Triggers in Source Database Select how to handle triggers. Required only when both Schema Migration and Incremental Data Migration are selected. For details, see Configure the method to synchronize or migrate triggers. Enable Migration Assessment Select Yesalert notification settings to evaluate schema compatibility (index length, stored procedures, dependent tables) before migration. Available only when Schema Migration is selected. Selecting Yes increases precheck time; the assessment result does not affect the precheck result. Processing Mode of Conflicting Tables Precheck and Report Errors (default): The precheck fails if the source and destination have tables with identical names. Use object name mapping to resolve conflicts. For details, see Map object names. Ignore Errors and Proceed: Skips the precheck. During full migration, DTS keeps the existing destination row if a primary key conflict occurs. During incremental migration, DTS overwrites the existing row. Use with caution — incompatible schemas may cause partial migration or task failure. Whether to migrate Event Select Yes to migrate events. If you select Yes, complete the additional steps described in Synchronize or migrate events. Capitalization of Object Names in Destination Instance By default, DTS default policy is selected. Change this if you need object name capitalization to match the source or destination. For details, see Specify the capitalization of object names in the destination instance. Source Objects Select objects from the Source Objects section and click
to move them to Selected Objects. Selecting tables or columns excludes other object types (views, triggers, stored procedures) from migration.Selected Objects To rename a single object in the destination, right-click it in Selected Objects. For details, see Map the name of a single object. To rename multiple objects at once, click Batch Edit. For details, see Map multiple object names at a time. NoteObject name mapping may cause migration to fail for objects that depend on the renamed object. To filter rows using a WHERE clause, right-click the table in Selected Objects. For details, see Set a filter condition. To select specific DML or DDL operations for incremental migration at the database or table level, right-click the object in Selected Objects and configure the operations.
Click Next: Advanced Settings and configure the following:
Setting Description Dedicated Cluster for Task Scheduling By default, DTS uses a shared cluster. Purchase a dedicated cluster for improved stability. For details, see What is a DTS dedicated cluster. Copy the temporary table of the Online DDL tool that is generated in the source table to the destination database. Controls how DTS handles temporary tables generated by online DDL tools. Importantpt-online-schema-change is not supported — using it causes task failure. Select one of: Yes — migrates data from temporary tables (may introduce latency if temporary tables are large). No, Adapt to DMS Online DDL — migrates only the original DDL operations from DMS (may lock destination tables). No, Adapt to gh-ost — migrates only the original DDL operations from gh-ost (may lock destination tables; supports custom regular expressions to filter shadow tables).
Whether to Migrate Accounts Select Yes to migrate account information from the source, then select the accounts and confirm their permissions. Retry Time for Failed Connections The time range (in minutes) during which DTS retries if the source or destination connection fails. Valid range: 10–1,440. Default: 720. Set this to a value greater than 30. If DTS reconnects within this window, the task resumes automatically. NoteIf multiple tasks share the same database, the most recently specified retry time applies. DTS charges for the instance during the retry period.
Retry Time for Other Issues The time range (in minutes) during which DTS retries failed DDL or DML operations. Valid range: 1–1,440. Default: 10. Set this to a value greater than 10. This value must be less than Retry Time for Failed Connections. Enable Throttling for Full Data Migration Limits resource usage during full data migration by setting Queries per second (QPS) to the source database, RPS of Full Data Migration, and Data migration speed for full migration (MB/s). Available only when Full Data Migration is selected. Enable Throttling for Incremental Data Migration Limits resource usage during incremental migration by setting RPS of Incremental Data Migration and Data migration speed for incremental migration (MB/s). Available only when Incremental Data Migration is selected. Environment Tag Optional. Select a tag to identify the instance by environment. Whether to delete SQL operations on heartbeat tables of forward and reverse tasks Controls whether DTS writes heartbeat SQL to the source database. Yes — does not write heartbeat SQL (latency may appear on the DTS instance). No — writes heartbeat SQL to the source (may affect physical backup and cloning of the source). Configure ETL Select Yes to enable the extract, transform, and load (ETL) feature and enter data processing statements. For an overview, see What is ETL?. For configuration steps, see Configure ETL in a data migration or data synchronization task. Monitoring and Alerting Select Yes to receive notifications when the task fails or migration latency exceeds your threshold. Configure the alert threshold and notification settings. For details, see Configure monitoring and alerting when you create a DTS task. Click Next Step: Data Verification to configure a data verification task. For details, see Configure a data verification task.
Step 4: Run the precheck
To preview the API parameters for this task configuration, hover over Next: Save Task Settings and Precheck and click Preview OpenAPI parameters.
Click Next: Save Task Settings and Precheck to save settings and start the precheck.
DTS runs a precheck before starting the migration. The task can start only after passing the precheck.
If the precheck fails, click View Details next to each failed item, resolve the issues, and click Precheck Again.
If an alert is raised during the precheck: if the alert cannot be ignored, fix the issue and rerun the precheck. If the alert can be ignored, click Confirm Alert Details, then click Ignore in the dialog box, confirm, and click Precheck Again. Ignoring alerts may cause data inconsistency.
Step 5: Purchase the instance and start migration
Wait until Success Rate reaches 100%, then click Next: Purchase Instance.
On the Purchase Instance page, configure the following:
Section Parameter Description New Instance Class Resource Group The resource group for the migration instance. Default: default resource group. For details, see What is Resource Management? Instance Class Select an instance class based on the required migration speed. For details, see Instance classes of data migration instances. Select the Data Transmission Service (Pay-as-you-go) Service Terms check box.
Click Buy and Start, then click OK in the confirmation dialog.
Track task progress on the Data Migration page.
Verify migration completion
After migration completes:
Check the task Status:
Full-only migration (schema + full): status shows Completed and the task stops automatically.
Migration with incremental data migration: status shows Running — the task does not stop automatically because it continuously syncs new changes.
Run
ANALYZE TABLE <table_name>on the destination tables. If a high availability (HA) switchover occurs in the destination MySQL database, data may be written to memory only and not persisted — this command confirms that data is fully written to the destination.Before switching your application workloads to the destination instance, stop or release the migration task, or revoke the DTS account's write permissions on the destination by running the
REVOKEcommand. This prevents the source from overwriting destination data if the task is automatically resumed.
Reference
Migration types
Schema migration: Migrates the schemas of selected objects — tables, views, triggers, stored procedures, and stored functions — from the source to the destination. Note that the
routine_bodyof stored procedures and stored functions, and theselect_statementof views cannot be modified during migration. DTS changes the SECURITY attribute from DEFINER to INVOKER for views, stored procedures, and functions, and sets the DEFINER to the destination database account. Note that the SECURITY attribute and DEFINER cannot be modified during migration. DTS does not migrate user information. To call a view, stored procedure, or stored function in the destination, grant read and write permissions to INVOKER.Full data migration: Migrates the historical data of selected objects from the source to the destination.
Incremental data migration: After full data migration completes, continuously syncs new changes from the source to the destination. This keeps your application online during the migration and lets you cut over at any time.
Billing
| Migration type | Instance fee | Internet traffic fee |
|---|---|---|
| Schema migration + Full data migration | Free | Charged only when Access Method is set to Public IP Address. For details, see Billing overview. |
| Incremental data migration | Charged. For details, see Billing overview. | — |
Supported SQL operations for incremental migration
| Operation type | SQL statements |
|---|---|
| DML | INSERT, UPDATE, DELETE |
| DDL | ALTER TABLE, ALTER VIEW, CREATE FUNCTION, CREATE INDEX, CREATE PROCEDURE, CREATE TABLE, CREATE VIEW, DROP INDEX, DROP TABLE, RENAME TABLE, TRUNCATE TABLE |
RENAME TABLE operations can cause data inconsistency. If you rename a table that is selected as a migration object, its data is not migrated to the destination. To prevent this, select the entire database (not individual tables) as the migration object, and make sure that both the pre-rename and post-rename database names are included in the migration scope.
Limitations
Foreign key behavior
During schema migration, DTS migrates foreign keys from the source to the destination.
During full and incremental data migration, DTS temporarily disables foreign key constraint checks and cascade operations at the session level. If you perform cascade update or delete operations on the source during migration, data inconsistency may occur.
Source database requirements
The source database server must have sufficient outbound bandwidth. Insufficient bandwidth reduces migration speed.
Tables must have primary keys or UNIQUE constraints with unique field values. Without these, duplicate rows may appear in the destination.
If you migrate at the table level and use object name mapping, a single task can migrate a maximum of 1,000 tables. If you exceed this limit, an error is reported after task submission. In that case, split the tables across multiple tasks or migrate the entire database instead.
Binary logging requirements for incremental migration
If you select incremental data migration, the source database must meet the following binary logging requirements:
| Parameter | Required value | Notes |
|---|---|---|
binlog_format | ROW | Required. If not set, the precheck fails. |
binlog_row_image | FULL | Required. If not set, the precheck fails. |
log_slave_updates | ON | Required only if the source is a self-managed MySQL in a dual-primary cluster. |
| Binary log retention (ApsaraDB RDS for MySQL) | At least 3 days | 7 days recommended. For configuration steps, see Configure parameters based on which the system automatically deletes the binary log files of an RDS instance. |
| Binary log retention (self-managed MySQL) | At least 7 days | Failure to meet this requirement may cause DTS to fail to obtain binary logs, which can result in task failure or, in extreme cases, data loss. |
A read-only ApsaraDB RDS for MySQL V5.6 instance (which does not record transaction logs) cannot be used as the source for incremental data migration.
Additional source limitations
Do not perform DDL operations during schema migration or full data migration. DDL changes cause the migration task to fail. During full migration, DTS queries the source database, which creates metadata locks that may also block DDL operations on the source.
If the source is MySQL 8.0.23 or later and the data includes invisible columns, DTS cannot read those columns and data loss occurs.
To make a column visible, run:
ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE;. For details, see Invisible Columns.Tables without primary keys auto-generate invisible primary keys. Make these visible before migration. For details, see Generated Invisible Primary Keys.
Data from binary log change operations — such as data restored from a physical backup or generated by cascade operations — is not recorded or migrated while the migration instance is running. If this data is missing in the destination, run a full migration again if your business allows it.
Other limitations
Use the same MySQL version for the source and destination when possible to ensure compatibility.
DTS does not migrate data that uses a parser defined by comments.
Online DDL changes made in temporary table mode (including merging multiple tables) may cause data loss in the destination or migration task failure.
If the destination is MySQL 8.0.23 or later and the destination table has invisible columns, DTS cannot find the target columns and the task fails with data loss. Make all destination invisible columns visible before migration.
If you skip schema migration, make sure that field types in the source and destination are compatible. For example, migrating a
textcolumn to avarchar(255)column can cause data truncation.If the data contains 4-byte characters (such as rare characters or emojis), the destination databases and tables must use the UTF8mb4 character set. If you use DTS schema migration, also set the
character_set_serverparameter in the destination to UTF8mb4.DTS periodically runs
CREATE DATABASE IF NOT EXISTS \test\`` on the source database to advance the binary log position.Run the migration during off-peak hours. Full data migration consumes read and write resources on both the source and destination databases.
FLOAT columns are migrated with a precision of 38 digits; DOUBLE columns with a precision of 308 digits (using
ROUND(COLUMN, PRECISION)). Verify that this precision meets your requirements.DTS auto-retries failed tasks for up to 7 days. Before switching workloads to the destination, stop or release the migration task, or run the
REVOKEcommand to revoke the write permissions of the DTS account on the destination. Otherwise, the source data may overwrite destination data when the task is automatically resumed.DTS automatically creates databases in ApsaraDB RDS for MySQL. If a database name does not comply with RDS MySQL naming conventions, create the database manually before configuring the migration task. For details, see Manage databases.
If DDL statements fail in the destination, the DTS task continues running. View the failed DDL statements in task logs. For details, see View task logs.
Column names that differ only in capitalization written to the same destination table may produce unexpected results because MySQL column names are not case-sensitive.
If an ApsaraDB RDS for MySQL instance has the EncDB feature enabled, full data migration is not supported. Instances with Transparent Data Encryption (TDE) enabled support schema migration, full data migration, and incremental data migration.
To migrate database accounts from the source, review the prerequisites and permissions first. For details, see Migrate database accounts.
If a DTS instance fails, the DTS helpdesk attempts recovery within 8 hours. Recovery operations may include restarting the instance and adjusting DTS instance parameters (not database parameters). For the parameters that may be modified, see Modify instance parameters.