Migrate data from an RDS for MySQL Enterprise Edition instance to an RDS for MySQL Cluster Edition instance - ApsaraDB RDS

When upgrading from an Enterprise Edition instance to a Cluster Edition instance, switching connections without data migration risks downtime and data loss. Use Data Transmission Service (DTS) to migrate data between the two instances with minimal downtime — keeping your application running throughout the process.

Prerequisites

Before you begin, make sure you have:

A source ApsaraDB RDS for MySQL Enterprise Edition instance and a destination Cluster Edition instance. See Create an ApsaraDB RDS for MySQL instance
Available storage space on the destination instance that exceeds the total data size on the source instance
binlog_format set to ROW and binlog_row_image set to FULL on the source instance. See Modify instance parameters. If these parameters are not set correctly, the precheck fails and the migration task cannot start

Permissions required

Grant the following permissions to the database accounts used by DTS before configuring the migration task.

Database	Schema migration	Full data migration	Incremental data migration
Source ApsaraDB RDS for MySQL Enterprise Edition instance	SELECT	SELECT	Read and write
Destination ApsaraDB RDS for MySQL Cluster Edition instance	Read and write	Read and write	Read and write

To create accounts and grant permissions, see Create an account and Modify the permissions of an account.

Migration types

DTS supports three migration types that you can combine depending on your requirements.

Schema migration copies the schemas of selected objects — tables, views, triggers, stored procedures, and stored functions — from the source to the destination. During schema migration:

DTS migrates foreign keys from the source to the destination.
The SECURITY attribute changes from DEFINER to INVOKER for views, stored procedures, and functions. To call these objects on the destination, grant read and write permissions to INVOKER.
DTS does not migrate user accounts.

During full data migration and incremental data migration, DTS temporarily disables the constraint check and cascade operations on foreign keys at the session level. If you perform cascade update and delete operations on the source database during data migration, data inconsistency may occur.

Full data migration copies all historical data in the selected objects from the source to the destination.

Incremental data migration continuously replicates changes from the source to the destination after full data migration completes. This keeps the two instances in sync and lets your application keep running during the migration.

Choose your migration approach

Goal	Select
One-time migration with a planned maintenance window	Schema migration + Full data migration
Minimize downtime — keep the application running during migration	Schema migration + Full data migration + Incremental data migration

If you run only full data migration (without incremental migration), stop all writes to the source database during migration to prevent data inconsistency.

SQL operations supported by incremental data migration

Operation type	SQL statements
DML	INSERT, UPDATE, DELETE
DDL	ALTER TABLE, ALTER VIEW; CREATE FUNCTION, CREATE INDEX, CREATE PROCEDURE, CREATE TABLE, CREATE VIEW; DROP INDEX, DROP TABLE; RENAME TABLE; TRUNCATE TABLE

Important

RENAME TABLE operations can cause data inconsistency. If you rename a table during migration and that table was selected as a migration object, its data will not be replicated to the destination. To avoid this, select the database (not individual tables) as the migration object, and make sure both the pre-rename and post-rename database names are included in the migration scope.

Billing

Migration type	Instance configuration fee	Internet traffic fee
Schema migration + full data migration	Free	Charged only when data is migrated from Alibaba Cloud over the Internet
Incremental data migration	Charged	—

For pricing details, see Billing overview.

Limitations

Source database

The source server must have sufficient outbound bandwidth. Insufficient bandwidth reduces migration speed.
Tables must have PRIMARY KEY or UNIQUE constraints, and all fields must be unique. Tables without these constraints may produce duplicate records in the destination.
If you select tables as migration objects and need to rename tables or columns in the destination, the limit is 1,000 tables per task. For larger migrations, split into multiple tasks or select the entire database as the migration object.
Binary log retention requirements:
- Incremental migration only: retain binary logs for more than 24 hours
- Full + incremental migration: retain binary logs for at least 7 days
- After full migration completes, you can reduce the retention period to more than 24 hours. Falling below these thresholds may cause DTS to fail to retrieve binary logs, which can lead to task failure or data loss. The DTS Service Level Agreement (SLA) does not cover service reliability if these requirements are not met.
During schema migration and full data migration: do not run DDL operations that change database or table schemas. The task will fail.
During full data migration only: do not write data to the source database. If you need writes to continue, include incremental data migration.

General

Use the same MySQL engine version for the source and destination instances to ensure compatibility.
Schedule the migration during off-peak hours. Full data migration uses read and write resources on both instances, which increases server load.
After full data migration, the tablespace on the destination may be larger than the source due to fragmentation from concurrent INSERT operations.
For FLOAT and DOUBLE columns, DTS uses ROUND(COLUMN, PRECISION) to read values. If you do not specify a precision, DTS defaults to 38 digits for FLOAT and 308 digits for DOUBLE. Verify these defaults meet your requirements before starting.
DTS automatically retries failed tasks for up to 7 days. Before switching workloads to the destination, either stop or release any failed tasks, or run REVOKE to remove DTS write access to the destination. Otherwise, source data may overwrite destination data when a failed task resumes.
If the source database name is invalid, manually create the destination database before configuring the task. See Manage databases. For valid database names, DTS creates the destination database automatically.

Configure the migration task

Step 1: Go to the Data Migration Tasks page

Log on to the Data Management (DMS) console.
In the top navigation bar, move the pointer over DTS, then choose DTS (DTS) > Data Migration.

Steps may vary based on the DMS console mode. See Simple mode and Customize the layout and style of the DMS console. Alternatively, go directly to the Data Migration page of the new DTS console.

Step 2: Select the region

From the drop-down list on the right side of Data Migration Tasks, select the region where your data migration instance resides.

In the new DTS console, select the region in the upper-left corner.

Step 3: Configure the source and destination databases

Click Create Task. On the Create Data Migration Task page, configure the source and destination databases.

Warning

After configuring the source and destination databases, read the Limits displayed at the top of the page before proceeding.

Source database

Parameter	Value
Select an existing DMS database instance	(Optional) Select an existing instance to auto-populate the parameters below.
Database Type	MySQL
Access Method	Alibaba Cloud Instance
Instance Region	The region where the source instance resides
Replicate Data Across Alibaba Cloud Accounts	No
RDS Instance ID	The ID of the source Enterprise Edition instance
Database Account	The database account with the required permissions (see Permissions required)
Database Password	The account password
Encryption	Select Non-encrypted or SSL-encrypted. To use SSL encryption, enable it on the RDS instance first. See Use a cloud certificate to enable SSL encryption.

Destination database

Parameter	Value
Select an existing DMS database instance	(Optional) Select an existing instance to auto-populate the parameters below.
Database Type	MySQL
Access Method	Alibaba Cloud Instance
Instance Region	The region where the destination instance resides
RDS Instance ID	The ID of the destination Cluster Edition instance
Database Account	The database account with read and write permissions
Database Password	The account password
Encryption	Select Non-encrypted or SSL-encrypted.

Step 4: Test connectivity

Click Test Connectivity and Proceed.

DTS automatically adds its server CIDR blocks to the IP address whitelist of ApsaraDB RDS instances. For self-managed databases on Elastic Compute Service (ECS) instances, DTS adds its CIDR blocks to the ECS security group rules. If the database runs on multiple ECS instances, manually add the DTS CIDR blocks to each instance's security group. For self-managed databases in data centers or on third-party clouds, manually add the DTS CIDR blocks to the database's IP address whitelist. For the full list of CIDR blocks, see Add the CIDR blocks of DTS servers.

Warning

Adding DTS server CIDR blocks to your whitelist or security group rules introduces security risks. Take preventive measures, including strengthening account credentials, limiting exposed ports, authenticating API calls, auditing whitelist and security group rules regularly, and considering private connectivity options such as Express Connect, VPN Gateway, or Smart Access Gateway.

Step 5: Select objects and configure migration settings

Configure the following parameters.

Parameter	Description
Migration Types	Select the migration types based on your approach. See Choose your migration approach.
Processing Mode of Conflicting Tables	Precheck and Report Errors: fails the precheck if the destination has tables with the same names as the source. Use this to catch conflicts before migration starts. Ignore Errors and Proceed: skips the name conflict check. During full data migration, existing records in the destination are retained. During incremental data migration, existing records are overwritten. If schemas differ between source and destination, only specific columns may be migrated, or the task may fail. Use this option with caution.
Capitalization of Object Names in Destination Instance	Defaults to DTS default policy. Adjust if the destination database has specific case requirements. See Specify the capitalization of object names in the destination instance.
Source Objects	Select one or more objects, then click the right-arrow icon to move them to Selected Objects. You can select columns, tables, or entire databases. Selecting tables or columns excludes views, triggers, and stored procedures from migration.
Selected Objects	To rename a single object, right-click it and select a rename option. To rename multiple objects, click Batch Edit. See Map object names. To filter rows with a WHERE condition, right-click the object and specify the condition. See Specify filter conditions. To select which DML or DDL operations to replicate incrementally, right-click the object and configure the operations.

Renaming an object may cause dependent objects to fail migration.

Step 6: Configure advanced settings

Click Next: Advanced Settings.

Data Verification Settings: to enable data verification, see Configure a data verification task.

Advanced Settings:

Parameter	Description
Monitoring and Alerting	Select Yes to receive alerts when the task fails or migration latency exceeds a threshold. Configure the alert threshold and notification settings. See Configure monitoring and alerting.
Copy the temporary table of the Online DDL tool that is generated in the source table to the destination database	Controls how DTS handles temporary tables from online DDL tools. Yes: migrates temporary table data. May cause latency if the data volume is large. No, Adapt to DMS Online DDL: migrates only the original DDL from DMS; temporary table data is skipped. Tables in the destination may be locked. No, Adapt to gh-ost: migrates only the original DDL from the gh-ost tool; temporary table data is skipped. Use default or custom regular expressions to filter shadow tables. Tables in the destination may be locked. Important Do not use pt-online-schema-change on the source database during migration — it will cause the DTS task to fail.
Retry Time for Failed Connections	The window during which DTS retries a lost connection. Valid range: 10 to 1,440 minutes. Default: 720 minutes. Set this to at least 30 minutes. If DTS reconnects within this window, the task resumes automatically; otherwise, it fails. DTS charges for the instance during retries. If multiple tasks share the same source or destination, the most recently set value applies.
Configure ETL	Select Yes to use the extract, transform, and load (ETL) feature to transform data during migration. See What is ETL? and Configure ETL in a data migration or data synchronization task.

Step 7: Run the precheck

Click Next: Save Task Settings and Precheck.

DTS runs a precheck automatically. The task cannot start until all precheck items pass.

If a precheck item fails, click View Details, fix the issue, and click Precheck Again.
If an alert is triggered and can be ignored, click Confirm Alert Details, then Ignore, then OK, and click Precheck Again. Ignoring alerts may lead to data inconsistency.

To view the API parameters for this configuration, hover over Next: Save Task Settings and Precheck and click Preview OpenAPI parameters.

Step 8: Purchase the migration instance

Wait until Success Rate reaches 100%, then click Next: Purchase Instance.

On the Purchase Instance page, configure the following:

Parameter	Description
Resource Group	The resource group for the migration instance. Defaults to default resource group. See What is Resource Management?
Instance Class	Select an instance class based on the required migration speed. Higher classes support faster migration. See Instance classes of data migration instances.

Step 9: Start the migration

Read and accept the Data Transmission Service (Pay-as-you-go) Service Terms by selecting the check box.
Click Buy and Start, then click OK in the confirmation message.

Track migration progress on the Data Migration page.