Data Transmission Service:Migrate data from an RDS for MySQL instance to a Tablestore instance

Source database requirements

• Primary keys or UNIQUE constraints: Tables must have primary keys or UNIQUE constraints with unique fields. Tables without these may produce duplicate data in the destination. Ensure all migration objects have primary keys before starting.

• Table limit per task: When selecting individual tables and editing them (such as mapping column names), a single task supports up to 1,000 tables. If you exceed this limit, split tables across multiple tasks or configure the task to migrate the entire database.

• Binary log requirements for incremental migration: The source database must meet all of the following conditions, or the precheck fails and the task cannot start: If DTS cannot obtain binary logs due to insufficient retention, the task fails and data inconsistency or loss may occur. The Service Level Agreement (SLA) guarantee does not apply in this case.

  • Binary logging is enabled.

  • binlog_format = row

  • binlog_row_image = full

  • For self-managed MySQL in a dual-primary cluster: log_slave_updates = ON

  • Binary log retention for RDS for MySQL: at least 3 days (7 days recommended). See Configure parameters based on which the system automatically deletes the binary log files of an RDS instance.

  • Binary log retention for self-managed MySQL: at least 7 days.

• DDL operations during migration: Do not run DDL operations on the source database during schema migration or full data migration — the task will fail.

  During full data migration, DTS queries the source database, which creates metadata locks that may block DDL operations on the source.

• Binary log change data not migrated: Data generated by change operations on binary logs — such as data restored from a physical backup or data from a cascade operation — is not recorded and migrated to the destination database while the data migration instance is running.

  If this change data is not migrated to the destination database, you can perform a full data migration again, provided your business is not affected.

• Read-only RDS for MySQL V5.6: Instances that do not record transaction logs (such as read-only ApsaraDB RDS for MySQL V5.6) cannot be used as the source for incremental migration.

• Invisible columns (MySQL 8.0.23 and later): DTS cannot read invisible columns, which causes data loss. Make a column visible before migrating:

  ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE;

  Tables without explicit primary keys automatically generate invisible primary keys. Make these visible before migrating. See Invisible Columns and Generated Invisible Primary Keys.

• EncDB feature: Full data migration is not supported for RDS for MySQL instances with the EncDB feature enabled. Instances with Transparent Data Encryption (TDE) support schema, full, and incremental migration.

Destination database requirements

• Table limit: The destination Tablestore instance supports a maximum of 64 tables. If your migration exceeds this limit, contact Alibaba Cloud support to raise the quota for the destination instance.

• Naming conventions: Table and column names must follow Tablestore naming rules, or schema migration fails:

  • Allowed characters: letters, digits, and underscores (_). Names must start with a letter or underscore.

  • Length: 1–255 characters.

• Data Write Mode and Batch Write Mode: Configure these Tablestore-specific settings carefully. See the parameter descriptions in Configure the objects to migrate.

Other limitations

• Do not use pt-online-schema-change or similar tools for online DDL on migration objects during the task.

• For FLOAT and DOUBLE columns, DTS reads values using ROUND(COLUMN, PRECISION). Default precision: FLOAT = 38, DOUBLE = 308. Confirm these values meet your requirements before starting.

• Run migrations during off-peak hours. Full data migration consumes read and write resources on both databases and may increase loads.

• After full data migration, the destination table storage is larger than the source due to fragmentation from concurrent INSERT operations.

• DTS automatically retries failed tasks for up to 7 days. Before switching your business to the destination, stop or release the DTS instance — or run REVOKE to remove write permissions from the DTS account — to prevent automatic retries from overwriting destination data with source data.

• DTS executes CREATE DATABASE IF NOT EXISTS `test` in the source database periodically to advance the binary log position.

• If a DTS instance fails, support will attempt to recover it within 8 hours. During recovery, the instance may restart and parameters may be adjusted (only DTS instance parameters, not database parameters). See Modify instance parameters.

Self-managed MySQL

• If a primary/secondary switchover occurs on the source self-managed MySQL database while the task is running, the task fails.

• Migration latency is calculated from the timestamp of the latest migrated record in the destination against the current source timestamp. If no DML operations occur on the source for a long time, the displayed latency may be inaccurate. To correct it, perform a DML operation on the source — or create a heartbeat table that is updated every second if you are migrating an entire database.

Step 1: Go to the Data Migration page

Open the Data Migration page using either the DTS console or DMS console.

DTS console

1. Log on to the DTS console.

2. In the left-side navigation pane, click Data Migration.

3. In the upper-left corner, select the region where the data migration instance resides.

DMS console

The actual steps may vary based on the DMS console mode and layout. See Simple mode and Customize the layout and style of the DMS console.

1. Log on to the DMS console.

2. In the top navigation bar, move the pointer over Data + AI > DTS (DTS) > Data Migration.

3. From the drop-down list next to Data Migration Tasks, select the region where the data migration instance resides.

Step 2: Create a task and configure endpoints

Click Create Task to open the task configuration page, then configure the parameters below.

Task name

DTS auto-generates a name. Specify a descriptive name for easy identification — the name does not need to be unique.

Source database

Destination database

Step 3: Test connectivity

Click Test Connectivity and Proceed at the bottom of the page.

DTS server CIDR blocks must be added to the security settings of both the source and destination databases. See Add DTS server IP addresses to a whitelist. For self-managed databases, click Test Connectivity in the CIDR Blocks of DTS Servers dialog box.

Step 4: Configure the objects to migrate

On the Configure Objects page, set the migration parameters.

Migration types and object selection

Parameter	Description
Migration Types	Select Schema Migration and Full Data Migration for a one-time migration. Select all three (Schema Migration, Full Data Migration, and Incremental Data Migration) to keep the source and destination in sync during migration.
Source Objects	Select one or more objects from the Source Objects section, then click to move them to Selected Objects. Migration granularity is at the database and table level. You can only select tables from a single database per task.
Selected Objects	Right-click an object to rename it in the destination. For bulk renaming, click Batch Edit. To filter rows, right-click a table and set a WHERE clause. See Map object names and Set filter conditions. Hover over a table and click the Edit icon to set column data types for the Tablestore instance.

If you skip Schema Migration, create the target tables in Tablestore manually before starting and enable object name mapping in Selected Objects. Only table names support mapping — mapping objects that other objects depend on may cause migration failures for those dependents.

Conflict handling

Tablestore write settings

Step 5: Configure advanced settings

Click Next: Advanced Settings and configure these parameters as needed.

After configuring advanced settings, click Next: Configure Database And Table Fields to set primary key columns for Tablestore tables.

Step 6: Run the precheck

Click Next: Save Task Settings and Precheck.

To preview the OpenAPI parameters for this task configuration, hover over Next: Save Task Settings and Precheck and click Preview OpenAPI parameters before proceeding.

DTS runs a precheck before starting the task. If the precheck fails:

• Click View Details next to each failed item, fix the issue, then click Precheck Again.

• For alert items (non-blocking): click Confirm Alert Details > Ignore > OK > Precheck Again. Ignoring alerts may lead to data inconsistency.

Step 7: Purchase the instance and start the task

1. Wait until Success Rate reaches 100%, then click Next: Purchase Instance.

2. On the Purchase Instance page, set the following:

   Parameter	Description
   Resource Group	The resource group for the migration instance. Default: default resource group. See What is Resource Management?
   Instance Class	Select an instance class based on your required migration speed. See Instance classes of data migration instances.

3. Read and accept the Data Transmission Service (Pay-as-you-go) Service Terms.

4. Click Buy and Start, then click OK in the confirmation dialog.