Use Data Transmission Service (DTS) to migrate data from a PolarDB for MySQL cluster to an ApsaraDB for SelectDB instance for large-scale analytics workloads. DTS supports schema migration, full data migration, and incremental data migration, so you can migrate data with minimal downtime.
Prerequisites
Before you begin, make sure that you have:
A source PolarDB for MySQL cluster
A target ApsaraDB for SelectDB instance
Database accounts with the required permissions on both instances (see Permissions required)
Billing
| Migration type | Link configuration fees | Data transfer fees |
|---|---|---|
| Schema migration + full data migration | Free | Free (unless the destination access method is Public IP Address — see Billing overview) |
| Incremental data migration | Charged — see Billing overview | — |
Permissions required
| Database | Schema migration | Full migration | Incremental migration |
|---|---|---|---|
| Source: PolarDB for MySQL | SELECT | SELECT | Access control list |
| Destination: SelectDB | USAGE_PRIV, SELECT_PRIV, LOAD_PRIV, ALTER_PRIV, CREATE_PRIV, DROP_PRIV | USAGE_PRIV, SELECT_PRIV, LOAD_PRIV, ALTER_PRIV, CREATE_PRIV, DROP_PRIV | USAGE_PRIV, SELECT_PRIV, LOAD_PRIV, ALTER_PRIV, CREATE_PRIV, DROP_PRIV |
To create accounts and grant permissions:
PolarDB for MySQL: see Create and manage database accounts and Manage the password of a database account.
SelectDB: see Cluster permission management and Basic permission management.
Limitations
Source database
| Limitation | Details |
|---|---|
| Bandwidth | The source database server must have at least 100 Mb/s outbound bandwidth. Lower bandwidth reduces migration speed. |
| Unsupported objects | Read-only nodes, OSS external tables, indexes, partitions, views, procedures, functions, triggers, and foreign keys cannot be migrated. |
| Table limit for column mapping | If you map column names or otherwise edit migration objects, a single task supports up to 1,000 tables. To migrate more, split the tables across multiple tasks or migrate the entire database without column-level edits. |
| Binary logging (incremental migration only) | Enable binary logging and set the loose_polar_log_bin parameter to on. Without this, the precheck fails and migration cannot start. See Enable binary logging and Modify parameters. Enabling binary logging consumes storage and incurs fees. |
| Binary log retention (incremental migration only) | Retain binary logs for at least 3 days (7 days recommended). If DTS cannot retrieve binary logs, migration may fail or cause data inconsistency or loss. This is not covered by the DTS Service-Level Agreement (SLA). See Modify the retention period. |
Destination database (SelectDB)
| Limitation | Details |
|---|---|
| Supported engines | Only tables using the Unique or Duplicate engine are supported. |
| Unique engine | All unique keys in the destination table must exist in the source table and be included in the migration objects. Otherwise, data inconsistency may occur. |
| Duplicate engine | Duplicate rows may appear when a retry occurs, the instance restarts, or two or more DML operations are performed on the same row after the instance starts. Use the additional columns _is_deleted, _version, and _record_id to identify and remove duplicates. DTS converts UPDATE and DELETE statements to INSERT statements for Duplicate engine tables. |
bucket_count parameter | In the Selected Objects box, you can only configure the bucket_count parameter (number of buckets). The value must be a positive integer; the default is auto. |
| Naming rules | Database and table names must start with a letter. Object names containing Chinese characters (databases, tables, or columns) must be renamed using the object name mapping feature before migration. |
| VARCHAR length conversion | In PolarDB for MySQL, VARCHAR(M) is character length. In SelectDB, VARCHAR(N) is byte length. If you do not use schema migration, set SelectDB VARCHAR field lengths to four times the PolarDB for MySQL values. |
| Multi-table merge | When migrating from multiple source tables into a single destination table, the source table schemas must be identical. Schema differences cause data inconsistency or task failure. |
Operational limits during migration
Avoid the following actions while migration is running:
| Prohibited action | Risk |
|---|---|
| DDL operations that change schemas (during schema migration or full migration) | The migration task fails. |
| DDL operations that modify multiple columns at once, or the same table in succession | Not supported; task fails. |
| Online DDL operations using pt-online-schema-change on the source | Cannot be migrated; may cause data loss or task failure. |
| Creating clusters in the destination SelectDB instance | Task fails. Restart the migration instance to resume. |
| Adding backend (BE) nodes to SelectDB | Task fails. Restart the migration instance to resume. |
| Writing data from sources other than DTS to the destination | May cause data inconsistency. |
| Writing new data to the source (full migration only, no incremental) | Causes data inconsistency between source and destination. |
| Primary/secondary failover during full migration | DTS does not support failover during full migration. Reconfigure the task if failover occurs. |
Other considerations
Online DDL with DMS or gh-ost: DTS migrates only the original DDL statements — not temporary table data. This may lock destination tables.
Non-binary-log operations: Data changes not recorded in binary logs (such as physical backup recovery and cascade operations) are not migrated. Run a full data migration again if this occurs.
RENAME TABLE: RENAME TABLE during incremental migration may cause data inconsistency. If you rename a table that is a migration object, its data may not be migrated. To avoid this, select the database — not individual tables — as the migration object, and make sure the database appears in the migration objects both before and after the rename.
Incremental synchronization latency: DTS writes to each sync object at most once every 5 seconds by default. Normal latency is typically within 10 seconds. To reduce latency, adjust the
selectdb.reservoir.timeout.millisecondsparameter in the DTS console (valid range: 1,000–10,000 ms). Shorter intervals increase write frequency and may raise destination load and write response time (RT).Performance impact: Full migration reads all source data and writes to the destination concurrently, which increases load on both databases. Run migrations during off-peak hours and enable throttling when needed.
Tablespace size after full migration: Concurrent INSERT operations during full migration cause fragmentation. The destination tablespace may be larger than the source after migration completes.
Instance recovery: If the migration instance fails, the DTS helpdesk attempts recovery within 8 hours. Recovery may involve restarting the instance or adjusting DTS instance parameters (not database parameters). See Modify instance parameters for the parameters that may be modified.
SQL operations supported for incremental migration
| Operation type | SQL statements |
|---|---|
| DML | INSERT, UPDATE, DELETE |
| DDL | ADD COLUMN, MODIFY COLUMN, CHANGE COLUMN, DROP COLUMN, DROP TABLE, TRUNCATE TABLE, RENAME TABLE |
RENAME TABLE may cause data inconsistency. If you select a table as a migration object and rename it during migration, its data may not migrate to the destination. To prevent this, select the database (not individual tables) as the migration object, and make sure the database appears in the migration objects both before and after the rename.
Create a migration task
Step 1: Open the Data Migration page
Use one of the following consoles:
DTS console
Log on to the DTS console.DTS console
In the left-side navigation pane, click Data Migration.
In the upper-left corner, select the region where the migration instance resides.
DMS console
The exact steps may vary based on your DMS console mode and layout. See Simple mode and Customize the layout and style of the DMS console.
Log on to the DMS console.DMS console
In the top navigation bar, go to Data + AI > DTS (DTS) > Data Migration.
From the drop-down list to the right of Data Migration Tasks, select the region where the migration instance resides.
Step 2: Configure source and destination databases
Click Create Task.
On the task configuration page, set the following parameters.
General
| Parameter | Description |
|---|---|
| Task Name | A name for the DTS task. DTS auto-generates a name. Specify a descriptive name to identify the task. The name does not need to be unique. |
Source Database
| Parameter | Description |
|---|---|
| Select Existing Connection | If the source instance is already registered with DTS, select it from the drop-down list — DTS populates the remaining parameters automatically. In the DMS console, use the Select a DMS database instance drop-down. If the instance is not registered, configure the parameters below manually. |
| Database Type | Select PolarDB for MySQL. |
| Access Method | Select Alibaba Cloud Instance. |
| Instance Region | The region where the source PolarDB for MySQL cluster resides. |
| Replicate Data Across Alibaba Cloud Accounts | Select No for migration within the same account. To migrate across accounts, first configure RAM authorization. |
| PolarDB Cluster ID | The ID of the source PolarDB for MySQL cluster. |
| Database Account | The database account for the source cluster. See Permissions required. |
| Database Password | The password for the database account. |
| Encryption | Select an option based on your PolarDB for MySQL cluster settings. For Secure Sockets Layer (SSL) encryption details, see Configure SSL encryption. |
Destination Database
| Parameter | Description |
|---|---|
| Select Existing Connection | If the destination instance is already registered with DTS, select it from the drop-down list. In the DMS console, use the Select a DMS database instance drop-down. If not registered, configure the parameters below manually. |
| Database Type | Select SelectDB. |
| Access Method | Select Alibaba Cloud Instance. |
| Instance Region | The region where the destination SelectDB instance resides. |
| Replicate Data Across Alibaba Cloud Accounts | Select No for migration within the same account. |
| Instance ID | The ID of the destination SelectDB instance. |
| Database Account | The database account for the destination SelectDB instance. See Permissions required. |
| Database Password | The password for the database account. |
Click Test Connectivity and Proceed.
DTS server CIDR blocks must be added to the security settings of both databases before the connectivity test can pass. See Add DTS server IP addresses to a whitelist.
Step 3: Select migration objects
On the Configure Objects page, configure the following settings.
| Configuration | Description |
|---|---|
| Migration Types | Select the migration types based on your requirements: Full migration only: Select Schema Migration and Full Data Migration. Migration with minimal downtime: Select Schema Migration, Full Data Migration, and Incremental Data Migration. |
| Processing Mode of Conflicting Tables | Precheck and Report Errors (default): DTS checks for tables with the same name in the destination. If duplicates exist, the precheck fails and the task does not start. To resolve without deleting the destination table, use object name mapping. Ignore Errors and Proceed: DTS skips the duplicate table check. Use with caution — if schemas match, source records overwrite destination records with the same primary key; if schemas differ, some columns may not migrate or the task may fail. |
| Capitalization of Object Names in Destination Instance | Controls capitalization of database, table, and column names in the destination. Default is DTS default policy. See Specify the capitalization of object names in the destination instance. |
| Source Objects | Select databases or tables to migrate. Click the arrow icon to add them to Selected Objects. |
| Selected Objects | Right-click a migration object to rename it, set the bucket_count parameter (when Schema Migration is selected), select specific incremental SQL operations, or set SQL filter conditions for row-level filtering. See Map object names and Set filter conditions. |
If you do not select Schema Migration, create tables using the Unique or Duplicate key model in the destination SelectDB instance before starting migration. See Data type mappings, Additional columns, and Data model.
If you do not select Incremental Data Migration, do not write new data to the source instance during migration.
Using object name mapping may cause dependent objects to fail migration.
Click Next: Advanced Settings and configure the following.
| Configuration | Description |
|---|---|
| Dedicated Cluster for Task Scheduling | By default, DTS uses a shared cluster. For higher stability, purchase a dedicated cluster. See What is a DTS dedicated cluster. |
| Retry Time for Failed Connections | How long DTS retries failed connections. Valid range: 10–1,440 minutes. Default: 720 minutes. Set to at least 30 minutes. If DTS reconnects within this window, migration resumes; otherwise, the task fails. Note that different tasks sharing the same source or destination use the most recently set value, and DTS charges for the instance during retry. |
| Retry Time for Other Issues | How long DTS retries failed DDL or DML operations. Valid range: 1–1,440 minutes. Default: 10 minutes. Set to at least 10 minutes. Must be less than Retry Time for Failed Connections. |
| Enable Throttling for Full Data Migration | Limits read and write load during full migration. Configure Queries per second (QPS) to the source database, RPS of Full Data Migration, and Data migration speed for full migration (MB/s). Available only when Full Data Migration is selected. |
| Enable Throttling for Incremental Data Migration | Limits load during incremental migration. Configure RPS of Incremental Data Migration and Data migration speed for incremental migration (MB/s). Available only when Incremental Data Migration is selected. |
| Environment Tag | (Optional) A tag to identify the instance. |
| Whether to delete SQL operations on heartbeat tables of forward and reverse tasks | Yesalert notification settings: does not write heartbeat operations to the source (a latency indicator may appear on the DTS instance). No: writes heartbeat operations (may affect physical backup and cloning of the source). |
| Configure ETL | Yes: configure extract, transform, and load (ETL) processing with data processing statements in the code editor. See Configure ETL in a data migration or data synchronization task and What is ETL?. No: skip ETL. |
| Monitoring and Alerting | Yes: configure alerts for task failure or latency exceeding a threshold. Set the alert threshold and notification contacts. See Configure monitoring and alerting when you create a DTS task. No: no alerting. |
(Optional) Click Next: Configure Database and Table Fields to set the Primary Key Column, Distribution Key, and Engine for destination tables.
This step is available only when Schema Migration is selected. Set Definition Status to All to view all tables.
Select one or more columns as Primary Key Column. At least one primary key column must also be selected as the Distribution Key.
For tables without primary keys or UNIQUE constraints, set Engine to duplicate. Otherwise, migration may fail or data may be lost.
Step 4: Run a precheck
Click Next: Save Task Settings and Precheck.
To preview the API parameters for this task, hover over Next: Save Task Settings and Precheck and click Preview OpenAPI parameters.
DTS runs a precheck before migration starts. The task can only start after the precheck passes.
If a check item fails, click View Details next to the failed item, fix the issue, then click Precheck Again.
If an alert is triggered:
If it cannot be ignored, click View Details, fix the issue, and run the precheck again.
If it can be ignored, click Confirm Alert Details, click Ignore in the dialog, click OK, then click Precheck Again.
Ignoring precheck alerts may result in data inconsistency or other risks.
Step 5: Purchase and start the migration instance
Wait until Success Rate reaches 100%, then click Next: Purchase Instance.
On the Purchase Instance page, configure the following.
| Section | Parameter | Description |
|---|---|---|
| New Instance Class | Resource Group | The resource group for the migration instance. Default: default resource group. See What is Resource Management? |
| Instance Class | Determines migration speed. Select based on your data volume and migration window. See Instance classes of data migration instances. |
Read and accept Data Transmission Service (Pay-as-you-go) Service Terms.
Click Buy and Start, then click OK in the confirmation dialog.
Track migration progress on the Data Migration page:
Full migration only: The task stops automatically when complete. Status shows Completed.
Incremental migration: The task runs continuously and does not stop automatically. Status shows Running.
Data type mappings
The following table shows how PolarDB for MySQL data types map to SelectDB data types.
| Category | PolarDB for MySQL | SelectDB |
|---|---|---|
| Numeric | TINYINT | TINYINT |
| TINYINT UNSIGNED | SMALLINT | |
| SMALLINT | SMALLINT | |
| SMALLINT UNSIGNED | INT | |
| MEDIUMINT | INT | |
| MEDIUMINT UNSIGNED | BIGINT | |
| INT | INT | |
| INT UNSIGNED | BIGINT | |
| BIGINT | BIGINT | |
| BIGINT UNSIGNED | LARGEINT | |
| BIT(M) | INT | |
| Decimal | Decimal | Decimal (Zerofill not supported) |
| Numeric | Decimal | |
| Float | Float | |
| Double | DOUBLE | |
| BOOL / BOOLEAN | BOOLEAN | |
| Date and time | DATE | DATEV2 |
| DATETIME[(fsp)] | DATETIMEV2 | |
| Timestamp[(fsp)] | DATETIMEV2 | |
| Time[(fsp)] | VARCHAR | |
| YEAR[(4)] | INT | |
| String | CHAR / VARCHAR | VARCHAR |
| BINARY / VARBINARY | STRING | |
| TINYTEXT / TEXT / MEDIUMTEXT / LONGTEXT | STRING | |
| TINYBLOB / BLOB / MEDIUMBLOB / LONGBLOB | STRING | |
| ENUM | STRING | |
| SET | STRING | |
| JSON | STRING |
To prevent data loss, CHAR and VARCHAR(n) are converted to VARCHAR(4*n) during migration to SelectDB.
If no length is specified, the default is VARCHAR(65533).
Data longer than 65,533 characters is converted to STRING.
Additional columns
For tables using the Duplicate engine, DTS automatically adds the following columns to destination tables. If you create destination tables manually, add these columns yourself.
| Column name | Data type | Default value | Description |
|---|---|---|---|
_is_deleted | Int | 0 | Indicates whether the row is deleted. 0 for INSERT and UPDATE; 1 for DELETE. |
_version | Bigint | 0 | 0 for full migration. For incremental migration, the timestamp in seconds from the source binary log. |
_record_id | Bigint | 0 | 0 for full migration. For incremental migration, the unique auto-incrementing record ID from the incremental log entry. |
What's next
After migration completes, verify data consistency between the source and destination databases. For incremental migration, stop writes to the source, let DTS synchronize the remaining changes, then switch your application traffic to SelectDB.