Data Transmission Service:Migrate data from a PolarDB for MySQL cluster to a PolarDB-X 2.0 instance

Source database

Binary logging requirements for incremental data migration:

Enabling binary logging on a PolarDB for MySQL cluster incurs storage charges for binary log files. For setup instructions, see Enable binary logging and Modify the retention period.

Other limitations

• DTS does not migrate read-only nodes of the source PolarDB for MySQL cluster.

• DTS does not migrate Object Storage Service (OSS) external tables from the source cluster.

• During schema migration, DTS migrates foreign keys from the source database to the destination database. During full data migration and incremental data migration, DTS temporarily disables foreign key constraint checks and cascade operations at the session level. If cascade update or delete operations are performed on the source database during migration, data inconsistency may occur.

• During full data migration, concurrent INSERT operations cause tablespace fragmentation. The used tablespace of the destination database will be larger than that of the source database after migration completes.

• DTS retrieves values from FLOAT and DOUBLE columns using ROUND(COLUMN,PRECISION). If no precision is specified, DTS sets precision to 38 digits for FLOAT and 308 digits for DOUBLE. Verify that these defaults meet your requirements.

• DTS attempts to resume failed tasks for up to 7 days. Before switching workloads to the destination database, stop or release any failed tasks, or run REVOKE to revoke write permissions from the DTS accounts. Otherwise, resumed failed tasks may overwrite data in the destination database.

• DTS runs CREATE DATABASE IF NOT EXISTS \`test\` on the source database periodically to advance the binary log position.

• If a task fails, DTS technical support will attempt to restore it within 8 hours. The task may be restarted and task parameters (not database parameters) may be modified during restoration.

Step 1: Go to the Data Migration page

Use one of the following methods to open the Data Migration page and select the region where your migration instance resides.

DTS console:

1. Log on to the DTS console.

2. In the left-side navigation pane, click Data Migration.

3. In the upper-left corner, select the region where the data migration instance resides.

DMS console:

The actual steps may vary depending on the mode and layout of your DMS console. See Simple mode and Customize the layout and style of the DMS console.

1. Log on to the DMS console.

2. In the top navigation bar, move the pointer over Data + AI > DTS (DTS) > Data Migration.

3. From the drop-down list to the right of Data Migration Tasks, select the region where the data migration instance resides.

Step 2: Configure source and destination databases

1. Click Create Task to go to the task configuration page.

2. Configure the source and destination databases using the following parameters.

   Warning

   After configuring the source and destination databases, read the Limits displayed at the top of the page. Skipping this step may cause task failures or data inconsistency.

   General settings

   Parameter	Description
   Task Name	A name for the DTS task. DTS auto-generates a name. Specify a descriptive name to identify the task easily. The name does not need to be unique.

   Source Database

   Parameter	Description
   Select Existing Connection	If the source instance is registered with DTS, select it from the drop-down list. DTS auto-fills the remaining parameters. In the DMS console, select the instance from Select a DMS database instance. If the instance is not registered, configure the parameters below manually.
   Database Type	Select PolarDB for MySQL.
   Access Method	Select Alibaba Cloud Instance.
   Instance Region	The region where the source PolarDB for MySQL cluster resides.
   PolarDB Instance ID	The ID of the source PolarDB for MySQL cluster.
   Database Account	The account for the source cluster. See Required permissions.
   Database Password	The password for the database account.
   Encryption	Whether to encrypt the connection. Configure based on your requirements. See Configure SSL encryption.

   Destination Database

   Parameter	Description
   Select Existing Connection	If the destination instance is registered with DTS, select it from the drop-down list. DTS auto-fills the remaining parameters. In the DMS console, select the instance from Select a DMS database instance. If the instance is not registered, configure the parameters below manually.
   Database Type	Select PolarDB-X 2.0.
   Access Method	Select Alibaba Cloud Instance.
   Instance Region	The region where the PolarDB-X 2.0 instance resides.
   Instance ID	The ID of the destination PolarDB-X 2.0 instance.
   Database Account	The account for the destination instance. See Required permissions.
   Database Password	The password for the database account.

3. Click Test Connectivity and Proceed.

   Make sure the CIDR blocks of DTS servers are added to the security settings of your source and destination databases. See Add the CIDR blocks of DTS servers.

Step 3: Configure migration objects

1. On the Configure Objects page, configure the following parameters.

   If you do not select Schema Migration, create the target database and tables in the destination database beforehand and enable object name mapping in Selected Objects. If you do not select Incremental Data Migration, do not write to the source database during migration to maintain data consistency. Renaming objects using the object name mapping feature may cause dependent objects to fail to migrate.

   Parameter	Description
   Migration Types	Select the migration types based on your needs: - For a one-time migration, select Schema Migration and Full Data Migration. - For continuous sync with minimal downtime (recommended), select Schema Migration, Full Data Migration, and Incremental Data Migration.
   Processing Mode of Conflicting Tables	- Precheck and Report Errors (recommended): checks for tables in the destination database with the same names as source tables. The precheck fails if conflicts exist. If conflicting destination tables cannot be deleted or renamed, use the object name mapping feature to rename migrated tables. See Map object names. - Ignore Errors and Proceed: skips the name conflict precheck. Use with caution — during full migration, DTS skips records that conflict with existing destination records; during incremental migration, DTS overwrites conflicting destination records. If schemas differ, only specific columns may be migrated or the task may fail.
   Source Objects	Select objects from the Source Objects section, click the rightwards arrow icon, and add them to the Selected Objects section. Selectable objects include columns, tables, and schemas. Selecting tables or columns excludes other object types such as views, triggers, and stored procedures.
   Selected Objects	- To rename a single object: right-click it and see Map the name of a single object. - To rename multiple objects at once: click Batch Edit. See Map multiple object names at a time. - To filter rows: right-click an object and specify WHERE conditions. See Specify filter conditions. - To select specific SQL operations for a table: right-click the object and select the operations.

2. Click Next: Advanced Settings and configure the following parameters.

   If multiple tasks share the same source or destination database and have different retry time settings, the most recently configured value applies. DTS charges for instance usage during retry periods. We recommend setting the retry time based on your business requirements and releasing the DTS instance promptly after migration completes.

   Parameter	Description
   Dedicated Cluster for Task Scheduling	By default, DTS schedules tasks to the shared cluster. For higher stability, purchase a dedicated cluster. See What is a DTS dedicated cluster.
   Retry Time for Failed Connections	How long DTS retries failed connections after the task starts. Valid values: 10–1,440 minutes. Default: 720 minutes. Set to at least 30 minutes. If DTS reconnects within this period, the task resumes; otherwise, it fails.
   Retry Time for Other Issues	How long DTS retries failed DDL or DML operations. Valid values: 1–1,440 minutes. Default: 10 minutes. Set to at least 10 minutes. This value must be smaller than Retry Time for Failed Connections.
   Enable Throttling for Full Data Migration	Limits read/write load during full migration. When enabled, configure Queries per second (QPS) to the source database, RPS of Full Data Migration, and Data migration speed for full migration (MB/s). Available only when Full Data Migration is selected.
   Enable Throttling for Incremental Data Migration	Limits load during incremental migration. When enabled, configure RPS of Incremental Data Migration and Data migration speed for incremental migration (MB/s). Available only when Incremental Data Migration is selected.
   Environment Tag	An optional tag to identify the DTS instance.
   Whether to delete SQL operations on heartbeat tables of forward and reverse tasks	Controls whether DTS writes heartbeat table operations to the source database. Yes: does not write heartbeat operations (a latency display may appear). No: writes heartbeat operations (may affect physical backup and cloning of the source database).
   Configure ETL	Whether to enable extract, transform, and load (ETL). Yes: opens the ETL code editor. See Configure ETL in a data migration or data synchronization task and What is ETL?. No: disables ETL.
   Monitoring and Alerting	Whether to receive alerts for task failures or latency thresholds. Yes: configure alert thresholds and notification settings. See Configure monitoring and alerting when you create a DTS task. No: no alerts.

3. Click Next Step: Data Verification to set up a data verification task. See Configure a data verification task.

Step 4: Run the precheck

• To preview the API parameters for this task, hover over Next: Save Task Settings and Precheck and click Preview OpenAPI parameters.

• Click Next: Save Task Settings and Precheck to save settings and start the precheck.

DTS runs a precheck before starting the migration. The task starts only after the precheck passes.

If the precheck fails:

• Click View Details next to any failed item, review the cause, fix the issue, and click Precheck Again.

• For alert items that can be ignored: click Confirm Alert Details > View Details > Ignore > OK, then click Precheck Again.

Ignoring alert items may result in data inconsistency or other risks.

Step 5: Purchase and start the migration instance

1. Wait until Success Rate reaches 100%, then click Next: Purchase Instance.

2. On the Purchase Instance page, configure the following parameters.

   Parameter	Description
   Resource Group	The resource group for the migration instance. Default: default resource group. See What is Resource Management?
   Instance Class	The migration instance class determines migration speed. Select a class based on your workload. See Instance classes of data migration instances.

3. Read and agree to Data Transmission Service (Pay-as-you-go) Service Terms by selecting the check box.

4. Click Buy and Start, then click OK in the confirmation dialog.