Type | Description |
Source database limits | Bandwidth requirements: The server that hosts the source database must have sufficient outbound bandwidth. Otherwise, the migration speed is affected. The tables to be migrated must have a primary key or a UNIQUE constraint, and the fields must be unique. Otherwise, duplicate data may exist in the destination database. If you migrate data at the table level and need to edit the tables, such as mapping column names, a single data migration task supports a maximum of 1,000 tables. If you exceed this limit, an error is reported after you submit the task. In this case, split the tables into multiple migration tasks or configure a task to migrate the entire database. If you perform incremental migration, note the following for binary logs: Binary logging must be enabled. The binlog_format parameter must be set to row, and the binlog_row_image parameter must be set to full. Otherwise, the precheck fails, and the data migration task cannot start.
Important If the source self-managed MySQL database is in a dual-primary cluster where each instance is a primary and a secondary of the other, you must enable the log_slave_updates parameter. This ensures that DTS can obtain all binary logs. The binary logs of an RDS for MySQL instance must be retained for at least 3 days. We recommend that you retain them for 7 days. The binary logs of a self-managed MySQL database must be retained for at least 7 days. Otherwise, DTS may fail because it cannot obtain the binary logs. In extreme cases, this can cause data inconsistency or data loss. Issues caused by a binary log retention period shorter than the required period are not covered by the DTS Service-Level Agreement (SLA).
Source database operation limits: During schema migration and full data migration, do not perform DDL operations to change the schema of databases or tables. Otherwise, the data migration task fails.
Note During full data migration, DTS queries the source database. This creates a metadata lock, which may block DDL operations on the source database. If you perform only full data migration, do not write new data to the source instance. Otherwise, data inconsistency occurs between the source and destination. To ensure real-time data consistency, select schema migration, full data migration, and incremental data migration.
Data changes from operations that are not recorded in binary logs during the migration are not migrated to the destination database. Examples of such operations include data recovery using physical backup and cascade operations.
Note If this occurs, you can perform a full data migration again when your business permits. If the source database is MySQL 8.0.23 or later and the data to be migrated contains invisible columns, data loss may occur because the data in these columns cannot be obtained.
Note You can run the ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE; command to make the invisible columns visible. For more information, see Invisible Columns.
|
Other limits | We recommend that you use the same MySQL version for the source and destination databases to ensure compatibility. If online DDL operations that use temporary tables, such as merging multiple tables, are performed on the source database, data loss may occur in the destination database or the migration instance may fail. Migration of parsers defined with comment syntax is not supported. If a primary key or unique key conflict occurs during the migration: If the source and destination databases have the same schema, and a data record has the same primary key as an existing data record in the destination database, the following scenarios may occur: During full data migration, DTS does not migrate the data record to the destination database. The existing data record in the destination database is retained. During incremental data migration, DTS migrates the data record to the destination database. The existing data record in the destination database is overwritten.
If the source and destination databases have different schemas, only specific columns are migrated or the data migration task fails. Proceed with caution.
If the destination database is MySQL 8.0.23 or later and the target columns for receiving data contain invisible columns, the DTS instance may fail or data loss may occur. This is because the target columns for writing data cannot be found.
Note You can run the ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE; command to make the invisible columns visible. For more information, see Invisible Columns. If you do not use the schema migration feature of DTS, you must ensure field compatibility yourself. Otherwise, the instance may fail or data loss may occur. For example, if a source table field is of the text type and the destination field is varchar(255), data truncation may occur if the source table contains large objects. If the data to be migrated contains content that requires four-byte storage, such as uncommon characters or emojis, the destination database and tables that receive the data must use the utf8mb4 charset.
Note If you use DTS for schema migration, you must set the instance-level parameter character_set_server of the destination database to utf8mb4. Before data migration, evaluate the performance of the source and destination databases. We recommend that you perform data migration during off-peak hours. During full data migration, DTS consumes some read and write resources of the source and destination databases, which may increase the database load. Full data migration involves concurrent INSERT operations, which cause table fragmentation in the destination database. After the full migration is complete, the table storage space in the destination database is larger than that in the source instance. Confirm whether the migration precision of DTS for columns of the FLOAT or DOUBLE data type meets your business requirements. DTS reads the values of these columns using ROUND(COLUMN,PRECISION). If the precision is not explicitly defined, DTS migrates FLOAT values with a precision of 38 digits and DOUBLE values with a precision of 308 digits. DTS attempts to resume a data migration task that failed within the last seven days. Therefore, before you switch your services to the destination instance, you must end or release the task, or use the revoke command to revoke the write permissions of the DTS account on the destination instance. Otherwise, data in the source instance overwrites the data in the destination instance after the task is automatically resumed. If a DDL statement fails to be written to the destination database, the DTS task continues to run. You need to check the task logs for the failed DDL statement. For more information about how to view task logs, see Query task logs. If you write fields whose column names differ only in case to the same table in the destination MySQL database, the migration result may not be as expected. This is because column names in MySQL databases are case-insensitive. After the data migration is complete (the Status of the instance is Completed), we recommend that you run the analyze table <table_name> command to confirm that all data has been written to the destination table. For example, if an HA switchover is triggered in the destination MySQL database, data may be written only to the memory, causing data loss. If the always-encrypted (EncDB) feature is enabled for the RDS for MySQL instance, full data migration is not supported.
Note RDS for MySQL instances with Transparent Data Encryption (TDE) enabled support schema migration, full data migration, and incremental data migration. To migrate database accounts from the source database, you must meet the prerequisites and understand the related notes. For more information, see Migrate database accounts. If an instance fails, DTS helpdesk will try to recover the instance within 8 hours. During the recovery process, operations such as restarting the instance or adjusting its parameters may be performed.
Note When parameters are adjusted, only the parameters of the DTS instance are modified. The parameters in the database are not modified. The parameters that may be modified include but are not limited to those described in Modify instance parameters.
|
Special cases | When the source database is a self-managed MySQL database: A primary/secondary switchover in the source database during migration causes the migration task to fail. The latency of DTS is calculated by comparing the timestamp of the last migrated data record with the current timestamp. If no DML operations are performed on the source database for a long time, the latency information may be inaccurate. If the displayed latency is too high, you can perform a DML operation on the source database to update the latency information.
Note If you choose to migrate the entire database, you can also create a heartbeat table. The heartbeat table is updated or written to every second. DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command on the source database to advance the binary log offset. If the source database is an Amazon Aurora MySQL instance or another clustered MySQL instance, ensure that the domain name or IP address configured for the task and its resolved result always point to the read/write (RW) node. Otherwise, the migration task may not run as expected.
When the source database is an RDS for MySQL instance: To migrate incremental data, RDS for MySQL instances that do not record transaction logs, such as read-only instances of RDS for MySQL 5.6, cannot be used as the source database. DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command on the source database to advance the binary log offset.
When the destination database is an RDS for MySQL instance: DTS automatically creates a database in the RDS for MySQL instance. If the name of the database to be migrated does not comply with the naming conventions of RDS for MySQL, you must create the database in the RDS for MySQL instance before you configure the migration task. For more information, see Manage databases.
|