All Products
Search

This Product

    Data Transmission Service:Considerations and limits when synchronizing from a MySQL source database

    Document Center

    Data Transmission Service:Considerations and limits when synchronizing from a MySQL source database

    Last Updated:Feb 07, 2026

    If your source database is MySQL—such as a self-managed MySQL database or ApsaraDB RDS for MySQL—review these considerations and limits before configuring a synchronization task to help ensure smooth operation.

    Overview of MySQL-based synchronization solutions

    Review the considerations and limits for your synchronization task based on the solution you use:

    Note

    DTS disables foreign key constraints by default when synchronizing to a destination database. Therefore, cascading operations and deletions in the source database are not synchronized to the following destination databases:

    • MySQL (ApsaraDB RDS for MySQL, self-managed MySQL)

    • PolarDB for MySQL

    • AnalyticDB for MySQL

    • AnalyticDB for PostgreSQL

    • Elasticsearch

    • Tablestore

    • PolarDB-X 2.0

    • Oracle

    • PostgreSQL

    Synchronize between MySQL databases

    If your destination database is MySQL—such as ApsaraDB RDS for MySQL or a self-managed MySQL database—follow these considerations and limits:

    One-way synchronization between MySQL databases

    Type

    Description

    Source database limits

    • The tables to synchronize must have a primary key or a unique constraint, and the fields must be unique. Otherwise, duplicate data may appear in the destination database.

    • If you synchronize at the table level and need to edit mappings (such as column name mapping), each synchronization task supports up to 1,000 tables. If you exceed this limit, the task fails with an error. To fix this, split the tables across multiple tasks or configure a full-database synchronization task.

    • Binary logs:

      • ApsaraDB RDS for MySQL enables binary logging by default. Ensure that the binlog_row_image parameter is set to full. Otherwise, the precheck fails and the synchronization task cannot start. For instructions, see Configure instance parameters.

        Important

        • If your source instance is a self-managed MySQL database, enable binary logging and set binlog_format to row and binlog_row_image to full.

        • If your self-managed MySQL database is a dual-primary cluster (where both nodes act as primary and secondary), enable the log_slave_updates parameter so DTS can capture all binary log events. For instructions, see Create an account and configure binary logging for a self-managed MySQL database.

      • The local binary logs for an ApsaraDB RDS for MySQL instance must be retained for at least three days (seven days is recommended). For a self-managed MySQL database, retain local binary logs for at least seven days. Otherwise, DTS may fail to retrieve binary logs, causing the task to fail. In extreme cases, this may cause data inconsistency or data loss. Issues caused by binary log retention periods shorter than DTS requires are not covered under the DTS SLA.

        Note

        To configure the retention period for local binary logs on an ApsaraDB RDS for MySQL instance, see Automatically delete local logs.

    • Do not run DDL operations that change database or table schemas during schema synchronization or full synchronization. Otherwise, the synchronization task fails.

      Note

      During full synchronization, DTS queries the source database. This creates metadata locks that may block DDL operations on the source database.

    • Data generated by changes that do not write to binary logs—such as data restored from physical backups or created by cascade operations—is not synchronized to the destination database.

      Note

      If this occurs, remove the affected database or table from the synchronization objects. Then add it back. You can do this only if your business allows it. For more information, see Modify synchronization objects.

    • If your source database is MySQL 8.0.23 or later and contains invisible hidden columns, DTS may not read those columns. This may cause data loss.

      Note

      Run the ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE; command to make the hidden column visible. For more information, see Invisible Columns.

    Other limits

    • Use the same version for the source and destination databases, or upgrade from a lower version to a higher version to ensure compatibility. Synchronizing from a higher version to a lower version may cause compatibility issues.

    • If your source database uses online DDL operations in temporary table mode—including but not limited to multi-table merge scenarios—or adds function-based indexes to unique key columns, data loss or task failure may occur in the destination database.

    • Resolvers defined by comment syntax do not support synchronous use.

    • If a primary key or unique key conflict occurs while the task is running:

      • If the table schemas are consistent and a record in the destination database has the same primary key or unique key value as a record in the source database:

        • During full data synchronization, DTS retains the destination record and skips the source record.

        • During incremental synchronization, DTS overwrites the destination record with the source record.

      • If the table schemas are inconsistent, data initialization may fail. This can result in only partial data synchronization or a complete synchronization failure. Use with caution.

    • If your destination database is MySQL 8.0.23 or later and contains invisible hidden columns in the receiving columns, DTS may fail to locate the target column. This may cause task failure or data loss.

      Note

      Run the ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE; command to make the hidden column visible. For more information, see Invisible Columns.

    • If you do not use DTS to synchronize table schemas, ensure field compatibility yourself. Otherwise, the task may fail or data may be lost. For example, if the source table uses the text data type and the destination column uses varchar(255), large fields in the source table may be truncated.

    • If your data includes four-byte characters—such as rare Chinese characters or emojis—the destination database and table must use the utf8mb4 charset.

      Note

      If you use DTS to synchronize table schemas, set the character_set_server parameter at the instance level in the destination database to utf8mb4.

    • Assess the performance of both the source and destination databases before starting synchronization. Run synchronization during off-peak hours. Otherwise, full initialization consumes read and write resources on both databases and may increase database load.

    • Full initialization runs INSERT operations concurrently. This fragments destination tables. After full initialization, the tablespace of the destination instance is larger than that of the source instance.

    • If you synchronize one or more tables—not a full database—do not use tools like pt-online-schema-change to perform online DDL operations on the source tables. Otherwise, synchronization fails.

      You can use Data Management (DMS) to perform online DDL operations. For more information, see Online DDL without locking tables.

    • Do not write data to the destination database except through DTS while synchronization is running. Otherwise, data inconsistency may occur between the source and destination databases. For example, if you use DMS to perform online DDL operations while other data is written to the destination database, data loss may occur.

    • If a DDL operation fails in the destination database, the DTS task continues to run. Check the failed DDL in the task logs. For instructions, see View task logs.

    • If you write columns with names that differ only in case to the same table in the destination MySQL database, unexpected results may occur because MySQL column names are case-insensitive.

    • After data synchronization completes (the instance's Status is Completed), you should use the ANALYZE TABLE <table_name> command to confirm that all data is written to the target table. For example, after the HA failover mechanism is triggered in the target MySQL database, data might be written only to memory, which can cause data loss.

    • If your ApsaraDB RDS for MySQL instance has Always-Encrypted enabled, full data synchronization is not supported.

      Note

      ApsaraDB RDS for MySQL instances with Transparent Data Encryption (TDE) enabled support schema synchronization, full data synchronization, and incremental data synchronization.

    • To synchronize accounts from the source database, meet the prerequisites and review related considerations. For more information, see Migrate database accounts.

    • If a task fails, DTS support staff will attempt to restore it within eight hours. During restoration, they may restart the task or adjust its parameters.

      Note

      Only DTS task parameters are modified—not database parameters. Parameters that may be adjusted include those listed in Modify instance parameters.

    Other considerations

    • For a self-managed MySQL source database:

      • If a primary/secondary switchover occurs in the source database during synchronization, the task fails.

      • DTS calculates latency by comparing the timestamp of the last synchronized record with the current time. If no DML operations run for a long time in the source database, latency reporting may become inaccurate. If latency appears too high, run a DML operation in the source database to update the latency.

        Note

        If you select a full database for synchronization, create a heartbeat table. Update or write to this table every second.

      • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

      • If your source database is Amazon Aurora MySQL or another clustered MySQL instance, ensure the domain name or IP address used in the task configuration—and its DNS resolution—always points to a read/write (RW) node. Otherwise, synchronization may fail.

    • For an ApsaraDB RDS for MySQL source database:

      • Read-only instances—such as ApsaraDB RDS for MySQL 5.6 read-only instances—that do not record transaction logs cannot serve as source databases.

      • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

    Two-way synchronization between MySQL databases

    Type

    Description

    Source and destination database limits

    • If the tables to synchronize lack primary keys and unique constraints, enable Exactly-Once write in the task configuration. Otherwise, duplicate data may appear in the destination database. For more information, see Synchronize tables without primary keys or unique constraints.

    • If you synchronize at the table level and need to edit mappings (such as column name mapping), each synchronization task supports up to 1,000 tables. If you exceed this limit, the task fails with an error. To fix this, split the tables across multiple tasks or configure a full-database synchronization task.

    • Binary logs:

      • ApsaraDB RDS for MySQL enables binary logging by default. Ensure that the binlog_row_image parameter is set to full. Otherwise, the precheck fails and the synchronization task cannot start. For instructions, see Configure instance parameters.

        Important

        • If your source instance is a self-managed MySQL database, enable binary logging and set binlog_format to row and binlog_row_image to full.

        • If your self-managed MySQL database is a dual-primary cluster (where both nodes act as primary and secondary), enable the log_slave_updates parameter so DTS can capture all binary log events. For instructions, see Create an account and configure binary logging for a self-managed MySQL database.

      • The local binary logs for an ApsaraDB RDS for MySQL instance must be retained for at least three days (seven days is recommended). For a self-managed MySQL database, retain local binary logs for at least seven days. Otherwise, DTS may fail to retrieve binary logs, causing the task to fail. In extreme cases, this may cause data inconsistency or data loss. Issues caused by binary log retention periods shorter than DTS requires are not covered under the DTS SLA.

        Note

        To configure the retention period for local binary logs on an ApsaraDB RDS for MySQL instance, see Automatically delete local logs.

    • Do not run DDL operations that change database or table schemas during schema synchronization or full synchronization. Otherwise, the synchronization task fails.

      Note

      During full synchronization, DTS queries the source database. This creates metadata locks that may block DDL operations on the source database.

    • Data generated by changes that do not write to binary logs—such as data restored from physical backups or created by cascade operations—is not synchronized to the destination database.

      Note

      If this occurs, remove the affected database or table from the synchronization objects. Then add it back. You can do this only if your business allows it. For more information, see Modify synchronization objects.

    • Ensure neither the source nor destination database contains invisible hidden columns in the columns to synchronize. Otherwise, the DTS task may fail or data may be lost.

      Note

      Run the ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE; command to make the hidden column visible. For more information, see Invisible Columns.

    Other limits

    • Use the same version for the source and destination databases, or upgrade from a lower version to a higher version to ensure compatibility. Synchronizing from a higher version to a lower version may cause compatibility issues.

    • If your source database uses online DDL operations in temporary table mode—including but not limited to multi-table merge scenarios—or adds function-based indexes to unique key columns, data loss or task failure may occur in the destination database.

    • Resolvers defined by comment syntax do not support synchronous use.

    • If a primary key or unique key conflict occurs while the task is running:

      • If the table schemas are consistent and a record in the destination database has the same primary key or unique key value as a record in the source database:

        • During full data synchronization, DTS retains the destination record and skips the source record.

        • During incremental synchronization, DTS overwrites the destination record with the source record.

      • If the table schemas are inconsistent, data initialization may fail. This can result in only partial data synchronization or a complete synchronization failure. Use with caution.

    • If you do not use DTS to synchronize table schemas, ensure field compatibility yourself. Otherwise, the task may fail or data may be lost. For example, if the source table uses the text data type and the destination column uses varchar(255), large fields in the source table may be truncated.

    • While a two-way synchronization task runs, DTS creates a database named dts in the destination databases of both forward and reverse tasks. This prevents circular synchronization. Do not modify this database while the task runs. Ensure the database account used by the task has read and write permissions on this database.

      Note

      The dts database may interfere with normal primary/secondary replication in MySQL. Check whether the dts database and its tables exist in the secondary database. If they do not, manually create them in the secondary database using the same schema as in the primary database.

    • If your data includes four-byte characters—such as rare Chinese characters or emojis—the destination database and table must use the utf8mb4 charset.

      Note

      If you use DTS to synchronize table schemas, set the character_set_server parameter at the instance level in the destination database to utf8mb4.

    • Assess the performance of both the source and destination databases before starting synchronization. Run synchronization during off-peak hours. Otherwise, full initialization consumes read and write resources on both databases and may increase database load.

    • Full initialization runs INSERT operations concurrently. This fragments destination tables. After full initialization, the tablespace of the destination instance is larger than that of the source instance.

    • If you synchronize one or more tables—not a full database—do not use tools like pt-online-schema-change to perform online DDL operations on the source tables. Otherwise, synchronization fails.

      You can use Data Management (DMS) to perform online DDL operations. For more information, see Online DDL without locking tables.

    • DTS disables foreign key constraints by default when synchronizing to a destination database. Therefore, cascading operations and deletions in the source database are not synchronized to the destination database.

    • A two-way synchronization task includes forward and reverse synchronization tasks. When you configure or reset the task, if the destination object of one task matches the synchronization object of the other task:

      • Allow only one task to synchronize full and incremental data. The other task supports only incremental synchronization.

      • Data from the source of the current task synchronizes only to the destination of the current task. It does not serve as source data for the other task.

    • If a DDL operation fails in the destination database, the DTS task continues to run. Check the failed DDL in the task logs. For instructions, see View task logs.

    • If you write columns with names that differ only in case to the same table in the destination MySQL database, unexpected results may occur because MySQL column names are case-insensitive.

    • After data synchronization completes (the instance's Status is Completed), you should use the ANALYZE TABLE <table_name> command to confirm that all data is written to the target table. For example, after the HA failover mechanism is triggered in the target MySQL database, data might be written only to memory, which can cause data loss.

    • If your ApsaraDB RDS for MySQL instance has Always-Encrypted enabled, full data synchronization is not supported.

      Note

      ApsaraDB RDS for MySQL instances with Transparent Data Encryption (TDE) enabled support schema synchronization, full data synchronization, and incremental data synchronization.

    • To synchronize accounts from the source database, meet the prerequisites and review related considerations. For more information, see Migrate database accounts.

    • If a task fails, DTS support staff will attempt to restore it within eight hours. During restoration, they may restart the task or adjust its parameters.

      Note

      Only DTS task parameters are modified—not database parameters. Parameters that may be adjusted include those listed in Modify instance parameters.

    Special cases

    • For a self-managed MySQL source database:

      • If a primary/secondary switchover occurs in the source database during synchronization, the task fails.

      • DTS calculates latency by comparing the timestamp of the last synchronized record with the current time. If no DML operations run for a long time in the source database, latency reporting may become inaccurate. If latency appears too high, run a DML operation in the source database to update the latency.

        Note

        If you select a full database for synchronization, create a heartbeat table. Update or write to this table every second.

      • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

      • If the self-managed MySQL serves as the source database for both a bidirectional sync task and other tasks, you must add the line log_slave_updates=1 to the /etc/my.cnf file of the source database and then restart the database. Otherwise, data changes from the reverse task will not be recorded in the binary log of the self-managed MySQL, causing data inconsistency in other tasks.

      • If your source database is Amazon Aurora MySQL or another clustered MySQL instance, ensure the domain name or IP address used in the task configuration—and its DNS resolution—always points to a read/write (RW) node. Otherwise, synchronization may fail.

    • For an ApsaraDB RDS for MySQL source database:

      • Read-only instances—such as ApsaraDB RDS for MySQL 5.6 read-only instances—that do not record transaction logs cannot serve as source databases.

      • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

    Synchronize from MySQL to PolarDB for MySQL

    One-way synchronization from MySQL to PolarDB for MySQL

    Type

    Description

    Source database limits

    • The tables to synchronize must have a primary key or a unique constraint, and the fields must be unique. Otherwise, duplicate data may appear in the destination database.

    • If you synchronize at the table level and need to edit mappings (such as column name mapping), each synchronization task supports up to 1,000 tables. If you exceed this limit, the task fails with an error. To fix this, split the tables across multiple tasks or configure a full-database synchronization task.

    • Binary logs:

      • ApsaraDB RDS for MySQL enables binary logging by default. Ensure that the binlog_row_image parameter is set to full. Otherwise, the precheck fails and the synchronization task cannot start. For instructions, see Configure instance parameters.

        Important

        • If your source instance is a self-managed MySQL database, enable binary logging and set binlog_format to row and binlog_row_image to full.

        • If your self-managed MySQL database is a dual-primary cluster (where both nodes act as primary and secondary), enable the log_slave_updates parameter so DTS can capture all binary log events. For instructions, see Create an account and configure binary logging for a self-managed MySQL database.

      • The local binary logs for an ApsaraDB RDS for MySQL instance must be retained for at least three days (seven days is recommended). For a self-managed MySQL database, retain local binary logs for at least seven days. Otherwise, DTS may fail to retrieve binary logs, causing the task to fail. In extreme cases, this may cause data inconsistency or data loss. Issues caused by binary log retention periods shorter than DTS requires are not covered under the DTS SLA.

        Note

        To configure the retention period for local binary logs on an ApsaraDB RDS for MySQL instance, see Automatically delete local logs.

    • Do not run DDL operations that change database or table schemas during schema synchronization or full synchronization. Otherwise, the synchronization task fails.

      Note

      During full synchronization, DTS queries the source database. This creates metadata locks that may block DDL operations on the source database.

    • Data generated by changes that do not write to binary logs—such as data restored from physical backups or created by cascade operations—is not synchronized to the destination database.

      Note

      If this occurs, remove the affected database or table from the synchronization objects. Then add it back. You can do this only if your business allows it. For more information, see Modify synchronization objects.

    • If your source database is MySQL 8.0.23 or later and contains invisible hidden columns, DTS may not read those columns. This may cause data loss.

      Note

      Run the ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE; command to make the hidden column visible. For more information, see Invisible Columns.

    Other limits

    • Use the same version for the source and destination databases, or upgrade from a lower version to a higher version to ensure compatibility. Synchronizing from a higher version to a lower version may cause compatibility issues.

    • If your source database uses online DDL operations in temporary table mode—including but not limited to multi-table merge scenarios—or adds function-based indexes to unique key columns, data loss or task failure may occur in the destination database.

    • Resolvers defined by comment syntax do not support synchronous use.

    • If a primary key or unique key conflict occurs while the task is running:

      • If the table schemas are consistent and a record in the destination database has the same primary key or unique key value as a record in the source database:

        • During full data synchronization, DTS retains the destination record and skips the source record.

        • During incremental synchronization, DTS overwrites the destination record with the source record.

      • If the table schemas are inconsistent, data initialization may fail. This can result in only partial data synchronization or a complete synchronization failure. Use with caution.

    • Assess the performance of both the source and destination databases before starting synchronization. Run synchronization during off-peak hours. Otherwise, full initialization consumes read and write resources on both databases and may increase database load.

    • Full initialization runs INSERT operations concurrently. This fragments destination tables. After full initialization, the tablespace of the destination instance is larger than that of the source instance.

    • If your data includes four-byte characters—such as rare Chinese characters or emojis—the destination database and table must use the utf8mb4 charset.

      Note

      If you use DTS to synchronize table schemas, set the character_set_server parameter at the instance level in the destination database to utf8mb4.

    • If you synchronize one or more tables—not a full database—do not use tools like pt-online-schema-change to perform online DDL operations on the source tables. Otherwise, synchronization fails.

      You can use Data Management (DMS) to perform online DDL operations. For more information, see Online DDL without locking tables.

    • Do not write data to the destination database except through DTS while synchronization is running. Otherwise, data inconsistency may occur between the source and destination databases. For example, if you use DMS to perform online DDL operations while other data is written to the destination database, data loss may occur.

    • You cannot adjust the full migration rate.

    • You cannot convert datetime data to varchar.

    • If a DDL operation fails in the destination database, the DTS task continues to run. Check the failed DDL in the task logs. For instructions, see View task logs.

    • If your ApsaraDB RDS for MySQL instance has Always-Encrypted enabled, full data synchronization is not supported.

      Note

      ApsaraDB RDS for MySQL instances with Transparent Data Encryption (TDE) enabled support schema synchronization, full data synchronization, and incremental data synchronization.

    • To synchronize accounts from the source database, meet the prerequisites and review related considerations. For more information, see Migrate database accounts.

    • If a task fails, DTS support staff will attempt to restore it within eight hours. During restoration, they may restart the task or adjust its parameters.

      Note

      Only DTS task parameters are modified—not database parameters. Parameters that may be adjusted include those listed in Modify instance parameters.

    Special cases

    • For a self-managed MySQL source database:

      • If a primary/secondary switchover occurs in the source database during synchronization, the task fails.

      • DTS calculates latency by comparing the timestamp of the last synchronized record with the current time. If no DML operations run for a long time in the source database, latency reporting may become inaccurate. If latency appears too high, run a DML operation in the source database to update the latency.

        Note

        If you select a full database for synchronization, create a heartbeat table. Update or write to this table every second.

      • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

      • If your source database is Amazon Aurora MySQL or another clustered MySQL instance, ensure the domain name or IP address used in the task configuration—and its DNS resolution—always points to a read/write (RW) node. Otherwise, synchronization may fail.

    • For an ApsaraDB RDS for MySQL source database:

      • Read-only instances—such as ApsaraDB RDS for MySQL 5.6 read-only instances—that do not record transaction logs cannot serve as source databases.

      • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

    Two-way synchronization from MySQL to PolarDB for MySQL

    Type

    Description

    Source and destination database limits

    • The tables to synchronize must have a primary key or a unique constraint, and the fields must be unique. Otherwise, duplicate data may appear in the destination database.

    • If you synchronize at the table level and need to edit mappings (such as column name mapping), each synchronization task supports up to 1,000 tables. If you exceed this limit, the task fails with an error. To fix this, split the tables across multiple tasks or configure a full-database synchronization task.

    • Binary logs:

      MySQL

      • ApsaraDB RDS for MySQL enables binary logging by default. Ensure that the binlog_row_image parameter is set to full. Otherwise, the precheck fails and the synchronization task cannot start. For instructions, see Configure instance parameters.

        Important

        • If your source instance is a self-managed MySQL database, enable binary logging and set binlog_format to row and binlog_row_image to full.

        • If your self-managed MySQL database is a dual-primary cluster (where both nodes act as primary and secondary), enable the log_slave_updates parameter so DTS can capture all binary log events. For instructions, see Create an account and configure binary logging for a self-managed MySQL database.

      • The local binary logs for an ApsaraDB RDS for MySQL instance must be retained for at least three days (seven days is recommended). For a self-managed MySQL database, retain local binary logs for at least seven days. Otherwise, DTS may fail to retrieve binary logs, causing the task to fail. In extreme cases, this may cause data inconsistency or data loss. Issues caused by binary log retention periods shorter than DTS requires are not covered under the DTS SLA.

        Note

        To configure the retention period for local binary logs on an ApsaraDB RDS for MySQL instance, see Automatically delete local logs.

      PolarDB for MySQL

      • You must enable binary logging and set the loose_polar_log_bin parameter to ON. Otherwise, an error is reported during the precheck and the DTS instance fails to start. For more information about how to enable binary logging and modify parameters, see Enable binary logging and Set cluster and node parameters.

        Note

        Enabling binary logging for a PolarDB for MySQL cluster consumes storage space and incurs fees.

      • The binary logs of the PolarDB for MySQL cluster must be retained for at least 3 days. We recommend that you retain them for 7 days. Otherwise, the DTS task may fail because DTS cannot obtain the binary logs. In extreme cases, this may cause data inconsistency or data loss. Issues caused by a binary log retention period shorter than the required period are not covered by the DTS Service-Level Agreement (SLA).

        Note

        For more information about how to set the retention period for binary logs of a PolarDB for MySQL cluster, see Modify the retention period.

    • Do not run DDL operations that change database or table schemas during schema synchronization or full synchronization. Otherwise, the synchronization task fails.

      Note

      During full synchronization, DTS queries the source database. This creates metadata locks that may block DDL operations on the source database.

    • Data generated by changes that do not write to binary logs—such as data restored from physical backups or created by cascade operations—is not synchronized to the destination database.

      Note

      If this occurs, remove the affected database or table from the synchronization objects. Then add it back. You can do this only if your business allows it. For more information, see Modify synchronization objects.

    • If your source database is MySQL 8.0.23 or later and contains invisible hidden columns, DTS may not read those columns. This may cause data loss.

      Note

      Run the ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE; command to make the hidden column visible. For more information, see Invisible Columns.

    Other limits

    • Use the same version for the source and destination databases, or upgrade from a lower version to a higher version to ensure compatibility. Synchronizing from a higher version to a lower version may cause compatibility issues.

    • If your source database uses online DDL operations in temporary table mode—including but not limited to multi-table merge scenarios—or adds function-based indexes to unique key columns, data loss or task failure may occur in the destination database.

    • Resolvers defined by comment syntax do not support synchronous use.

    • If a primary key or unique key conflict occurs while the task is running:

      • If the table schemas are consistent and a record in the destination database has the same primary key or unique key value as a record in the source database:

        • During full data synchronization, DTS retains the destination record and skips the source record.

        • During incremental synchronization, DTS overwrites the destination record with the source record.

      • If the table schemas are inconsistent, data initialization may fail. This can result in only partial data synchronization or a complete synchronization failure. Use with caution.

    • While a two-way synchronization task runs, DTS creates a database named dts in the destination databases of both forward and reverse tasks. This prevents circular synchronization. Do not modify this database while the task runs. Ensure the database account used by the task has read and write permissions on this database.

    • If your data includes four-byte characters—such as rare Chinese characters or emojis—the destination database and table must use the utf8mb4 charset.

      Note

      If you use DTS to synchronize table schemas, set the character_set_server parameter at the instance level in the destination database to utf8mb4.

    • Assess the performance of both the source and destination databases before starting synchronization. Run synchronization during off-peak hours. Otherwise, full initialization consumes read and write resources on both databases and may increase database load.

    • Full initialization runs INSERT operations concurrently. This fragments destination tables. After full initialization, the tablespace of the destination instance is larger than that of the source instance.

    • If you synchronize one or more tables—not a full database—do not use tools like pt-online-schema-change to perform online DDL operations on the source tables. Otherwise, synchronization fails.

      You can use Data Management (DMS) to perform online DDL operations. For more information, see Online DDL without locking tables.

    • Do not write data to the destination database except through DTS while synchronization is running. Otherwise, data inconsistency may occur between the source and destination databases. For example, if you use DMS to perform online DDL operations while other data is written to the destination database, data loss may occur.

    • DTS disables foreign key constraints by default when synchronizing to a destination database. Therefore, cascading operations and deletions in the source database are not synchronized to the destination database.

    • A two-way synchronization task includes forward and reverse synchronization tasks. When you configure or reset the task, if the destination object of one task matches the synchronization object of the other task:

      • Allow only one task to synchronize full and incremental data. The other task supports only incremental synchronization.

      • Data from the source of the current task synchronizes only to the destination of the current task. It does not serve as source data for the other task.

    • You cannot convert datetime data to varchar.

    • If a DDL operation fails in the destination database, the DTS task continues to run. Check the failed DDL in the task logs. For instructions, see View task logs.

    • If your ApsaraDB RDS for MySQL instance has Always-Encrypted enabled, full data synchronization is not supported.

      Note

      ApsaraDB RDS for MySQL instances with Transparent Data Encryption (TDE) enabled support schema synchronization, full data synchronization, and incremental data synchronization.

    • To synchronize accounts from the source database, meet the prerequisites and review related considerations. For more information, see Migrate database accounts.

    • If a task fails, DTS support staff will attempt to restore it within eight hours. During restoration, they may restart the task or adjust its parameters.

      Note

      Only DTS task parameters are modified—not database parameters. Parameters that may be adjusted include those listed in Modify instance parameters.

    Special cases

    • For a self-managed MySQL source database:

      • If a primary/secondary switchover occurs in the source database during synchronization, the task fails.

      • DTS calculates latency by comparing the timestamp of the last synchronized record with the current time. If no DML operations run for a long time in the source database, latency reporting may become inaccurate. If latency appears too high, run a DML operation in the source database to update the latency.

        Note

        If you select a full database for synchronization, create a heartbeat table. Update or write to this table every second.

      • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

      • If your source database is Amazon Aurora MySQL or another clustered MySQL instance, ensure the domain name or IP address used in the task configuration—and its DNS resolution—always points to a read/write (RW) node. Otherwise, synchronization may fail.

    • For an ApsaraDB RDS for MySQL source database:

      • Read-only instances—such as ApsaraDB RDS for MySQL 5.6 read-only instances—that do not record transaction logs cannot serve as source databases.

      • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

    Synchronize from MySQL to AnalyticDB for MySQL

    Type

    Description

    Source database limits

    • The tables to synchronize must have a primary key or a unique constraint, and the fields must be unique. Otherwise, duplicate data may appear in the destination database.

    • If you synchronize at the table level and need to edit mappings (such as column name mapping), each synchronization task supports up to 1,000 tables. If you exceed this limit, the task fails with an error. To fix this, split the tables across multiple tasks or configure a full-database synchronization task.

    • Binary logs:

      • ApsaraDB RDS for MySQL enables binary logging by default. Ensure that the binlog_row_image parameter is set to full. Otherwise, the precheck fails and the synchronization task cannot start. For instructions, see Configure instance parameters.

        Important

        • If your source instance is a self-managed MySQL database, enable binary logging and set binlog_format to row and binlog_row_image to full.

        • If your self-managed MySQL database is a dual-primary cluster (where both nodes act as primary and secondary), enable the log_slave_updates parameter so DTS can capture all binary log events. For instructions, see Create an account and configure binary logging for a self-managed MySQL database.

      • The local binary logs for an ApsaraDB RDS for MySQL instance must be retained for at least three days (seven days is recommended). For a self-managed MySQL database, retain local binary logs for at least seven days. Otherwise, DTS may fail to retrieve binary logs, causing the task to fail. In extreme cases, this may cause data inconsistency or data loss. Issues caused by binary log retention periods shorter than DTS requires are not covered under the DTS SLA.

        Note

        To configure the retention period for local binary logs on an ApsaraDB RDS for MySQL instance, see Automatically delete local logs.

    • Do not run DDL operations that modify primary keys or add comments—such as ALTER TABLE table_name COMMENT='Table comment';—while synchronization is running. Otherwise, the DDL operation fails.

    • Do not run DDL operations that change database or table schemas during schema synchronization or full synchronization. Otherwise, the synchronization task fails.

      Note

      During full synchronization, DTS queries the source database. This creates metadata locks that may block DDL operations on the source database.

    • Data generated by changes that do not write to binary logs—such as data restored from physical backups or created by cascade operations—is not synchronized to the destination database.

      Note

      If this occurs, remove the affected database or table from the synchronization objects. Then add it back. You can do this only if your business allows it. For more information, see Modify synchronization objects.

    • If your source database is MySQL 8.0.23 or later and contains invisible hidden columns, DTS may not read those columns. This may cause data loss.

      Note

      Run the ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE; command to make the hidden column visible. For more information, see Invisible Columns.

    Other limits

    • DTS does not support synchronizing prefix indexes. If your source database contains prefix indexes, synchronization may fail.

    • If your source database uses online DDL operations in temporary table mode—including but not limited to multi-table merge scenarios—or adds function-based indexes to unique key columns, data loss or task failure may occur in the destination database.

    • DTS does not support synchronizing indexes, partitions, views, stored procedures, functions, triggers, or foreign keys.

    • If a primary key or unique key conflict occurs while the task is running:

      • If the table schemas are consistent and a record in the destination database has the same primary key or unique key value as a record in the source database:

        • During full data synchronization, DTS retains the destination record and skips the source record.

        • During incremental synchronization, DTS overwrites the destination record with the source record.

      • If the table schemas are inconsistent, data initialization may fail. This can result in only partial data synchronization or a complete synchronization failure. Use with caution.

    • The destination database must contain a custom primary key. Or, in the Configurations for Databases, Tables, and Columns step, configure a Primary Key Column. Otherwise, synchronization may fail.

    • Due to usage limits in AnalyticDB for MySQL, if disk usage exceeds 80% in an AnalyticDB for MySQL cluster, DTS tasks may experience delays or errors. Estimate required space based on your synchronization objects and ensure sufficient storage in the destination cluster.

    • If the destination AnalyticDB for MySQL 3.0 cluster is backing up while the DTS task runs, the task fails.

    • Assess the performance of both the source and destination databases before starting synchronization. Run synchronization during off-peak hours. Otherwise, full initialization consumes read and write resources on both databases and may increase database load.

    • Full initialization runs INSERT operations concurrently. This fragments destination tables. After full initialization, the tablespace of the destination instance is larger than that of the source instance.

    • If you synchronize one or more tables—not a full database—do not use tools like pt-online-schema-change to perform online DDL operations on the source tables. Otherwise, synchronization fails.

      You can use Data Management (DMS) to perform online DDL operations. For more information, see Online DDL without locking tables.

    • For table-level synchronization, if no data other than DTS writes to the destination AnalyticDB for MySQL, you can use Data Management (DMS) to perform online DDL operations. For more information, see Online DDL without locking tables.

    • Do not write data to the destination database except through DTS while synchronization is running. Otherwise, data inconsistency may occur between the source and destination databases. For example, if you use DMS to perform online DDL operations while other data is written to the destination database, data loss may occur.

    • If a DDL operation fails in the destination database, the DTS task continues to run. Check the failed DDL in the task logs. For instructions, see View task logs.

    • If your ApsaraDB RDS for MySQL instance has Always-Encrypted enabled, full data synchronization is not supported.

      Note

      ApsaraDB RDS for MySQL instances with Transparent Data Encryption (TDE) enabled support schema synchronization, full data synchronization, and incremental data synchronization.

    • If a task fails, DTS support staff will attempt to restore it within eight hours. During restoration, they may restart the task or adjust its parameters.

      Note

      Only DTS task parameters are modified—not database parameters. Parameters that may be adjusted include those listed in Modify instance parameters.

    Special cases

    • For a self-managed MySQL source database:

      • If a primary/secondary switchover occurs in the source database during synchronization, the task fails.

      • DTS calculates latency by comparing the timestamp of the last synchronized record with the current time. If no DML operations run for a long time in the source database, latency reporting may become inaccurate. If latency appears too high, run a DML operation in the source database to update the latency.

        Note

        If you select a full database for synchronization, create a heartbeat table. Update or write to this table every second.

      • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

      • If your source database is Amazon Aurora MySQL or another clustered MySQL instance, ensure the domain name or IP address used in the task configuration—and its DNS resolution—always points to a read/write (RW) node. Otherwise, synchronization may fail.

    • For an ApsaraDB RDS for MySQL source database:

      • Read-only instances—such as ApsaraDB RDS for MySQL 5.6 read-only instances—that do not record transaction logs cannot serve as source databases.

      • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

    Synchronize from MySQL to AnalyticDB for PostgreSQL

    Type

    Description

    Source database limits

    • The tables to synchronize must have a primary key or a unique constraint, and the fields must be unique. Otherwise, duplicate data may appear in the destination database.

    • If you synchronize at the table level and need to edit mappings (such as column name mapping), each synchronization task supports up to 1,000 tables. If you exceed this limit, the task fails with an error. To fix this, split the tables across multiple tasks or configure a full-database synchronization task.

    • Binary logs:

      • ApsaraDB RDS for MySQL enables binary logging by default. Ensure that the binlog_row_image parameter is set to full. Otherwise, the precheck fails and the synchronization task cannot start. For instructions, see Configure instance parameters.

        Important

        • If your source instance is a self-managed MySQL database, enable binary logging and set binlog_format to row and binlog_row_image to full.

        • If your self-managed MySQL database is a dual-primary cluster (where both nodes act as primary and secondary), enable the log_slave_updates parameter so DTS can capture all binary log events. For instructions, see Create an account and configure binary logging for a self-managed MySQL database.

      • The local binary logs for an ApsaraDB RDS for MySQL instance must be retained for at least three days (seven days is recommended). For a self-managed MySQL database, retain local binary logs for at least seven days. Otherwise, DTS may fail to retrieve binary logs, causing the task to fail. In extreme cases, this may cause data inconsistency or data loss. Issues caused by binary log retention periods shorter than DTS requires are not covered under the DTS SLA.

        Note

        To configure the retention period for local binary logs on an ApsaraDB RDS for MySQL instance, see Automatically delete local logs.

    • Do not run DDL operations that modify primary keys or add comments—such as ALTER TABLE table_name COMMENT='Table comment';—while synchronization is running. Otherwise, the DDL operation fails.

    • If your source database contains date values of 0000-00-00 00:00:00, the task may fail.

      Note

      DTS converts this date to null in the destination database. Temporarily change the source data to 0001-01-01 00:00:00 or set the corresponding destination column to allow null values.

    • Do not run DDL operations that change database or table schemas during schema synchronization or full synchronization. Otherwise, the synchronization task fails.

      Note

      During full synchronization, DTS queries the source database. This creates metadata locks that may block DDL operations on the source database.

    • Data generated by changes that do not write to binary logs—such as data restored from physical backups or created by cascade operations—is not synchronized to the destination database.

      Note

      If this occurs, remove the affected database or table from the synchronization objects. Then add it back. You can do this only if your business allows it. For more information, see Modify synchronization objects.

    • If your source database is MySQL 8.0.23 or later and contains invisible hidden columns, DTS may not read those columns. This may cause data loss.

      Note

      Run the ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE; command to make the hidden column visible. For more information, see Invisible Columns.

    Other limits

    • Synchronization object requirements:

      • Only table-level synchronization is supported.

      • DTS does not support synchronizing VARBIT, GEOMETRY, ARRAY, UUID, TSQUERY, TSVECTOR, TXID_SNAPSHOT, or POINT data types.

      • DTS does not support synchronizing prefix indexes. If your source database contains prefix indexes, synchronization may fail.

      • DTS does not support synchronizing partitions, views, stored procedures, functions, triggers, foreign keys, or indexes.

    • If your source database uses online DDL operations in temporary table mode—including but not limited to multi-table merge scenarios—or adds function-based indexes to unique key columns, data loss or task failure may occur in the destination database.

    • If a primary key or unique key conflict occurs while the task is running:

      • If the table schemas are consistent and a record in the destination database has the same primary key or unique key value as a record in the source database:

        • During full data synchronization, DTS retains the destination record and skips the source record.

        • During incremental synchronization, DTS overwrites the destination record with the source record.

      • If the table schemas are inconsistent, data initialization may fail. This can result in only partial data synchronization or a complete synchronization failure. Use with caution.

    • If the table to synchronize has a primary key, the primary key column in the destination table must match the source table. If the table to synchronize lacks a primary key, the primary key column in the destination table must match the distribution key.

    • The unique key in the destination table—including the primary key column—must include all columns in the distribution key.

    • Assess the performance of both the source and destination databases before starting synchronization. Run synchronization during off-peak hours. Otherwise, full initialization consumes read and write resources on both databases and may increase database load.

    • Full initialization runs INSERT operations concurrently. This fragments destination tables. After full initialization, the tablespace of the destination instance is larger than that of the source instance.

    • If you synchronize one or more tables—not a full database—do not use tools like pt-online-schema-change to perform online DDL operations on the source tables. Otherwise, synchronization fails.

      You can use Data Management (DMS) to perform online DDL operations. For more information, see Online DDL without locking tables.

    • Do not write data to the destination database except through DTS while synchronization is running. Otherwise, data inconsistency may occur between the source and destination databases. For example, if you use DMS to perform online DDL operations while other data is written to the destination database, data loss may occur.

    • The destination table does not support append-optimized (AO) tables.

    • If you use column mapping and do not synchronize full tables—or if the source and destination table schemas do not match—data in columns missing from the destination side is lost.

    • If your ApsaraDB RDS for MySQL instance has Always-Encrypted enabled, full data synchronization is not supported.

      Note

      ApsaraDB RDS for MySQL instances with Transparent Data Encryption (TDE) enabled support schema synchronization, full data synchronization, and incremental data synchronization.

    • If a task fails, DTS support staff will attempt to restore it within eight hours. During restoration, they may restart the task or adjust its parameters.

      Note

      Only DTS task parameters are modified—not database parameters. Parameters that may be adjusted include those listed in Modify instance parameters.

    Special cases

    • For a self-managed MySQL source database:

      • If a primary/secondary switchover occurs in the source database during synchronization, the task fails.

      • DTS calculates latency by comparing the timestamp of the last synchronized record with the current time. If no DML operations run for a long time in the source database, latency reporting may become inaccurate. If latency appears too high, run a DML operation in the source database to update the latency.

        Note

        If you select a full database for synchronization, create a heartbeat table. Update or write to this table every second.

      • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

      • If your source database is Amazon Aurora MySQL or another clustered MySQL instance, ensure the domain name or IP address used in the task configuration—and its DNS resolution—always points to a read/write (RW) node. Otherwise, synchronization may fail.

    • For an ApsaraDB RDS for MySQL source database:

      • Read-only instances—such as ApsaraDB RDS for MySQL 5.6 read-only instances—that do not record transaction logs cannot serve as source databases.

      • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

    Synchronize from MySQL to DataHub

    Type

    Description

    Source database limits

    • The tables to synchronize must have a primary key or a unique constraint, and the fields must be unique. Otherwise, duplicate data may appear in the destination database.

    • If you synchronize at the table level and need to edit mappings (such as column name mapping), each synchronization task supports up to 1,000 tables. If you exceed this limit, the task fails with an error. To fix this, split the tables across multiple tasks or configure a full-database synchronization task.

    • Binary logs:

      • ApsaraDB RDS for MySQL enables binary logging by default. Ensure that the binlog_row_image parameter is set to full. Otherwise, the precheck fails and the synchronization task cannot start. For instructions, see Configure instance parameters.

        Important

        • If your source instance is a self-managed MySQL database, enable binary logging and set binlog_format to row and binlog_row_image to full.

        • If your self-managed MySQL database is a dual-primary cluster (where both nodes act as primary and secondary), enable the log_slave_updates parameter so DTS can capture all binary log events. For instructions, see Create an account and configure binary logging for a self-managed MySQL database.

      • The local binary logs for an ApsaraDB RDS for MySQL instance must be retained for at least three days (seven days is recommended). For a self-managed MySQL database, retain local binary logs for at least seven days. Otherwise, DTS may fail to retrieve binary logs, causing the task to fail. In extreme cases, this may cause data inconsistency or data loss. Issues caused by binary log retention periods shorter than DTS requires are not covered under the DTS SLA.

        Note

        To configure the retention period for local binary logs on an ApsaraDB RDS for MySQL instance, see Automatically delete local logs.

    • Do not run DDL operations that change database or table schemas during schema synchronization or full synchronization. Otherwise, the synchronization task fails.

      Note

      During full synchronization, DTS queries the source database. This creates metadata locks that may block DDL operations on the source database.

    • Data generated by changes that do not write to binary logs—such as data restored from physical backups or created by cascade operations—is not synchronized to the destination database.

      Note

      If this occurs, remove the affected database or table from the synchronization objects. Then add it back. You can do this only if your business allows it. For more information, see Modify synchronization objects.

    • If your source database is MySQL 8.0.23 or later and contains invisible hidden columns, DTS may not read those columns. This may cause data loss.

      Note

      Run the ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE; command to make the hidden column visible. For more information, see Invisible Columns.

    Other limits

    • A single String field in DataHub supports up to 2 MB.

    • DTS does not support synchronizing indexes, partitions, views, stored procedures, functions, triggers, or foreign keys.

    • If your source table contains a record_id column, use field mapping to rename it in the destination instance. Otherwise, the task fails. For instructions, see Database and table column name mapping.

    • Do not write data to the destination database except through DTS while synchronization is running. Otherwise, data inconsistency may occur between the source and destination databases. For example, if you use DMS to perform online DDL operations while other data is written to the destination database, data loss may occur.

    • If your ApsaraDB RDS for MySQL instance has Always-Encrypted enabled, full data synchronization is not supported.

      Note

      ApsaraDB RDS for MySQL instances with Transparent Data Encryption (TDE) enabled support schema synchronization, full data synchronization, and incremental data synchronization.

    • If a task fails, DTS support staff will attempt to restore it within eight hours. During restoration, they may restart the task or adjust its parameters.

      Note

      Only DTS task parameters are modified—not database parameters. Parameters that may be adjusted include those listed in Modify instance parameters.

    Special cases

    • For a self-managed MySQL source database:

      • If a primary/secondary switchover occurs in the source database during synchronization, the task fails.

      • DTS calculates latency by comparing the timestamp of the last synchronized record with the current time. If no DML operations run for a long time in the source database, latency reporting may become inaccurate. If latency appears too high, run a DML operation in the source database to update the latency.

        Note

        If you select a full database for synchronization, create a heartbeat table. Update or write to this table every second.

      • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

      • If your source database is Amazon Aurora MySQL or another clustered MySQL instance, ensure the domain name or IP address used in the task configuration—and its DNS resolution—always points to a read/write (RW) node. Otherwise, synchronization may fail.

    • For an ApsaraDB RDS for MySQL source database:

      • Read-only instances—such as ApsaraDB RDS for MySQL 5.6 read-only instances—that do not record transaction logs cannot serve as source databases.

      • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

    Synchronize from MySQL to Elasticsearch

    Type

    Description

    Source database limits

    • The tables to synchronize must have a primary key or a unique constraint, and the fields must be unique. Otherwise, duplicate data may appear in the destination database.

    • If you synchronize at the table level and need to edit mappings (such as column name mapping), each synchronization task supports up to 1,000 tables. If you exceed this limit, the task fails with an error. To fix this, split the tables across multiple tasks or configure a full-database synchronization task.

    • Binary logs:

      • ApsaraDB RDS for MySQL enables binary logging by default. Ensure that the binlog_row_image parameter is set to full. Otherwise, the precheck fails and the synchronization task cannot start. For instructions, see Configure instance parameters.

        Important

        • If your source instance is a self-managed MySQL database, enable binary logging and set binlog_format to row and binlog_row_image to full.

        • If your self-managed MySQL database is a dual-primary cluster (where both nodes act as primary and secondary), enable the log_slave_updates parameter so DTS can capture all binary log events. For instructions, see Create an account and configure binary logging for a self-managed MySQL database.

      • The local binary logs for an ApsaraDB RDS for MySQL instance must be retained for at least three days (seven days is recommended). For a self-managed MySQL database, retain local binary logs for at least seven days. Otherwise, DTS may fail to retrieve binary logs, causing the task to fail. In extreme cases, this may cause data inconsistency or data loss. Issues caused by binary log retention periods shorter than DTS requires are not covered under the DTS SLA.

        Note

        To configure the retention period for local binary logs on an ApsaraDB RDS for MySQL instance, see Automatically delete local logs.

    • Do not run DDL operations that change database or table schemas during schema synchronization or full synchronization. Otherwise, the synchronization task fails.

      Note

      During full synchronization, DTS queries the source database. This creates metadata locks that may block DDL operations on the source database.

    • Data generated by changes that do not write to binary logs—such as data restored from physical backups or created by cascade operations—is not synchronized to the destination database.

      Note

      If this occurs, remove the affected database or table from the synchronization objects. Then add it back. You can do this only if your business allows it. For more information, see Modify synchronization objects.

    • If your source database is MySQL 8.0.23 or later and contains invisible hidden columns, DTS may not read those columns. This may cause data loss.

      Note

      Run the ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE; command to make the hidden column visible. For more information, see Invisible Columns.

    Other limits

    • DTS does not support synchronizing indexes, partitions, views, stored procedures, functions, triggers, or foreign keys.

    • You cannot synchronize data to Elasticsearch indexes that contain parent-child relationships or Join field types. Doing so may cause task errors or query failures in the destination.

    • Assess the performance of both the source and destination databases before starting synchronization. Run synchronization during off-peak hours. Otherwise, full initialization consumes read and write resources on both databases and may increase database load.

    • Full initialization runs INSERT operations concurrently. This fragments destination tables. After full initialization, the tablespace of the destination instance is larger than that of the source instance.

    • If you synchronize one or more tables—not a full database—do not use tools like pt-online-schema-change to perform online DDL operations on the source tables. Otherwise, synchronization fails.

    • If your source table needs to add a column, first modify the corresponding mapping in the Elasticsearch instance. Then run the DDL operation in the source MySQL database. Finally, pause and restart the DTS synchronization task.

    • Do not write data to the destination database except through DTS while synchronization is running. Otherwise, data inconsistency may occur between the source and destination databases. For example, if you use DMS to perform online DDL operations while other data is written to the destination database, data loss may occur.

    • If data synchronized from MySQL to Elasticsearch contains null characters, DTS converts them to LONG type values. This causes the task to fail.

    • If data synchronized from MySQL to Elasticsearch contains location information with reversed latitude and longitude values, writing to Elasticsearch fails.

    • Development and test specifications of Elasticsearch instances are not supported.

    • If your ApsaraDB RDS for MySQL instance has Always-Encrypted enabled, full data synchronization is not supported.

      Note

      ApsaraDB RDS for MySQL instances with Transparent Data Encryption (TDE) enabled support schema synchronization, full data synchronization, and incremental data synchronization.

    • If a task fails, DTS support staff will attempt to restore it within eight hours. During restoration, they may restart the task or adjust its parameters.

      Note

      Only DTS task parameters are modified—not database parameters. Parameters that may be adjusted include those listed in Modify instance parameters.

    Special cases

    • For a self-managed MySQL source database:

      • If a primary/secondary switchover occurs in the source database during synchronization, the task fails.

      • DTS calculates latency by comparing the timestamp of the last synchronized record with the current time. If no DML operations run for a long time in the source database, latency reporting may become inaccurate. If latency appears too high, run a DML operation in the source database to update the latency.

        Note

        If you select a full database for synchronization, create a heartbeat table. Update or write to this table every second.

      • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

      • If your source database is Amazon Aurora MySQL or another clustered MySQL instance, ensure the domain name or IP address used in the task configuration—and its DNS resolution—always points to a read/write (RW) node. Otherwise, synchronization may fail.

    • For an ApsaraDB RDS for MySQL source database:

      • Read-only instances—such as ApsaraDB RDS for MySQL 5.6 read-only instances—that do not record transaction logs cannot serve as source databases.

      • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

    Synchronize from MySQL to MaxCompute

    Type

    Description

    Source database limits

    • The tables to synchronize must have a primary key or a unique constraint, and the fields must be unique. Otherwise, duplicate data may appear in the destination database.

    • If you synchronize at the table level and need to edit mappings (such as column name mapping), each synchronization task supports up to 1,000 tables. If you exceed this limit, the task fails with an error. To fix this, split the tables across multiple tasks or configure a full-database synchronization task.

    • Binary logs:

      • ApsaraDB RDS for MySQL enables binary logging by default. Ensure that the binlog_row_image parameter is set to full. Otherwise, the precheck fails and the synchronization task cannot start. For instructions, see Configure instance parameters.

        Important

        • If your source instance is a self-managed MySQL database, enable binary logging and set binlog_format to row and binlog_row_image to full.

        • If your self-managed MySQL database is a dual-primary cluster (where both nodes act as primary and secondary), enable the log_slave_updates parameter so DTS can capture all binary log events. For instructions, see Create an account and configure binary logging for a self-managed MySQL database.

      • The local binary logs for an ApsaraDB RDS for MySQL instance must be retained for at least three days (seven days is recommended). For a self-managed MySQL database, retain local binary logs for at least seven days. Otherwise, DTS may fail to retrieve binary logs, causing the task to fail. In extreme cases, this may cause data inconsistency or data loss. Issues caused by binary log retention periods shorter than DTS requires are not covered under the DTS SLA.

        Note

        To configure the retention period for local binary logs on an ApsaraDB RDS for MySQL instance, see Automatically delete local logs.

    • Do not run DDL operations that change database or table schemas during schema synchronization or full synchronization. Otherwise, the synchronization task fails.

      Note

      During full synchronization, DTS queries the source database. This creates metadata locks that may block DDL operations on the source database.

    • Data generated by changes that do not write to binary logs—such as data restored from physical backups or created by cascade operations—is not synchronized to the destination database.

      Note

      If this occurs, remove the affected database or table from the synchronization objects. Then add it back. You can do this only if your business allows it. For more information, see Modify synchronization objects.

    • If your source database is MySQL 8.0.23 or later and contains invisible hidden columns, DTS may not read those columns. This may cause data loss.

      Note

      Run the ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE; command to make the hidden column visible. For more information, see Invisible Columns.

    Other limits

    • Assess the performance of both the source and destination databases before starting synchronization. Run synchronization during off-peak hours. Otherwise, full initialization consumes read and write resources on both databases and may increase database load.

    • Full initialization runs INSERT operations concurrently. This fragments destination tables. After full initialization, the tablespace of the destination instance is larger than that of the source instance.

    • If you synchronize one or more tables—not a full database—do not use tools like pt-online-schema-change to perform online DDL operations on the source tables. Otherwise, synchronization fails.

      You can use Data Management (DMS) to perform online DDL operations. For more information, see Online DDL without locking tables.

    • DTS does not support synchronizing indexes, partitions, views, stored procedures, functions, triggers, or foreign keys.

    • MaxCompute does not support primary key constraints. If DTS retries due to network issues or other reasons, duplicate records may appear in MaxCompute.

    • Do not write data to the destination database except through DTS while synchronization is running. Otherwise, data inconsistency may occur between the source and destination databases. For example, if you use DMS to perform online DDL operations while other data is written to the destination database, data loss may occur.

    • If your ApsaraDB RDS for MySQL instance has Always-Encrypted enabled, full data synchronization is not supported.

      Note

      ApsaraDB RDS for MySQL instances with Transparent Data Encryption (TDE) enabled support schema synchronization, full data synchronization, and incremental data synchronization.

    • If a task fails, DTS support staff will attempt to restore it within eight hours. During restoration, they may restart the task or adjust its parameters.

      Note

      Only DTS task parameters are modified—not database parameters. Parameters that may be adjusted include those listed in Modify instance parameters.

    Special cases

    • For a self-managed MySQL source database:

      • If a primary/secondary switchover occurs in the source database during synchronization, the task fails.

      • DTS calculates latency by comparing the timestamp of the last synchronized record with the current time. If no DML operations run for a long time in the source database, latency reporting may become inaccurate. If latency appears too high, run a DML operation in the source database to update the latency.

        Note

        If you select a full database for synchronization, create a heartbeat table. Update or write to this table every second.

      • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

      • If your source database is Amazon Aurora MySQL or another clustered MySQL instance, ensure the domain name or IP address used in the task configuration—and its DNS resolution—always points to a read/write (RW) node. Otherwise, synchronization may fail.

    • For an ApsaraDB RDS for MySQL source database:

      • Read-only instances—such as ApsaraDB RDS for MySQL 5.6 read-only instances—that do not record transaction logs cannot serve as source databases.

      • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

    Synchronize from MySQL to ClickHouse clusters

    Type

    Description

    Source database limits

    • Tables without a primary key cannot be synchronized.

    • If you synchronize data at the table level and need to edit objects, such as mapping table or column names, a single data synchronization task supports a maximum of 1,000 tables. If you exceed this limit, an error is reported after you submit the task. In this case, split the tables into multiple tasks or configure a task to synchronize the entire database.

    • Binary logs:

      • ApsaraDB RDS for MySQL enables binary logging by default. Ensure that the binlog_row_image parameter is set to full. Otherwise, the precheck fails and the synchronization task cannot start. For instructions, see Configure instance parameters.

        Important

        • If your source instance is a self-managed MySQL database, enable binary logging and set binlog_format to row and binlog_row_image to full.

        • If your self-managed MySQL database is a dual-primary cluster (where both nodes act as primary and secondary), enable the log_slave_updates parameter so DTS can capture all binary log events. For instructions, see Create an account and configure binary logging for a self-managed MySQL database.

      • The local binary logs for an ApsaraDB RDS for MySQL instance must be retained for at least three days (seven days is recommended). For a self-managed MySQL database, retain local binary logs for at least seven days. Otherwise, DTS may fail to retrieve binary logs, causing the task to fail. In extreme cases, this may cause data inconsistency or data loss. Issues caused by binary log retention periods shorter than DTS requires are not covered under the DTS SLA.

        Note

        To configure the retention period for local binary logs on an ApsaraDB RDS for MySQL instance, see Automatically delete local logs.

    • Do not run DDL operations that change database or table schemas during schema synchronization or full synchronization. Otherwise, the synchronization task fails.

      Note

      During full synchronization, DTS queries the source database. This creates metadata locks that may block DDL operations on the source database.

    • Data generated by changes that do not write to binary logs—such as data restored from physical backups or created by cascade operations—is not synchronized to the destination database.

      Note

      If this occurs, remove the affected database or table from the synchronization objects. Then add it back. You can do this only if your business allows it. For more information, see Modify synchronization objects.

    • If your source database is MySQL 8.0.23 or later and contains invisible hidden columns, DTS may not read those columns. This may cause data loss.

      Note

      Run the ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE; command to make the hidden column visible. For more information, see Invisible Columns.

    Other limits

    • If the DDL statements of the source RDS MySQL instance do not follow standard MySQL syntax, the synchronization task may fail or data may be lost.

    • If your source database uses online DDL operations in temporary table mode—including but not limited to multi-table merge scenarios—or adds function-based indexes to unique key columns, data loss or task failure may occur in the destination database.

    • When you use DMS or the gh-ost tool to perform online DDL operations on the source, DTS synchronizes only the original DDL statements to the destination. In this scenario, DTS does not need to synchronize a large amount of temporary table data, but this may cause tables to be locked at the destination.

      Note

      DTS does not support synchronizing online DDL changes made with tools like pt-online-schema-change. If such changes exist at the source, data may be lost at the destination, or the synchronization instance may fail.

    • Synchronization of INDEX, PARTITION, VIEW, PROCEDURE, FUNCTION, TRIGGER, and FK is supported.

    • The RENAME TABLE operation is not supported.

    • If a primary key or unique key conflict occurs while the task is running:

      • If the table schemas are consistent and a record in the destination database has the same primary key or unique key value as a record in the source database:

        • During full data synchronization, DTS retains the destination record and skips the source record.

        • During incremental synchronization, DTS overwrites the destination record with the source record.

      • If the table schemas are inconsistent, data initialization may fail. This can result in only partial data synchronization or a complete synchronization failure. Use with caution.

    • Time-type data in ApsaraDB for ClickHouse has range limitations. If the time data in RDS MySQL is outside this range, the time synchronized to ApsaraDB for ClickHouse will be incorrect. For information about the range limitations, see Time information.

    • The Partition Key cannot be a nullable field. Otherwise, the synchronization task will fail.

      Note

      Only fields of the BIGINT, INT, TIMESTAMP, DATETIME, and DATE types are supported as partition keys.

    • The number of databases to be synchronized does not exceed the limit of ApsaraDB for ClickHouse, which is 256.

    • The names of the databases, tables, and columns to be synchronized comply with the naming conventions of ApsaraDB for ClickHouse. For more information about the conventions, see Object naming conventions.

    • During schema synchronization, DTS adds the _sign, _is_deleted, and _version fields to the destination tables. If you do not select Schema Synchronization when you configure Synchronization Types, you must manually create the tables for receiving data at the destination and add the extra fields to the tables. For information about table creation requirements and field details, see Table and field information.

    • Before synchronizing data, evaluate the performance of the source and destination databases. We recommend that you perform data synchronization during off-peak hours. Otherwise, initial full data synchronization will consume read and write resources of the source and destination databases, which may increase database loads.

    • If you synchronize one or more tables instead of the entire database, do not use tools like pt-online-schema-change to perform online DDL operations on the synchronization objects in the source database. Otherwise, the synchronization will fail.

      You can use Data Management (DMS) to perform online DDL operations. For more information, see Change schemas without locking tables.

    • During DTS synchronization, do not allow any data other than that from DTS to be written to the destination database. Otherwise, data inconsistency between the source and destination databases will occur.

    • If your ApsaraDB RDS for MySQL instance has Always-Encrypted enabled, full data synchronization is not supported.

      Note

      ApsaraDB RDS for MySQL instances with Transparent Data Encryption (TDE) enabled support schema synchronization, full data synchronization, and incremental data synchronization.

    • If a task fails, DTS support staff will attempt to restore it within eight hours. During restoration, they may restart the task or adjust its parameters.

      Note

      Only DTS task parameters are modified—not database parameters. Parameters that may be adjusted include those listed in Modify instance parameters.

    Special cases

    • For a self-managed MySQL source database:

      • If a primary/secondary switchover occurs in the source database during synchronization, the task fails.

      • DTS calculates latency by comparing the timestamp of the last synchronized record with the current time. If no DML operations run for a long time in the source database, latency reporting may become inaccurate. If latency appears too high, run a DML operation in the source database to update the latency.

        Note

        If you select a full database for synchronization, create a heartbeat table. Update or write to this table every second.

      • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

      • If your source database is Amazon Aurora MySQL or another clustered MySQL instance, ensure the domain name or IP address used in the task configuration—and its DNS resolution—always points to a read/write (RW) node. Otherwise, synchronization may fail.

    • For an ApsaraDB RDS for MySQL source database:

      • Read-only instances—such as ApsaraDB RDS for MySQL 5.6 read-only instances—that do not record transaction logs cannot serve as source databases.

      • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

    Synchronize from MySQL to Tablestore

    Type

    Description

    Source database limits

    • The tables to synchronize must have a primary key or a unique constraint, and the fields must be unique. Otherwise, duplicate data may appear in the destination database.

    • If you synchronize at the table level and need to edit mappings (such as column name mapping), each synchronization task supports up to 1,000 tables. If you exceed this limit, the task fails with an error. To fix this, split the tables across multiple tasks or configure a full-database synchronization task.

    • Binary logs:

      • ApsaraDB RDS for MySQL enables binary logging by default. Ensure that the binlog_row_image parameter is set to full. Otherwise, the precheck fails and the synchronization task cannot start. For instructions, see Configure instance parameters.

        Important

        • If your source instance is a self-managed MySQL database, enable binary logging and set binlog_format to row and binlog_row_image to full.

        • If your self-managed MySQL database is a dual-primary cluster (where both nodes act as primary and secondary), enable the log_slave_updates parameter so DTS can capture all binary log events. For instructions, see Create an account and configure binary logging for a self-managed MySQL database.

      • The local binary logs for an ApsaraDB RDS for MySQL instance must be retained for at least three days (seven days is recommended). For a self-managed MySQL database, retain local binary logs for at least seven days. Otherwise, DTS may fail to retrieve binary logs, causing the task to fail. In extreme cases, this may cause data inconsistency or data loss. Issues caused by binary log retention periods shorter than DTS requires are not covered under the DTS SLA.

        Note

        To configure the retention period for local binary logs on an ApsaraDB RDS for MySQL instance, see Automatically delete local logs.

    • Do not run DDL operations that change database or table schemas during schema synchronization or full synchronization. Otherwise, the synchronization task fails.

      Note

      During full synchronization, DTS queries the source database. This creates metadata locks that may block DDL operations on the source database.

    • Data generated by changes that do not write to binary logs—such as data restored from physical backups or created by cascade operations—is not synchronized to the destination database.

      Note

      If this occurs, remove the affected database or table from the synchronization objects. Then add it back. You can do this only if your business allows it. For more information, see Modify synchronization objects.

    • If your source database is MySQL 8.0.23 or later and contains invisible hidden columns, DTS may not read those columns. This may cause data loss.

      Note

      Run the ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE; command to make the hidden column visible. For more information, see Invisible Columns.

    Other limits

    • Assess the performance of both the source and destination databases before starting synchronization. Run synchronization during off-peak hours. Otherwise, full initialization consumes read and write resources on both databases and may increase database load.

    • Full initialization runs INSERT operations concurrently. This fragments destination tables. After full initialization, the tablespace of the destination instance is larger than that of the source instance.

    • If you synchronize one or more tables—not a full database—do not use tools like pt-online-schema-change to perform online DDL operations on the source tables. Otherwise, synchronization fails.

      You can use Data Management (DMS) to perform online DDL operations. For more information, see Online DDL without locking tables.

    • Do not write data to the destination database except through DTS while synchronization is running. Otherwise, data inconsistency may occur between the source and destination databases. For example, if you use DMS to perform online DDL operations while other data is written to the destination database, data loss may occur.

    • If a primary key or unique key conflict occurs while the task is running:

      • If the table schemas are consistent and a record in the destination database has the same primary key or unique key value as a record in the source database:

        • During full data synchronization, DTS retains the destination record and skips the source record.

        • During incremental synchronization, DTS overwrites the destination record with the source record.

      • If the table schemas are inconsistent, data initialization may fail. This can result in only partial data synchronization or a complete synchronization failure. Use with caution.

    • Names of tables or columns to synchronize must comply with Tablestore naming conventions:

      • Names consist of uppercase and lowercase letters, digits, or underscores (_), and must begin with a letter or underscore.

      • Names must be 1 to 255 characters long.

    • If your ApsaraDB RDS for MySQL instance has Always-Encrypted enabled, full data synchronization is not supported.

      Note

      ApsaraDB RDS for MySQL instances with Transparent Data Encryption (TDE) enabled support schema synchronization, full data synchronization, and incremental data synchronization.

    • If a task fails, DTS support staff will attempt to restore it within eight hours. During restoration, they may restart the task or adjust its parameters.

      Note

      Only DTS task parameters are modified—not database parameters. Parameters that may be adjusted include those listed in Modify instance parameters.

    Special cases

    • For a self-managed MySQL source database:

      • If a primary/secondary switchover occurs in the source database during synchronization, the task fails.

      • DTS calculates latency by comparing the timestamp of the last synchronized record with the current time. If no DML operations run for a long time in the source database, latency reporting may become inaccurate. If latency appears too high, run a DML operation in the source database to update the latency.

        Note

        If you select a full database for synchronization, create a heartbeat table. Update or write to this table every second.

      • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

      • If your source database is Amazon Aurora MySQL or another clustered MySQL instance, ensure the domain name or IP address used in the task configuration—and its DNS resolution—always points to a read/write (RW) node. Otherwise, synchronization may fail.

    • For an ApsaraDB RDS for MySQL source database:

      • Read-only instances—such as ApsaraDB RDS for MySQL 5.6 read-only instances—that do not record transaction logs cannot serve as source databases.

      • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

    Synchronize from MySQL to ApsaraDB for Kafka or a self-managed Kafka cluster

    Type

    Description

    Source database limits

    • The tables to synchronize must have a primary key or a unique constraint, and the fields must be unique. Otherwise, duplicate data may appear in the destination database.

    • If you synchronize at the table level and need to edit mappings (such as column name mapping), each synchronization task supports up to 1,000 tables. If you exceed this limit, the task fails with an error. To fix this, split the tables across multiple tasks or configure a full-database synchronization task.

    • Binary logs:

      • ApsaraDB RDS for MySQL enables binary logging by default. Ensure that the binlog_row_image parameter is set to full. Otherwise, the precheck fails and the synchronization task cannot start. For instructions, see Configure instance parameters.

        Important

        • If your source instance is a self-managed MySQL database, enable binary logging and set binlog_format to row and binlog_row_image to full.

        • If your self-managed MySQL database is a dual-primary cluster (where both nodes act as primary and secondary), enable the log_slave_updates parameter so DTS can capture all binary log events. For instructions, see Create an account and configure binary logging for a self-managed MySQL database.

      • The local binary logs for an ApsaraDB RDS for MySQL instance must be retained for at least three days (seven days is recommended). For a self-managed MySQL database, retain local binary logs for at least seven days. Otherwise, DTS may fail to retrieve binary logs, causing the task to fail. In extreme cases, this may cause data inconsistency or data loss. Issues caused by binary log retention periods shorter than DTS requires are not covered under the DTS SLA.

        Note

        To configure the retention period for local binary logs on an ApsaraDB RDS for MySQL instance, see Automatically delete local logs.

    • Do not run DDL operations that change database or table schemas during schema synchronization or full synchronization. Otherwise, the synchronization task fails.

      Note

      During full synchronization, DTS queries the source database. This creates metadata locks that may block DDL operations on the source database.

    • Data generated by changes that do not write to binary logs—such as data restored from physical backups or created by cascade operations—is not synchronized to the destination database.

      Note

      If this occurs, remove the affected database or table from the synchronization objects. Then add it back. You can do this only if your business allows it. For more information, see Modify synchronization objects.

    • If your source database is MySQL 8.0.23 or later and contains invisible hidden columns, DTS may not read those columns. This may cause data loss.

      Note

      Run the ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE; command to make the hidden column visible. For more information, see Invisible Columns.

    Other limits

    • Assess the performance of both the source and destination databases before starting synchronization. Run synchronization during off-peak hours. Otherwise, full initialization consumes read and write resources on both databases and may increase database load.

    • Full initialization runs INSERT operations concurrently. This fragments destination tables. After full initialization, the tablespace of the destination instance is larger than that of the source instance.

    • If you synchronize one or more tables—not a full database—do not use tools like pt-online-schema-change to perform online DDL operations on the source tables. Otherwise, synchronization fails.

      You can use Data Management (DMS) to perform online DDL operations. For more information, see Online DDL without locking tables.

    • DTS does not support synchronizing indexes, partitions, views, stored procedures, functions, triggers, or foreign keys.

    • Do not write data to the destination database except through DTS while synchronization is running. Otherwise, data inconsistency may occur between the source and destination databases.

    • If the destination Kafka cluster scales out or scales in during synchronization, restart the task.

    • If your ApsaraDB RDS for MySQL instance has Always-Encrypted enabled, full data synchronization is not supported.

      Note

      ApsaraDB RDS for MySQL instances with Transparent Data Encryption (TDE) enabled support schema synchronization, full data synchronization, and incremental data synchronization.

    • If a task fails, DTS support staff will attempt to restore it within eight hours. During restoration, they may restart the task or adjust its parameters.

      Note

      Only DTS task parameters are modified—not database parameters. Parameters that may be adjusted include those listed in Modify instance parameters.

    Special cases

    • For a self-managed MySQL source database:

      • If a primary/secondary switchover occurs in the source database during synchronization, the task fails.

      • DTS calculates latency by comparing the timestamp of the last synchronized record with the current time. If no DML operations run for a long time in the source database, latency reporting may become inaccurate. If latency appears too high, run a DML operation in the source database to update the latency.

        Note

        If you select a full database for synchronization, create a heartbeat table. Update or write to this table every second.

      • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

      • If your source database is Amazon Aurora MySQL or another clustered MySQL instance, ensure the domain name or IP address used in the task configuration—and its DNS resolution—always points to a read/write (RW) node. Otherwise, synchronization may fail.

    • For an ApsaraDB RDS for MySQL source database:

      • Read-only instances—such as ApsaraDB RDS for MySQL 5.6 read-only instances—that do not record transaction logs cannot serve as source databases.

      • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

    Synchronize from MySQL to PolarDB-X 2.0

    Type

    Description

    Source database limits

    • The tables to synchronize must have a primary key or a unique constraint, and the fields must be unique. Otherwise, duplicate data may appear in the destination database.

    • If you synchronize at the table level and need to edit mappings (such as column name mapping), each synchronization task supports up to 1,000 tables. If you exceed this limit, the task fails with an error. To fix this, split the tables across multiple tasks or configure a full-database synchronization task.

    • Binary logs:

      • ApsaraDB RDS for MySQL enables binary logging by default. Ensure that the binlog_row_image parameter is set to full. Otherwise, the precheck fails and the synchronization task cannot start. For instructions, see Configure instance parameters.

        Important

        • If your source instance is a self-managed MySQL database, enable binary logging and set binlog_format to row and binlog_row_image to full.

        • If your self-managed MySQL database is a dual-primary cluster (where both nodes act as primary and secondary), enable the log_slave_updates parameter so DTS can capture all binary log events. For instructions, see Create an account and configure binary logging for a self-managed MySQL database.

      • The local binary logs for an ApsaraDB RDS for MySQL instance must be retained for at least three days (seven days is recommended). For a self-managed MySQL database, retain local binary logs for at least seven days. Otherwise, DTS may fail to retrieve binary logs, causing the task to fail. In extreme cases, this may cause data inconsistency or data loss. Issues caused by binary log retention periods shorter than DTS requires are not covered under the DTS SLA.

        Note

        To configure the retention period for local binary logs on an ApsaraDB RDS for MySQL instance, see Automatically delete local logs.

    • Do not run DDL operations that change database or table schemas during schema synchronization or full synchronization. Otherwise, the synchronization task fails.

      Note

      During full synchronization, DTS queries the source database. This creates metadata locks that may block DDL operations on the source database.

    • Data generated by changes that do not write to binary logs—such as data restored from physical backups or created by cascade operations—is not synchronized to the destination database.

      Note

      If this occurs, remove the affected database or table from the synchronization objects. Then add it back. You can do this only if your business allows it. For more information, see Modify synchronization objects.

    • If your source database is MySQL 8.0.23 or later and contains invisible hidden columns, DTS may not read those columns. This may cause data loss.

      Note

      Run the ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE; command to make the hidden column visible. For more information, see Invisible Columns.

    Other limits

    • If a primary key or unique key conflict occurs while the task is running:

      • If the table schemas are consistent and a record in the destination database has the same primary key or unique key value as a record in the source database:

        • During full data synchronization, DTS retains the destination record and skips the source record.

        • During incremental synchronization, DTS overwrites the destination record with the source record.

      • If the table schemas are inconsistent, data initialization may fail. This can result in only partial data synchronization or a complete synchronization failure. Use with caution.

    • Synchronization object requirements:

      • DTS does not support synchronizing BIT, VARBIT, GEOMETRY, ARRAY, UUID, TSQUERY, TSVECTOR, or TXID_SNAPSHOT data types.

      • DTS does not support synchronizing prefix indexes. If your source database contains prefix indexes, synchronization may fail.

    • If your source database uses online DDL operations in temporary table mode—including but not limited to multi-table merge scenarios—or adds function-based indexes to unique key columns, data loss or task failure may occur in the destination database.

    • Assess the performance of both the source and destination databases before starting synchronization. Run synchronization during off-peak hours. Otherwise, full initialization consumes read and write resources on both databases and may increase database load.

    • Full initialization runs INSERT operations concurrently. This fragments destination tables. After full initialization, the tablespace of the destination instance is larger than that of the source instance.

    • If you synchronize one or more tables—not a full database—do not use tools like pt-online-schema-change to perform online DDL operations on the source tables. Otherwise, synchronization fails.

      You can use Data Management (DMS) to perform online DDL operations. For more information, see Online DDL without locking tables.

    • Do not write data to the destination database except through DTS while synchronization is running. Otherwise, data inconsistency may occur between the source and destination databases. For example, if you use DMS to perform online DDL operations while other data is written to the destination database, data loss may occur.

    • If your ApsaraDB RDS for MySQL instance has Always-Encrypted enabled, full data synchronization is not supported.

      Note

      ApsaraDB RDS for MySQL instances with Transparent Data Encryption (TDE) enabled support schema synchronization, full data synchronization, and incremental data synchronization.

    • If a task fails, DTS support staff will attempt to restore it within eight hours. During restoration, they may restart the task or adjust its parameters.

      Note

      Only DTS task parameters are modified—not database parameters. Parameters that may be adjusted include those listed in Modify instance parameters.

    Special cases

    • For a self-managed MySQL source database:

      • If a primary/secondary switchover occurs in the source database during synchronization, the task fails.

      • DTS calculates latency by comparing the timestamp of the last synchronized record with the current time. If no DML operations run for a long time in the source database, latency reporting may become inaccurate. If latency appears too high, run a DML operation in the source database to update the latency.

        Note

        If you select a full database for synchronization, create a heartbeat table. Update or write to this table every second.

      • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

      • If your source database is Amazon Aurora MySQL or another clustered MySQL instance, ensure the domain name or IP address used in the task configuration—and its DNS resolution—always points to a read/write (RW) node. Otherwise, synchronization may fail.

    • For an ApsaraDB RDS for MySQL source database:

      • Read-only instances—such as ApsaraDB RDS for MySQL 5.6 read-only instances—that do not record transaction logs cannot serve as source databases.

      • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

    Synchronize from MySQL to Oracle

    Type

    Description

    Source database limits

    • The tables to synchronize must have a primary key or a unique constraint, and the fields must be unique. Otherwise, duplicate data may appear in the destination database.

    • If you synchronize at the table level and need to edit mappings (such as column name mapping), each synchronization task supports up to 1,000 tables. If you exceed this limit, the task fails with an error. To fix this, split the tables across multiple tasks or configure a full-database synchronization task.

    • Binary logs:

      • ApsaraDB RDS for MySQL enables binary logging by default. Ensure that the binlog_row_image parameter is set to full. Otherwise, the precheck fails and the synchronization task cannot start. For instructions, see Configure instance parameters.

        Important

        • If your source instance is a self-managed MySQL database, enable binary logging and set binlog_format to row and binlog_row_image to full.

        • If your self-managed MySQL database is a dual-primary cluster (where both nodes act as primary and secondary), enable the log_slave_updates parameter so DTS can capture all binary log events. For instructions, see Create an account and configure binary logging for a self-managed MySQL database.

      • The local binary logs for an ApsaraDB RDS for MySQL instance must be retained for at least three days (seven days is recommended). For a self-managed MySQL database, retain local binary logs for at least seven days. Otherwise, DTS may fail to retrieve binary logs, causing the task to fail. In extreme cases, this may cause data inconsistency or data loss. Issues caused by binary log retention periods shorter than DTS requires are not covered under the DTS SLA.

        Note

        To configure the retention period for local binary logs on an ApsaraDB RDS for MySQL instance, see Automatically delete local logs.

    • Do not run DDL operations that change database or table schemas during schema synchronization or full synchronization. Otherwise, the synchronization task fails.

      Note

      During full synchronization, DTS queries the source database. This creates metadata locks that may block DDL operations on the source database.

    • Data generated by changes that do not write to binary logs—such as data restored from physical backups or created by cascade operations—is not synchronized to the destination database.

      Note

      If this occurs, remove the affected database or table from the synchronization objects. Then add it back. You can do this only if your business allows it. For more information, see Modify synchronization objects.

    • If your source database is MySQL 8.0.23 or later and contains invisible hidden columns, DTS may not read those columns. This may cause data loss.

      Note

      Run the ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE; command to make the hidden column visible. For more information, see Invisible Columns.

    Other limits

    • Multi-column attribute changes are not supported.

    • Changes to basic data types are not supported.

    • If your source database uses online DDL operations in temporary table mode—including but not limited to multi-table merge scenarios—or adds function-based indexes to unique key columns, data loss or task failure may occur in the destination database.

    • If your self-managed Oracle database uses Real Application Clusters (RAC), do not configure ScanIP. Instead, configure only one virtual IP address (VIP) in the connection information. With this configuration, RAC node failover is not supported.

    • If a primary key or unique key conflict occurs while the task is running:

      • If the table schemas are consistent and a record in the destination database has the same primary key or unique key value as a record in the source database:

        • During full data synchronization, DTS retains the destination record and skips the source record.

        • During incremental synchronization, DTS overwrites the destination record with the source record.

      • If the table schemas are inconsistent, data initialization may fail. This can result in only partial data synchronization or a complete synchronization failure. Use with caution.

    • If your ApsaraDB RDS for MySQL instance has Always-Encrypted enabled, full data synchronization is not supported.

      Note

      ApsaraDB RDS for MySQL instances with Transparent Data Encryption (TDE) enabled support schema synchronization, full data synchronization, and incremental data synchronization.

    • If a task fails, DTS support staff will attempt to restore it within eight hours. During restoration, they may restart the task or adjust its parameters.

      Note

      Only DTS task parameters are modified—not database parameters. Parameters that may be adjusted include those listed in Modify instance parameters.

    Special cases

    • For a self-managed MySQL source database:

      • If a primary/secondary switchover occurs in the source database during synchronization, the task fails.

      • DTS calculates latency by comparing the timestamp of the last synchronized record with the current time. If no DML operations run for a long time in the source database, latency reporting may become inaccurate. If latency appears too high, run a DML operation in the source database to update the latency.

        Note

        If you select a full database for synchronization, create a heartbeat table. Update or write to this table every second.

      • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

      • If your source database is Amazon Aurora MySQL or another clustered MySQL instance, ensure the domain name or IP address used in the task configuration—and its DNS resolution—always points to a read/write (RW) node. Otherwise, synchronization may fail.

    • For an ApsaraDB RDS for MySQL source database:

      • Read-only instances—such as ApsaraDB RDS for MySQL 5.6 read-only instances—that do not record transaction logs cannot serve as source databases.

      • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

    Synchronize from MySQL to PostgreSQL

    Type

    Description

    Source database limits

    • The tables to synchronize must have a primary key or a unique constraint, and the fields must be unique. Otherwise, duplicate data may appear in the destination database.

    • If you synchronize at the table level and need to edit mappings (such as column name mapping), each synchronization task supports up to 1,000 tables. If you exceed this limit, the task fails with an error. To fix this, split the tables across multiple tasks or configure a full-database synchronization task.

    • Binary logs:

      • ApsaraDB RDS for MySQL enables binary logging by default. Ensure that the binlog_row_image parameter is set to full. Otherwise, the precheck fails and the synchronization task cannot start. For instructions, see Configure instance parameters.

        Important

        • If your source instance is a self-managed MySQL database, enable binary logging and set binlog_format to row and binlog_row_image to full.

        • If your self-managed MySQL database is a dual-primary cluster (where both nodes act as primary and secondary), enable the log_slave_updates parameter so DTS can capture all binary log events. For instructions, see Create an account and configure binary logging for a self-managed MySQL database.

      • The local binary logs for an ApsaraDB RDS for MySQL instance must be retained for at least three days (seven days is recommended). For a self-managed MySQL database, retain local binary logs for at least seven days. Otherwise, DTS may fail to retrieve binary logs, causing the task to fail. In extreme cases, this may cause data inconsistency or data loss. Issues caused by binary log retention periods shorter than DTS requires are not covered under the DTS SLA.

        Note

        To configure the retention period for local binary logs on an ApsaraDB RDS for MySQL instance, see Automatically delete local logs.

    • Do not run DDL operations that change database or table schemas during schema synchronization or full synchronization. Otherwise, the synchronization task fails.

      Note

      During full synchronization, DTS queries the source database. This creates metadata locks that may block DDL operations on the source database.

    • Data generated by changes that do not write to binary logs—such as data restored from physical backups or created by cascade operations—is not synchronized to the destination database.

      Note

      If this occurs, remove the affected database or table from the synchronization objects. Then add it back. You can do this only if your business allows it. For more information, see Modify synchronization objects.

    • If your source database is MySQL 8.0.23 or later and contains invisible hidden columns, DTS may not read those columns. This may cause data loss.

      Note

      Run the ALTER TABLE <table_name> ALTER COLUMN <column_name> SET VISIBLE; command to make the hidden column visible. For more information, see Invisible Columns.

    Other limits

    • Synchronization object requirements:

      • Only table-level synchronization is supported.

      • DTS does not support synchronizing BIT, VARBIT, GEOMETRY, ARRAY, UUID, TSQUERY, TSVECTOR, TXID_SNAPSHOT, or POINT data types.

      • DTS does not support synchronizing prefix indexes. If your source database contains prefix indexes, synchronization may fail.

      • DTS does not support synchronizing partitions, views, stored procedures, functions, or triggers.

    • If your source database uses online DDL operations in temporary table mode—including but not limited to multi-table merge scenarios—or adds function-based indexes to unique key columns, data loss or task failure may occur in the destination database.

    • If a primary key or unique key conflict occurs while the task is running:

      • If the table schemas are consistent and a record in the destination database has the same primary key or unique key value as a record in the source database:

        • During full data synchronization, DTS retains the destination record and skips the source record.

        • During incremental synchronization, DTS overwrites the destination record with the source record.

      • If the table schemas are inconsistent, data initialization may fail. This can result in only partial data synchronization or a complete synchronization failure. Use with caution.

    • Assess the performance of both the source and destination databases before starting synchronization. Run synchronization during off-peak hours. Otherwise, full initialization consumes read and write resources on both databases and may increase database load.

    • Full initialization runs INSERT operations concurrently. This fragments destination tables. After full initialization, the tablespace of the destination instance is larger than that of the source instance.

    • If you synchronize one or more tables—not a full database—do not use tools like pt-online-schema-change to perform online DDL operations on the source tables. Otherwise, synchronization fails.

      You can use Data Management (DMS) to perform online DDL operations. For more information, see Online DDL without locking tables.

    • Do not write data to the destination database except through DTS while synchronization is running. Otherwise, data inconsistency may occur between the source and destination databases. For example, if you use DMS to perform online DDL operations while other data is written to the destination database, data loss may occur.

    • If your ApsaraDB RDS for MySQL instance has Always-Encrypted enabled, full data synchronization is not supported.

      Note

      ApsaraDB RDS for MySQL instances with Transparent Data Encryption (TDE) enabled support schema synchronization, full data synchronization, and incremental data synchronization.

    • For a full or incremental synchronization task, if the tables to be synchronized in the source database contain foreign keys, triggers, or event triggers, DTS temporarily sets the `session_replication_role` parameter to `replica` at the session level if the destination database account is a privileged account or has superuser permissions. If the destination database account does not have these permissions, you must manually set the `session_replication_role` parameter to `replica` in the destination database. During this period (while `session_replication_role` is `replica`), if cascade update or delete operations occur in the source database, data inconsistency may occur. After the DTS task is released, you can change the `session_replication_role` parameter back to `origin`.

    • If a task fails, DTS support staff will attempt to restore it within eight hours. During restoration, they may restart the task or adjust its parameters.

      Note

      Only DTS task parameters are modified—not database parameters. Parameters that may be adjusted include those listed in Modify instance parameters.

    Special cases

    • For a self-managed MySQL source database:

      • If a primary/secondary switchover occurs in the source database during synchronization, the task fails.

      • DTS calculates latency by comparing the timestamp of the last synchronized record with the current time. If no DML operations run for a long time in the source database, latency reporting may become inaccurate. If latency appears too high, run a DML operation in the source database to update the latency.

        Note

        If you select a full database for synchronization, create a heartbeat table. Update or write to this table every second.

      • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.

      • If your source database is Amazon Aurora MySQL or another clustered MySQL instance, ensure the domain name or IP address used in the task configuration—and its DNS resolution—always points to a read/write (RW) node. Otherwise, synchronization may fail.

    • For an ApsaraDB RDS for MySQL source database:

      • Read-only instances—such as ApsaraDB RDS for MySQL 5.6 read-only instances—that do not record transaction logs cannot serve as source databases.

      • DTS periodically runs the CREATE DATABASE IF NOT EXISTS `test` command in the source database to advance the binary log offset.