All Products
Search

This Product

    Data Transmission Service:MySQL as a source: Data synchronization user guide

    Document Center

    Data Transmission Service:MySQL as a source: Data synchronization user guide

    Last Updated:Dec 13, 2025

    This document guides you through the process of synchronizing data from a MySQL database using Alibaba Cloud Data Transmission Service (DTS). The process consists of seven core steps.

    Step 1: Preparations

    Before you configure the synchronization task, complete the following preparations to ensure that the task runs smoothly.

    1. (Optional) Connect to Alibaba Cloud: Ensure that your environment can connect to Alibaba Cloud. You can ignore this step if your Alibaba Cloud instance or environment supports public network access.

    2. Access authorization:

    3. Network and security settings: Ensure that the DTS server has the required permissions to access the source and destination databases. For a complete list of IP addresses for the destination region, see Add the IP addresses of DTS servers to a whitelist.

    4. Source and destination databases: For more information about how to configure the source and destination databases and view the limits, see MySQL as a source: Data synchronization limits.

    Step 2: Create and configure a synchronization task

    1. Log on to the Data Transmission Service (DTS) console.

    2. In the navigation pane on the left, click Data Synchronization. In the upper-left corner of the page, select the region where the data synchronization instance is located.

    3. You can click Create Task to open the task configuration page.

      1. Task Name: DTS automatically generates a name for the task, which does not need to be unique. We recommend that you specify a descriptive name for easy identification.

      2. Configure the source and destination databases:

        Source Database

        The configuration items vary depending on the Access Method. You can select the type that meets your needs.

        Alibaba Cloud Instance

        Configuration

        Description

        Example

        Select Existing Connection

        • If you want to use a database instance that is added to the system (newly created or saved), select the database instance from the drop-down list. The database information is automatically configured.

          Note

          If you are in the DMS console, the configuration item is Select a DMS database instance..

        • If the database instance is not added to the system, or you do not want to use an added database instance, manually configure the database information.

        image

        Database Type

        Select MySQL.

        Access Method

        Select Alibaba Cloud Instance.

        Instance Region

        Select the region where the source RDS for MySQL instance resides.

        Replicate Data Across Alibaba Cloud Accounts

        -

        Specifies whether to use a different Alibaba Cloud account. If you select Yes, you need to configure Alibaba Cloud Account and RAM Role Name.

        Alibaba Cloud Account

        Enter the Alibaba Cloud Account ID.

        RAM Role Name

        Enter the name of the RAM role.

        RDS Instance ID

        Select the ID of the source RDS for MySQL instance.

        Database Account

        Enter the account of the source RDS for MySQL instance.

        Database Password

        Enter the password for the database account.

        Encryption

        Please select Non-encrypted or SSL-encrypted based on your requirements.

        Express Connect, VPN Gateway, or Smart Access Gateway

        Configuration

        Description

        Example

        Select Existing Connection

        • If you want to use a database instance that is added to the system (newly created or saved), select the database instance from the drop-down list. The database information is automatically configured.

          Note

          If you are in the DMS console, the configuration item is Select a DMS database instance..

        • If the database instance is not added to the system, or you do not want to use an added database instance, manually configure the database information.

        image

        Database Type

        Select MySQL.

        Access Method

        Select Express Connect, VPN Gateway, or Smart Access Gateway.

        Instance Region

        Select the region where the VPC connected to the source database resides.

        Replicate Data Across Alibaba Cloud Accounts

        -

        Specify whether to use a cross-account configuration based on your requirements. If you select Yes, you must configure Alibaba Cloud Account and RAM Role Name.

        Alibaba Cloud Account

        Enter the Alibaba Cloud Account ID.

        RAM Role Name

        Enter the name of the RAM role.

        Connected VPC

        Select the VPC that is connected to the source database.

        Domain Name or IP

        Enter the endpoint of the source database.

        Port Number

        Enter the port of the source database.

        Database Account

        Enter the account of the source database.

        Database Password

        Enter the password for the database account.

        Encryption

        Select Non-encrypted or SSL-encrypted based on your requirements. If the source database has SSL-encrypted enabled, you must also upload a CA Certificate and enter the CA Key.

        Primary/Secondary vSwitch in VPC Tunnel

        Select the vSwitch (Primary) and vSwitch (Secondary) to allow DTS to access your data source through the VPC data channel.

        Public IP Address

        Configuration

        Description

        Example

        Select Existing Connection

        • If you want to use a database instance that is added to the system (newly created or saved), select the database instance from the drop-down list. The database information is automatically configured.

          Note

          If you are in the DMS console, the configuration item is Select a DMS database instance..

        • If the database instance is not added to the system, or you do not want to use an added database instance, manually configure the database information.

        image

        Database Type

        Select MySQL.

        Access Method

        Select Public IP Address.

        Instance Region

        Select a region that is close to the source database.

        Domain Name or IP

        Enter the endpoint of the source database.

        Port Number

        Enter the port of the source database.

        Database Account

        Enter the account of the source database.

        Database Password

        Enter the password for the database account.

        Encryption

        Select Non-encrypted or SSL-encrypted. If SSL-encrypted is enabled for the source database, you also need to upload the CA Certificate and enter the CA Key.

        Self-managed Database on ECS

        Configuration

        Description

        Example

        Select Existing Connection

        • If you want to use a database instance that is added to the system (newly created or saved), select the database instance from the drop-down list. The database information is automatically configured.

          Note

          If you are in the DMS console, the configuration item is Select a DMS database instance..

        • If the database instance is not added to the system, or you do not want to use an added database instance, manually configure the database information.

        image

        Database Type

        Select MySQL.

        Access Method

        Select Self-managed Database on ECS.

        Instance Region

        Select the region where the ECS instance resides.

        Replicate Data Across Alibaba Cloud Accounts

        -

        Specify whether to use resources across different Alibaba Cloud accounts. If you select Yes, you must configure Alibaba Cloud Account and RAM Role Name.

        Alibaba Cloud Account

        Enter the Alibaba Cloud Account ID.

        RAM Role Name

        Enter the name of the RAM role.

        ECS Instance ID

        Select the ECS instance ID.

        Port Number

        Enter the port of the self-managed MySQL database on the ECS instance.

        Database Account

        Enter the account of the source database.

        Database Password

        Enter the password for the database account.

        Encryption

        Select Non-encrypted or SSL-encrypted. If SSL-encrypted is enabled for the source database, you also need to upload the CA Certificate and enter the CA Key.

        Cloud Enterprise Network (CEN)

        Configuration

        Description

        Example

        Select Existing Connection

        • If you want to use a database instance that is added to the system (newly created or saved), select the database instance from the drop-down list. The database information is automatically configured.

          Note

          If you are in the DMS console, the configuration item is Select a DMS database instance..

        • If the database instance is not added to the system, or you do not want to use an added database instance, manually configure the database information.

        image

        Database Type

        Select MySQL.

        Access Method

        Select Cloud Enterprise Network (CEN).

        Instance Region

        Select the region where the source database resides.

        Replicate Data Across Alibaba Cloud Accounts

        -

        Specifies whether to use multiple Alibaba Cloud accounts. If you select Yes, you must configure Alibaba Cloud Account and RAM Role Name.

        Alibaba Cloud Account

        Enter Alibaba Cloud Account ID.

        RAM Role Name

        Enter the name of the RAM role.

        CEN Instance ID

        Select the CEN instance ID.

        Connected VPC

        Select the VPC in the CEN instance that is connected to the source database.

        Domain Name or IP

        Enter the endpoint of the source database.

        Port Number

        Enter the port of the source database.

        Database Account

        Enter the account of the source database.

        Database Password

        Enter the password for the database account.

        Encryption

        Select Non-encrypted or SSL-encrypted. If SSL-encrypted is enabled for the source database, you also need to upload the CA Certificate and enter the CA Key.

        Database Gateway

        Configuration

        Description

        Example

        Select Existing Connection

        • If you want to use a database instance that is added to the system (newly created or saved), select the database instance from the drop-down list. The database information is automatically configured.

          Note

          If you are in the DMS console, the configuration item is Select a DMS database instance..

        • If the database instance is not added to the system, or you do not want to use an added database instance, manually configure the database information.

        image

        Database Type

        Select MySQL.

        Access Method

        Select Database Gateway.

        Instance Region

        Select the region where the Database Gateway resides.

        Database Gateway ID

        Select the Database Gateway ID.

        Database Endpoint

        Select the database endpoint under the Database Gateway.

        Database Account

        Enter the account of the source database.

        Database Password

        Enter the password for the database account.

        Encryption

        Select Non-encrypted or SSL-encrypted as needed. If an SSL-encrypted is enabled for the source database, you must also upload a CA Certificate and enter the CA Key.

        Destination Database

        The Access Method for a source database can be different from the Access Method for a destination database. The options available in the console take precedence. The following section lists the parameters that you must configure for different Access Method.

        Note

        You cannot connect to the destination database using a Public IP Address. To do so, select a data migration task.

        Alibaba Cloud Instance

        Configuration

        Description

        Example

        Select Existing Connection

        • If you want to use a database instance that is added to the system (newly created or saved), select the database instance from the drop-down list. The database information is automatically configured.

          Note

          If you are in the DMS console, the configuration item is Select a DMS database instance..

        • If the database instance is not added to the system, or you do not want to use an added database instance, manually configure the database information.

        image

        Database Type

        Select a database type based on your needs. This topic uses MySQL as an example.

        Access Method

        Select Alibaba Cloud Instance.

        Instance Region

        Select the region where the destination database resides.

        Replicate Data Across Alibaba Cloud Accounts

        -

        Specify whether to operate across different Alibaba Cloud accounts based on your requirements. If you select Yes, you must configure Alibaba Cloud Account and RAM Role Name.

        Alibaba Cloud Account

        Enter the Alibaba Cloud Account ID.

        RAM Role Name

        Enter the name of the RAM role.

        RDS Instance ID

        Select the ID of the destination RDS for MySQL instance. If you select a different Database Type, select the instance ID of the corresponding type.

        Database Account

        Enter the account of the destination RDS for MySQL instance.

        Database Password

        Enter the password for the database account.

        Encryption

        Select Non-encrypted or SSL-encrypted based on your requirements.

        Express Connect, VPN Gateway, or Smart Access Gateway

        Configuration

        Description

        Example

        Select Existing Connection

        • If you want to use a database instance that is added to the system (newly created or saved), select the database instance from the drop-down list. The database information is automatically configured.

          Note

          If you are in the DMS console, the configuration item is Select a DMS database instance..

        • If the database instance is not added to the system, or you do not want to use an added database instance, manually configure the database information.

        image

        Database Type

        Select a database type based on your requirements. This topic uses MySQL as an example.

        Access Method

        Select Express Connect, VPN Gateway, or Smart Access Gateway.

        Instance Region

        Select the region where the VPC connected to the destination database resides.

        Connected VPC

        Select the VPC that is connected to the destination database.

        Domain Name or IP

        Enter the endpoint of the destination database.

        Port Number

        Enter the port of the destination database.

        Database Account

        Enter the account of the destination database.

        Database Password

        Enter the password for the database account.

        Encryption

        Select Non-encrypted or SSL-encrypted based on your requirements. If the target database has SSL-encrypted enabled, you also need to upload the CA Certificate and enter the CA Key.

        Primary/Secondary vSwitch in VPC Tunnel

        Select the vSwitch (Primary) and vSwitch (Secondary) that DTS uses to create a VPC data channel to access your data source.

        Self-managed Database on ECS

        Configuration

        Description

        Example

        Select Existing Connection

        • If you want to use a database instance that is added to the system (newly created or saved), select the database instance from the drop-down list. The database information is automatically configured.

          Note

          If you are in the DMS console, the configuration item is Select a DMS database instance..

        • If the database instance is not added to the system, or you do not want to use an added database instance, manually configure the database information.

        image

        Database Type

        Select a database type based on your needs. This topic uses MySQL as an example.

        Access Method

        Select Self-managed Database on ECS.

        Instance Region

        Select the region where the ECS instance resides.

        ECS Instance ID

        Select the ECS instance ID.

        Port Number

        Enter the port of the self-managed MySQL database on the ECS instance.

        Database Account

        Enter the account of the self-managed MySQL database.

        Database Password

        Enter the password for the database account.

        Encryption

        Select Non-encrypted or SSL-encrypted as needed. If SSL-encrypted is enabled for the target database, you also need to upload a CA Certificate and enter the CA Key.

        Cloud Enterprise Network (CEN)

        Configuration

        Description

        Example

        Select Existing Connection

        • If you want to use a database instance that is added to the system (newly created or saved), select the database instance from the drop-down list. The database information is automatically configured.

          Note

          If you are in the DMS console, the configuration item is Select a DMS database instance..

        • If the database instance is not added to the system, or you do not want to use an added database instance, manually configure the database information.

        image

        Database Type

        Select a database type based on your needs. This topic uses MySQL as an example.

        Access Method

        Select Cloud Enterprise Network (CEN).

        Instance Region

        Select the region where the destination database resides.

        CEN Instance ID

        Select the CEN instance ID.

        Connected VPC

        Select the VPC in the CEN instance that is connected to the destination database.

        Domain Name or IP

        Enter the endpoint of the destination database.

        Port Number

        Enter the port of the destination database.

        Database Account

        Enter the account of the destination database.

        Database Password

        Enter the password for the database account.

        Encryption

        Select Non-encrypted or SSL-encrypted as needed. If an SSL-encrypted is enabled for the target database, you also need to upload the CA Certificate and enter the CA Key.

        Database Gateway

        Configuration

        Description

        Example

        Select Existing Connection

        • If you want to use a database instance that is added to the system (newly created or saved), select the database instance from the drop-down list. The database information is automatically configured.

          Note

          If you are in the DMS console, the configuration item is Select a DMS database instance..

        • If the database instance is not added to the system, or you do not want to use an added database instance, manually configure the database information.

        image

        Database Type

        Select a database type based on your requirements. This topic uses MySQL as an example.

        Access Method

        Select Database Gateway.

        Instance Region

        Select the region where the Database Gateway resides.

        Database Gateway ID

        Select the Database Gateway ID.

        Database Endpoint

        Select the database endpoint under the Database Gateway.

        Database Account

        Enter the account of the destination database instance.

        Database Password

        Enter the password for the database account.

        Encryption

        Select Non-encrypted or SSL-encrypted based on your requirements. If the target database has SSL-encrypted enabled, you must also upload the CA Certificate and enter the CA Key.

    4. After you complete the configuration, click Test Connectivity and Proceed to verify the connectivity to the source and destination databases. If the CIDR Blocks of DTS Servers dialog box appears, add the IP addresses from the dialog box to the security settings of the source or destination database. Then, click Confirm authorization and test link. If the connection is successful, you are automatically redirected to configure the task object.

    Step 3: Configure synchronization objects and advanced options

    1. On the Configure Objects page, configure settings such as the synchronization type and the objects to synchronize. The available configuration items vary depending on the target database type.

      MySQL

      Configuration

      Description

      Example

      Synchronization Types

      Incremental Data Synchronization is always selected. By default, Schema Synchronization and Full Data Synchronization are also selected. After the precheck is complete, DTS synchronizes the full data of the selected objects from the source instance to the destination cluster. This data serves as the baseline for subsequent incremental data synchronization.

      image

      image

      Method to Migrate Triggers in Source Database

      Select a method to synchronize triggers as needed. If the objects to be synchronized do not involve triggers, you do not need to configure this parameter. For more information, see Configure a method to synchronize or migrate triggers.

      Note

      You can configure this option only when you set Synchronization Types to Schema Synchronization.

      Enable Migration Assessment

      Evaluate whether the structure of the source and destination databases (such as index length, stored procedures, and dependent tables) meets the requirements, and select Yes or No based on the actual situation.

      Note

      • This parameter can be configured only when Synchronization Types is set to Schema Synchronization.

      • If you select Yes, the precheck time may increase. You can view the Assessment Result during the precheck phase, and the assessment results do not affect the precheck results.

      Synchronization Topology

      Based on your business needs, you can select One-way Synchronization or Two-way Synchronization. Two-way Synchronization is supported only for specific source and destination endpoints. For more information, see Data Synchronization Topologies.

      Enable Exactly-Once Write

      If the objects to be synchronized include tables without primary keys or unique constraints, select Yes. For more information, see Synchronize tables without primary keys or unique constraints.

      Note

      This can be configured only when Synchronization Topology is set to Two-way Synchronization.

      Exclude DDL Operations

      • If you select Yes: DDL operations are not synchronized.

      • Select No: DDL operations are synchronized.

      Note

      • This can be configured only when Synchronization Topology is set to Two-way Synchronization.

      • To ensure the stability of the two-way synchronization link, you can select whether to synchronize DDL operations only for the forward task (from source to destination). For the reverse task (from destination to source), DDL operations are automatically filtered.

      Global Conflict Resolution Policy

      If a conflict occurs, select an appropriate conflict resolution policy as needed.

      Click to expand and view supported conflict detection types

      To ensure data consistency, you must make sure that a record with the same primary key, business primary key, or unique key is updated at only one node of the two-way synchronization. If the record is updated at both nodes at the same time, the system responds based on the conflict resolution policy that you configured for the data synchronization job.

      DTS uses conflict detection and resolution to maintain the stability of two-way synchronization instances. DTS can detect the following types of conflicts:

      • Uniqueness conflicts caused by INSERT

        A uniqueness constraint is violated when an INSERT statement is synchronized. For example, if the two nodes of a two-way synchronization INSERT a record with the same primary key value at the same time or at a very close time, the INSERT synchronization fails when the record is synchronized to the other node because a record with the same primary key value already exists.

      • Incomplete match for records to be updated by UPDATE

        • When the record to be updated by an UPDATE operation does not exist in the destination instance, DTS automatically converts the operation to an INSERT operation. This may cause a unique key conflict.

        • A primary key or unique key conflict occurs for the record to be updated by an UPDATE operation.

      • Record to be deleted by DELETE does not exist

        The record to be deleted by a DELETE operation does not exist in the destination instance. When this conflict occurs, DTS automatically ignores the DELETE operation, regardless of the configured conflict resolution policy.

      Important

      • Due to factors such as differences in system time between the two ends of data synchronization and synchronization latency, DTS cannot guarantee that the conflict detection mechanism can completely prevent data conflicts. When you use two-way synchronization, you need to make corresponding modifications at the business layer to ensure that a record with the same primary key, business primary key, or unique key is updated at only one node of the two-way synchronization.

      • DTS provides resolution policies for the preceding data synchronization conflicts. You can select a policy when you configure two-way synchronization.

      • Overwrite (If a conflict occurs, the conflicting record in the destination instance is overwritten): If a conflict occurs during data synchronization, the conflicting record in the destination database is overwritten.

      • TaskFailed (If a conflict occurs, an error is reported and the task is stopped): When a conflict occurs during data synchronization, the sync task immediately reports an error and exits. The sync task enters a failed state, and you must intervene to fix the task.

      • Ignore (If a conflict occurs, the conflicting record in the destination instance is used): When a data synchronization conflict occurs, the system skips the current synchronization statement, continues execution, and uses the conflicting record from the destination database.

      Note

      • This can be configured only when Synchronization Topology is set to Two-way Synchronization.

      • If the synchronization task is paused or restarted and there is a delay, these policies do not take effect during the delay. The data in the destination is overwritten by default.

      Processing Mode of Conflicting Tables

      • Precheck and Report Errors: Checks whether a table with the same name exists in the destination database. If no such table exists, the precheck is passed. Otherwise, an error is returned during the precheck, and the data synchronization task cannot be started.

        Note

        If it is not convenient to delete or rename the table with the same name in the destination database, you can change the name of the table in the destination database. For more information, see Map table and column names.

      • Ignore Errors and Proceed: Skips the check to determine whether a table with the same name exists in the target database.

        Warning

        If you select Ignore Errors and Proceed, it may cause data inconsistency and introduce risks to your business. For example:

        • If the table schemas are consistent and a record in the destination database has the same primary key or unique key value as a record in the source database:

          • During full synchronization, DTS retains the record in the destination cluster. The record from the source database is not synchronized to the destination database.

          • During incremental synchronization, DTS does not retain the record in the destination cluster. The record from the source database overwrites the record in the destination database.

        • If the table schemas are inconsistent, it may not be possible to initialize data, only some columns of data can be synchronized, or the synchronization may fail. Proceed with caution.

      Whether to migrate Event

      Select whether to synchronize events from the source database. If you select Yes, you must also follow the relevant requirements and perform the subsequent operations. For more information, see Synchronize or migrate events.

      Capitalization of Object Names in Destination Instance

      You can configure the case policy for the names of databases, tables, and columns for objects synchronized to the destination instance. By default, DTS default policy is selected. You can also select Consistent with the source database or Consistent with the default policy of the destination database. For more information, see Case policy for destination object names.

      Source Objects

      In the Source Objects box, click the objects to sync, and then click Right arrow to move them to the Selected Objects box.

      Note

      The granularity for selecting synchronization objects is database, table, and column. If you select a table or column as the synchronization object, other objects such as views, triggers, and stored procedures are not synchronized to the destination database.

      Selected Objects

      Note

      • If you use the object name mapping feature, other objects that depend on the mapped object may fail to be synchronized.

      • To select SQL operations for synchronization at the database or table level, right-click the object to be synchronized in the Selected Objects section, and in the dialog box that appears, select the desired SQL operations.

      • To set a WHERE condition to filter data, right-click the table to be synchronized in Selected Objects, and set the filter condition in the dialog box that appears. For more information, see Set filter conditions.

      Kafka

      Configuration

      Description

      Example

      Synchronization Types

      Incremental Data Synchronization is always selected. By default, Schema Synchronization and Full Data Synchronization are also selected. After the precheck is complete, DTS synchronizes the full data of the selected objects from the source instance to the destination cluster. This data serves as the baseline for subsequent incremental data synchronization.

      Note

      If the Access Method for the target Kafka instance is Alibaba Cloud Instance, Schema Synchronization is not supported.

      image

      image

      Processing Mode of Conflicting Tables

      • Precheck and Report Errors: Checks whether a table with the same name exists in the destination database. If no such table exists, the precheck is passed. Otherwise, an error is returned during the precheck, and the data synchronization task cannot be started.

        Note

        If it is not convenient to delete or rename the table with the same name in the destination database, you can change the name of the table in the destination database. For more information, see Map table and column names.

      • Ignore Errors and Proceed: Skips the check to determine whether a table with the same name exists in the target database.

        Warning

        If you select Ignore Errors and Proceed, it may cause data inconsistency and introduce risks to your business. For example:

        • If the table schemas are consistent and a record in the destination database has the same primary key or unique key value as a record in the source database:

          • During full synchronization, DTS retains the record in the destination cluster. The record from the source database is not synchronized to the destination database.

          • During incremental synchronization, DTS does not retain the record in the destination cluster. The record from the source database overwrites the record in the destination database.

        • If the table schemas are inconsistent, it may not be possible to initialize data, only some columns of data can be synchronized, or the synchronization may fail. Proceed with caution.

      Data Format in Kafka

      Select the data storage format for data synchronized to the Kafka instance.

      Synchronize all fields

      Select whether to synchronize all fields.

      Kafka Data Compression Format

      Select a compression format for Kafka messages based on your requirements.

      • LZ4 (default): Provides fast compression with a lower compression ratio.

      • GZip: Provides a high compression ratio but has a low compression speed, which results in high CPU usage.

      • Snappy: Offers a medium compression ratio and speed.

      Policy for Shipping Data to Kafka Partitions

      Select a policy based on your requirements.

      Message acknowledgement mechanism

      Select a message acknowledgment mechanism based on your requirements.

      Topic That Stores DDL Information

      From the drop-down list, select a topic to store DDL information.

      Note

      If you do not select a topic, DDL information is stored in the topic that receives data by default.

      Capitalization of Object Names in Destination Instance

      You can configure the case policy for the names of databases, tables, and columns for objects synchronized to the destination instance. By default, DTS default policy is selected. You can also select Consistent with the source database or Consistent with the default policy of the destination database. For more information, see Case policy for destination object names.

      Source Objects

      In the Source Objects box, click the objects to sync, and then click Right arrow to move them to the Selected Objects box.

      Note

      The granularity for selecting synchronization objects is database, table, and column. If you select a table or column as the synchronization object, other objects such as views, triggers, and stored procedures are not synchronized to the destination database.

      Selected Objects

      No additional configuration is required for this example. You can use the mapping feature to set the topic name for the source table in the destination Kafka instance, and select the DML and DDL operations or column names to be synchronized.

      Note

      • To select the SQL operations to sync for a database or table, right-click the object in the Selected Objects section and select the operations in the dialog box that appears.

      • If you use the object name mapping feature, other objects that depend on the mapped object may fail to be synchronized.

      AnalyticDB for MySQL 3.0

      Configuration

      Description

      Example

      Synchronization Types

      Incremental Data Synchronization is always selected. By default, Schema Synchronization and Full Data Synchronization are also selected. After the precheck is complete, DTS synchronizes the full data of the selected objects from the source instance to the destination cluster. This data serves as the baseline for subsequent incremental data synchronization.

      Note

      If Full Data Synchronization is selected, tables created with CREATE TABLE statements, including their schemas and data, are synchronized to the destination database.

      image

      image

      DDL and DML Operations to Be Synchronized

      Select the DML and DDL operations to be synchronized at the instance level. By default, all operations are selected.

      Note

      To select the SQL operations to synchronize for a database or table, right-click the synchronization object in the Selected Objects pane and select the desired SQL operations from the shortcut menu.

      Merge Tables

      Select whether to synchronize multiple tables with the same schema from the source database (sharded tables) to a single table in the destination database.

      • Yes: In online transactional processing (OLTP) scenarios, sharding is often used to improve the responsiveness of business tables. However, in online analytical processing (OLAP) scenarios, you can store a large amount of data in a single table in the destination database to make single-table queries more convenient. In such scenarios, you can use the multi-table merge feature of DTS to synchronize data from multiple source tables that have the same schema (the sharded tables) to a single table in the destination database. For more information, see Enable multi-table merge.

        Note

        • After selecting multiple tables from the source database, you need to use the object name mapping feature to change their names to the same table name in the destination database. For more information about the object name mapping feature, see Map table and column names.

        • DTS adds the __dts_data_source column (of the text type) to the destination table in the destination database to store the data source. The column value is in the DTS instance ID:database name:schema name:table name format to identify the data source. Example: dts********:dtstestdata:testschema:customer1.

        • The multi-table merging feature is at the task level, which means that it does not support merging at the table level. To merge some tables but not others, you must create two separate data synchronization tasks.

        Warning

        Do not perform DDL operations that change the schema of a database or table in the source database. Otherwise, data inconsistency or synchronization task failure may occur.

      • No is the default option.

      Processing Mode of Conflicting Tables

      • Precheck and Report Errors: checks whether the destination database contains tables that have the same names as tables in the source database. If the source and destination databases do not contain tables that have identical table names, the precheck is passed. Otherwise, an error is returned during the precheck, and the data synchronization task cannot be started.

        Note

        If the source and destination databases contain tables with identical names and the tables in the destination database cannot be deleted or renamed, you can use the object name mapping feature to rename the tables that are synchronized to the destination database. For more information, see Map object names.

      • Ignore Errors and Proceed: skips the precheck for identical table names in the source and destination databases.

        Warning

        If you select Ignore Errors and Proceed, data inconsistency may occur and your business may be exposed to potential risks.

        • If the source and destination databases have the same schema and a data record in the destination database has the same primary key value or unique key value as a data record in the source database:

          • During full data synchronization, DTS does not synchronize the data record to the destination database. The existing data record in the destination database is retained.

          • During incremental data synchronization, DTS synchronizes the data record to the destination database. The existing data record in the destination database is overwritten.

        • If the source and destination databases have different schemas, data may fail to be initialized. In this case, only some columns are synchronized, or the data synchronization instance fails. Proceed with caution.

      Capitalization of Object Names in Destination Instance

      You can configure the case policy for the names of databases, tables, and columns for objects synchronized to the destination instance. By default, DTS default policy is selected. You can also select Consistent with the source database or Consistent with the default policy of the destination database. For more information, see Case policy for destination object names.

      Source Objects

      In the Source Objects box, click the objects to sync, and then click Right arrow to move them to the Selected Objects box.

      Note

      • The granularity for selecting synchronization objects is database, table, and column. If you select a table or column as the synchronization object, other objects such as views, triggers, and stored procedures are not synchronized to the destination database.

      • If you select an entire database as the synchronization object, the following default behavior applies:

        • If a table to be synchronized in the source database has a primary key (single-column or multi-column), that primary key column is used as the distribution key.

        • If a table to be synchronized in the source database does not have a primary key, an auto-increment primary key column is automatically generated. This may cause data inconsistency between the source and destination databases.

      Selected Objects

      Note

      • To select the SQL operations to sync for a database or table, right-click the object in the Selected Objects section and select the required SQL operations from the dialog box.

      • To filter data, right-click the table to be synchronized in Selected Objects and set the filter condition in the dialog box that appears. For more information, see Set filter conditions.

    2. Click Next to continue to Advanced Settings. The default configurations are sufficient for most scenarios.

      Click to expand and view detailed configuration descriptions

      Note

      The configuration items vary based on the destination database type. The options displayed in the console take precedence.

      Configuration

      Description

      Example

      Dedicated Cluster for Task Scheduling

      By default, DTS schedules tasks on a shared cluster, so you do not need to make a selection. For more stable task performance, you can purchase a dedicated cluster to run DTS synchronization tasks. For more information, see What is a DTS dedicated cluster?.

      image

      Copy the temporary table of the Online DDL tool that is generated in the source table to the destination database.

      If you use Data Management (DMS) or gh-ost to perform online DDL changes in the source database, you can choose whether to synchronize the temporary table data generated by the online DDL changes.

      Important

      DTS tasks do not support using tools like pt-online-schema-change to perform online DDL changes. Doing so will cause the DTS task to fail.

      • Yes: Synchronizes data in temporary tables generated by online DDL changes.

        Note

        If the temporary table data generated by online DDL changes is too large, it may cause synchronization task latency.

      • No, Adapt to DMS Online DDL: Temporary table data generated by Online DDL operations is not synchronized. Only the original DDL statements that are executed on the source database through Data Management (DMS) are synchronized.

        Note

        This option will cause locked tables in the destination database.

      • No, Adapt to gh-ost: Does not sync data from temporary tables generated by online DDL changes. Instead, it syncs only the original DDL statements executed by gh-ost in the source database. Additionally, you can use the default regular expressions or configure your own for gh-ost shadow tables and discarded tables.

        Note

        This option will cause locked tables in the destination database.

      Whether to Migrate Accounts

      You can specify whether to synchronize source database account information. If you select Yes, you also need to select the accounts to synchronize and confirm the account permissions. For more information, see Migrate database accounts.

      Retry Time for Failed Connections

      After a synchronization task starts, if the connection to the source or destination database fails, DTS reports an error and immediately begins to continuously retry the connection. The default retry duration is 720 minutes. You can also customize the retry duration within the range of 10 to 1440 minutes. We recommend that you set it to 30 minutes or more. If DTS reconnects to the source or destination database within the specified retry duration, the synchronization task automatically resumes. Otherwise, the synchronization task fails.

      Note

      • For multiple DTS instances with the same source or destination, such as DTS instance A and DTS instance B, if you set the network retry duration to 30 minutes for A and 60 minutes for B, the shorter duration of 30 minutes applies.

      • Because DTS charges for task runtime during connection retries, we recommend that you customize the retry duration based on your business needs or release the DTS instance as soon as possible after the source and destination database instances are released.

      Retry Time for Other Issues

      After a synchronization task starts, if a non-connectivity issue (such as a DDL or DML execution error) occurs in the source or destination database, DTS reports an error and immediately begins to continuously retry the operation. The default retry duration is 10 minutes. You can also customize the retry duration within the range of 1 to 1440 minutes. We recommend that you set it to 10 minutes or more. If the relevant operation succeeds within the specified retry duration, the synchronization task automatically resumes. Otherwise, the synchronization task fails.

      Important

      The value of Retry Time for Other Issues must be less than that of Retry Time for Failed Connections.

      Enable Throttling for Full Data Synchronization

      During full data synchronization, DTS uses the read and write resources of the source and destination databases, which may increase the load on the database servers. To reduce the load on the destination database, you can limit the rate of the full sync task by setting the Queries per second (QPS) to the source database, RPS of Full Data Migration, and Data migration speed for full migration (MB/s).

      Note

      • This configuration item is available only when Synchronization Types is set to Full Data Synchronization.

      • You can also adjust the full synchronization speed after the synchronization instance is running.

      Enable Throttling for Incremental Data Synchronization

      To reduce the load on the destination database, you can also set a rate limit for the incremental synchronization task by setting RPS of Incremental Data Synchronization and Data synchronization speed for incremental synchronization (MB/s).

      Whether to delete SQL operations on heartbeat tables of forward and reverse tasks

      Select whether to write heartbeat SQL information to the source database while the DTS instance is running.

      • Yes: Heartbeat SQL information is not written to the source database. This may cause the DTS instance to show a delay.

      • No: Heartbeat SQL information is written to the source database. This can affect source database features such as physical backup and cloning.

      Environment Tag

      You can select an environment tag to identify the instance if required.

      Scene Label

      You can select an environment tag to identify the instance if required.

      Configure ETL

      Select whether to configure the extract, transform, and load (ETL) feature. For more information about ETL, see What is ETL?.

      Sensitive Data Scanning and Masking

      Select whether to enable the sensitive data scan and desensitization feature to identify and process sensitive information in the transmitted data. For more information, see Sensitive data scan and desensitization.

      • Open

      • Close

      Monitoring and Alerting

      Select whether to set up alerts. If the synchronization fails or the latency exceeds the threshold, a notification is sent to the alert contact.

      • No: No alerts are set.

      • Yes: Enables the alert feature. You must also specify an alert threshold and an alert notification. For more information, see Configure Monitoring.

    Step 4: (Optional) Configure data validation

    If the target database type supports data validation, click Data Verification to configure the task. Otherwise, you are automatically advanced to Step 5: Precheck and Purchase.

    1. Select Data Verification Mode:

      Note

      • Full Data Verification: Verifies the data of a full data task. If the instance does not include an incremental task, verification starts after the instance is created and the full data migration is complete. If the instance includes an incremental task, verification starts after the instance is created and the first incremental data write occurs with zero latency.

      • Incremental Data Verification: Verifies the data of incremental tasks. Verification starts after the instance is created and when incremental data is first written without delay.

      • Schema Verification: Verifies the structure of the selected objects. If the instance does not include an incremental task, verification starts after the instance is created and the database, table schema, and full data are written. If the instance includes an incremental task, verification starts after the instance is created and the incremental data first achieves zero latency.

      Full validation

      If you select Full Data Verification, you must also configure the parameters described in the following table.

      Parameter

      Description

      Example

      Full Data Verification

      • Full field validation by row sampling: Validates all fields for a percentage of sampled rows. The sampling percentage must be an integer from 10 to 100.

      • Verify based on the number of table rows: Validates the number of rows for all data in the task, but does not validate the content of the data.

      Note

      The Verify based on the number of table rows full data verification mode is free of charge. The Full field validation by row sampling full data verification mode is charged based on the actual verified data volume.

      image

      Timeout Settings for Full Data Verification

      • No: The full data verification job will not be forcibly terminated if it times out.

      • Yes: Sets a timeout period for the full data verification job. The countdown begins after the job starts. If the job does not complete within the specified time, it is forcibly terminated. Valid values: 1 to 72. The value must be an integer.

      Full calibration reference

      • Default: Uses the combined data from the source and destination databases as the baseline to verify data consistency.

      • Source Database: Uses the source database as the baseline to verify data consistency. (Data that exists in the destination database but not in the source database is not verified.)

      • Destination Database: Uses the destination database as the baseline to verify data consistency between the source and destination databases (extra data in the source database is not verified).

      Maximum number of rows of data read per second by full verification (RPS)

      Full data validation consumes some read resources of the database. You can set a speed limit for the full validation task (rows and data volume read per second) to reduce the pressure on the database.

      Note

      A value of 0 indicates no limit. If both the Maximum number of rows of data read per second by full verification (RPS) and The maximum number of bytes read per second by full verification Byte/s. parameters are set to 0, the speed is not limited.

      The maximum number of bytes read per second by full verification Byte/s.

      Incremental validation

      If you select Incremental Data Verification, you must also set the parameters as described in the following table.

      Parameter

      Description

      Example

      Incremental Verification Benchmark

      You can select the DML operations to verify as needed. The supported operations are Source INSERT Operation, Source UPDATE Operation, and Source DELETE Operation.

      image

    2. Set Verification Objects: All objects are selected by default. You can remove the objects on the right that do not need to be verified.

    3. Configure validation alerting: Configure the parameters in the following table based on your requirements.

      Note

      You can also set or modify data validation alerts after the DTS instance is running.

      Parameter

      Description

      Example

      Full Data Verification Alert

      • No: Do not set alerts.

      • Yes: Configure alerting. You also need to select and configure alert rules. The alert rules are as follows:

        • Trigger an alert when the full validation task fails.

        • Set a threshold for data inconsistency. Trigger an alert when the number of inconsistent data points in the full validation task is greater than or equal to the set threshold.

      image

      Incremental Data Verification Alert

      • No: No alerts are set.

      • Yes: Configure alerting. You also need to select and configure alert rules. The alert rules are as follows:

        • Trigger an alert when the incremental validation task fails.

        • Set the number of periods, statistical period, and inconsistency count threshold. Trigger an alert when the number of inconsistent records in the incremental validation task is greater than or equal to the set threshold for the specified number of consecutive periods.

        • Set the number of periods, statistical period, and latency threshold. Trigger an alert when the latency of the incremental validation task is greater than or equal to the set threshold for the specified number of consecutive periods.

      Note

      If you set up validation alerts and need to receive alert notifications when a validation task triggers an alert, you must also configure alert message subscriptions on the CloudMonitor platform. For more information, see Set an alert rule for a DTS task on the CloudMonitor platform.

    Step 5: Precheck and purchase

    1. Save the task and perform a precheck to verify that the configurations, permissions, and network settings of the source and destination databases meet the synchronization requirements. If the precheck is successful, click Next: Purchase Instance.

      • To view the API parameters for configuring the instance, hover over the Next: Save Task Settings and Precheck button and click Preview OpenAPI parameters in the pop-up.

      • When you have finished reviewing the API parameters, click Next: Save Task Settings and Precheck at the bottom of the page.

      Note

      • Before the synchronization job officially starts, a precheck is performed. The synchronization job can start only after it passes the precheck.

      • If the precheck fails, click View Details for the failed item, resolve the issue as prompted, and then run the precheck again.

      • If a warning is generated during the precheck:

        • For failed check items that cannot be ignored, click View Details, fix the issue as prompted, and then rerun the precheck.

        • For check items that can be ignored without requiring a fix, you can click Confirm Alert Details, Ignore, OK, and Precheck Again in sequence to skip the item and rerun the precheck. If you ignore an alert item, data inconsistency may occur and your business may be exposed to potential risks.

      image

    2. On the purchase page, configure the billing method and link specification.

      • Billing method: The supported billing methods are Subscription, Pay-as-you-go, and Pay-as-you-go Serverless. For more information, see Billing methods.

      • Instance Class: Specifies the performance limit (RPS) for incremental synchronization. You can select a specification, such as small or medium, based on the frequency of data changes in the source database. For more information, see Specifications.

      • After you complete the configuration, read and select the Data Transmission Service (Pay-as-you-go) Service Terms.

      • Click Buy and Start. In the OK dialog box, click OK to purchase and automatically start the task.

    image

    Step 6: (Optional) Configure a remote sync task

    1. Wait until the forward sync task completes its initial synchronization and its Status is Running.

    2. Navigate to the remote sync task and click Configure Task.image

    3. Configure the remote sync task by following the steps above.

      Important

      • When you configure a remote sync task, you need to select the correct source and destination instances. In a remote sync task, the source instance is the destination instance of the forward synchronization task, and the destination instance is the source instance of the forward synchronization task. You must also carefully confirm that the instance information, such as the database name, account, and password, are consistent.

      • For a remote sync task, the Instance Region of the source and destination databases cannot be modified, and it requires fewer configuration parameters than a forward sync task. The actual parameters are displayed in the console.

      • The Processing Mode of Conflicting Tables setting for a remote sync task does not detect tables synced to the destination instance by the forward sync task.

      • Remote sync tasks cannot sync the objects in the Selected Objects section of the source task.

      • Do not use the mapping feature when you configure the remote sync task. Otherwise, data inconsistency may occur.

    4. When the Success Rate is 100%, click Back.

    Step 7: Monitor and verify

    After the task starts, the system automatically executes stages such as Incremental Data Collection, Schema Migration, Full Data Synchronization, Incremental Write, and Full Data Verification based on your task configuration.

    • View status: In the synchronization task list, you can view the current phase and running status of the task. When the status changes to Incremental Synchronization and the latency is stable in seconds, it indicates that the historical data has been synchronized. DTS is now synchronizing incremental data in real time.

    • Data validation: You can view data consistency in the Check Details section on a data verification tab, such as Full Data Verification or Schema Verification.

    image