This topic describes how to enable the data verification feature for a Data Transmission Service (DTS) task. The data verification feature helps you detect inconsistent data between the source and destination databases at the earliest opportunity.

Prerequisites

  • Full Data Synchronization or Full Data Migration is selected as Task Stages if you need to enable full data verification for a DTS task.
  • Incremental Data Synchronization or Incremental Data Migration is selected as Task Stages if you need to enable incremental data verification for a DTS task.

Usage notes

  • Data verification is not supported for database instances that are deployed on the classic network.
  • Data verification is not supported for DTS tasks that enable the multi-table merging feature.
  • If a table to be verified contains neither a primary key nor a unique index and the table contains more than 10,000 rows of data, DTS does not perform data verification on the table.
  • If the data to be verified has neither a primary key nor the UNIQUE constraint, DTS does not perform incremental data verification.
  • If you pause and then restart a data synchronization or migration task, the full data verification task configured for the DTS task is also restarted to perform data verification again.
  • Before you perform data verification, you must evaluate the performance of the source and destination databases. We recommend that you perform data verification during off-peak hours. For example, the CPU load of the source and destination databases is less than 30%. During data verification, DTS uses the read and write resources of the source and destination databases. This may increase the loads of the database servers. If you perform data verification when the database performance is poor, the database specifications are low, or the business volume is large, this causes high pressure on the databases. The database services may become unavailable.

Supported regions

  • Full data verification: All regions and cross-region tasks are supported. For more information about the supported regions, see Supported regions.
  • Incremental data verification: The China (Hangzhou), China (Shanghai), China (Qingdao), China (Beijing), and China (Zhangjiakou) regions are supported. The instances used for cross-region tasks are not supported.

Supported instances

  • Full data verification:
    • DTS instances that are used to synchronize or migrate data and whose source or destination database is an ApsaraDB for MySQL instance, an AnalyticDB for MySQL instance, a PolarDB for MySQL cluster, an ApsaraDB RDS for SQL Server instance, an ApsaraDB RDS for PostgreSQL instance, an AnalyticDB for PostgreSQL instance, a PolarDB for PostgreSQL cluster, or a self-managed Oracle database.
    • DTS instances that are used to synchronize or migrate data between ApsaraDB for MongoDB instances.
    • DTS instances that are used to synchronize or migrate data between ApsaraDB for Redis Enhanced Edition (Tair) and Redis databases.
    • DTS instances that are used to synchronize or migrate data from a self-managed Oracle database to a PolarDB for Oracle cluster.
    • DTS instances that are used to synchronize or migrate data from a Db2 for i (AS/400) database to a MySQL instance.
  • Incremental data verification:
    • DTS instances that are used to synchronize or migrate data and whose source or destination database is an ApsaraDB for MySQL instance, an AnalyticDB for MySQL instance, a PolarDB for MySQL cluster, an ApsaraDB RDS for SQL Server instance, an ApsaraDB RDS for PostgreSQL instance, an AnalyticDB for PostgreSQL instance, or a PolarDB for PostgreSQL cluster.
    • DTS instances that are used to migrate data from a self-managed Oracle database to an ApsaraDB RDS for MySQL instance, an AnalyticDB for MySQL instance, a PolarDB for MySQL cluster, or a PolarDB for Oracle cluster.
    • DTS instances that are used to synchronize data from a self-managed Oracle database to a PolarDB for Oracle cluster.
Note
  • Data verification is supported in the two-way data synchronization tasks between the preceding source and destination databases.
  • The preceding source and destination databases support Alibaba Cloud instances and self-managed database instances.

Configure the data verification feature for a data synchronization or migration task in DTS

  1. Go to the Advanced Settings step.
    • For more information about how to go to the Advanced Settings step when you create a DTS task, see the topics listed in Overview of data synchronization scenarios or Overview of data migration scenarios.
    • For an existing DTS task, perform the following steps:
      1. Go to the Data Synchronization or Data Migration page in the DTS console.
      2. Find the task for which you want to enable data verification and click the name of the task.
      3. On the Basic Information or Task Management page, click Create Verification Task.
  2. Select data verification modes.

    Select one or more data verification modes based on your business requirements. If you select Full Data Verification, you must configure the parameters that are described in the following table.

    Note
    • Full Data Verification: verifies the required data in the full data synchronization or migration task. Data verification starts after the data synchronization or migration task is created and full data is written to the destination database.
    • Incremental Data Verification: verifies the required data in the incremental data synchronization or migration task. Data verification starts after the data synchronization or migration task is created and incremental data is written to the destination database without latency for the first time.
    ParameterDescription
    Full Data Verification
    • Verify hash values based on the sampling ratio: verifies all fields of the data sampled based on the specified sampling percentage. You can specify an integer that ranges from 10 to 100 in the field as the sampling percentage.
    • Verify based on the number of table rows: verifies the number of table rows in the full data synchronization or migration task.
    Note If the full data verification mode is Verify based on the number of table rows, you are not charged. If the full data verification mode is Verify hash values based on the sampling ratio, you are charged based on the actual amount of data to be verified. You are not charged during the public preview phase.
    Full Data Verification Time RuleYou can set this parameter only to Start Now.
    Timeout Settings for Full Data Verification
    • No: does not specify a timeout period for full data verification. The system does not forcibly stop full data verification if the verification times out.
    • Yes: specifies a timeout period for full data verification. The system starts to count the duration when full data verification starts. If full data verification is not complete within the specified timeout period, the system forcibly stops the verification. The value must be an integer that ranges from 1 to 72.
  3. Specify the verification objects.
    You can select objects that do not require data verification in the Selected Objects section, and then click the Remove icon icon to remove the objects.
    Note By default, the objects to be synchronized or migrated are moved to the Selected Objects section.
  4. Configure verification alerts.

    Configure the parameters for configuring verification alerts based on your business requirements. The following table describes the parameters.

    ParameterDescription
    Full Data Verification Alert
    • No: does not configure alerts.
    • Yes: configures alerts. You must also select and configure alert rules. The following information describes the alert rules:
      • An alert is triggered if full data verification fails.
      • An alert is triggered if the volume of inconsistent data detected by full data verification is greater than or equal to the specified threshold.
    Incremental Data Verification Alert
    • No: does not configure alerts.
    • Yes: configures alerts. You must also select and configure alert rules. The following information describes the alert rules:
      • An alert is triggered if incremental data verification fails.
      • An alert is triggered if the volume of inconsistent data detected by incremental data verification within the specified consecutive periods is greater than or equal to the specified threshold. You can specify the number of consecutive periods, the statistical period, and the threshold of the inconsistent data volume.
      • An alert is triggered if the latency of the data migration or synchronization detected by incremental data verification within the specified consecutive periods is greater than or equal to the specified threshold. You can specify the number of consecutive periods, the statistical period, and the latency of the data migration or synchronization.
    Note If you set the Full Data Verification Alert or Incremental Data Verification Alert parameter to Yes, you must also set the Alert Contact parameter to the mobile phone number of the contact. If an alert is triggered upon data verification, DTS sends alert notifications to the contact by using text messages.
  5. Complete the subsequent steps as prompted.