Data Transmission Service:View data verification details

Prerequisites

A data verification task is created and configured. Configure data verification.

Usage notes

• Verification tasks in a synchronization or migration instance appear on both the instance details page and the Data Verification page. Standalone tasks appear only on the Data Verification page.

• View Check Details for full and schema verification tasks only after their status is Complete. For incremental verification tasks, view Check Details only when the Current Offset is after the time of the incremental data change.

• Incremental verification merges multiple changes to the same record within a period before checking consistency. The displayed record count may be lower than the actual number of incremental changes.

• Inconsistent data is retained for 30 days by default. Review and resolve inconsistencies promptly.

• If a full verification task detects inconsistencies, correct the data and run a manual verification or re-run the task to confirm consistency.

• DTS automatically re-verifies inconsistencies in incremental verification tasks approximately every 10 minutes. The interval may increase with large volumes of inconsistent data.

• You are charged for incremental validation tasks even when they are paused.

View details on the Data Verification page

1. Go to the data verification task list page. Choose one of the following methods.

   DTS console (quick)

   1. Go to the Data Verification page.

   2. Optional: In the upper-left corner of the page, select the region of the verification instance.

   DTS console (standard)

   1. Log on to the Data Transmission Service (DTS) console.

   2. In the left-side navigation pane, click Data Verification.

   3. Optional: In the upper-left corner of the page, select the region of the verification instance.

   DMS console

   Note

   The actual steps may vary based on the mode and layout of the DMS console. For more information, see Simplified mode console and Customize the layout and style of the DMS interface.

   1. Log on to the Data Management Service (DMS) console.

   2. In the top menu bar, choose Data + AI > Data Transmission (DTS) > Data Verification.

   3. Optional: To the right of Verification Task, select the region of the verification instance.

2. Click the instance ID to open its details page.

3. Optional: In the left-side navigation pane, click Task Management.

4. In the Progress section, click Full Data Verification, Incremental Data Verification, or Schema Verification to view the verification details.

   • Basic Information

     The Basic Information tab shows the task start time, verification time rule, status, progress, and error messages. Displayed content varies by task configuration.

     After the task starts, you can Restart, Pause, Stopped, or Delete the task. What to do next.

     Important

     The Status on the Basic Information tab reflects the task status. View verification results on the Check Details tab.

   • Check Details

     The Check Details tab shows source and destination database/table names, verification status, and verification time.

     Note

     • The results on the Check Details tab are sorted in descending order by the Number of Inconsistent Records.

     • You can download a revised SQL statement only for full data verification tasks. Tasks started before 14:04:52 (UTC+8) on November 20, 2023 must be re-run to enable this feature.

     • The Total Number of Records is an estimated value, while the Number of records completed by verification is the actual count of verified records. The Number of records completed by verification may be higher than the Total Number of Records.

     • If the Number of Inconsistent Records is 0, source and destination data is consistent. No report data is generated.

     • If the Number of Inconsistent Records is not 0, data or schema inconsistencies exist between source and destination. In the Actions column, click View Report, Download Report, or Download Revised SQL Statements to manually fix the inconsistencies using the downloaded report and revised SQL statement.

       Important

       Use caution when executing DELETE statements from a revised SQL statement.

View details in synchronization or migration instances

1. Go to the data synchronization or data migration task list using one of the following methods.

   From the DTS console

   1. Log in to the Data Transmission Service (DTS) console.

   2. In the left-side navigation pane, click Data Synchronization or Data Migration.

   3. Optional: In the upper-left corner, select the instance region.

   From the DMS console

   Note

   Steps may vary by DMS console mode and layout. Simple mode console, Customize the layout and style of the DMS console.

   1. Log in to the Data Management Service (DMS) console.

   2. In the top menu bar, choose Data + AI > Data Transmission (DTS) > Data Synchronization or Data + AI > Data Transmission (DTS) > Data Migration.

   3. Optional: To the right of Data Synchronization Tasks or Data Migration Tasks, select the region of the synchronization or migration instance.

2. Click the instance ID to open its details page.

3. Optional: In the left-side navigation pane, click Task Management.

4. To the right of Progress, clear the Hide Completed Modules checkbox.

5. In the Progress section, click Full Data Verification, Incremental Data Verification, or Schema Verification to view the verification details.

   • Basic Information

     The Basic Information tab shows the task start time, verification time rule, status, progress, and error messages. Displayed content varies by task configuration.

     After the task starts, you can Restart, Pause, Stopped, or Delete the task. What to do next.

     Important

     The Status on the Basic Information tab reflects the task status. View verification results on the Check Details tab.

   • Check Details

     The Check Details tab shows source and destination database/table names, verification status, and verification time.

     Note

     • The results on the Check Details tab are sorted in descending order by the Number of Inconsistent Records.

     • You can download a revised SQL statement only for full data verification tasks. Tasks started before 14:04:52 (UTC+8) on November 20, 2023 must be re-run to enable this feature.

     • The Total Number of Records is an estimated value, while the Number of records completed by verification is the actual count of verified records. The Number of records completed by verification may be higher than the Total Number of Records.

     • If the Number of Inconsistent Records is 0, source and destination data is consistent. No report data is generated.

     • If the Number of Inconsistent Records is not 0, data or schema inconsistencies exist between source and destination. In the Actions column, click View Report, Download Report, or Download Revised SQL Statements to manually fix the inconsistencies using the downloaded report and revised SQL statement.

       Important

       Use caution when executing DELETE statements from a revised SQL statement.

What to do next

After correcting inconsistencies, manage the verification task in the Progress section.

1. Go to the Task Management page of the target instance.

2. To the right of Progress, clear the Hide Completed Modules checkbox.

3. Click the Full Data Verification or Incremental Data Verification module.

4. Optional: Click the Basic Information tab.

5. Manage the verification task.

   • Pause a verification task

     Important

     • You can pause a verification task only while it is running.

     • Full data verification tasks cannot resume after being paused.

     To the right of Actions, click Pause, then click OK in the Note dialog box. The task Status changes to Pausing.

   • Restart a verification task

     Note

     • You can restart the verification task only when its Status is Pausing or Completed.

     • Restarting a full data verification task re-verifies all existing data in the source and destination databases.

     To the right of Actions, click Restart. In the Note dialog box that appears, click OK.

   • Stop a verification task

     Note

     Only incremental verification tasks can be stopped. After stopping, the task Status changes to Completed.

     To the right of Actions, click Stopped. In the Note dialog box that appears, click OK.

   • Delete a verification task

     Important

     Deleted verification tasks cannot be recovered. You can only create a new one.

     To the right of Actions, click Delete. In the Note dialog box that appears, click OK.

API reference

FAQ

• How do I recreate a full or incremental data verification task?

  Use one of the following methods:

  • On the Data Verification page, create a standalone verification task.

    Note

    You cannot create an incremental data verification task on this page.

  • Go to the Full Data Verification or Incremental Data Verification module of the target instance. On the Basic Information tab, click Delete, and then click Create Verification Task at the top of the page.

    Note

    DTS does not support data verification for instances whose Status is Completed.

• Why does the verification status show "Failed"?

  Possible reasons:

  • The table has no primary key or unique index and contains more than 10,000 rows. DTS skips full data verification for such tables because a table without a primary key cannot ensure data consistency between the source and destination tables. Skipping verification does not affect data synchronization, which continues to run as expected.

  • The data has neither a primary key nor a unique constraint. DTS skips incremental verification in this case.

  • Verifying materialized views is not supported.

  • The number of differing rows in a single table between source and destination exceeds 100,000, which can occur if synchronization is incomplete or conflicts exist.

  • The connection to the source or destination database is unstable, possibly due to network interruptions or permission changes.

  Check the verification details page for the failure reason. Usage notes.

• Why does a full or incremental data verification task not start?

  This depends on whether the instance includes an incremental write module (incremental synchronization or migration):

  • Without an incremental write module:

    Full data verification starts after the full data write is complete.

  • With an incremental write module:

    Note

    Full and incremental data verification tasks are independent and do not affect each other.

    • If the incremental task has no subtasks, both full and incremental verification start when the incremental write module first reaches zero latency.

    • If the incremental task has multiple subtasks (for example, MongoDB instance migration), both full and incremental verification start when all subtask latencies drop below 2 seconds.

• Why are the results of an incremental data verification task always empty?

  Incremental verification only records databases and tables with inconsistencies. If all incremental data is consistent, the results are empty.

• Why is the current offset in the incremental verification module not updated?

  The Current Offset in the incremental verification module indicates the last verification timestamp. If no new incremental data exists in the source database, the Current Offset is not updated or may show no timestamp.

• Why is inconsistent data not detected by the incremental data verification task even when incremental data is written to the destination database?

  Incremental verification uses the source database as the baseline. Inconsistencies are detected only when source changes are not replicated to the destination or when the replicated data does not match.

• How long does full data verification take?

  To estimate the duration of full data verification, set the sampling percentage to 10%, run the verification, and record the duration. Then estimate the duration of full verification proportionally. For example, if verification with 10% sampling takes X minutes, full verification (100%) is expected to take about 10X minutes. The actual duration is affected by the data volume, network bandwidth, and the performance of the source and destination databases. The estimate is for reference only.