Data verification issues - Data Transmission Service - Alibaba Cloud Documentation Center

Q: What are the causes of data inconsistency in a data verification task?
Common causes are as follows:
1. The data migration or synchronization task has latency.
2. The source database performs an ADD COLUMN operation with a default value while the task has latency.
3. The destination database receives data writes from sources other than DTS.
4. The source database of a task with the multi-table merging feature enabled performs a DDL operation.
5. The data migration or synchronization task uses the database, table, or column name mapping feature.
Q: Why does a schema verification task detect a difference in isRelHasoids?
PostgreSQL databases earlier than version 12 support creating tables with WITH OIDS to add a globally unique object identifier (OID) column. If the source of a schema verification task specifies WITH OIDS when creating tables and the destination is a later PostgreSQL version that does not support WITH OIDS, the task reports an isRelHasoids difference.
Q: Does an isRelHasoids difference reported by a schema verification task require attention?
No.
Q: Does DTS synchronize or migrate the object identifier (OID) column?
The object identifier (OID) column is automatically generated when WITH OIDS is specified. DTS does not synchronize or migrate this data, regardless of whether the destination supports this column.
Q: How do I check whether a table has an object identifier (OID) column?
Note
Replace <table_name> in the commands with the name of the table you want to query.
- SQL command: SELECT relname AS table_name, relhasoids AS has_oids FROM pg_class WHERE relname = '<table_name>' AND relkind = 'r';
- Client command: \d+ <table_name>
Q: How do I troubleshoot data inconsistency reported in a full data verification report?
You can troubleshoot by following these steps:
1. Review the verification report to identify the source and destination values for each column, and confirm which columns and values are inconsistent.
2. Based on the verification report, manually log on to the source and destination databases and compare the differing values to determine whether they are expected.
3. Confirm that no data was written to the source or destination during verification. If you cannot ensure this, we recommend that you pause writes to the source database during off-peak hours and then restart the verification.
4. If the differences persist, you can temporarily enable SQL Explorer (pay-as-you-go) on both the source and destination, and use audit logs to check whether the columns were modified.
5. Differences in some attributes (such as SECURITY_TYPE) are expected behavior and can be ignored.
6. A successful full data verification confirms data consistency after non-standard backend modifications.
7. Full data verification inconsistencies are usually unrelated to schema differences. We recommend that you manually run count(*) to compare the row counts between the source and destination databases for further verification.
Q: Why does DTS show the task progress as incomplete (for example, 41%) even though data transmission is finished?
The DTS task progress percentage covers two stages: data transmission (full data migration) and data verification. After full data migration is complete, the remaining progress reflects the data verification in progress. Data verification is an independent consistency check that runs after full data transmission finishes. The overall task is considered complete only when the verification progress reaches 100%.
If you need to confirm the data immediately after the transmission finishes, you can manually connect to the destination database to check the data without waiting for the verification to complete.
Q: Causes and solutions for slow SQL queries during the full data verification stage
Cause: During the full data verification stage, DTS reads large tables in shards by primary key ID. When the data volume is large, shard queries take longer and may cause slow SQL queries on the source database.
Solution:
1. First, verify whether the full data verification task has stopped. If it has, no such queries should be generated.
2. If the verification task is still running and affects your business, we recommend that you run the full data verification during off-peak hours, or suspend the verification task and restart it during off-peak hours.