Q: How does change tracking work?
A: For information about how change tracking works, see Architecture and features.
Q: What data can a change tracking instance track, and what information can I obtain?
You can track schema-level changes (creations, deletions, and modifications) of the tracked objects, as well as data updates (INSERT, DELETE, and UPDATE operations) on those objects.
NoteWhen some databases are used as the source, only data updates can be tracked.
Q: Is the consumer group deleted after a change tracking task expires?
A: After a DTS change tracking instance expires, the data consumer group is retained for 7 days. If the instance is not renewed within 7 days after expiration, the instance is released and the corresponding consumer group is deleted.
Q: Can a read-only instance be used as the source instance of a change tracking task?
There are two cases:
If the instance is a read-only instance that records transaction logs (for example, an ApsaraDB RDS for MySQL 5.7 or 8.0 instance), it can be used as the source instance.
If the instance is a read-only instance that does not record transaction logs (for example, an ApsaraDB RDS for MySQL 5.6 instance), it cannot be used as the source instance.
Q: How do I consume tracked data?
For more information, see Consume tracked data.
Q: Why does the date format change after data is transferred by using the change tracking feature?
A: The default storage format for date values in DTS is YYYY:MM:DD. YYYY-MM-DD is only the display format; the actual storage format is YYYY:MM:DD. Therefore, regardless of the format in which the data is written during transfer, it is ultimately converted to the default format.
Q: How do I troubleshoot change tracking task issues?
A: For information about how to troubleshoot change tracking tasks, see Troubleshoot issues with change tracking tasks.
Q: The SDK suddenly pauses during normal data downloading and cannot track data. How do I resolve this issue?
A: Check whether the SDK code calls the ackAsConsumed method to report the consumer offset. If ackAsConsumed is not called to report the offset, the record cache configured within the SDK is not cleared. When the cache is fully occupied, no new data can be pulled, causing the SDK to pause and stop tracking data.
Q: The SDK cannot successfully track data after being restarted. How do I resolve this issue?
A: Before you start the SDK, modify the consumer offset so that it falls within the data range. For information about how to modify the consumer offset, see Save and query consumption checkpoints.
Q: How does a client specify a point in time from which to consume data?
A: When you consume tracked data, specify a point in time by setting the initCheckpoint parameter. For more information, see Consume subscription data using SDK sample code.
Q: The DTS change tracking task has accumulated data. How do I reset the offset?
Open the corresponding code file based on the usage mode of the SDK client. For example, DTSConsumerAssignDemo.java or DTSConsumerSubscribeDemo.java.
NoteFor more information, see Consume subscription data using SDK sample code.
In the change tracking task list, in the Data Range column, view the modifiable offset range of the target change tracking instance.
Select a new consumer offset based on your actual needs and convert it into a Unix timestamp.
In the code file, replace the old consumer offset (the initCheckpoint parameter) with the new consumer offset.
Restart the client.
Q: The client cannot connect to the change tracking task through the VPC endpoint. How do I resolve this issue?
A: The machine on which the client runs may not reside in the VPC that you specified when you configured the change tracking task (for example, the client VPC has been changed). You must reconfigure the task.
Q: Why is the consumer offset shown in the console larger than the maximum value of the data range?
A: The data range of a change tracking channel is updated every 1 minute, whereas the consumer offset is updated every 10 seconds. Therefore, when data is consumed in real time, the value of the consumer offset may be larger than the maximum value of the data range of the change tracking channel.
Q: How does DTS ensure that the data tracked by the SDK is a complete transaction?
A: Based on the provided consumer offset, the DTS server searches for the complete transaction that corresponds to the offset and distributes data downstream starting from the BEGIN statement of the transaction. Therefore, you receive the complete transaction content.
Q: How do I confirm whether data is consumed normally?
A: If data is consumed normally, the consumer offset in the Data Transmission Service console advances as expected.
Q: What does usePublicIp=true mean in the change tracking SDK?
A: Setting usePublicIp=true in the change tracking SDK indicates that the SDK accesses the DTS change tracking channel over the Internet.
Q: Is my business affected when a primary/secondary switchover or a primary database restart occurs on the source RDS instance of a change tracking task?
A: When a primary/secondary switchover or a restart occurs on an ApsaraDB RDS for MySQL, ApsaraDB RDS for PostgreSQL, PolarDB for MySQL, PolarDB for PostgreSQL, or PolarDB-X 1.0 (with the RDS MySQL storage type) instance, DTS automatically adapts to the switchover, and your business is not affected.
Q: Does ApsaraDB RDS support automatic binlog downloading to an on-premises server?
A: DTS change tracking supports real-time tracking of ApsaraDB RDS binlogs. You can activate the DTS change tracking service and use the DTS SDK to track ApsaraDB RDS binlog data and synchronize the data to an on-premises server in real time.
Q: Does the real-time incremental data tracked by change tracking include only newly added data, or does it also include modified data?
A: The incremental data that DTS change tracking can track includes all inserts, deletions, and updates, as well as schema changes (DDL).
Q: A record on the consumer end of a change tracking task is not ACKed. Why does the SDK receive duplicate data after being restarted?
A: When the SDK has a message that is not ACKed, the server continues to push all messages in the buffer. After the push is complete, the SDK cannot receive any more messages. At this point, the consumer offset saved on the server is the offset of the last message before the un-ACKed one. When the SDK restarts, to ensure that no messages are lost, the server pushes data again starting from the offset that corresponds to the message before the un-ACKed one. Therefore, the SDK receives some duplicate messages.
Q: How often is the change tracking consumer offset updated? Why does the SDK sometimes receive duplicate data after being restarted?
A: After the change tracking SDK consumes each message, it must call ackAsConsumed to send an ACK to the server. After the server receives the ACK, it updates the consumer offset in memory and then persists the offset every 10 seconds. If the SDK is restarted before the latest ACK is persisted, the server pushes messages starting from the previously persisted consumer offset to ensure that no messages are lost. As a result, the SDK receives duplicate messages.
Q: Can a change tracking instance track multiple RDS instances?
A: No. A change tracking instance can currently track only one RDS instance.
Q: Can data inconsistency occur in a change tracking instance?
A: No. A change tracking task only obtains changes from the source database and does not involve data inconsistency. If the data consumed by the client does not match your expectations, troubleshoot the issue on your side.
Q: How do I handle the UserRecordGenerator message that appears when I consume tracked data?
A: When you consume tracked data, if a message such as
UserRecordGenerator: haven't receive records from generator for 5sappears, check whether the consumer offset falls within the offset range of the incremental data collection module and ensure that the consumer is running properly.Q: Does a topic support multiple partitions?
A: No. To ensure global message ordering, each change tracking topic has only one partition, which is fixed and assigned to partition 0.
Q: Does the change tracking SDK support Go?
A: Yes. For sample code, see dts-subscribe-demo.
Q: Does the change tracking SDK support Python?
A: Yes. For sample code, see dts-subscribe-demo.
Q: Does flink-dts-connector support multi-threaded concurrent consumption of tracked data?
A: No.
Q: What do I do if I forget the username or password of the consumer group of a change tracking instance?
A: If your business allows, you can delete the consumer group and then create a new one (with the original username and a new password).
Q: Can both a change tracking task and a data synchronization task be configured on the same RDS instance?
A: Yes. Both tasks can be configured at the same time without affecting each other. Change tracking and data synchronization are two independent task types that can coexist on the same RDS instance without interference.