The change tracking feature of Data Transmission Service (DTS) captures real-time incremental data from your databases. You can consume this data and write it to a destination as needed. Common use cases include:
Cache updates: Invalidate or refresh cached entries as source data changes.
Asynchronous service decoupling: Publish database changes to downstream consumers without tight coupling between services.
Real-time data synchronization: Keep disparate data sources in sync without batch jobs.
Complex extract, transform, and load (ETL) operations: Feed continuously arriving incremental data into transformation pipelines.
Source database limits
Before setting up change tracking, verify that your source database meets the following requirements. Exceeding these thresholds degrades instance performance or causes task failures.
Network requirements
| Requirement | Threshold | Notes |
|---|---|---|
| Network bandwidth | 100 Mb/s or higher | Lower bandwidth increases the risk of lag during high-traffic periods. |
| Round-trip time (RTT) between the source database and DTS | Less than 2 ms | A large physical distance between the database and the DTS service raises RTT above 2 ms and causes latency. For example, a database in the Singapore region connected to DTS through a VPN in the China (Hong Kong) region can produce RTT well above 2 ms. |
Service requirements
| Requirement | Threshold | Notes |
|---|---|---|
| Total log size | Less than 1 TB | By default, DTS pulls logs for the entire database instance. If objects not included in your change tracking task generate heavy write activity, that log volume still counts toward this limit and can cause delays. |
| Average log size per hour | Less than 50 GB | — |
| Peak traffic | Less than 15 MB/s | — |
| DDL operations | At most 10 DDL statements per second | Exceeding this rate delays the DTS task. |
| Large transactions | Total log size per transaction must be less than 100 GB | Transactions exceeding 100 GB may cause the DTS task to fail. |
Large field updates: Updating large volumes of data in CLOB, BLOB, or LONG fields can delay the DTS task. Perform such operations in smaller batches, or avoid performing such operations altogether.
Tables without primary keys: Frequent deletions or modifications to tables without primary keys can delay the DTS task. Minimize these operations where possible.
Change tracking solutions
Whether you can create a change tracking task across Alibaba Cloud accounts depends on the database type and connection type. To create a cross-account task, set the Replicate Data Across Alibaba Cloud Accounts parameter to Yes for the source database instance. For more information, see Configure a task across Alibaba Cloud accounts.
Select your source database from the tables below to find the corresponding configuration guide.
Alibaba Cloud managed databases
| Source database | Supported versions | Supported change types | Configuration guide |
|---|---|---|---|
| ApsaraDB RDS for MySQL | All versions | Data updates, schema updates | Create a change tracking task for an ApsaraDB RDS for MySQL instance |
| PolarDB for MySQL | All versions | Data updates, schema updates | Create a change tracking task for a PolarDB for MySQL cluster |
| PolarDB-X 1.0 | — | Data updates, schema updates | Create a change tracking task for a PolarDB-X 1.0 instance |
| PolarDB-X 2.0 | — | Data updates, schema updates | Create a change tracking task for a PolarDB-X 2.0 instance |
| PolarDB for PostgreSQL (Compatible with Oracle) | All versions | Data updates only | Create a change tracking task for a PolarDB for PostgreSQL (Compatible with Oracle) cluster |
| ApsaraDB RDS for PostgreSQL | 9.4, 10, 11, 12, 13, 14, 15, 16, 17 | Data updates only | Create a change tracking task for an ApsaraDB RDS for PostgreSQL instance |
| PolarDB for PostgreSQL | 11, 14, 15, 16 | Data updates, schema updates | Create a change tracking task for a PolarDB for PostgreSQL cluster |
| Data Management (DMS) logical database | — | Data updates only | Create a change tracking task for a DMS logical database |
PolarDB-X constraints: Databases in a PolarDB-X instance must be created from ApsaraDB RDS for MySQL instances. DTS does not support PolarDB-X databases created from PolarDB for MySQL clusters.
DMS logical database constraint: The logical database must be created from the database shards of multiple PolarDB for MySQL clusters.
Self-managed databases
| Source database | Supported versions | Supported change types | Configuration guide |
|---|---|---|---|
| Self-managed MySQL | 5.1, 5.5, 5.6, 5.7, 8.0 | Data updates, schema updates | Create a change tracking task for an ApsaraDB RDS for MySQL instance |
| Self-managed Oracle (non-RAC) | 9i, 10g, 11g | Data updates, schema updates | Create a change tracking task for an Oracle database |
| Self-managed PostgreSQL | 9.4.8 and later, 9.5, 9.6, 10.x, 11.x, 12.x, 13.x, 14.x, 15.x, 16.x, 17.x | Data updates only | Create a change tracking task for an ApsaraDB RDS for PostgreSQL instance |