Create a change tracking task for PolarDB for PostgreSQL - Data Transmission Service

Use Data Transmission Service (DTS) to track INSERT, UPDATE, and DELETE operations from a PolarDB for PostgreSQL cluster in real time. Once the task is running, downstream consumers can subscribe to the change stream using the SDK, Flink, or a Kafka client.

Limitations

Review these limitations before you start. Misconfigurations in this area are the most common cause of task failures and data inconsistency.

Source Database

Limitation	Impact	Action
Primary key or unique constraint	Source tables must have a PRIMARY KEY or UNIQUE constraint with all fields unique. Without this, some tracked changes may be duplicated.	Add a PRIMARY KEY or UNIQUE constraint to each tracked table before creating the task.
Table count	A single change tracking task can track up to 500 tables. Exceeding this limit causes a request error.	Create separate tasks for different table groups, or track the entire database instead of individual tables.
WAL log retention	WAL logs must be retained for more than 24 hours. Shorter retention may cause DTS to fail reading logs, which can result in task failure or data inconsistency.	Set the WAL log retention period on the source cluster to at least 24 hours before starting the task.
Read-only or temporary instances	If the source is a read-only or temporary instance, transaction logging must be enabled.	Enable transaction logging on the instance.

Other limits

Limitation	Details
One database per task	A single change tracking task tracks data from one database only. Create a separate task for each database.
Heartbeat table	DTS adds a heartbeat table named `dts_postgres_heartbeat` to the source database to maintain accurate latency measurement.
FLOAT and DOUBLE precision	DTS retrieves FLOAT and DOUBLE values using `ROUND(COLUMN,PRECISION)`. If no precision is specified, FLOAT defaults to 38 digits and DOUBLE defaults to 308 digits. Verify that these defaults meet your requirements before starting the task.

Prerequisites

Before you begin, ensure that you have:

A PolarDB for PostgreSQL cluster. See Create a cluster.
The wal_level parameter set to logical on the cluster. See Specify cluster parameters.
WAL log retention configured to more than 24 hours on the cluster.

Create a change tracking task

Step 1: Go to the Change Tracking Tasks page

Log on to the Data Management (DMS) console.
In the top navigation bar, click DTS.
In the left-side navigation pane, choose DTS (DTS) > Change Tracking.

If you use Simple Mode in the DMS console, move the pointer over the icon in the upper-left corner, then choose All functions > DTS > Change Tracking. Alternatively, open the DTS console directly.

Step 2: Select a region

To the right of Change Tracking Tasks, select the region where you want to create the task.

In the DTS console, select the region from the drop-down list to the right of Workbench on the Change Tracking Tasks page.

Step 3: Configure the source database and consumer network

Click Create Task. On the page that appears, configure the following parameters.

Warning

Read the Limits displayed at the top of the page before proceeding. Skipping this step may cause the task to fail or prevent tracked data from being consumed.

Source Database

Parameter	Description
Task Name	A name for the change tracking task. DTS assigns a default name. Use a descriptive name to make the task easy to identify. The name does not need to be unique.
Select an existing database connection	Optional. Select an existing connection to have DTS apply its settings automatically. If you skip this, configure the parameters below manually.
Database Type	Select PolarDB for PostgreSQL.
Access Method	Select Alibaba Cloud Instance.
Instance Region	The region where the source PolarDB for PostgreSQL cluster resides.
Replicate Data Across Alibaba Cloud Accounts	Select No. You can select only No.
Instance ID	The ID of the PolarDB for PostgreSQL cluster.
Database Name	The name of the source database.
Database Account	The privileged account for the PolarDB for PostgreSQL database.
Database Password	The password for the database account.

Consumer Network Type

Parameter	Description
Network Type	Fixed to VPC. Select a VPC and a vSwitch. For details, see VPCs.

The consumer network type cannot be changed after the task is created. If your change tracking client runs in a VPC, select the same VPC and vSwitch to minimize network latency.

Step 4: Test connectivity

Click Test Connectivity and Proceed.

DTS automatically adds its CIDR blocks to the whitelist of Alibaba Cloud database instances or the security group rules of ECS-hosted databases. For self-managed databases in data centers or hosted by third-party providers, manually add the DTS CIDR blocks to your database security settings. See Add the CIDR blocks of DTS servers.

Warning

Adding DTS CIDR blocks to a whitelist or security group introduces security exposure. Before using DTS, acknowledge the risks and apply appropriate controls, such as strong credentials, restricting exposed ports, authenticating API calls, reviewing whitelist rules regularly, and removing unauthorized CIDR blocks. Consider connecting the database to DTS over Express Connect, VPN Gateway, or Smart Access Gateway instead of the public internet.

Step 5: Select objects

Select the databases or tables to track.

Parameter	Description
Data Change Types	Fixed to Data Update, which covers INSERT, DELETE, and UPDATE operations.
Source Objects	Select objects from the Source Objects list and click the icon to move them to Selected Objects.

Tracking scope:

Database-level: DTS tracks all objects in the database, including tables added after the task starts.
Table-level: DTS tracks only the specified tables. To add more tables later, see Modify the objects for change tracking.

Step 6: Configure advanced settings

Click Next: Advanced Settings.

Parameter	Description
Monitoring and Alerting	Select Yes to receive notifications when the task fails or latency exceeds a threshold. Specify the alert threshold and contacts. Select No to skip alerts.
Retry Time for Failed Connections	The duration DTS retries a failed connection before the task fails. Range: 10–1440 minutes. Default: 120 minutes. Set this to at least 30 minutes. If DTS reconnects within this window, the task resumes automatically.
Configure ETL	Select Yes to apply extract, transform, and load (ETL) transformations. Enter processing statements in the code editor. See Configure ETL in a data migration or synchronization task. Select No to skip ETL.

If one database instance serves as the source for multiple change tracking tasks, the shortest retry time configured across those tasks applies to all of them. The DTS instance is billed while retrying.

Step 7: Run the precheck

Click Next: Save Task Settings and Precheck.

DTS runs a precheck before starting the task. The task only starts after the precheck passes.

If a precheck item fails:

Click View Details next to the failed item, fix the issue based on the error message, then click Precheck Again.
If an item triggers an alert that can be ignored, click Confirm Alert Details. In the dialog, click Ignore, then click OK, then click Precheck Again.

To preview the OpenAPI parameters for this task configuration, move the pointer over Next: Save Task Settings and Precheck and click Preview OpenAPI parameters.

Step 8: Wait for the precheck to complete

Wait until Success Rate reaches 100%, then click Next: Purchase Instance.

Step 9: Choose a billing method

On the Purchase Instance page, select a billing method.

Billing method	Best for	Details
Subscription	Long-term use	Pay upfront for a fixed period. More cost-effective than pay-as-you-go for sustained workloads. Set the subscription duration (1–9 months or 1–3 years) and enable auto-renewal if needed.
Pay-as-you-go	Short-term use	Billed hourly. Release the instance when you no longer need it to stop charges.

Step 10: Accept the service terms

Read and select the Data Transmission Service (Pay-as-you-go) Service Terms.

Step 11: Start the task

Click Buy and Start. The task appears in the task list where you can monitor its progress.

What's next

When the change tracking task is running, set up consumer groups and start consuming the tracked data.

Create consumer groups. See Create consumer groups.
Choose a consumption method: