Set up a change tracking task to capture INSERT, UPDATE, and DELETE operations from a PolarDB-X 2.0 instance and stream them to downstream consumers via the change tracking client.
Prerequisites
Before you begin, make sure that:
A PolarDB-X 2.0 instance is created and a database exists in the instance. See Create a PolarDB-X instance and Create a database.
If the ApsaraDB RDS for MySQL instance is of the classic network type, an internal endpoint is configured for the ApsaraDB RDS for MySQL instance.
Binary logging is enabled and
binlog_row_imageis set tofull. See Parameter settings.Binary logs are retained for more than 24 hours. If the retention period is shorter, DTS may fail to read the logs and the task will fail.
The database account used for this task has read permissions on all objects to be tracked.
Limitations
Source database
Source tables must have a primary key or UNIQUE constraint with all unique fields. Tables without these constraints may produce duplicate change records.
Table-level tracking supports up to 500 tables per task. For more than 500 tables, use multiple tasks or switch to database-level tracking.
If the source instance is a read-only or temporary instance, make sure the instance records transaction logs.
During the task
A primary/secondary switchover on the source instance causes the task to fail.
Do not scale the source instance, migrate frequently accessed tables, change shards, or run DDL operations while the task is running.
If the source instance is also used by another running task (for example, a data migration task), DTS may capture changes from additional objects. Filter the data manually on the change tracking client.
DTS uses
ROUND(COLUMN, PRECISION)to read FLOAT and DOUBLE columns. The default precision is 38 digits for FLOAT and 308 digits for DOUBLE. Verify these defaults meet your requirements.DTS does not capture DDL operations run via
pt-online-schema-change. If schema changes are made with this tool, the change tracking client may fail to write data due to schema conflicts.
Required permissions
| Instance | Required permission |
|---|---|
| Source PolarDB-X 2.0 instance | Read permissions on the objects for change tracking |
Create a change tracking task
Go to the Change Tracking Tasks page.
Log on to the Data Management (DMS) console.
In the top navigation bar, click DTS.
In the left-side navigation pane, choose DTS (DTS) > Change Tracking.
Alternatively, in Simple Mode, move the pointer over the icon in the upper-left corner and choose All functions > DTS > Change Tracking. You can also use the new DTS console.
Select the region where you want to create the change tracking task from the drop-down list to the right of Change Tracking Tasks.
In the new DTS console, select the region from the drop-down list to the right of Workbench.
Click Create Task. Configure the source database and consumer network type.
WarningRead the Limits displayed at the top of the page before proceeding. Skipping this step may cause the task to fail or prevent downstream consumers from reading the tracked data.
Section Parameter Description None Task Name A name for the task. DTS assigns a default name. Specify a descriptive name to identify the task easily. The name does not need to be unique. Source Database Select an existing database connection Select an existing connection or configure a new one. If you use an existing connection, DTS applies its saved settings automatically. Database Type Select PolarDB-X 2.0. Access Method Fixed to Alibaba Cloud Instance. Instance Region The region where the source PolarDB-X instance is located. Instance ID The ID of the source PolarDB-X instance. Database Account The account for the PolarDB-X instance. The account must have read permissions on all objects to be tracked. Database Password The password for the database account. Consumer Network Type Network Type Fixed to VPC. Select the VPC and vSwitch to use. If the change tracking client runs in a VPC, select the same VPC and vSwitch as the client to minimize latency. This setting cannot be changed after the task is created. Click Test Connectivity and Proceed. DTS automatically adds its CIDR blocks to the whitelist of Alibaba Cloud database instances and to the security group rules of ECS-hosted databases. For self-managed databases in data centers or on third-party cloud platforms, manually add the DTS server CIDR blocks to your database's security settings. For the full list of CIDR blocks, see Add the CIDR blocks of DTS servers.
WarningAdding public CIDR blocks to a database whitelist or security group exposes the database to potential security risks. Take preventive measures such as strengthening account credentials, limiting exposed ports, auditing API access, reviewing whitelist rules regularly, and using Express Connect, VPN Gateway, or Smart Access Gateway for private connectivity.
Configure the objects and advanced settings.
Basic settings
Parameter Description Data Change Types Select the types of changes to track:
Data Update: tracks DML operations — INSERT, DELETE, and UPDATE.
Schema Update: tracks the create, delete, and modify operations that are performed on all object schemas of the source instance. You must use the change tracking client to filter the data to be tracked.
Source Objects Select objects from the Source Objects section and click the arrow icon to move them to the Selected Objects section. Advanced settings
Parameter Description Monitoring and Alerting Configure alerts for task failures or latency exceeding a threshold. Select Yes to specify an alert threshold and alert contacts, or No to skip alerting. Retry Time for Failed Connections How long DTS retries a failed connection before marking the task as failed. Valid values: 10–1440 minutes. Default: 120 minutes. Set this to at least 30 minutes. If DTS reconnects within this window, the task resumes automatically. Note: If an instance serves as the source or destination database of multiple change tracking tasks, the shortest retry time range that is set for the instance takes precedence. When DTS retries a connection, you are charged for the DTS instance. We recommend that you specify the retry time range based on your business requirements. You can also release the DTS instance at your earliest opportunity after the source and destination instances are released. Configure ETL Enable extract, transform, and load (ETL) processing. Select Yes to write an ETL policy in the code editor. See What is ETL? Click Next: Save Task Settings and Precheck. To preview the API parameters for this configuration, move the pointer over Next: Save Task Settings and Precheck and click Preview OpenAPI parameters.
DTS runs a precheck before the task can start. The task starts only after the precheck passes. If an item fails, click View Details next to it, fix the issue, and click Precheck Again. If an alert item is not critical, click Confirm Alert Details, then Ignore, then OK, and run the precheck again. Ignoring an alert may cause data inconsistency.
Wait for Success Rate to reach 100%, then click Next: Purchase Instance.
On the Purchase page, select a billing method.
Parameter Description Billing method Subscription: Pay upfront for a fixed term. More cost-effective for long-term use. Available durations: 1–9 months, or 1, 2, 3, or 5 years.
Pay-as-you-go: Billed hourly. Use for short-term or trial deployments. Release the instance when no longer needed to stop charges.
Resource Group Settings The resource group for the instance. Default: default resource group. See What is Resource Management? Subscription Duration Available only for the Subscription billing method. Set the duration and the number of instances to purchase. Read and select Data Transmission Service (Pay-as-you-go) Service Terms.
Click Buy and Start. The task appears in the task list and begins tracking changes.
What's next
After the task is running, create consumer groups and connect your downstream clients to consume the tracked data.
Connect a client to consume the tracked data: