Create a change tracking task in Data Transmission Service (DTS) to capture real-time data changes from a PolarDB-X 1.0 instance.
How it works
DTS reads binary logs from the ApsaraDB RDS for MySQL instances attached to your PolarDB-X 1.0 instance to capture data changes. Because PolarDB-X 1.0 distributes data across multiple ApsaraDB RDS for MySQL instances, DTS runs a subtask for each attached instance. Binary log settings on those RDS instances directly affect whether the change tracking task can start and stay healthy.
Prerequisites
Before you begin, make sure you have:
-
A PolarDB-X 1.0 instance whose storage type is ApsaraDB RDS for MySQL (either a custom ApsaraDB RDS instance or a purchased one). PolarDB for MySQL cannot be used as the storage type.
-
If the attached ApsaraDB RDS for MySQL instance uses the classic network type, an internal endpoint configured for it.
-
A Virtual Private Cloud (VPC) and a vSwitch for the change tracking instance.
-
Binary logging enabled on each attached ApsaraDB RDS for MySQL instance, with
binlog_row_imageset tofulland binary logs retained for more than 24 hours. See Create a PolarDB-X 1.0 instance and Create a databaseCreate a database.
Without binlog_row_image=full, the precheck fails and the task cannot start. Without 24+ hours of binary log retention, DTS may fail to read logs, causing task failure or data loss. These settings are not covered by the DTS SLA if misconfigured.
Limitations
Source database
| Limitation | Details |
|---|---|
| Primary key or UNIQUE constraint required | Source tables must have primary key or UNIQUE constraints, with all fields unique. Without these, DTS may track duplicate changes. Schema updates are not supported for tables that have only UNIQUE constraints. Use tables with primary keys for full support. |
| 500-table limit per task | Tracking more than 500 tables in a single task causes a request error. Split the tables across multiple tasks, or track the entire database instead. |
| Binary log requirements | Binary logging must be enabled. binlog_row_image must be full. Binary logs must be retained for more than 24 hours. |
| Read-only or temporary instances | The instance must record transaction logs. |
Other limitations
| Limitation | Details |
|---|---|
| Table-level tracking only | Change tracking is supported only at the table level, not column or row level. |
| Objects cannot be changed after configuration | After you configure the task, the tracked objects cannot be reselected. Create a new task to track additional tables. |
| No primary/secondary switchover while running | If a primary/secondary switchover occurs during the task, the task fails. |
| No scaling or DDL operations while running | Do not scale in or out the source instance, migrate frequently-accessed tables, change shards, or perform DDL operations while the task is running. These actions cause task failure or data inconsistency. |
| Concurrent tasks may introduce noise | If the source database is used in another running task (such as a data migration task), DTS may track data changes from objects outside your selection. Filter the tracked data manually on the change tracking client. |
| FLOAT/DOUBLE precision | DTS uses ROUND(COLUMN,PRECISION) to read values from FLOAT and DOUBLE columns. Default precision: 38 digits for FLOAT, 308 digits for DOUBLE. Confirm these defaults meet your requirements. |
| pt-online-schema-change not supported | DTS does not track DDL operations performed using pt-online-schema-change. This may cause schema conflicts when writing consumed data to destination tables. |
| Duplicate DDL operations from sharding | PolarDB-X 1.0 applies DDL operations to all table shards. Because subtask progress may vary, the tracked data can contain duplicate DDL operations. Handle this on the change tracking client to avoid exceptions. |
Create a change tracking task
Step 1: Go to the Change Tracking Tasks page
-
Log on to the Data Management (DMS) console.
-
In the top navigation bar, click DTS.
-
In the left-side navigation pane, choose DTS (DTS) > Change Tracking.
If you log on to the DMS console and click the Enter Simple Mode icon in the upper-right corner, you can move the pointer over theicon in the upper-left corner, then choose All functions > DTS > Change Tracking. You can also use the new DTS console directly.
Step 2: Select a region and create a task
-
To the right of Change Tracking Tasks, select the region where you want to create the task.
In the new DTS console, select the region from the drop-down list to the right of Workbench on the Change Tracking Tasks page.
-
Click Create Task.
After you specify the source database, review the Limits displayed at the top of the page before proceeding. Skipping this step may cause the task to fail or prevent the tracked data from being consumed.
Step 3: Configure the source database and consumer network
Set the following parameters. If you have an existing database connection template, select it under Select an existing database connection and DTS fills in the parameters automatically.
Source Database
| Parameter | Description |
|---|---|
| Task Name | A name for the change tracking task. DTS assigns a name automatically. Use a descriptive name to identify the task easily. The name does not need to be unique. |
| Database Type | Select PolarDB-X 1.0. |
| Access Method | Select Alibaba Cloud Instance. |
| Instance Region | The region where the PolarDB-X 1.0 instance resides. |
| Instance ID | The ID of the PolarDB-X 1.0 instance. |
| Database Account | The database account for the PolarDB-X 1.0 instance. The account must have read permissions on the objects to be tracked. |
| Database Password | The password for the database account. |
| Save as Instance or Edit Template | If you are not using an existing connection, click Save as Instance to save the connection as a reusable template. If you selected an existing connection, click Edit Template to rename it. Changes to the template take effect the next time you select it and do not affect the currently configured instance. |
Consumer Network Type
| Parameter | Description |
|---|---|
| Network Type | Fixed to VPC. Select the VPC and vSwitch for the change tracking instance. If your change tracking client is in a VPC, select the same VPC and vSwitch to minimize network latency. This setting cannot be changed after the task is configured. |
For more information about VPCs, see VPCs.
Step 4: Test connectivity and proceed
Click Test Connectivity and Proceed.
DTS automatically adds its server CIDR blocks to the whitelist of Alibaba Cloud database instances (such as ApsaraDB RDS for MySQL) or to the security group rules of ECS-hosted databases. For self-managed databases in data centers or third-party cloud environments, manually add the CIDR blocks. See Add the CIDR blocks of DTS servers.
Adding DTS server CIDR blocks to a whitelist or security group introduces security exposure. Before proceeding, take preventive measures such as strengthening credentials, limiting exposed ports, authenticating API calls, and auditing whitelist rules regularly. Consider using Express Connect, VPN Gateway, or Smart Access Gateway to connect the database to DTS over a private network.
Step 5: Configure tracked objects and advanced settings
Basic settings
| Parameter | Description |
|---|---|
| Data Change Types | Data Update: tracks INSERT, DELETE, and UPDATE operations on selected objects. Schema Update: tracks create, delete, and modify operations on all object schemas in the source instance. Filter the relevant changes on the change tracking client. |
| Source Objects | Select objects from the Source Objects section and click  to move them to Selected Objects. Tracking is at the table level only. Objects cannot be reselected after the task is configured. |
Advanced settings
| Parameter | Description |
|---|---|
| Monitoring and Alerting | Select Yes to receive alerts when the task fails or latency exceeds a threshold. Configure the alert threshold and notification settings. See Configure monitoring and alerting when you create a DTS task. Select No to disable alerting. |
| Retry Time for Failed Connections | The time range, in minutes, during which DTS retries a failed connection. Valid values: 10–1440. Default: 720. Set this to at least 30 minutes. If DTS reconnects within this window, the task resumes automatically; otherwise, the task fails. If multiple tasks share the same source instance, the shortest retry time across all tasks takes precedence. Fees are charged during retry. Release the DTS instance promptly if the source instance is no longer needed. |
Step 6: Run the precheck
Click Next: Save Task Settings and Precheck. DTS runs a precheck before the task starts.
-
If a check item fails, click View Details, fix the issue, then click Precheck Again.
-
If an alert is generated for an item that can be ignored, click Confirm Alert Details > Ignore > OK, then click Precheck Again. Ignoring alerts may cause data inconsistency.
To review the underlying API call parameters before saving, hover over Next: Save Task Settings and Precheck and click Preview OpenAPI parameters.
Step 7: Purchase the change tracking instance
Wait until Success Rate reaches 100%, then click Next: Purchase Instance.
On the Purchase page, configure the billing settings:
| Parameter | Description |
|---|---|
| Billing method | Subscription: prepaid, billed upfront. More cost-effective for long-term use, with lower prices for longer durations. Pay-as-you-go: billed hourly. Use this for short-term needs; release the instance when no longer needed to avoid unnecessary charges. |
| Resource Group Settings | The resource group for the instance. Default: default resource group. See What is Resource Management?. |
| Subscription Duration | Available for the Subscription billing method only. Options: 1–9 months, or 1, 2, 3, or 5 years. |
Read and select the Data Transmission Service (Pay-as-you-go) Service Terms, then click Buy and Start.
The task appears in the task list. Monitor subtask progress on the Task Topology page—DTS creates one subtask per attached ApsaraDB RDS for MySQL instance.
What to do next
After the change tracking task starts, create consumer groups and begin consuming the tracked data:
-
Create and manage consumer groups: Create consumer groups.
-
Consume the tracked data using one of the following methods: