Consumer groups let multiple independent consumers track change data from the same source database simultaneously. This helps you reduce the cost of tracking data changes.
Prerequisites
Before you begin, ensure that you have:
A change tracking instance with a status of Normal. For more information, see Overview of change tracking scenarios.
Limits
Each change tracking instance supports a maximum of 20 consumer groups.
Each consumer group supports only one consumer.
If a single tracked row exceeds 16 MB, an out of memory (OOM) error may occur on the change tracking client.
DTS reads incremental data through the DStore module or a client. Switching between them may cause data duplication.
Create a consumer group
Go to the Change Tracking Tasks page using one of the following methods, then select the region where your change tracking instance resides.
DTS console
Log on to the DTS console.
In the left-side navigation pane, click Change Tracking.
In the upper-left corner, select the region where your change tracking instance resides.
DMS console
NoteThe navigation paths in the DMS console vary depending on your console mode and layout. For more information, see Simple mode and Customize the layout and style of the DMS console.
Log on to the DMS console.
In the top navigation bar, choose Data + AI > DTS (DTS) > Change Tracking.
To the right of Change Tracking Tasks, select the region where your change tracking instance resides.
Find the change tracking instance and click its instance ID.
In the left-side navigation pane, click Consume Data.
On the Consume Data page, click Add Consumer Group in the upper-left corner.
In the Add Consumer Group dialog box, configure the following parameters.
Parameter Description Consumer Group Name Enter a name that identifies the consumer group and its purpose. Account Enter a username for the consumer group. The username must be 1–16 characters and can contain uppercase letters, lowercase letters, digits, and underscores (_). Password Enter a password for the account. The password must be 8–32 characters and contain at least two of the following: uppercase letters, lowercase letters, digits, and special characters. Confirm Password Re-enter the password to confirm. Click OK.
What to do next
After creating consumer groups, choose a consumption mode based on your source database type and client.
| Consumption mode | Applicable databases | Notes |
|---|---|---|
| Use the SDK demo to consume tracked data from a PolarDB-X 1.0 instance | PolarDB-X 1.0, DMS LogicDB | The only supported consumption mode for these database types. |
| Use the SDK demo to consume tracked data | MySQL, PolarDB for MySQL, PostgreSQL, Oracle, PolarDB (Compatible with Oracle), PolarDB for PostgreSQL, PolarDB-X 2.0 | Recommended for most scenarios. |
| Use flink-dts-connector to consume tracked data | — | Use this mode when consuming tracked data with Apache Flink. Apache Flink is a framework and distributed processing engine. |
| Use a Kafka client to consume tracked data | — | Use this mode when consuming tracked data with a Kafka client and custom source code. |