After you configure a change tracking task, you can use the SDK demo code that is provided by Data Transmission Service (DTS) to track and consume data. This topic describes how to use the SDK demo code to consume tracked data.
Procedure
Notice If you use a RAM user to track and consume data, the RAM user must have the AliyunDTSFullAccess permission and the permissions to access the source objects. For more information,
see Use a system policy to authorize a RAM user to manage DTS instances and Grant permissions to a RAM user.
IntelliJ IDEA (Community Edition 2020.1 Windows) is used in this example.
Save and query the consumer offset
When the SDK client is started for the first time or restarted, or an internal retry occurs, you need to query and specify the consumer offset to start or resume data consumption. The following table describes how to manage and query the consumer offset in different situations. To implement on-demand data consumption, you must ensure that no data is lost or duplicate.Scenario | SDK usage mode | Query method |
---|---|---|
Query the consumer offset | ASSIGN and SUBSCRIBE |
|
When you start the SDK client for the first time, you must specify the consumer offset to consume data. | ASSIGN and SUBSCRIBE | Select the DTSConsumerAssignDemo.java or DTSConsumerSubscribeDemo.java file based on the mode in which you use the SDK client. Then, specify the initCheckpoint parameter. For more information, see Step 8 and Step 9.
|
If an internal retry occurs, you must specify the consumer offset of the last data record to resume data consumption. | ASSIGN | Perform the following steps to find the consumer offset of the last data record:
|
SUBSCRIBE | Perform the following steps to find the consumer offset of the last data record:
|
|
After the SDK client is restarted, you must specify the consumer offset of the last data record to resume data consumption. | ASSIGN | Check the setting of the setForceUseCheckpoint parameter in the consumerContext.java file and query the consumer offset.
|
SUBSCRIBE | In this mode, the setting of the setForceUseCheckpoint parameter in the consumerContext.java file does not take effect. Perform the following steps to find the consumer offset
of the last data record:
|
Troubleshooting
Issue | Error message | Cause | Solution |
---|---|---|---|
Connection failed |
|
The specified brokerUrl is invalid.
|
Enter the valid brokerUrl , userName , and password . For more information, see Table 1.
|
|
The broker address cannot be redirected to the real IP address. | ||
|
The specified username and password are invalid. | ||
|
The setUseCheckpoint parameter in the consumerContext.java file is set to true , but the consumer offset is not within the data range of the change tracking instance.
|
Enter the consumer offset within the data range of the change tracking instance. For more information, see Table 1. | |
The response time of data consumption is increased. | None | You can analyze the cause by querying the DStoreRecordQueue and DefaultUserRecordQueue parameters. For more information, see Table 2.
|