This topic describes how to troubleshoot issues with the new version of change tracking tasks by using a dedicated Data Transmission Service (DTS) toolkit on your change tracking client.

Background information

It is difficult to troubleshoot issues with change tracking tasks in the following scenarios: tracked data is consumed without using SDKs, lots of changes are made to SDKs, and the change tracking client runs in different environments. DTS provides a toolkit to help troubleshoot issues with your change tracking tasks.

Important If you use the toolkit, the consumer offset of the specified consumer group in the console is updated, but the consumer offset recorded on the server in SUBSCRIBE mode is not updated.

Prerequisites

Java Development Kit (JDK) is installed on your change tracking client. We recommend that you install JDK 1.8 or later.

Procedure

  1. Download the dts_subscribe_sdk_dep_demo toolkit to the client and decompress it.
  2. Modify the config configuration file by configuring the parameters described in the following table.
    Table 1. Parameters in the configuration file
    ParameterDescriptionMethod to obtain the parameter value
    brokerUrlThe endpoint and port number of the change tracking instance.
    Note If you track data changes over internal networks, the network latency is minimal. This is applicable if the Elastic Compute Service (ECS) instance where you deploy the SDK client belongs to the classic network or the same virtual private cloud (VPC) as the change tracking instance.
    		In the DTS console, click the instance ID. On the Task Management page, you can obtain the endpoint and port number. Endpoint
    topicThe name of the topic of the change tracking instance. In the DTS console, click the instance ID. On the Task Management page, you can obtain the tracked Topic. Topic
    sidThe ID of the consumer group. In the DTS console, click the instance ID. In the left-side navigation pane, click Consume Data. You can obtain the ID and account of the consumer group.
    Note The password of the consumer group account is automatically specified when you create the consumer group.
    Data consumption
    userNameThe account of the consumer group.
    Warning If you are not using the SDK client that is used in this topic, you must specify this parameter in the following format: <Username>-<Consumer group ID>. Otherwise, the connection fails. Example: dtstest-dtsae******bpv.
    passwordThe password of the account.
    initCheckpointThe consumer offset. It is the time when the SDK client consumes the first data record. The value is a UNIX timestamp. Example: 1620962769.
    Important
    • This parameter takes effect only if the subscribeMode parameter is set to ASSIGN and the isForceUseInitCheckpoint parameter is set to true.
    • The consumer offset can be used in the following scenarios:
      • After the consumption process is interrupted, you can specify the consumer offset to resume data consumption. This allows you to prevent data loss.
      • When you start the change tracking client, you can specify the consumer offset to consume data on demand.
    		The consumer offset must be within the data range of the change tracking instance. The following figure shows the data range. The consumer offset must be converted to a UNIX timestamp. Data range
    Note You can use a search engine to obtain a UNIX timestamp converter.
    subscribeModeThe mode in which you use the SDK client. Valid values:
    • ASSIGN: Only a single SDK client can be used for a consumer group.
    • SUBSCRIBE: Multiple SDK clients can be used for a consumer group to implement disaster recovery.
    		N/A
    isForceUseInitCheckpointSpecifies whether to forcefully use the specified offset to track data changes. Valid values: true and false. N/A
  3. Run the following command in the directory in which the toolkit is decompressed:
    java -jar dts_subscribe_sdk_dep_demo-1.0-SNAPSHOT-jar-with-dependencies.jar config
  4. View the dts-new-subscribe.log log file in the directory in which the command is run. Refer to the following table to troubleshoot issues.
    Log dataDescriptionSolution
    [2022-01-04 17:10:53.949] [INFO ] 
[com.aliyun.dts.subscribe.clients.recordprocessor.EtlRecordProcessor] 
[com.aliyun.dts.subscribe.clients.recordprocessor.DefaultRecordPrintListener:49] -
RecordID [13082769]
RecordTimestamp [1641284702]
Source [{"sourceType": "MySQL", "version": "5.6.16-log"}]
RecordType [HEARTBEAT]
    		The change tracking task is running as expected. N/A
    ERROR CheckResult{isOk=false, errMsg='telnet dts-cn-hangzhou.aliyuncs.com:18009 failed, please check the network and if the brokerUrl is correct'} (com.aliyun.dts.subscribe.clients.DefaultDTSConsumer)
    		The specified endpoint of the change tracking instance is invalid. Specify a valid value for the brokerUrl parameter in the config configuration file. 
    telnet real node xxx failed, please check the network
    		No connections can be established to the change tracking instance. Specify a valid value for the brokerUrl parameter in the config configuration file. 
    ERROR CheckResult{isOk=false, errMsg='build kafka consumer failed, error: org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata, probably the user name or password is wrong'} (com.aliyun.dts.subscribe.clients.DefaultDTSConsumer)
    		The specified username or password is invalid. Specify a valid value for the userName and password parameters in the config configuration file. 
    com.aliyun.dts.subscribe.clients.exception.TimestampSeekException: RecordGenerator:seek timestamp for topic [cn_hangzhou_rm_bp11tv2923n87081s_rdsdt_dtsacct-0] with timestamp [1610249501] failed
    		The consumer offset falls because the consumer offset is not within the data range of the change tracking instance. Specify a value within the data range of the change tracking instance for the initCheckpoint parameter in the config configuration file.
    Note If the value of the DStoreRecordQueue and DefaultUserRecordQueue parameters remains 0, data is read from the server at a slow rate. If the value of these parameters remains 512, the client consumes tracked data changes at a slow rate.