Tablestore:Errors that may occur when OTSStream Reader is running

Problem description

When you run an OTSStreamReader task, you receive the following error message:

com.alibaba.datax.common.exception.DataXException: [  code:OTSStreamReaderError, messageOTS Stream Reader Error] – com.alibaba.datax.plugin.reader.otsstreamreader.internal.OTSStreamReaderException: Must set date or time range millis or time range string, please check your config. – com.alibaba.datax.plugin.reader.otsstreamreader.internal.OTSStreamReaderException: Must set date or time range millis or time range string, please check your config.
        at com.alibaba.datax.plugin.reader.otsstreamreader.internal.config.OTSStreamReaderConfig.load(OTSStreamReaderConfig.java:201)
        at com.alibaba.datax.plugin.reader.otsstreamreader.internal.OTSStreamReader$Job.init(OTSStreamReader.java:30)
        at com.alibaba.datax.core.job.JobContainer.initJobReader(JobContainer.java:1085)
        at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:497)
        at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:220)
        at com.alibaba.datax.core.Engine.start(Engine.java:100)

Possible cause

The parameters that specify the timestamp range of incremental data are not configured in the configuration script of OTSStream Reader.

Solutions

Specify the timestamp range of incremental data by using one of the following methods:

• Set the parameter.date parameter to the date of the incremental data that you want to export.

• Set the parameter.startTimestampMillis parameter to the start timestamp of the timestamp range of incremental data and the parameter.endTimestampMillis parameter to the end timestamp of the timestamp range of incremental data.

For more information about how to configure the timestamp range of incremental data, see Synchronize incremental data to OSS.

Problem description

When you run an OTSStreamReader task, you receive the following error message:

[ code:OTSStreamReaderError, messageOTS Stream Reader Error] – com.alibaba.datax.plugin.reader.otsstreamreader.internal.OTSStreamReaderException: The stream of data table is not enabled. – com.alibaba.datax.plugin.reader.otsstreamreader.internal.OTSStreamReaderException: The stream of data table is not enabled.
        at com.alibaba.datax.plugin.reader.otsstreamreader.internal.core.OTSStreamReaderChecker.checkStreamEnabledAndTimeRangeOK(OTSStreamReaderChecker.java:49)
        at com.alibaba.datax.plugin.reader.otsstreamreader.internal.OTSStreamReaderMasterProxy.init(OTSStreamReaderMasterProxy.java:38)
        at com.alibaba.datax.plugin.reader.otsstreamreader.internal.OTSStreamReader$Job.init(OTSStreamReader.java:31)
        at com.alibaba.datax.core.job.JobContainer.initJobReader(JobContainer.java:1085)
        at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:497)
        at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:220)
        at com.alibaba.datax.core.Engine.start(Engine.java:100)
        at com.alibaba.datax.core.Engine.entry(Engine.java:321)
        at com.alibaba.datax.core.Engine.main(Engine.java:354)

Possible cause

The Stream feature is disabled for the data table that is specified by the parameter.table parameter in OTSStream Reader.

Solutions

Enable the Stream feature for the data table that is specified by the parameter.table parameter in OTSStream Reader by using one of the following methods:

Problem description

When you run an OTSStreamReader task, you receive the following error message:

com.alibaba.datax.common.exception.DataXException: [ code:OTSStreamReaderError, messageOTS Stream Reader Error] – com.alibaba.datax.plugin.reader.otsstreamreader.internal.OTSStreamReaderException: As expiration time is 25200000, so the start timestamp must greater than 2021-09-25T20:09Z(1632600562281) – com.alibaba.datax.plugin.reader.otsstreamreader.internal.OTSStreamReaderException: As expiration time is 25200000, so the start timestamp must greater than 2021-09-25T20:09Z(1632600562281)
        at com.alibaba.datax.plugin.reader.otsstreamreader.internal.core.OTSStreamReaderChecker.checkStreamEnabledAndTimeRangeOK(OTSStreamReaderChecker.java:60)
        at com.alibaba.datax.plugin.reader.otsstreamreader.internal.OTSStreamReaderMasterProxy.init(OTSStreamReaderMasterProxy.java:38)
        at com.alibaba.datax.plugin.reader.otsstreamreader.internal.OTSStreamReader$Job.init(OTSStreamReader.java:31)
        at com.alibaba.datax.core.job.JobContainer.initJobReader(JobContainer.java:1085)
        at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:497)
        at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:220)
        at com.alibaba.datax.core.Engine.start(Engine.java:100)
        at com.alibaba.datax.core.Engine.entry(Engine.java:321)

Possible cause

The startTime parameter that specifies the start timestamp of the timestamp range of incremental logs is not correctly configured based on the log expiration time.

Solutions

Set the startTime parameter in the synchronization script to a value that is greater than the value that is obtained by using the following formula: Synchronization task start time - Log expiration time + 10 minutes.

You can perform the following steps to view the log expiration time:

1. Go to the Manage Table page.

   1. Log on to the Tablestore console.

   2. In the upper part of the page, select a resource group and a region.

   3. On the Overview page, click the name of the instance.

   4. On the Instance Details tab, in the Tables area, click the name of the data table.

2. On the Manage Table page, click the Basic Information tab to view the log expiration time.

Problem description

When you run an OTSStreamReader task, you receive the following error message:

com.alibaba.datax.common.exception.DataXException: [ code:OTSStreamReaderError, messageOTS Stream Reader Error] – com.alibaba.datax.plugin.reader
.otsstreamreader.internal.OTSStreamReaderException: To avoid timing error between different machines, the end timestamp must smaller than 2021-09
-26T03:59Z(1632628796433) – com.alibaba.datax.plugin.reader.otsstreamreader.internal.OTSStreamReaderException: To avoid timing error between diff
erent machines, the end timestamp must smaller than 2021-09-26T03:59Z(1632628796433)
        at com.alibaba.datax.plugin.reader.otsstreamreader.internal.core.OTSStreamReaderChecker.checkStreamEnabledAndTimeRangeOK(OTSStreamReaderC
hecker.java:65)
        at com.alibaba.datax.plugin.reader.otsstreamreader.internal.OTSStreamReaderMasterProxy.init(OTSStreamReaderMasterProxy.java:38)
        at com.alibaba.datax.plugin.reader.otsstreamreader.internal.OTSStreamReader$Job.init(OTSStreamReader.java:31)
        at com.alibaba.datax.core.job.JobContainer.initJobReader(JobContainer.java:1085)
        at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:497)
        at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:220)
        at com.alibaba.datax.core.Engine.start(Engine.java:100)
        at com.alibaba.datax.core.Engine.entry(Engine.java:321)
        at com.alibaba.datax.core.Engine.main(Engine.java:354)
        at com.alibaba.datax.common.exception.DataXException.asDataXException(DataException.java:41)

Possible cause

The value of the endTime parameter is invalid.

Solutions

Set the endTime parameter in the synchronization script to a value that is smaller than the value that is obtained by using the following formula: Synchronization task start time - 5 minutes.

Problem description

The "The item of column must be map object" error message appears when OTSStream Reader is running. The following sample code provides an example of the error message.

[code:OTSStreamReaderError,messageOTSStreamReaderError]-com.alibaba.datax.plugin.reader.otsstreamreader.internal.OTSStreamReaderException:Parsecolumnfail,pleasecheckyourconfig.-com.alibaba.datax.plugin.reader.otsstreamreader.internal.OTSStreamReaderException:Parsecolumnfail,pleasecheckyourconfig.
atcom.alibaba.datax.plugin.reader.otsstreamreader.internal.config.OTSStreamReaderConfig.parseConfigForSingleVersionAndUpdateOnlyMode(OTSStreamReaderConfig.java:178)
atcom.alibaba.datax.plugin.reader.otsstreamreader.internal.config.OTSStreamReaderConfig.load(OTSStreamReaderConfig.java:267)
atcom.alibaba.datax.plugin.reader.otsstreamreader.internal.OTSStreamReader$Job.init(OTSStreamReader.java:30)
atcom.alibaba.datax.core.job.JobContainer.initJobReader(JobContainer.java:1083)
atcom.alibaba.datax.core.job.JobContainer.init(JobContainer.java:497)
atcom.alibaba.datax.core.job.JobContainer.start(JobContainer.java:220)
atcom.alibaba.datax.core.Engine.start(Engine.java:100)
atcom.alibaba.datax.core.Engine.entry(Engine.java:318)
atcom.alibaba.datax.core.Engine.main(Engine.java:351)
Causedby:java.lang.IllegalArgumentException:Theitemofcolumnmustbemapobject,pleasecheckyourinput.

Possible cause

The format of the parameter.column parameter is invalid.

Solutions

Make sure that the parameter.column parameter is in the correct format The following sample code provides examples of the format:

• Invalid format

  "column":[
    "col1",
    "col2"
  ]

• Valid format

  "column":[
    {
       "name" : "col1"
    },
    {
       "name" : "col2"
    }
  ]