Tablestore:Synchronize incremental data to OSS

Step 1: Add a Tablestore data source

Configure a Tablestore data source in DataWorks to connect to the source data.

1. Log on to the DataWorks console. Switch to the target region. In the left-side navigation pane, choose Data integration > Data integration. In the drop-down list, select the desired workspace and click Go to Data Integration.

2. In the left-side navigation pane, click Data Source.

3. On the Data Sources page, click Add Data Source.

4. In the Add Data Source dialog box, search for and select Tablestore as the data source type.

5. In the Add OTS Data Source dialog box, configure the data source parameters as described in the following table.

   Parameter	Description
   Data source name	The name of the data source. The name can contain letters, digits, and underscores (_), but cannot start with a digit or an underscore (_).
   Data source description	A brief description of the data source. The description can be up to 80 characters in length.
   Region	Select the region where the Tablestore instance is located.
   Tablestore Instance Name	The name of the Tablestore instance.
   Endpoint	The endpoint of the Tablestore instance. We recommend that you use the VPC Address.
   AccessKey ID	The AccessKey ID and AccessKey Secret of your Alibaba Cloud account or RAM user.
   AccessKey Secret

6. Test the resource group connectivity.

   You must test the resource group's connectivity to the data source. The synchronization task cannot run if the resource group cannot connect to the data source.

   1. In the Connection Configuration section, click Test Network Connectivity in the Connection Status column for the resource group.

   2. After the connectivity test passes, the Connection Status changes to Connected. Click Complete. You can view the new data source in the data source list.

      If the connectivity test result is Failed, you can use the Network Connectivity Diagnostic Tool to resolve the issue yourself.

Step 2: Add an OSS data source

Configure the OSS data source as the destination for data exporting.

1. Click Add Data Source again. In the dialog box, search for and select OSS as the data source type, and then configure the parameters.

   Parameter	Description
   Data source name	The name can contain only letters, digits, and underscores (_) and cannot start with a digit or an underscore.
   Data source description	Enter a brief description of the data source, no longer than 80 characters.
   Access Mode	RAM Role Authorization Mode: The DataWorks service account accesses the data source by assuming a RAM role. If this is your first time selecting this mode, follow the on-screen instructions to grant the required permissions. Access Key Mode: Access the data source by using the AccessKey ID and AccessKey Secret of an Alibaba Cloud account or RAM user.
   Select Role	You only need to select a RAM role when the Access Mode is RAM Role Authorization Mode.
   AccessKey ID	The AccessKey ID and AccessKey Secret of an Alibaba Cloud account or a RAM user are required only when Access Mode is set to AccessKey Mode.
   AccessKey secret
   Region	The region where the bucket is located.
   Endpoint	For OSS access domain names, see Regions and Endpoints.
   Bucket	The name of the bucket.

2. After you configure the parameters and the connectivity test passes, click Complete.

Step 3: Configure a batch synchronization task

Create and configure a data synchronization task to define the data transfer rules from Tablestore to OSS.

{
    "type": "job",
    "version": "2.0",
    "steps": [
        {
            "stepType": "otsstream",
            "parameter": {
                "statusTable": "TableStoreStreamReaderStatusTable",
                "maxRetries": 31,
                "isExportSequenceInfo": false,
                "datasource": "source_data",
                "column": [
                    "id",
                    "name",
                    "colName",
                    "version",
                    "colValue",
                    "opType",
                    "sequenceInfo"
                ],
                "startTimeString": "${startTime}",
                "table": "source_table",
                "endTimeString": "${endTime}"
            },
            "name": "Reader",
            "category": "reader"
        },
        {
            "stepType": "oss",
            "parameter": {
                "dateFormat": "yyyy-MM-dd HH:mm:ss",
                "datasource": "target_data",
                "writeSingleObject": false,
                "column": [
                    "0",
                    "1",
                    "2",
                    "3",
                    "4",
                    "5",
                    "6"
                ],
                "writeMode": "truncate",
                "encoding": "UTF-8",
                "fieldDelimiter": ",",
                "fileFormat": "csv",
                "object": "tablestore/source_table.csv"
            },
            "name": "Writer",
            "category": "writer"
        }
    ],
    "setting": {
        "errorLimit": {
            "record": "0"
        },
        "speed": {
            "concurrent": 2,
            "throttle": false
        }
    },
    "order": {
        "hops": [
            {
                "from": "Reader",
                "to": "Writer"
            }
        ]
    }
}

{
    "type": "job",
    "version": "2.0",
    "steps": [
        {
            "stepType": "otsstream",
            "parameter": {
                "statusTable": "TableStoreStreamReaderStatusTable",
                "maxRetries": 31,
                "isExportSequenceInfo": false,
                "datasource": "source_data",
                "column": [
                  {
                    "name": "_m_name"
                  },
                  {
                    "name": "_data_source"
                  },
                  {
                    "name": "_tags"
                  },
                  {
                    "name": "_time"
                  },
                  {
                    "name": "value",
                    "type": "int"
                  }
                ],
                "startTimeString": "${startTime}",
                "table": "source_series",
                "isTimeseriesTable":"true",
                "mode": "single_version_and_update_only",
                "endTimeString": "${endTime}"
            },
            "name": "Reader",
            "category": "reader"
        },
        {
            "stepType": "oss",
            "parameter": {
                "dateFormat": "yyyy-MM-dd HH:mm:ss",
                "datasource": "target_data",
                "writeSingleObject": false,
                "writeMode": "truncate",
                "encoding": "UTF-8",
                "fieldDelimiter": ",",
                "fileFormat": "csv",
                "object": "tablestore/source_series.csv"
            },
            "name": "Writer",
            "category": "writer"
        }
    ],
    "setting": {
        "errorLimit": {
            "record": "0"
        },
        "speed": {
            "concurrent": 2,
            "throttle": false
        }
    },
    "order": {
        "hops": [
            {
                "from": "Reader",
                "to": "Writer"
            }
        ]
    }
}

Step 4: View the synchronization result

After the synchronization task is complete, you can view the execution status in the log and check the resulting file in the OSS bucket.

1. View the task execution status and result at the bottom of the page. The following log information indicates that the synchronization task is successful.

   2025-11-18 11:16:23 INFO Shell run successfully!
   2025-11-18 11:16:23 INFO Current task status: FINISH
   2025-11-18 11:16:23 INFO Cost time is: 77.208s

2. View the file in the destination bucket.

   Go to the Bucket List, click the destination bucket, and then view or download the result file.