Tablestore:Synchronize incremental data to OSS

Step 1: Add a Tablestore data source

Configure a Tablestore data source in DataWorks to connect to the source data.

1. Log on to the DataWorks console. Switch to the destination region. In the navigation pane on the left, choose Data Integration > Data Integration. From the drop-down list, select the workspace and click Go To Data Integration.

2. In the navigation pane on the left, click Data source.

3. On the Data Sources page, click Add Data Source.

4. In the Add Data Source dialog box, search for and select Tablestore as the data source type.

5. In the Add OTS Data Source dialog box, configure the data source parameters as described in the following table.

   Parameter	Description
   Data Source Name	The data source name must be a combination of letters, digits, and underscores (_). It cannot start with a digit or an underscore (_).
   Data Source Description	A brief description of the data source. The description cannot exceed 80 characters in length.
   Region	Select the region where the Tablestore instance resides.
   Tablestore Instance Name	The name of the Tablestore instance.
   Endpoint	The endpoint of the Tablestore instance. Use the VPC address.
   AccessKey ID	The AccessKey ID and AccessKey secret of the Alibaba Cloud account or RAM user.
   AccessKey Secret

6. Test the resource group connectivity.

   When you create a data source, you must test the connectivity of the resource group to ensure that the resource group for the sync task can connect to the data source. Otherwise, the data sync task cannot run.

   1. In the Connection Configuration section, click Test Network Connectivity in the Connection Status column for the resource group.

   2. After the connectivity test passes, click Complete. The new data source appears in the data source list.

      If the connectivity test fails, use the Network Connectivity Diagnostic Tool to troubleshoot the issue.

Step 2: Add an OSS data source

Configure the OSS data source as the destination for data exporting.

1. Click Add Data Source again. In the dialog box, search for and select OSS as the data source type, and then configure the data source parameters.

   Parameter	Description
   Data Source Name	The data source name must consist of letters, digits, and underscores (_). It cannot start with a digit or an underscore (_).
   Data Source Description	A brief description of the data source. The description cannot exceed 80 characters.
   Access Mode	RAM Role Authorization Mode: The DataWorks service account accesses the data source by assuming a RAM role. If this is the first time you select this mode, follow the on-screen instructions to grant the required permissions. AccessKey Mode: Access the data source using the AccessKey ID and AccessKey secret of an Alibaba Cloud account or RAM user.
   Role	This parameter is required only when you set Access Mode to RAM Role Authorization Mode.
   AccessKey ID	These parameters are required only when you set Access Mode to AccessKey Mode. The AccessKey ID and AccessKey secret of the Alibaba Cloud account or RAM user.
   AccessKey Secret
   Region	The region where the bucket is located.
   Endpoint	The OSS domain name. For more information, see OSS regions and endpoints.
   Bucket	The name of the bucket.

2. After you configure the parameters and the connectivity test passes, click Complete to add the data source.

Step 3: Configure a batch synchronization task

Create and configure a data synchronization task to define the data transfer rules from Tablestore to OSS.

{
    "type": "job",
    "version": "2.0",
    "steps": [
        {
            "stepType": "otsstream",
            "parameter": {
                "statusTable": "TableStoreStreamReaderStatusTable",
                "maxRetries": 31,
                "isExportSequenceInfo": false,
                "datasource": "source_data",
                "column": [
                    "id",
                    "name",
                    "colName",
                    "version",
                    "colValue",
                    "opType",
                    "sequenceInfo"
                ],
                "startTimeString": "${startTime}",
                "table": "source_table",
                "endTimeString": "${endTime}"
            },
            "name": "Reader",
            "category": "reader"
        },
        {
            "stepType": "oss",
            "parameter": {
                "dateFormat": "yyyy-MM-dd HH:mm:ss",
                "datasource": "target_data",
                "writeSingleObject": false,
                "column": [
                    "0",
                    "1",
                    "2",
                    "3",
                    "4",
                    "5",
                    "6"
                ],
                "writeMode": "truncate",
                "encoding": "UTF-8",
                "fieldDelimiter": ",",
                "fileFormat": "csv",
                "object": "tablestore/source_table.csv"
            },
            "name": "Writer",
            "category": "writer"
        }
    ],
    "setting": {
        "errorLimit": {
            "record": "0"
        },
        "speed": {
            "concurrent": 2,
            "throttle": false
        }
    },
    "order": {
        "hops": [
            {
                "from": "Reader",
                "to": "Writer"
            }
        ]
    }
}

{
    "type": "job",
    "version": "2.0",
    "steps": [
        {
            "stepType": "otsstream",
            "parameter": {
                "statusTable": "TableStoreStreamReaderStatusTable",
                "maxRetries": 31,
                "isExportSequenceInfo": false,
                "datasource": "source_data",
                "column": [
                  {
                    "name": "_m_name"
                  },
                  {
                    "name": "_data_source"
                  },
                  {
                    "name": "_tags"
                  },
                  {
                    "name": "_time"
                  },
                  {
                    "name": "value",
                    "type": "int"
                  }
                ],
                "startTimeString": "${startTime}",
                "table": "source_series",
                "isTimeseriesTable":"true",
                "mode": "single_version_and_update_only",
                "endTimeString": "${endTime}"
            },
            "name": "Reader",
            "category": "reader"
        },
        {
            "stepType": "oss",
            "parameter": {
                "dateFormat": "yyyy-MM-dd HH:mm:ss",
                "datasource": "target_data",
                "writeSingleObject": false,
                "writeMode": "truncate",
                "encoding": "UTF-8",
                "fieldDelimiter": ",",
                "fileFormat": "csv",
                "object": "tablestore/source_series.csv"
            },
            "name": "Writer",
            "category": "writer"
        }
    ],
    "setting": {
        "errorLimit": {
            "record": "0"
        },
        "speed": {
            "concurrent": 2,
            "throttle": false
        }
    },
    "order": {
        "hops": [
            {
                "from": "Reader",
                "to": "Writer"
            }
        ]
    }
}

Step 4: View the synchronization result

After the synchronization task is complete, you can view the execution status in the log and check the resulting file in the OSS bucket.

1. View the task running status and result at the bottom of the page. The following log information indicates that the sync task ran successfully.

   2025-11-18 11:16:23 INFO Shell run successfully!
   2025-11-18 11:16:23 INFO Current task status: FINISH
   2025-11-18 11:16:23 INFO Cost time is: 77.208s

2. View the file in the destination bucket.

   Go to the Bucket List. Click the destination bucket to view or download the result file.