This topic describes the data types and parameters that are supported by Tablestore Writer and how to configure Tablestore Writer by using the codeless user interface (UI) and code editor.

Tablestore is a NoSQL database service that is built on the Apsara distributed operating system. The service allows you to store and access large volumes of structured data in real time. The data is stored in the tables that are created in Tablestore instances. Tablestore uses the data sharding and load balancing technologies to seamlessly expand the data scale.

Tablestore Writer connects to and writes data to the Tablestore server by using Tablestore SDK for Java. Tablestore Writer provides some features to optimize the write process, such as retry after write timeouts, retry after exceptions, and batch submission.

Tablestore Writer supports all Tablestore data types. The following table lists the data types that are supported by Tablestore Writer.
Category Tablestore data type
Integer INTEGER
Floating point DOUBLE
String STRING
Boolean BOOLEAN
Binary BINARY
Note To write data of the INTEGER type, set the data type to INT in the code editor. During data synchronization, Tablestore Writer converts the data from the INT data type to the INTEGER data type. If you set the data type to INTEGER, an error is reported in the log, and the synchronization node fails.

Parameters

Parameter Description Required Default value
datasource The name of the data source. It must be the same as the name of the added data source. You can add data sources by using the code editor. Yes No default value
endPoint The endpoint of the Tablestore server. Yes No default value
accessId The AccessKey ID of the account that you use to connect to the Tablestore server. Yes No default value
accessKey The AccessKey secret of the account that you use to connect to the Tablestore server. Yes No default value
instanceName The name of the Tablestore instance.

After you activate the Tablestore service, you must create an instance in the Tablestore console before you create and manage tables. Tablestore instances are the basic units that you can use to manage your Tablestore resources. Access control and resource measurement for applications are performed at the instance level.

Yes No default value
table The name of the table to which you want to write data. You can specify only one table. Data synchronization to multiple tables is not supported for Tablestore. Yes No default value
primaryKey The primary keys of the destination table in Tablestore. Specify the primary keys in a JSON array. Tablestore is a NoSQL database service. If you want to use Tablestore Writer to write data, you must specify the primary keys of the destination table.
Data Integration supports data type conversion. Tablestore Writer can convert data from a data type other than STRING and INT to the STRING or INT data type. The following code provides a configuration example: 
"primaryKey" : [
    {"name":"pk1", "type":"string"},
    {"name":"pk2", "type":"int"}
                    ],
Note
  • The primary keys in Tablestore must be of the STRING or INT type. Therefore, you must set the data type of a primary key to STRING or INT in the code editor.

  • You must set the primaryKey parameter to a value that is different from the value of the column parameter.

Yes No default value
column The names of the columns to which you want to write data. Specify the names in a JSON array.
Specify this parameter in the following format:
"column" : [
{"name":"col2", "type":"INT"},
{"name":"col3", "type":"STRING"}
                    ],
The name parameter specifies the name of the column to which you want to write. The type parameter specifies the data type of the column. Tablestore supports the following data types: STRING, INT, DOUBLE, BOOLEAN, and BINARY.
Note

You must set the primaryKey parameter to a value that is different from the value of the column parameter.

Yes No default value
writeMode
The write mode. Valid values:
  • PutRow: the PutRow API operation for Tablestore, which is used to insert data to a specific row. If the row does not exist, a new row is added. If the row exists, the row is overwritten.
  • UpdateRow: the UpdateRow API operation for Tablestore, which is used to update the data of a specific row. If the row does not exist, a new row is added. If the row exists, the values of the specified columns in the row are added, modified, or removed based on actual conditions.
Yes No default value
requestTotalSizeLimitation The maximum size of data that can be written to a single row in Tablestore. The parameter value must be of a numeric data type. No 1MB
attributeColumnSizeLimitation The maximum size of data that can be written to a single attribute column in Tablestore. The parameter value is of a numeric data type. No 2MB
primaryKeyColumnSizeLimitation The maximum size of data that can be written to a single primary key column in Tablestore. The parameter value is of a numeric data type. No 1KB
attributeColumnMaxCount The maximum number of attribute columns to which data can be written in Tablestore. The parameter value is of a numeric data type. No 1,024

Configure Tablestore Writer by using the codeless UI

This method is not supported.

Configure Tablestore Writer by using the code editor

In the following code, a synchronization node is configured to write data to Tablestore. For more information about how to configure a synchronization node by using the code editor, see Create a sync node by using the code editor.
Notice Delete the comments from the following code before you run the code. 
{
    "type":"job",
    "version":"2.0",// The version number. 
    "steps":[
        { 
            "stepType":"stream",
            "parameter":{},
            "name":"Reader",
            "category":"reader"
        },
        {
            "stepType":"ots",// The writer type. 
            "parameter":{
                "datasource":"",// The name of the data source. 
                "column":[// The names of the columns to which you want to write data. 
                    {
                        "name":"columnName1",// The name of a column. 
                        "type":"INT"// The data type of a column. 
                    },
                    {
                        "name":"columnName2",
                        "type":"STRING"
                    },
                    {
                        "name":"columnName3",
                        "type":"DOUBLE"
                    },
                    {
                        "name":"columnName4",
                        "type":"BOOL"
                    },
                    {
                        "name":"columnName5",
                        "type":"BINARY"
                    }
                ],
                "writeMode":"",// The write mode. 
                "table":"",// The name of the table to which you want to write data. 
                "primaryKey":[// The primary keys of the destination table in Tablestore. 
                    {
                        "name":"pk1",
                        "type":"STRING"
                    },
                    {
                        "name":"pk2",
                        "type":"INT"
                    }
                ]
            },
            "name":"Writer",
            "category":"writer"
        }
    ],
    "setting":{
        "errorLimit":{
            "record":"0"// The maximum number of dirty data records allowed. 
        },
        "speed":{
            "throttle":true,// Specifies whether to enable bandwidth throttling. The value false indicates that bandwidth throttling is disabled, and the value true indicates that bandwidth throttling is enabled. The mbps parameter takes effect only when the throttle parameter is set to true. 
            "concurrent":1, // The maximum number of parallel threads. 
            "mbps":"12"// The maximum transmission rate.
        }
    },
    "order":{
        "hops":[
            {
                "from":"Reader",
                "to":"Writer"
            }
        ]
    }
}