This topic describes the data types and parameters that are supported by Tablestore Writer. For example, you can configure the write mode, field mappings, and connections for Tablestore Writer. This topic also provides an example to describe how to configure Tablestore Writer.

Tablestore is a NoSQL database service that is built on the Apsara distributed operating system. Tablestore allows you to store and access large amounts of structured data in real time. Tablestore organizes data into instances and tables. It seamlessly expands the data scale by using data sharding and load balancing technologies.

Tablestore Writer connects to a Tablestore server by using the official Java SDK and writes data to the Tablestore server by using the SDK. Tablestore Writer greatly optimizes the write process, including retry upon write timeout, retry upon exceptions, and batch submission.

Tablestore Writer supports all Tablestore data types. The following table describes the data types that are supported by Tablestore Writer.
Category Tablestore data type
Integer INTEGER
Floating point DOUBLE
String STRING
Boolean BOOLEAN
Binary BINARY
Note To write data of the INTEGER type, set the data type to INT in the code editor. Then, DataWorks converts the INT type to the INTEGER type. If you set the data type to INTEGER, an error is reported in the log and the node fails.

Parameters

Parameter Description Required Default value
datasource The name of the connection. It must be the same as the name of the added connection. You can add connections in the code editor. Yes N/A
endPoint The endpoint of the Tablestore server. Yes N/A
accessId The AccessKey ID of the account that you can use to connect to the Tablestore server. Yes N/A
accessKey The AccessKey secret of the account that you can use to connect to the Tablestore server. Yes N/A
instanceName The name of the Tablestore instance.

The instance is an entity for you to use and manage Tablestore. After you activate the Tablestore service, you must create an instance in the console before you create and manage tables. Instances are the basic units for managing Tablestore resources. All access control and resource measurement for applications are implemented at the instance level.

Yes N/A
table The name of the destination table. You can specify only one table as the destination table. Multi-table synchronization is not required for Tablestore. Yes N/A
primaryKey The primary key of the destination table in Tablestore. The primary keys are described in a JSON array. Tablestore is a NoSQL database service. You must specify the primary key of the destination table for Tablestore Writer to import data.
Note The primary keys in Tablestore must be of the STRING or INT type. The BINARY type is not supported. Therefore, Tablestore Writer supports only the STRING and INT types.
Data Integration can convert data types. Tablestore Writer can convert data that is not of the STRING or INT type to the STRING or INT type. Example:
"primaryKey" : [
    {"name":"pk1", "type":"string"},
    {"name":"pk2", "type":"int"}
                    ],
Yes N/A
column The columns in the destination table to which data is written. The columns are described in a JSON array.

Format:

{"name":"col2", "type":"INT"},

The name parameter specifies the name of the column to which data is written. The type parameter specifies the data type of the column. Tablestore supports the STRING, INT, DOUBLE, BOOL, and BINARY types.

Yes N/A
writeMode
The write mode. Valid values:
  • PutRow: the PutRow API operation for Tablestore, which is used to insert data to a specified row. If the specified row does not exist, a new row is added. Otherwise, the original row is overwritten.
  • UpdateRow: the UpdateRow API operation for Tablestore, which is used to update the data of a specified row. If the specified row does not exist, a new row is added. Otherwise, the values of the specified columns are added, modified, or deleted as requested.
Yes N/A
requestTotalSizeLimitation The maximum size of data in a row to be written to Tablestore. The value is a number. No 1 MB
attributeColumnSizeLimitation The maximum size of data in an attribute column to be written to Tablestore. The value is a number. No 2 MB
primaryKeyColumnSizeLimitation The maximum size of data in a primary key column to be written to Tablestore. The value is a number. No 1 KB
attributeColumnMaxCount The maximum number of attribute columns to be written to Tablestore. The value is a number. No 1,024

Codeless UI mode

The codeless user interface (UI) mode is not supported.

Code editor mode

The following example shows how to configure a node to write data to Tablestore by using the code editor. For more information, see Create a sync node by using the code editor.
Notice Delete the comments from the following code before you run the code.
{
    "type":"job",
    "version":"2.0",// The version number.
    "steps":[
        { 
            "stepType":"stream",
            "parameter":{},
            "name":"Reader",
            "category":"reader"
        },
        {
            "stepType":"ots",// The writer type.
            "parameter":{
                "datasource":"",// The connection name.
                "column":[// The columns to which data is written.
                    {
                        "name":"columnName1",// The name of the column.
                        "type":"INT"// The data type.
                    },
                    {
                        "name":"columnName2",
                        "type":"STRING"
                    },
                    {
                        "name":"columnName3",
                        "type":"DOUBLE"
                    },
                    {
                        "name":"columnName4",
                        "type":"BOOL"
                    },
                    {
                        "name":"columnName5",
                        "type":"BINARY"
                    }
                ],
                "writeMode":"",// The write mode.
                "table":"",// The name of the table.
                "primaryKey":[// The primary key of the destination table in Tablestore.
                    {
                        "name":"pk1",
                        "type":"STRING"
                    },
                    {
                        "name":"pk2",
                        "type":"INT"
                    }
                ]
            },
            "name":"Writer",
            "category":"writer"
        }
    ],
    "setting":{
        "errorLimit":{
            "record":"0"// The number of dirty data records.
        },
        "speed":{
            "throttle":false,// A value of false indicates that the bandwidth is not throttled. A value of true indicates that the bandwidth is throttled. The maximum transmission rate takes effect only if you set this parameter to true.
            "concurrent":1// The number of concurrent threads.
        }
    },
    "order":{
        "hops":[
            {
                "from":"Reader",
                "to":"Writer"
            }
        ]
    }
}