This topic describes the parameters supported by KingbaseES Writer and how to configure it by using the codeless user interface (UI) or code editor.

Background information

KingbaseES Writer writes data to destination tables of a KingbaseES database. At the underlying layer, KingbaseES Writer connects to a remote KingbaseES database by using Java Database Connectivity (JDBC). KingbaseES Writer executes the INSERT INTO or REPLACE INTO statement to write data to KingbaseES. KingbaseES uses the InnoDB engine so that data is written in batches.

Users such as database administrators can use KingbaseES Writer as a data migration tool. KingbaseES Writer obtains protocol data from a data integration reader, and writes the data to the destination database based on the value of the writeMode parameter.
Note A synchronization node that uses KingbaseES Writer must have at least the permission to execute the INSERT INTO or REPLACE INTO statement. Whether other permissions are required depends on the SQL statements that you specify in the preSql and postSql parameters when you configure the node.

Parameters

Parameter Description Required Default value
datasource The name of the data source. It must be the same as the name of the added data source. You can add data sources in the code editor. Yes No default value
table The name of the table you want to synchronize. Yes No default value
column The columns in the destination table. Separate the columns with commas (,). Example: "column":["id","name","age"].

To write data to all the columns in the destination table, set the value to an asterisk (*), for example, "column":["*"].

Yes No default value
preSql The SQL statement to execute before the sync node is run. You can execute only one SQL statement on the codeless UI and multiple SQL statements in the code editor. For example, to clear old data from the table before execution:
truncate table tablename
.
Note If you specify multiple SQL statements, they may not be executed in the same transaction.
No No default value
postSql The SQL statement to execute after the sync node is run. You can execute only one SQL statement on the codeless UI and multiple SQL statements in the code editor. For example, you can execute the alter table tablename add colname timestamp DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP statement to add a timestamp after data synchronization. No No default value
batchSize The number of records that can be submitted at a time. You can set this parameter to reduce the frequency of interaction between Data Integration and SAP HANA on the network, and increase the overall throughput. An excessively large value may lead to the out of memory (OOM) error during the data synchronization process. No 1024

Configure KingbaseES Writer by using the codeless UI

  1. Configure a data source.
    Configure a data source for Target.KingbaseES data source
    Parameter Description
    Data source This parameter corresponds to the datasource parameter that is described in the Parameters section. Select the name of a data source that you have configured.
    Table The table parameter in the preceding parameter description.
    Statement Run Before Writing The preSql parameter in the preceding parameter description. Enter an SQL statement to execute before the sync node is run.
    Statement Run After Writing The postSql parameter in the preceding parameter description. Enter an SQL statement to execute after the sync node is run.
  2. Configure field mapping. It is equivalent to setting the column parameter in the preceding parameter description. Fields in the source table on the left have a one-to-one mapping with fields in the destination table on the right.Field Mapping section
    GUI element Description
    The same name mapping Click The same name mapping to establish a mapping between fields with the same name. Note that the data types of the fields must match.
    Peer mapping Click Peer mapping to establish a mapping between fields in the same row. Note that the data types of the fields must match.
    Unmap Click Unmap to remove mappings that have been established.
    Automatic typesetting Click Automatic typesetting to sort the fields based on specified rules.
  3. Configure channel control policies.Channel control section
    Parameter Description
    Maximum number of concurrent tasks expected The maximum number of concurrent threads that the sync node uses to read data from or write data to data stores. You can configure the concurrency for the node on the codeless UI.
    Synchronization rate Specifies whether to enable bandwidth throttling. You can enable bandwidth throttling and set a maximum transmission rate to avoid heavy read workload of the source. We recommend that you enable bandwidth throttling and set the maximum transmission rate to a proper value.
    The number of error records exceeds The maximum number of dirty data records allowed.

Configure KingbaseES Writer by using the code editor

The following code shows a configuration example:
{
    "type":"job",
    "version":"2.0",// The version number.
    "steps":[
        {
            "stepType":"stream",
            "parameter":{},
            "name":"Reader",
            "category":"reader"
        },
        {
            "stepType":"kingbasees",// The writer type.
            "parameter":{
                "postSql":[],// The SQL statement to execute after the synchronization node is run.
                "datasource":"",// The data source.
                "column":[// The columns to which data is written.
                    "id",
                    "value"
                ],
                "batchSize":1024,// The number of data records to write at a time.
                "table":"",// The name of the destination table.
                "preSql":[
                     "delete from XXX;" // The SQL statement to execute before the synchronization node is run.
                   ]
            },
            "name":"Writer",
            "category":"writer"
        }
    ],
    "setting":{
        "errorLimit":{// The maximum number of dirty data records allowed.
            "record":"0"
        },
        "speed":{
            "throttle":false,// Specifies whether to enable throttling.
            "concurrent":1 // The maximum number of concurrent jobs.
        }
    },
    "order":{
        "hops":[
            {
                "from":"Reader",
                "to":"Writer"
            }
        ]
    }
}