DRDS Writer

DRDS Writer allows you to write data to tables that are stored in Distributed Relational Database Service (DRDS) databases. DRDS Writer connects to the proxy of a remote DRDS database by using Java Database Connectivity (JDBC), and executes a REPLACE INTO statement to write data to the DRDS database.

DRDS Writer obtains data from a Data Integration reader, and executes the REPLACE INTO statement to write the data to the destination database. If no primary key conflict or unique index conflict occurs, the action is the same as the action of the INSERT INTO statement. If a conflict occurs, original rows are replaced by new rows. DRDS Writer sends data to the DRDS proxy when the amount of buffered data reaches a specific threshold. The proxy determines whether to write the data to one or more tables and how to route the data when it is written to multiple tables.

The following table describes the data types that DRDS Writer supports.

Category	DRDS data type
Integer	INT, TINYINT, SMALLINT, MEDIUMINT, BIGINT, and YEAR
Floating point	FLOAT, DOUBLE, and DECIMAL
String	VARCHAR, CHAR, TINYTEXT, TEXT, MEDIUMTEXT, and LONGTEXT
Date and time	DATE, DATETIME, TIMESTAMP, and TIME
Boolean	BIT and BOOLEAN
Binary	TINYBLOB, MEDIUMBLOB, BLOB, LONGBLOB, and VARBINARY

Complete the following steps on the configuration tab of the node:

1. Configure the connections.
   Configure the connections to the source and destination data stores for the sync node.

   Parameter	Description
   Data source	The datasource parameter in the preceding parameter description. Select a connection type and select the name of a connection that you have configured in DataWorks.
   Table	The table parameter in the preceding parameter description.
   Prepare statement before import	The preSql parameter in the preceding parameter description. Enter an SQL statement to execute before the sync node is run.
   Complete statement after import	The postSql parameter in the preceding parameter description. Enter an SQL statement to execute after the sync node is run.
   Constraint conflict	The writeMode parameter in the preceding parameter description. Select the required write mode.

2. Configure field mapping. It is equivalent to setting the column parameter in the preceding parameter description. Fields in the source table on the left have a one-to-one mapping with fields in the destination table on the right.

   Parameter	Description
   The same name mapping	Click The same name mapping to establish a mapping between fields with the same name. Note that the data types of the fields must match.
   Peer mapping	Click Peer mapping to establish a mapping between fields in the same row. Note that the data types of the fields must match.
   Unmap	Click Unmap to remove mappings that have been established.
   Automatic typesetting	Click Automatic typesetting to sort the fields based on specified rules.

3. Configure channel control policies.

   Parameter	Description
   Maximum number of concurrent tasks expected	The maximum number of concurrent threads that the sync node uses to read data from or write data to data stores. You can configure the concurrency for the node on the codeless UI.
   Synchronization rate	Specifies whether to enable bandwidth throttling. You can enable bandwidth throttling and set a maximum transmission rate to avoid heavy read workload of the source. We recommend that you enable bandwidth throttling and set the maximum transmission rate to a proper value.
   The number of error records exceeds	The maximum number of dirty data records allowed.

The following example shows how to configure a sync node to write data to a DRDS database. For more information, see Create a sync node by using the code editor.

{
    "type":"job",
    "version":"2.0", // The version number.
    "steps":[
        {
            "stepType":"stream",
            "parameter":{},
            "name":"Reader",
            "category":"reader"
                },
        {
            "stepType":"drds", // The writer type.
            "parameter":{
                "postSql":[], // The SQL statement to execute after the sync node is run.
                "datasource":"", // The connection name.
                "column":[ // The columns to which data is written.
                "id"
                ],
                "writeMode":"insert ignore",
                "batchSize":"1024", // The number of data records to write at a time.
                "table":"test", // The name of the destination table.
                "preSql":[] // The SQL statement to execute before the sync node is run.
                },
            "name":"Writer",
            "category":"writer"
                }
                ],
    "setting":{
        "errorLimit":{
        "record":"0" // The maximum number of dirty data records allowed.
            },
        "speed":{
            "throttle":false, // Specifies whether to enable bandwidth throttling.
            "concurrent":1 // The maximum number of concurrent threads.
                }
            },
    "order":{
        "hops":[
            {
                "from":"Reader",
                "to":"Writer"
                }
            ]
        }
    }