This topic describes the data types and parameters that are supported by AnalyticDB for MySQL 3.0 Writer. For example, you can configure field mappings and connections for AnalyticDB for MySQL 3.0 Writer. This topic also provides an example to describe how to configure AnalyticDB for MySQL 3.0 Writer.

You must configure an AnalyticDB for MySQL 3.0 connection before you configure AnalyticDB for MySQL 3.0 Writer. For more information, see Configure an AnalyticDB for MySQL 3.0 connection.

Data types

The following table describes the AnalyticDB for MySQL 3.0 data types that are supported by AnalyticDB for MySQL 3.0 Writer.
Category AnalyticDB for MySQL 3.0 data type
Integer INT, INTEGER, TINYINT, SMALLINT, and BIGINT
Floating point FLOAT, DOUBLE, and DECIMAL
String VARCHAR
Date and time DATE, DATETIME, TIMESTAMP, and TIME
Boolean BOOLEAN

Parameters

Parameter Description Required Default value
datasource The connection name. It must be the same as the name of the added connection. You can add connections in the code editor. Yes N/A
table The name of the table to be synchronized. Yes N/A
writeMode The write mode. Valid values: insert into and replace into.
  • insert into: If a primary key conflict or unique index conflict occurs, data cannot be written to the conflicting rows and is regarded as dirty data.
  • replace into: If no primary key conflict or unique index conflict occurs, the action is the same as that of insert into. If a conflict occurs, original rows are deleted and new rows are inserted. This means that all fields of the original rows are replaced with those of the new rows.
No insert
column The columns in the destination table to which data is written. Separate the columns with commas (,). Example: "column":["id","name","age"]. To write data to all the columns, set the value to an asterisk (*). Example: "column":["*"].
Note If the field name contains select, enclose the field name in grave accents (` `). For example, item_select_no is presented as `item_select_no`.
Yes N/A
preSql The SQL statement to execute before the sync node is run. For example, you can clear outdated data. You can execute only one SQL statement on the codeless user interface (UI) and multiple SQL statements in the code editor.
Note If you specify multiple SQL statements, they may not be executed in the same transaction.
No N/A
postSql The SQL statement to execute after the sync node is run. For example, you can add a timestamp. You can execute only one SQL statement on the codeless UI and multiple SQL statements in the code editor.
Note If you specify multiple SQL statements, they may not be executed in the same transaction.
No N/A
batchSize The number of data records to write at a time. A proper value can greatly reduce the interactions between Data Integration and the AnalyticDB for MySQL 3.0 database on the network, and increase the throughput. However, an excessively large value may lead to the out of memory (OOM) error during the data synchronization process. No 1,024

Codeless UI mode

  1. Configure the connections.

    Configure the connections to the source and destination data stores for the sync node.

    Parameter Description
    Connection The datasource parameter in the preceding parameter description. Select the name of a connection that you have configured.
    Table The table parameter in the preceding parameter description.
    Statement Run Before Writing The preSql parameter in the preceding parameter description. Enter an SQL statement to execute before the sync node is run.
    Statement Run After Writing The postSql parameter in the preceding parameter description. Enter an SQL statement to execute after the sync node is run.
    Solution to Primary Key Violation The writeMode parameter in the preceding parameter description. Select the required write mode.
    Data Records Per Write The batchSize parameter in the preceding parameter description. This parameter takes effect only when the wirteMode parameter is set to insert.
  2. Configure field mapping. It is equivalent to setting the column parameter in the preceding parameter description. Fields in the source table on the left have a one-to-one mapping with fields in the destination table on the right.
  3. Configure channel control policies.Channel
    Parameter Description
    Expected Maximum Concurrency The maximum number of concurrent threads that the sync node uses to read data from the source data store and write data to the destination data store. You can configure the concurrency for the sync node on the codeless UI.
    Bandwidth Throttling You can enable bandwidth throttling and set a maximum transmission rate to avoid heavy read workload of the source data store. We recommend that you enable bandwidth throttling and set the maximum transmission rate to a proper value based on the configurations of the source data store.
    Dirty Data Records Allowed The maximum number of dirty data records that are allowed.

Code editor mode

For more information about the code editor mode, see Create a sync node by using the code editor.

The following example shows how to use the code editor. For more information about the parameters, see the preceding parameter description.
Notice Delete the comments from the following code before you run the code.
{
    "type": "job",
    "steps": [
        {
            "stepType": "stream",
            "parameter": {},
            "name": "Reader",
            "category": "reader"
        },
        {
            "stepType": "analyticdb_for_mysql", // The writer type.
             "parameter": {
                "postSql": [], // The SQL statement to execute after the sync node is run.
                 "tableType": null, // The reserved field. Default value: null.
                 "datasource": "hangzhou_ads", // The connection name.
                 "column": [ // The columns to which data is synchronized.
                     "id",
                    "value"
                ],
                "guid": null,
                "writeMode": "insert", // The write mode. For more information, see the description of the writeMode parameter.
                 "batchSize": 2048, // The number of data records to write at a time. For more information, see the description of the batchSize parameter.
                 "encoding": "UTF-8", // The encoding format.
                 "table": "t5", // The name of the destination table.
                 "preSql": [] // The SQL statement to execute before the sync node is run.
             },
            "name": "Writer",
            "category": "writer"
        }
    ],
    "version": "2.0", // The version number of the configuration file.
     "order": {
        "hops": [
            {
                "from": "Reader",
                "to": "Writer"
            }
        ]
    },
    "setting": {
        "errorLimit": {
            "record":"0"// The number of dirty data records.
         },
        "speed": {
            "concurrent": 2, // The number of concurrent threads.
             "throttle": false // A value of false indicates that the bandwidth is not throttled. In this case, the specified maximum transmission rate does not take effect. A value of true indicates that the bandwidth is throttled.
         }
    }
}