All Products
Search

This Product

    DataHub:Synchronize data with RDS, MySQL, and AnalyticDB

    Document Center

    DataHub:Synchronize data with RDS, MySQL, and AnalyticDB

    Last Updated:Jun 15, 2026

    Synchronize data to RDS

    Prerequisites

    1. Prepare an RDS instance and table structure Create an RDS instance in the RDS console. You can use either a classic network or a VPC network. Select the correct network type when you create the DataHub synchronization task. During synchronization, DataHub maps its field data types to the corresponding RDS data types as shown below.

    DataHub

    RDS

    TINYINT

    TINYINT

    SMALLINT

    SMALLINT

    INTEGER

    INT

    BIGINT

    BIGINT

    STRING

    VARCHAR

    BOOLEAN

    BOOLEAN / TINYINT

    FLOAT

    FLOAT

    DOUBLE

    DOUBLE

    TIMESTAMP

    TIMESTAMP / BIGINT

    DECIMAL

    DECIMAL

    The TINYINT, SMALLINT, INTEGER, and FLOAT data types in DataHub are supported starting from Java SDK version 2.16.1-public. 2. Configure the RDS IP whitelist and internal address To use the DataHub synchronization feature, add the DataHub service IP addresses to the IP whitelist in the RDS console. This allows DataHub to access your RDS service. For the required DataHub service IP addresses, see Overview. You must also provide the RDS internal address when creating the synchronization task.

    3. Synchronization notes

    1. DataHub can only synchronize TUPLE data to RDS.

    2. The TIMESTAMP type in DataHub, which is based on microseconds, is converted to the RDS TIMESTAMP type by default. Pay attention to data precision.

    3. To prevent lock contention from concurrent reads and writes on the same primary key, ensure that data with the same primary key is written to the same DataHub shard.

    4. If you use a VPC network, ensure that the DataHub topic and the RDS instance are in the same region.

    Create a synchronization task

    1. Navigate to the Project List/Project Details/Topic Details page.

    2. Click the + Sync button in the upper-right corner to create a synchronization task. In the Create Connector dialog box that appears, configure connection information such as Host, Port (3306 by default), Database, Table, User, and Password, and synchronization parameters such as Write Mode, Imported Fields, and Network Type. If you select VPC Network, you must also specify the VPC ID and Instance ID. Set the Start Time and Timestamp Unit (MICROSECOND by default), and then click Create.

    The following are key console configuration settings. For advanced options, see the SDK documentation.

    1. For Host, specify the RDS internal address.

    2. Imported Fields: You can select specific columns from DataHub to synchronize to the RDS table.

    3. Write Mode: Includes IGNORE and OVERWRITE modes.

      • IGNORE: Skips duplicate data, corresponding to INSERT IGNORE INTO writes.

      • OVERWRITE: Replaces duplicate data, corresponding to a REPLACE INTO write.

    4. VPC ID and Instance ID: If your RDS instance is in a VPC network, you must provide the corresponding VPC ID and RDS instance ID.

      Synchronization example

    5. Create an RDS instance and table structure. In the RDS console, create an RDS instance and a data table. In this example, the table is named wurao_tuple_test_001 and has three columns: field1 (varchar, length 255), field2 (bigint, length 20), and field3 (double).

    6. Create a DataHub topic. In DataHub, create a topic of type TUPLE. In this example, the schema has three fields: field1 (STRING), field2 (BIGINT), and field3 (DOUBLE). None of the fields can be null.

    7. Create the synchronization task. Set the Write Mode to IGNORE, and import all fields: field1, field2, and field3. Select CLASSIC as the Network Type, and then click Create.

    8. Write TUPLE data to DataHub. Write a total of four records. The data consists of two unique records, each written twice. The values for field1 are f_1 and f_2, for field2 are 1 and 2, and for field3 are 1.0 and 2.0.

    9. Verify the synchronized data. Use a MySQL client to connect to your RDS service and query the table. The result is shown below.

      mysql> select * from wurao_tuple_test_001;
+--------+--------+--------+
| field1 | field2 | field3 |
+--------+--------+--------+
| f_1    |      1 |   1.00 |
| f_2    |      2 |   2.00 |
| f_1    |      1 |   1.00 |
| f_2    |      2 |   2.00 |
+--------+--------+--------+
4 rows in set (0.00 sec)

    Synchronize data to MySQL

    Prerequisites

    1. Prepare a MySQL instance and table structure Create a MySQL instance in the MySQL console.

    During synchronization, DataHub maps its field data types to the corresponding MySQL data types as shown below.

    DataHub

    MySQL

    TINYINT

    TINYINT

    SMALLINT

    SMALLINT

    INTEGER

    INT

    BIGINT

    BIGINT

    STRING

    VARCHAR

    BOOLEAN

    BOOLEAN / TINYINT

    FLOAT

    FLOAT

    DOUBLE

    DOUBLE

    TIMESTAMP

    TIMESTAMP / BIGINT

    DECIMAL

    DECIMAL

    The TINYINT, SMALLINT, INTEGER, and FLOAT data types in DataHub are supported starting from Java SDK version 2.16.1-public.

    2. Synchronization notes 1) DataHub supports synchronizing only TUPLE data to MySQL services. 2) By default, TIMESTAMP values in DataHub are converted to the MySQL TIMESTAMP type based on microseconds. Manage the data precision accordingly. 3) To avoid lock contention from concurrent reads and writes on the same primary key, ensure that data with the same primary key is written to the same shard in DataHub.

    Create a synchronization task

    1. Go to the Project List > Project Details > Topic Details page.

    2. Click the + Sync button in the upper-right corner to create a synchronization task.

    In the New Connector dialog box, configure the connection information, including Host (the MySQL internal address), Port (default is 3306), Database, Table, User, and Password. Then, configure the synchronization parameters, such as write mode, imported fields, and network type (VPC network or classic network). If you select a VPC network, you must also provide the VPC ID and instance ID of your MySQL server. After you complete the configuration, click Create.

    The following are key console configuration settings. For advanced options, see the SDK documentation.

    1. For Host, specify the MySQL internal address.

    2. Imported Fields: You can select specific columns from DataHub to synchronize to the MySQL table.

    3. Write Mode: Includes IGNORE and OVERWRITE modes.

      • IGNORE: Skips duplicate data by using INSERT IGNORE INTO to write data.

      • OVERWRITE: Replaces duplicate data through a REPLACE INTO write operation.

    Synchronization example

    1. Create a MySQL instance and table structure. In the MySQL console, create a MySQL instance and a data table.

    2. Create a DataHub topic. In DataHub, create a topic of type TUPLE. The schema is shown below.

    In this example, the topic is named wurao_tuple_test_001 and its schema has three fields: field1 (STRING), field2 (BIGINT), and field3 (DOUBLE). None of the fields can be null.

    1. Create the synchronization task. Set the Write Mode to IGNORE and import all fields.

    1. Write TUPLE data to DataHub. Write a total of four records.

    The data consists of two unique records, each written twice. The values for field1 are f_1 and f_2, for field2 are 1 and 2, and for field3 are 1.0 and 2.0.

    1. Verify the synchronized data. Use a MySQL client to connect to your MySQL service and query the table. The result is shown below.

    mysql> select * from  wurao_tuple_test_001;
+--------+--------+--------+
| field1 | field2 | field3 |
+--------+--------+--------+
| f_1    |      1 |   1.00 |
| f_2    |      2 |   2.00 |
| f_1    |      1 |   1.00 |
| f_2    |      2 |   2.00 |
+--------+--------+--------+
4 rows in set (0.00 sec)

    Synchronize data to AnalyticDB for MySQL 3.0

    Prerequisites

    1. Prepare an AnalyticDB for MySQL instance and table structure Create an AnalyticDB for MySQL instance in the console. You can use either a classic network or a VPC network. Select the correct network type when you create the DataHub synchronization task. During synchronization, DataHub maps its field data types to the corresponding AnalyticDB for MySQL data types as shown below.

    DataHub

    AnalyticDB

    TINYINT

    TINYINT

    SMALLINT

    SMALLINT

    INTEGER

    INT

    BIGINT

    BIGINT

    STRING

    VARCHAR

    BOOLEAN

    BOOLEAN / TINYINT

    FLOAT

    FLOAT

    DOUBLE

    DOUBLE

    TIMESTAMP

    TIMESTAMP / BIGINT

    DECIMAL

    DECIMAL

    The TINYINT, SMALLINT, INTEGER, and FLOAT data types in DataHub are supported from Java SDK 2.16.1-public. 2. ADS IP whitelist and internal address To use the DataHub synchronization feature, configure an IP whitelist in the RDS console to allow DataHub to access your ADS service. For the IP whitelist of the DataHub service, see FAQ. You must also provide the ADS internal address when creating the synchronization task. 3. Synchronization notes 1) DataHub supports synchronizing only TUPLE data to ADS. 2) By default, the TIMESTAMP data type in DataHub is converted to the ADS TIMESTAMP type with microsecond precision. Manage the data precision accordingly. 3) To prevent lock contention from concurrent read and write operations on data with the same primary key, ensure that data with the same primary key is written to the same DataHub Shard. 4) When you use a VPC network, ensure that the DataHub topic and the ADS instance are in the same region.

    Create a synchronization task

    1. Go to the Project List > Project Details > Topic Details page

    2. Click the + Sync button in the upper-right corner to create a synchronization task. The Create Connector dialog box appears. In the dialog box, configure connection information such as Host, Port (default: 3306), Database, Table, User, and Password, and synchronization parameters such as Write Mode, Imported Fields, and Network Type. If you select the VPC network type, you must also enter the VPC ID and Instance ID. After you complete the configuration, click Create.

    The following are key console configuration settings. For advanced options, see the SDK documentation.

    1. For Host, specify the ADS internal address.

    2. Imported Fields: You can select specific columns from DataHub to synchronize to the AnalyticDB for MySQL table.

    3. Write Mode: Includes IGNORE and OVERWRITE modes.

      • IGNORE: Skips duplicate data by performing INSERT IGNORE INTO writes.

      • OVERWRITE: Replaces duplicate data by performing a REPLACE INTO write.

    4. How to get the instance ID

      1. Click Query the detailed attributes of a cluster.

      2. Click Debug. On the debugging page, select a service endpoint, enter the DBClusterId (the cluster ID of your AnalyticDB for MySQL instance), and click Initiate Call.

        1. Go to the DescribeDBClusterAttribute API reference page, which provides an API description and a Debug button on the right.

        2. In addition to the DBClusterId, you must also enter the region ID for your cluster (for example, cn-beijing) in the RegionId field. Then, click Initiate Call.

      3. The value of the VPCCloudInstanceId field in the response is the instance ID you need.

        1. When the call succeeds (indicated by a status code of 200), search the returned JSON result for the keyword vpccloud to find the VPCCloudInstanceId field.

    Synchronization example

    1. Create an AnalyticDB for MySQL instance and table structure.

    2. Create a DataHub topic. In DataHub, create a topic of type TUPLE. In this example, the schema has three fields: field1 (STRING), field2 (BIGINT), and field3 (DOUBLE). None of the fields can be null.

    3. Create the synchronization task. Set the Write Mode to IGNORE and import all fields: field1, field2, and field3. Select CLASSIC as the Network Type, and then click Create.

    4. Write TUPLE data to DataHub. Write a total of four records. The data consists of two unique records, each written twice. The values for field1 are f_1 and f_2, for field2 are 1 and 2, and for field3 are 1.0 and 2.0.

    5. Verify the synchronized data. Use a MySQL client to connect to your AnalyticDB for MySQL service and query the table. The result is shown below.

      mysql> select * from  wurao_tuple_test_001;
+--------+--------+--------+
| field1 | field2 | field3 |
+--------+--------+--------+
| f_1    |      1 |   1.00 |
| f_2    |      2 |   2.00 |
| f_1    |      1 |   1.00 |
| f_2    |      2 |   2.00 |
+--------+--------+--------+
4 rows in set (0.00 sec)