ApsaraDB for OceanBase:Use the data transmission service to synchronize data

Procedure

1. Create a data synchronization project.

   1. Log on to the ApsaraDB for OceanBase console.

   2. In the left-side navigation pane, choose Data Transmission > Data Synchronization.

   3. On the Data Synchronization page, click Create Synchronization Project in the upper-right corner.

2. On the Select Source and Destination page, configure the related parameters.

   Parameter	Description
   Synchronization Project Name	We recommend that you set it to a combination of digits and letters. It must not contain any spaces and cannot exceed 64 characters in length.
   Source	If you have created an OceanBase data source, select it from the drop-down list. Otherwise, click New Data Source in the drop-down list to create one in the dialog box on the right side. For more information about parameters, see Create an OceanBase data source.
   Destination	If you have created a Kafka data source, select it from the drop-down list. Otherwise, click New Data Source in the drop-down list to create one in the dialog box on the right side. For more information, see Create a Kafka data source.
   Transmission Instance	Select a purchased transmission instance from the drop-down list. If you have not purchased a transmission instance, click Purchase on the right to purchase one. For more information, see Purchase a transmission instance.

3. Click Next. On the Select Synchronization Type page, specify the synchronization type for the current data synchronization project.

   Options for Synchronization Type are Schema Synchronization, Full Synchronization, and Incremental Synchronization. Options for Incremental Synchronization are DML Synchronization and DDL Synchronization.

4. Click Next. On the Select Synchronization Objects page, select the target topic type and objects.

   When you synchronize data from an OceanBase database to a Kafka instance, you can synchronize data from multiple tables to multiple topics.

   1. In the left-side pane, select the objects to be synchronized.

   2. Click >.

   3. In the Map Object to Topic dialog box, select a mapping method.

   4. Click OK.

   After you specify the objects to be synchronized, you can change the topics of the objects, set row filters, or remove a single object or all objects. Objects in the destination database are listed in the structure of Topic > Database > Table.

5. Click Next. On the Synchronization Options page, configure the parameters.

   Parameter	Description
   Incremental Synchronization Start Timestamp	If you have selected Full Synchronization as the synchronization type, the default value of this parameter is the project startup time and cannot be modified. If you do not select Full Synchronization as the synchronization type, set this parameter to a certain point of time, which is the current system time by default.
   Serialization Method	The message format for synchronizing data to a Kafka instance. Valid values: Default, Canal, DataWorks (version 2.0 supported), SharePlex, DefaultExtendColumnType, Debezium, DebeziumFlatten, and DebeziumSmt. For more information, see Data formats. Important Only MySQL tenants of OceanBase Database support Debezium, DebeziumFlatten, and DebeziumSmt. If the message format is set to DataWorks, DDL operations COMMENT ON TABLE and ALTER TABLE…TRUNCATE PARTITION cannot be synchronized.
   Partitioning Rules	The rule for synchronizing data from an OceanBase database to a Kafka topic. The data transmission service supports Hash, Table, and One.
   Business System Identification (Optional)	Identifies the source business system of data. The business system identifier consists of 1 to 20 characters.

6. Click Precheck.

   During the precheck, the data transmission service detects the connection with the destination Kafka instance. If an error is returned during the precheck:

   • You can identify and troubleshoot the problem and then perform the precheck again.

   • You can also click Skip in the Actions column of the failed precheck item. A dialog box appears, prompting you the impact. If you want to skip this operation, click OK.

7. After the precheck succeeds, click Start Project.

   If you do not need to start the project now, click Save. You can manually start the project on the Synchronization Projects page or by performing batch operations later. For more information about the batch operations, see Perform batch operations on data synchronization projects.

   The data transmission service allows you to modify the synchronization objects when a synchronization project is running. For more information, see View and modify synchronization objects. After the data synchronization project is started, it will be executed based on the selected synchronization types. For more information, see View synchronization details.

Video tutorial