After you prepare data sources, network environments, and resources, you can create a real-time data synchronization node to synchronize data to Hologres. This topic describes how to create a real-time data synchronization node and view the running status of the node.

Prerequisites

Before you create a real-time data synchronization node, make sure that the following operations are performed:

Limits

  • You can run real-time data synchronization nodes to synchronize data only from PolarDB, Oracle, or MySQL to Hologres.

Create a real-time data synchronization node

  1. Log on to the DataWorks console.
  2. In the left-side navigation pane, click Workspaces.
  3. After you select the region in which the workspace that you want to manage resides, find the workspace and click Data Analytics in the Actions column.
  4. Create a workflow.
    If you have a workflow, skip this step.
    1. Move the pointer over the Create icon and select Workflow.
    2. In the Create Workflow dialog box, set the Workflow Name parameter.
    3. Click Create.
  5. Create a real-time data synchronization node.
    1. In the Create Node dialog box, configure the parameters.Create a real-time data synchronization node (synchronize data to Hologres)
      Parameter Description
      Node Type The type of the node. Default value: Real-time synchronization.
      Sync Method Set the value to Migration to Hologres. In this case, partial or all tables in your desired database are migrated to Hologres.
      Node Name The name of the node. The name must be 1 to 128 characters in length and can contain letters, digits, underscores (_), and periods (.).
      Location The directory in which the real-time data synchronization node is stored.
  6. Select a source data source and configure synchronization rules.
    1. In the Data source section, specify the Type and Data source parameters.
      Note You can set Type only to MySQL, Oracle, or PolarDB.
    2. In the Select the source table for synchronization section, select the tables whose data you want to synchronize in the Source Table section, and click the Icon icon to move the tables to the Selected Source table section.
      Source Table
      The Source Table section displays all the tables in the source data source. You can select all or some tables.
      Notice If a selected table does not have a primary key, the table cannot be synchronized in real time.
    3. In the Set synchronization rules section, click Add rule and select an option to configure the naming rules for destination tables.
      Supported options include Conversion Rule for Table Name and Rule for Destination Table name.
      • Conversion Rule for Table Name: the rule used to convert the names of source tables to those of destination tables.
      • Rule for Destination Table name: the rule used to add a prefix and a suffix to the converted names of destination tables.
    4. Click Next Step.
  7. Select a data source as the destination and configure the formats for the destination tables.
    1. In the Set Destination Table step, specify Target Hologres data source and Schema, and specify whether to enable Table name case sensitive.
    2. Click Refresh source table and Hologres Table mapping to configure the mappings between the source tables and destination Hologres tables.
    3. View the mapping progress, source tables, and mapped destination tables.View the mapping progress, source tables, and mapped destination tables.
      Serial number Description
      1
      The progress of mapping the source tables to the destination tables.
      Note The mapping may require a long period of time if you want to synchronize data from a large number of tables.
      2
      • If the tables in the source database contain primary keys, the system removes duplicate data based on the primary keys during the synchronization.
      • If the tables in the source database do not contain primary keys, you can click the Edit icon to customize primary keys. You can use one field or a combination of several fields as the primary keys of the tables. This way, the system removes duplicate data based on the primary keys during the synchronization.
      Note
      3
    4. Click Next.
      If you set Table creation method to Create Table, you must click Start table building in the Create tables automatically dialog box to create destination Hologres tables.
  8. Configure rules for processing DDL messages.
    Data definition language (DDL) statements exist in the source. Before you synchronize data, you can configure synchronization rules for different DDL statements based on your business requirements.
    Note The rules apply when a real-time data synchronization node is run for the first time. If you want to modify the rules in subsequent operations, go to the configuration page of the real-time data synchronization node.
    1. Click Next Step.