You can build a real-time data warehouse by using the real-time write capability of Hologres.

Prerequisites

A reader or conversion node is configured. For more information, see Plug-ins for data sources that support real-time synchronization.

Procedure

  1. Go to the DataStudio page.
    1. Log on to the DataWorks console.
    2. In the left-side navigation pane, click Workspaces.
    3. In the top navigation bar, select the region in which the workspace that you want to manage resides. Find the workspace and click Data Development in the Actions column.
  2. In the Scheduled Workflow pane, move the pointer over the Create icon icon and choose Data Integration > Real-time synchronization.
    Alternatively, click the required workflow, right-click Data Integration, and then choose Create > Real-time synchronization.
  3. In the Create Node dialog box, set the Sync Method parameter to End-to-end ETL and set the Node Name and Location parameters.
    Notice The node name can be up to 128 characters in length and can contain letters, digits, underscores (_), and periods (.).
  4. Click Commit.
  5. On the configuration tab of the real-time synchronization node, drag Hologres in the Output section to the canvas on the right. Then, draw a line to connect it to the configured reader or conversion node.
  6. Click the Hologres node. In the panel that appears, set the parameters as required.
    Parameter Description
    Data source The Hologres data source that you configured. You can select only a Hologres data source.

    If no data source is available, click New data source on the right to add a data source on the Data Source page. For more information, see Add a Hologres data source.

    Table The name of the Hologres table to which you want to write data.

    You can click One-Click table creation on the right to create a table, or click Data preview to preview the selected table.

    Dynamic Time Partition If the Hologres table is a partitioned table, you must specify a dynamic time-based partition.

    The dynamic time-based partition parses the value of a source field in the yyyymmdddhhmmss format. After the value is parsed, you can use the dynamic partition whose name is a string of variables in the destination table. The destination partition varies based on the value of the source field.

    For example, the value of the source field is 20200816, and the name of the destination partition is in the {yyyy}-{mm}-{dd} format. In this case, the value is written to the 2020-08-16 partition.

    Job Type The type of the data write operation. Valid values: Replay (replay operation log to restore data) and Insert (direct archive save).
    • Replay (replay operation log to restore data): Hologres Writer performs the same operation on the Hologres destination as that performed on the source. For example, if the INSERT statement is executed to add a record to the source, Hologres Writer executes the INSERT statement to add the same record to the Hologres destination. If the UPDATE or DELETE statement is executed in the source, Hologres Writer executes the UPDATE or DELETE statement in the Hologres destination.
    • Insert (direct archive save): Hologres Writer uses the Hologres destination as streaming data storage. Data is synchronized from the source to the Hologres destination by using the INSERT statement.
    Policy for Write Conflict The solution to data write conflicts. Valid values: Cover (Overwrite) and Ignore (Ignore).
    • Cover (Overwrite): Hologres Writer uses the new data synchronized from the source to overwrite the existing data in the Hologres destination.
    • Ignore (Ignore): Hologres Writer ignores the new data synchronized from the source and retains the existing data in the Hologres destination.
    Field Mapping The field mappings between the source and Hologres destination. Click Field Mapping to configure field mappings. The real-time synchronization node synchronizes data based on the field mappings.
  7. Click the Save icon icon in the top toolbar.