You can build a real-time data warehouse by using the real-time writing capability of Hologres.

Prerequisites

The required reader or transformation connection is configured. For more information, see Supported data stores.

Procedure

  1. Go to the DataStudio page.
    1. Log on to the DataWorks console.
    2. In the left-side navigation pane, click Workspaces.
    3. Select the region where the required workspace resides, find the workspace, and then click Data Analytics.
  2. Move the pointer over the Create icon and choose Data Integration > Real-time synchronization.
    Alternatively, you can click the required workflow, right-click Data Integration, and then choose Create > Real-time synchronization.
  3. In the Create Node dialog box, set the Node Name and Location parameters.
    Notice The node name must be 1 to 128 characters in length. It can contain letters, digits, underscores (_), and periods (.).
  4. Click Commit.
  5. On the configuration tab of the real-time sync node, drag Hologres under Output to the canvas. Connect it to the configured reader or transformation node in the canvas.
  6. Click the Hologres node. Set the required parameters in the Node configuration section.
    Parameter Description
    Data source The connection to the Hologres data store. In this example, you can select only a Hologres connection.

    If no connection is configured, click New data source on the right to create one on the Workspace Management > Data Source page. For more information, see Configure a Hologres connection.

    Table The name of the Hologres table to which data is written.

    You can click One-Click table creation on the right to create a table, or click Data preview to preview the selected table.

    Dynamic Time Partition If the Hologres table is a partitioned table, you must specify a dynamic time-based partition.

    The dynamic time-based partition parses the value of a source field in the yyyymmdddhhmmss format. After the value is parsed, you can use the dynamic partition whose name is a string of variables in the destination table. The destination partition varies with the value of the source field.

    Assume that the value of the source field is 20200816 and the name of the destination partition is in the {yyyy}-{mm}-{dd} format. In this case, the value is written to the 2020-08-16 partition.

    Job type Valid values: Replay (replay operation log to restore data) and Insert (direct archive save).
    • Replay (replay operation log to restore data): the mirror feature. For example, if the INSERT statement is executed to add a record to the source data store, the INSERT statement is also executed to add the same record to the Hologres data store. If the UPDATE or DELETE statement is executed in the source data store, the UPDATE or DELETE statement is also executed in the Hologres data store.
    • Insert (direct archive save): indicates that the Hologres data store is used as streaming data storage. Data is synchronized from the source data store to the Hologres data store by executing the INSERT statement.
    Write conflict policy Valid values: Cover (Overwrite) and Ignore (Ignore)
    • Cover (Overwrite): uses the new data that is synchronized from the source data store to overwrite the existing data.
    • Ignore (Ignore): ignores the new data that is synchronized from the source data store and retains the existing data.
    Field Mapping Click Field Mapping and configure field mappings between the source and destination data stores. The sync node synchronizes data based on the field mappings.
  7. Click the Save icon in the toolbar.