The Hologres output component is designed for writing data to a Hologres data source. When synchronizing data from various sources to a Hologres data source, it is necessary to configure the target data source within the Hologres output component after setting up the source data information. This topic provides guidance on configuring the Hologres output component.
Prerequisites
Ensure a Hologres data source is created and write-through permission is obtained:
For more information on creating a data source, see Create Hologres Data Source.
For detailed instructions on how to obtain write-through permissions for a data source, please refer to Request Data Source Permission.
Limits
The Hologres database does not support writing to views.
Procedure
Navigate to the Dataphin home page, select Development > Data Integration from the top menu bar.
At the top menu bar of the integration page, choose Project (select an environment if using the Dev-Prod pattern).
In the navigation pane on the left, click Batch Pipeline. Then, from the Batch Pipeline list, select the Offline Pipeline you want to develop to access its configuration page.
To open the Component Library panel, click Component Library in the upper right corner of the page.
Within the Component Library panel's left-side navigation pane, select Output, locate the Hologres component in the list on the right, and drag it onto the canvas.
Connect the target input component to the Hologres output component by clicking and dragging the
icon.Click the
icon on the Hologres output component card to open the Hologres Output Configuration dialog box.
In the Hologres Output Configuration dialog box, set the following parameters:
Parameter
Description
Basic Settings
Step Name
Specify the name for the Hologres output component. Dataphin automatically generates a step name, but you can customize it to suit your business scenario. The name must adhere to the following criteria:
May only include Chinese characters, letters, underscores (_), and numbers.
Cannot exceed 64 characters in length.
Datasource
Choose a Hologres data source.
If you do not have a data source of the Hologres type, click Create to create a data source. For specific operations, see Create Hologres Data Source.
Schema
You can select tables across different schemas. Choose the schema where the table is located. If not specified, the schema set in the data source will be used by default. If the data source does not define a schema, tables from all schemas will be available.
Table
Select the desired table from the Hologres data source.
Write Mode
Choose the method for writing data from the source to the Hologres data source. The write modes are SDK and SQL. Recommended usage is as follows:
SDK mode is advised for Hologres version 0.8.
SQL mode is recommended for Hologres version 0.9 and above.
Loading Policy
Select the policy for loading data into the Hologres data source. Options include overwrite and ignore, with the following considerations:
Overwrite: New data replaces existing data when a primary key is present; otherwise, data is appended.
Ignore: New data is disregarded when a primary key is present; without a primary key, data is appended.
Field Mapping
Input Field
Shows the fields retrieved from the source table.
Output Field
Shows the fields of the destination table.
Mapping
Define how input fields from the source table correspond to output fields in the destination table. Mapping options include same-name and same-row, with the following guidelines:
Same-name mapping: Aligns fields that share the same name.
For instance, after selecting same-name mapping, the data from the input field id is mapped to the output field id.
Same-row mapping: Maps fields that correspond to the same row but have different names.
For example, the data from the endpoint_type field is mapped to the id field.
Click Confirm to finalize the configuration of the Hologres Output Component.