The Hologres input component is designed to read data from a Hologres data source. To synchronize data from the Hologres data source to other destinations, you must first configure the Hologres input component to read the data source, followed by setting up the target data source for synchronization. This topic provides guidance on configuring the Hologres input component.
Limits
The Hologres input component cannot read views from the Hologres database.
Prerequisites
A Hologres data source has been successfully established. For more information, see Create Hologres Data Source.
To configure the properties of the Hologres input component, the account must possess read-through permission for the data source. Should you lack the necessary permission, it is essential to obtain authorization from the data source. For additional details, see Request, Renew, and Return Data Source Permissions.
Procedure
Navigate to the Dataphin home page, select Development > Data Integration from the top menu bar.
At the top menu bar of the integration page, select Project (Dev-Prod mode requires selecting an environment).
In the left-side navigation pane, click Batch Pipeline, then in the Batch Pipeline list, select the Offline Pipeline you wish to develop to access its configuration page.
Click the Component Library at the top right corner of the page to open the Component Library panel.
In the Component Library panel's left-side navigation pane, select Input. Locate the Hologres component in the list on the right and drag it onto the canvas.
Click the
icon on the Hologres input component card to open the Hologres Input Configuration dialog box.In the Hologres Input Configuration dialog box, set the following parameters:
Parameter
Description
Step Name
The Hologres input component's name is automatically generated by Dataphin and can be customized to fit your business scenario. The naming convention allows:
Only Chinese characters, letters, underscores (_), and numbers.
A maximum length of 64 characters.
Datasource
The data source dropdown list includes all Hologres-type data sources, both those you have read-through permission for and those you do not.
If the data source lacks read-through permission, select Request adjacent to the data source to apply for access. For additional details, refer to Request, Renew, and Return Data Source Permissions.
If you do not yet have a Hologres-type data source, click Create to create a data source. For more information, see Create Hologres Data Source.
Schema
You can select tables across different schemas. By default, the schema set in the data source is used. If no schema is specified, tables from all schemas will be available for selection.
Table
Choose the data table from the Hologres data source.
Input Filter
Specify the filter criteria for input fields, such as
ds=${bizdate}. The Input Filter is useful in scenarios involving:Filtering a fixed subset of data.
Applying parameter-based filtering.
Output Fields
The output fields section lists all fields from the selected table and filter conditions. To exclude fields from being output to downstream components, you can remove them as follows:
Single Field Deletion Scenario: To delete a few specific fields, simply click the
icon in the operation column.Batch Field Deletion Scenario: To delete multiple fields at once, click Field Management, choose the desired fields within the Field Management dialog box, then use the
shift-left icon to transfer the selected fields to the unselected group. Confirm the action by clicking Confirm to finalize the batch deletion process.
Click Confirm to finalize the configuration of the Hologres Input Component.