The Hologres input component reads data from a Hologres data source. When synchronizing Hologres data to other destinations, configure this input component before setting up the target data source.
Limits
The Hologres input component cannot read views from the Hologres database.
Prerequisites
-
A Hologres data source is created. Create Hologres Data Source.
-
The account has read-through permission for the data source. If not, apply through Request, Renew, and Return Data Source Permissions.
Procedure
-
On the Dataphin home page, choose Development > Data Integration from the top menu bar.
-
In the top menu bar, select a Project. In Dev-Prod mode, also select an environment.
-
In the left navigation pane, click Batch Pipeline, then select the target Offline Pipeline from the Batch Pipeline list to open its configuration page.
-
In the upper-right corner, click Component Library to open the Component Library panel.
-
In the Component Library panel, select Input in the left pane, then drag the Hologres component onto the canvas.
-
On the Hologres input component card, click the
icon to open the Hologres Input Configuration dialog box. -
In the Hologres Input Configuration dialog box, set the following parameters:
Parameter
Description
Step Name
Auto-generated by Dataphin. You can customize the name. Naming rules:
-
Only Chinese characters, letters, underscores (_), and numbers.
-
A maximum length of 64 characters.
Datasource
Lists all Hologres-type data sources, regardless of your read-through permission status.
-
If you lack read-through permission, click Request next to the data source to apply. Request, Renew, and Return Data Source Permissions.
-
If no Hologres data source exists, click Create to add one. Create Hologres Data Source.
Schema
Select tables across different schemas. Defaults to the schema set in the data source. If unspecified, tables from all schemas are available.
Table
Choose the data table from the Hologres data source.
Input Filter
Specify filter criteria for input fields, such as
ds=${bizdate}. The Input Filter supports:-
Filtering a fixed subset of data.
-
Applying parameter-based filtering.
Output Fields
Lists all fields from the selected table and filter conditions. To exclude fields from downstream output, remove them:
-
Single field: Click the
icon in the operation column. -
Batch fields: Click Field Management, select fields in the Field Management dialog box, click the
icon to move them to the unselected group, and click Confirm.
-
-
Click Confirm to save the Hologres Input Component configuration.