Dataphin:Configure Lindorm Input Component

Prerequisites

• A Lindorm data source has been created. For more information, see Create a Lindorm Data Source.

• The account used to configure the Lindorm input component must have read-through permission for the data source. If you do not have the required permission, request it. For more information, see Request Data Source Permission.

Procedure

1. On the Dataphin home page, select Development > Data Integration from the top menu bar.

2. In the integration page's top menu bar, select Project (Dev-Prod mode requires selecting an environment).

3. In the left-side navigation pane, click Batch Pipeline. In the Batch Pipeline list, click the offline pipeline you want to develop to open its configuration page.

4. Click Component Library in the upper right corner of the page to open the Component Library panel.

5. In the Component Library panel's left-side navigation pane, select Input. Find the Lindorm component in the list of input components on the right and drag it to the canvas.

6. Click the icon on the Lindorm input component card to open the Lindorm Input Configuration dialog box.

7. In the Lindorm Input Configuration dialog box, configure the parameters.

   Parameter	Description
   Step Name	The name of the Lindorm input component. Dataphin automatically generates a step name, which you can modify as needed. The naming convention is as follows: Can only contain Chinese characters, letters, underscores (_), and numbers. Cannot exceed 64 characters.
   Datasource	The drop-down list displays all Lindorm-type data sources in Dataphin, including those you have read-through permission for and those you do not. Click the icon to copy the data source name. For data sources without read-through permission, you can click Request after the data source to request read-through permission. For more information, see Request Data Source Permission. If you do not have a Lindorm-type data source yet, click Create Data Source to create a data source. For more information, see Create a Lindorm Data Source.
   Table	Select the source table for data synchronization. Click the icon to copy the name of the currently selected table. Important Tables in iceberg format with a primary key are not supported.
   Input Filter	Filtering is supported only for the iceberg storage format. Logical operators: and, or, not. Relational operators: >, >=, <, <=, =, !=, like, not like, is null, is not null. The like operator matches only records that start with a specified string. The not like operator matches only records that do not start with a specified string, such as name like 'abc%'.
   Partition	If the source table is a partitioned table, you need to configure partition information. Single or multiple partitions are supported, such as ds=20230101 or /*query*/ds>=20230101 and ds<=20230107. Parameters are supported, such as ds=${bizdate}.
   File Encoding	Select the file encoding of the source table. UTF-8 and GBK are supported.
   Compression Format	If the file is compressed, select the compression format so Dataphin can decompress it. This field is optional. Supported formats: gzip, bzip2, lzo, lzo_deflate, hadoop-snappy, framing-snappy, zip, zlib. The default format for ORC tables is zlib. For other decompression formats, you must specify them explicitly. Non-ORC tables have no default format.
   Field Separator	The separator used when writing to the target. Default: \u0001.
   Output Fields	Displays all fields matched by the selected table and filter criteria. To exclude fields from downstream components, delete them: Single Field Deletion: Click the icon in the operation column to remove individual fields. Batch Field Deletion: Click Field Management, select fields in the Field Management dialog box, then click the left shift icon to move them to unselected fields and click Confirm.

8. Click Confirm to complete the Lindorm input component configuration.