Dataphin:Configure Vertica input component

Prerequisites

• A Vertica data source is created. Create a Vertica data source.

• The account used to configure the Vertica input component has read-through permission on the data source. If not, request permission first. Request, renew, and return data source permissions.

Procedure

1. In the top navigation bar of the Dataphin homepage, choose Develop > Data Integration.

2. In the top navigation bar of the integration page, select Project. In Dev-Prod mode, also select an environment.

3. In the left-side navigation pane, click Batch Pipeline. In the Batch Pipeline list, click the target offline pipeline to open its configuration page.

4. Click Component Library in the upper-right corner of the page to open the Component Library panel.

5. In the Component Library panel, select Inputs in the left-side navigation pane. Find Vertica in the input component list and drag it to the canvas.

6. Click the icon on the Vertica input component card to open the Vertica Input Configuration dialog box.

7. In the Vertica Input Configuration dialog box, configure the parameters.

   Parameter	Description
   Step Name	The name of the Vertica input component. Dataphin generates a default name. You can modify it. The name must meet the following requirements: It can contain only Chinese characters, letters, underscores (_), and digits. It cannot exceed 64 characters in length.
   Datasource	The dropdown list displays all Vertica data sources in the current Dataphin instance, including those you have read-through permission for and those you do not. Click the icon to copy the data source name. For data sources without read-through permission, click Request next to the data source to request permission. Request, renew, and return data source permissions. If no Vertica data source exists, click Create Data Source to create one. Create a Vertica data source.
   Table	Enter a table name keyword to search, or enter an exact table name and click Exact Match. After you select a table, the system checks the table status. Click the icon to copy the selected table name.
   Shard Key (optional)	Use an integer-type column from the source table as the shard key. Using the primary key or an indexed column is recommended. The system shards data based on this key to enable concurrent reads, improving synchronization efficiency.
   Batch Read Count (optional)	The number of records read per batch. Instead of reading records one by one, set a batch size (such as 1,024) to reduce data source interactions, improve I/O efficiency, and reduce network latency.
   Input Filter (optional)	The filter condition for extracting data: Static field: extract data matching a fixed value, for example, ds=20210101. Variable parameter: extract data dynamically, for example, ds=${bizdate}.
   Output Fields	Displays all fields matching the selected table and filter conditions. You can perform the following operations: Field Management: Remove fields that you do not need in downstream components: Delete a single field: Click the icon in the Operation column to delete a field. Delete multiple fields in batch: Click Field Management. In the Field Management dialog box, select the fields, click the left arrow icon to move them to unselected fields, and then click OK. Batch Add: Click Batch Add to configure fields in JSON, TEXT, or DDL format in batch. Note After you complete the batch addition and click OK, the system overwrites the existing field configuration. JSON format example: // Example: [{ "index": 1, "name": "id", "type": "int(10)", "mapType": "Long", "comment": "comment1" }, { "index": 2, "name": "user_name", "type": "varchar(255)", "mapType": "String", "comment": "comment2" }] Note index is the column number, name is the field name after import, and type is the field type after import. For example, "index":3,"name":"user_id","type":"String" imports the fourth column with field name user_id and type String. TEXT format example: // Example: 1,id,int(10),Long,comment1 2,user_name,varchar(255),Long,comment2 Row delimiter: separates field entries. Default: line feed (\n). Supported: line feed (\n), semicolon (;), or period (.). Column delimiter: separates field name and type. Default: comma (,). Supported delimiter: ','. The field type can be left empty. Default: ','. DDL format example: CREATE TABLE tablename ( user_id serial, username VARCHAR(50), password VARCHAR(50), email VARCHAR (255), created_on TIMESTAMP, ); Create an output field: Click + Create Output Field, fill in Column, Type, and Description, select Mapping Type, and click the icon to save.

8. Click OK to complete the property configuration of the Vertica input component.