Dataphin:Configure the KingbaseES Input Component

Prerequisites

• You have created a KingbaseES data source. For more information, see Create a KingbaseES Data Source.

• The account that you use to configure the KingbaseES Input component must have sync-read permission for the data source. If the account does not have this permission, you must request it. For more information, see Request, Renew, or Release Data Source Permissions.

Procedure

1. On the Dataphin homepage, choose Develop > Data Integration from the top menu bar.

2. On the Integration page, select a Project from the top menu bar. If you are in Dev-Prod mode, you must also select an environment.

3. In the navigation pane on the left, click Batch Pipeline. In the Batch Pipeline list, click the offline pipeline that you want to develop. The configuration page for the pipeline opens.

4. In the upper-right corner of the page, click Component Library to open the Component Library panel.

5. In the navigation pane on the left of the Component Library panel, click Input. From the list of input components on the right, find the KingbaseES component and drag it onto the canvas.

6. Click the icon on the KingbaseES Input widget card to open the KingbaseES Input Configuration dialog box.

7. In the KingbaseES Input Configuration dialog box, configure the parameters.

   Parameter	Description
   Step Name	The name of the KingbaseES input component. Dataphin generates a step name automatically. You can change it based on your business scenario. Use these naming rules: Use only Chinese characters, letters, underscores (_), and digits. Keep the name under 64 characters.
   Datasource	The drop-down list shows all KingbaseES data sources in Dataphin. It includes data sources for which you have sync-read permission and those for which you do not. Click the icon to copy the current data source name. If you lack sync-read permission for a data source, request read permission for it. For instructions, see Request, Renew, or Release Data Source Permissions. If you do not have a KingbaseES data source yet, click Create Data Source to create one. For details, see Create a KingbaseES Data Source.
   Source Table Count	Select the number of source tables. Options are Single Table and Multiple Tables: Single Table: Use this option when syncing business data from one source table to one target table. Multiple Tables: Use this option when syncing business data from multiple source tables to one target table. When writing data from multiple tables into one table, the system uses the union algorithm.
   Table Matching Method	You can select only Generic Rule. Note This setting is available only when you select Multiple Tables for Source Table Count.
   Table	Select the source table: If you selected Single Table for Source Table Count, search by entering a keyword for the table name. Or enter the exact table name and click Exact Match. After you select a table, the system automatically checks its status. Click the icon to copy the selected table name. If you selected Multiple Tables for Source Table Count, add tables as follows: In the input box, enter an expression to filter tables with the same structure. The system supports enumeration, regex-like patterns, and mixed patterns. For example: table_[001-100];table_102. Click Exact Match. In the Confirm Match Details dialog box, review the list of matched tables. Click Confirm.
   Shard Key (Optional)	The system splits data based on the shard key field you specify. Use this with concurrency settings to enable concurrent reads. You can use any column from the source table as the shard key. For best performance, use a primary key or an indexed column. Important If you select a date-time type, the system performs brute-force splitting across the full time range using the max and min values. This method does not guarantee even distribution.
   Batch Read Size (Optional)	The number of records to read at once. Instead of reading records one by one, set a batch size such as 1024. This reduces interactions with the data source, improves I/O efficiency, and lowers network latency.
   Input Filter (Optional)	Enter filter conditions for input fields. For example: ds=${bizdate}. Use Input Filter for these scenarios: A fixed subset of data. Parameter-based filtering.
   Output Fields	This section lists all fields from the selected table and matching filters. You can perform these actions: Field Management: Remove fields you do not need to pass to downstream components: Remove individual fields: Click the icon in the Actions column to delete extra fields. Batch field deletion scenario: To delete multiple fields, you can click Field Management, select multiple fields in the Field Management dialog box, click the left-moving icon, move the selected input fields to the unselected input fields, and click OK to complete batch field deletion. Batch Add: Click Batch Add to configure output fields in JSON, TEXT, or DDL format. Note After you click OK, the batch-add operation overwrites existing field configurations. JSON format example: // Example: [{ "index": 1, "name": "id", "type": "int(10)", "mapType": "Long", "comment": "comment1" }, { "index": 2, "name": "user_name", "type": "varchar(255)", "mapType": "String", "comment": "comment2" }] Note index specifies the 0-based column index of the specified object, name specifies the name of the field after import, and type specifies the type of the field after import. For example, "index":3,"name":"user_id","type":"String" indicates that the fourth column of the file is imported, the field name is user_id, and the field type is String. TEXT format example: // Example: 1,id,int(10),Long,comment1 2,user_name,varchar(255),Long,comment2 The row delimiter separates each field’s information. By default, it is a line feed (\n). You can also use a semicolon (;) or period (.). The column delimiter separates field names from field types. By default, it is a comma (,). You can also use ','. Field types are optional and default to ','. You can perform batch configurations in DDL format. For example: CREATE TABLE tablename ( user_id serial, username VARCHAR(50), password VARCHAR(50), email VARCHAR (255), created_on TIMESTAMP, ); Add a new output field: Click + Add Output Field. Enter the Column, Type, and Comment. Select a Mapping Type. Click the icon to save the row.

8. Click OK to finish configuring the KingbaseES Input Component.