Dataphin:Configure the ClickHouse output component

Prerequisites

A ClickHouse data source and data table are created:

• A ClickHouse data source is created. Create a ClickHouse data source.

• The account that configures the ClickHouse output component has write-through permission on the data source. To request permission, go to Request, renew, and return data source permissions.

Procedure

1. In the top navigation bar of the Dataphin homepage, choose Development > Data Integration.

2. In the top navigation bar of the Integration page, select a project. In Dev-Prod mode, also select an environment.

3. In the left-side navigation pane, click Batch Pipeline. In the Batch Pipeline list, click the target offline pipeline to open its configuration page.

4. In the upper-right corner, click Component Library to open the Component Library panel.

5. In the left pane of Component Library, select Outputs. Find ClickHouse in the list and drag it to the canvas.

6. Drag the icon from an upstream input, transform, or flow component to connect it to the ClickHouse output component.

7. Click the icon on the ClickHouse output component to open the ClickHouse Output Configuration dialog box.

8. In the ClickHouse Output Configuration dialog box, configure the parameters.

   Parameter		Description
   Basic Configuration	Step Name	The component name. Dataphin auto-generates a name that you can modify. Requirements: Can contain only Chinese characters, letters, underscores (_), and digits. Cannot exceed 64 characters.
   	Datasource	Lists all ClickHouse data sources in Dataphin, with or without write-through permission. Click to copy the selected table name. For data sources without write-through permission, click Request next to the data source. Request data source permissions. To add a data source, click Create Data Source. Create a ClickHouse data source.
   	Table	Select the target table for output data. Enter a keyword to search, or enter the exact name and click Exact Match. The system automatically validates the selected table. Click to copy the table name.
   	Loading Policy	The write policy for the target table. Only Append Data is supported, which appends new data without modifying existing records.
   	Batch Write Size (optional)	Maximum data size per batch write. Works with Batch Write Records — the system writes when either limit is reached first. Default: 32M.
   	Batch Write Records (optional)	Maximum records per batch write. Default: 2048 records. The batch write strategy uses both Batch Write Records and Batch Write Size — the system writes when either limit is reached first. When accumulated data reaches either limit (size or record count), the system flushes the batch to the destination. Recommended batch write size: 32 MB. Set the record limit higher than the byte limit divided by your average record size so that byte-based flushing takes priority. Example: for 1 KB records with a 16 MB byte limit, set the record limit above 16,384 (for example, 20000 records). The system then flushes at 16 MB regardless of record count.
   	Prepare Statement (optional)	The SQL script to be executed on the database before data import. Example: to avoid downtime, the script creates Target_A, writes data to it, then swaps tables — rename Service_B to Temp_C, rename Target_A to Service_B, and drop Temp_C.
   	Post Statement (optional)	The SQL script to be executed on the database after data import.
   Field Mapping	Input Fields	Shows input fields from the upstream component output.
   	Output Fields	Displays the output fields. Available operations: Field Management: Click Field Management to select output fields. Click the icon to move Selected Input Fields to Unselected Input Fields. Click the icon to move Unselected Input Fields to Selected Input Fields. Batch Add: Click Batch Add to configure in JSON, TEXT, or DDL format. Batch configuration in JSON format, for example: // Example: [{ "name": "user_id", "type": "String" }, { "name": "user_name", "type": "String" }] Note name: the field name. type: the data type after import. Example: "name":"user_id","type":"String" imports field user_id as String. Batch configuration in TEXT format, for example: // Example: user_id,String user_name,String Row delimiter separates fields. Default: line feed (\n). Supported: line feed (\n), semicolon (;), period (.). Column delimiter separates the field name and type. Default: comma (,). Batch configuration in DDL format, for example: CREATE TABLE tablename ( id INT PRIMARY KEY, name VARCHAR(50), age INT ); Create new output field: Click +Create New Output Field, enter the Column name and select a Type, then click to save.
   	Mapping	Map upstream input fields to target table fields. Quick Mapping supports Same Row Mapping and Same Name Mapping. Same Name Mapping: Maps fields with the same name. Same Row Mapping: Maps fields by row position when field names differ between source and target.

9. Click OK.