Dataphin:Configure the GoldenDB output component

Prerequisites

• You have created a GoldenDB data source. For more information, see Create a GoldenDB data source.

• The account used to configure the GoldenDB output component properties has the write-through permission for the data source. If you do not have the permission, you need to apply for the data source permission. For more information, see Apply for data source permissions.

Procedure

1. In the top navigation bar of the Dataphin homepage, choose Develop > Data Integration.

2. In the top navigation bar of the Integration page, select Project (In Dev-Prod mode, you need to select Environment).

3. In the navigation pane on the left, click Batch Pipeline. In the Batch Pipeline list, click the offline pipeline that you want to develop to open its configuration page.

4. Click Component Library in the upper-right corner of the page to open the Component Library panel.

5. In the navigation pane on the left of the Component Library panel, select Outputs. Find the GoldenDB component in the output component list on the right side, and drag it to the canvas.

6. Click and drag the icon of the target input, transform, or flow component to connect it to the current GoldenDB output component.

7. On the GoldenDB output component, click the icon to open the GoldenDB Output Configuration dialog box.

8. In the GoldenDB Output Configuration dialog box, configure the parameters.

   Parameter		Description
   Basic Settings	Step Name	The name of the GoldenDB output component. Dataphin generates a default step name that you can modify. The name must meet the following requirements: It can contain only Chinese characters, letters, underscores (_), and digits. It cannot exceed 64 characters.
   	Datasource	The dropdown list displays all GoldenDB data sources, regardless of whether you have write-through permissions. Click the icon to copy the data source name. For data sources for which you do not have write-through permissions, you can click Apply next to the data source to apply for write-through permissions. For more information, see Apply for data source permissions. If you do not have a GoldenDB data source, click Create Data Source to create one. For more information, see Create a GoldenDB data source.
   	Table	Select the target table for output data. You can enter a table name keyword to search, or enter the exact table name and click Exact Match. After selecting a table, the system automatically checks the table status. Click the icon to copy the name of the currently selected table.
   	Loading Policy	Specifies how data is written to the target GoldenDB table. Available policies: Append Data: When a primary key or constraint violation occurs, the system prompts a dirty data error. Overwrite Data: When a primary key or constraint violation occurs, the system first deletes the original data and then inserts the entire new row of data.
   	Batch Write Size (optional)	The maximum data size per batch write. Works together with Batch Write Count: the system writes a batch whenever either limit is reached first. The default is 32M.
   	Batch Write Count (optional)	The maximum number of records per batch write. Default: 2048 records. The system uses both Batch Write Count and Batch Write Size as thresholds. A batch is flushed to the target whenever the accumulated data reaches either the size limit or the count limit. We recommend setting the batch write size to 32 MB. Adjust the batch write count based on the actual single-record size, typically setting it higher to maximize batch efficiency. For example, if a single record is about 1 KB, you can set the batch write size to 16 MB and the batch write count to a value greater than 16 MB / 1 KB = 16,384 records (for example, 20000 records). With this configuration, the system triggers a batch write whenever the accumulated data reaches 16 MB.
   	Prepare Statement (optional)	The SQL script to be executed on the database before data import. For example, to ensure continuous service availability: before writing data, create a target table Target_A and write to it. After writing completes, rename the active table Service_B to Temp_C, rename Target_A to Service_B, and then delete Temp_C.
   	Post Statement (optional)	The SQL script to be executed on the database after data import.
   Field Mapping	Input Fields	Displays the input fields based on the output of the upstream component.
   	Output Fields	Displays the output fields. You can perform the following operations: Field Management: Click Field Management to select output fields. Click the icon to move Selected Input Fields to Unselected Input Fields. Click the icon to move Unselected Input Fields to Selected Input Fields. Batch Add: Click Batch Add to configure in JSON, TEXT, or DDL format. Batch configuration in JSON format, for example: // Example: [{ "name": "user_id", "type": "String" }, { "name": "user_name", "type": "String" }] Note `name` is the name of the imported field. `type` is the data type of the field after import. For example, "name":"user_id","type":"String" imports the field named `user_id` and sets its data type to `String`. Batch configuration in TEXT format, for example: // Example: user_id,String user_name,String Row delimiters are used to separate information for each field. The default is a line feed (\n), and it supports line feed (\n), semicolon (;), and period (.). Column delimiters are used to separate field names from field types. The default is a comma (,). Batch configuration in DDL format, for example: CREATE TABLE tablename ( id INT PRIMARY KEY, name VARCHAR(50), age INT ); Create New Output Field: Click +Create New Output Field, fill in the Column and select the Type as prompted. After completing the configuration for the current row, click the icon to save.
   	Mapping	Map upstream input fields to target table fields. Mapping supports Same Row Mapping and Same Name Mapping. Same Name Mapping: Maps fields with the same name. Same Row Mapping: Maps fields by row position when the source and target field names differ.

9. Click Confirm to complete the property configuration of the GoldenDB output component.