Dataphin:Configure the StarRocks output component

Prerequisites

• You have added a StarRocks data source. For more information, see Create a StarRocks data source.

• Your account has write-through permission for the data source. If not, request permission. Request, renew, and revoke data source permissions.

Stream load latency

When you import data to StarRocks using stream load, the returned status can vary. A publish timeout may occur even when the job succeeds. Monitor the status in the operational log:

• Success: The import succeeded, and data is visible.

• Publish Timeout: The import job committed successfully, but data is not immediately visible for some reason. Consider it successful. No re-import is needed.

• Label Already Exists: Another job already uses this label. The import might have succeeded or is in progress.

• Fail: This import failed. You can retry the job by specifying a label.

Procedure

1. On the Dataphin home page, select Development > Data Integration from the top menu bar.

2. On the Integration page, in the top menu bar, select Project (In Dev-Prod mode, select an environment).

3. In the left navigation pane, click Offline Integration. In the Offline Integration list, click the target offline pipeline to open its configuration page.

4. Click Component Library in the upper-right corner of the page to open the Component Library panel.

5. In the left pane of Component Library, select Output, find StarRocks, and drag it to the canvas.

6. Drag the icon from the input component to the StarRocks output component to connect them.

7. Click the icon on the StarRocks output component to open the StarRocks output configuration dialog box.

   

8. In the StarRocks output configuration dialog box, configure the following parameters.

   Parameter		Description
   Basic Settings	Step Name	Auto-generated name of this component. You can modify it. Naming rules: Can contain only Chinese characters, letters, underscores (_), and digits. Length cannot exceed 64 characters.
   	Datasource	Lists all StarRocks data sources. Click to copy the data source name. If you lack write-through permissions, click Request. Request data source permissions. If you do not have a StarRocks data source, click New Data Source to create one. For more information, see Create a StarRocks data source.
   	Table	Select the target table for output data. Enter a table name keyword to search, or use Exact Search for an exact match. The system automatically checks the table status after selection. Click to copy the table name. If no target table exists, create one: Click One-Click Table Creation. Dataphin generates the table creation SQL, including the table name (defaults to the source table name) and field types (converted from Dataphin field types). Modify the SQL as needed, then click New. Dataphin sets the newly created table as the output target. Note If a table with the same name exists in the development environment, Dataphin reports an error. If no matching item is found, you can also integrate based on a manually entered table name. View selection is not supported in copy mode.
   	Production Table Missing Policy	How to handle a missing production table at publish time. Options: No Action or Automatic Creation. Default: Automatic Creation. Do Not Process: A warning appears if the target table is missing, but publishing proceeds. You must manually create the table in production before running the task. Automatic Creation: Click Edit The Table Creation Statement to customize the pre-filled SQL. Use ${table_name} as the table name placeholder. This is the only supported placeholder and is replaced at execution time. If the target table does not exist, the system creates it using this statement. If creation fails, the publish check fails — fix the statement and publish again. If the table already exists, no creation occurs. Note This option is supported only in Dev-Prod mode projects.
   	Data Format	You can select CSV or JSON. If you select CSV, also configure CSV import column delimiter and CSV import row delimiter.
   	CSV import column delimiter (Optional)	Column delimiter for StreamLoad CSV import. Default: _@dp@_. Leave unset to use the default. If your data contains _@dp@_, specify a different delimiter.
   	CSV import row delimiter (Optional)	Row delimiter for StreamLoad CSV import. Default: _#dp#_. Leave unset to use the default. If your data contains _#dp#_, specify a different delimiter.
   	Batch write data volume (Optional)	Data volume per write operation. Default: 32 MB. The system writes when either this limit or the Batch Write Count is reached first.
   	Number of batch write records (Optional)	Default: 2,048 records. Works with Number Of Records Per Batch and Data Volume Per Batch — the system writes when either limit is reached first. When either the data volume or record count limit is reached, the system writes the batch to the target. Tune the record count based on individual record size. Example: with ~1 KB records, set the batch byte size to 16 MB and the record count above 16,384 (e.g., 20,000 records). The system then writes each time the accumulated data reaches 16 MB.
   	Prepare Statement (Optional)	SQL script executed before data import. Example: for zero-downtime table swaps, create Target_A, write data to it, then rename Service_B → Temp_C, rename Target_A → Service_B, and drop Temp_C.
   	End Statement (Optional)	SQL script executed after data import.
   Field Mapping	Input Fields	Shows fields from upstream components.
   	Output Fields	Shows output fields. Supported operations: Field Management: Click Field Management to select output fields. Click the icon to move Selected input fields to Unselected input fields. Click the icon to move Unselected input fields to Selected input fields. Batch Add: Click Batch Add to configure fields in JSON, TEXT, or DDL format. JSON format example: // Example: [{ "name": "user_id", "type": "String" }, { "name": "user_name", "type": "String" }] Note name is the field name and type is the field type. Example: "name":"user_id","type":"String" imports field user_id as String. TEXT format example: // Example: user_id,String user_name,String Row delimiter: line feed (\n) by default. Also supports semicolon (;) and period (.). Column delimiter: comma (,) by default. DDL format example: CREATE TABLE tablename ( id INT PRIMARY KEY, name VARCHAR(50), age INT ); Create Output Field: Click +Create Output Field, fill in Column and select Type, then click to save.
   	Mapping	Map upstream input fields to target table fields. Mapping supports Row Mapping and Name Mapping. Name-based mapping: Matches fields by name. Row-wise mapping: Maps fields by row position, regardless of field names.

9. Click Confirm.