Dataphin:Configure TDH Inceptor Input Component

Limits

The TDH Inceptor input component supports data tables in formats such as ORC, Parquet, text, RC, and SEQ. However, it does not support the integration of transactional tables in ORC format.

Prerequisites

• A TDH Inceptor data source must be created. For more information, see Create TDH Inceptor Data Source.

• To configure the TDH Inceptor input component properties, the account must possess read-through permission for the data source. If you lack this permission, you must obtain it from the data source. For more information, see Request Data Source Permission.

Procedure

1. On the Dataphin home page, select Development > Data Integration from the top menu bar.

2. On the integration page's top menu bar, select Project (Dev-Prod mode requires selecting an environment).

3. In the left-side navigation pane, click Batch Pipeline. In the Batch Pipeline list, click the offline pipeline you want to develop to open its configuration page.

4. Click the Component Library in the upper right corner of the page to open the Component Library panel.

5. In the Component Library panel's left-side navigation pane, select Input. Find the TDH Inceptor component in the list on the right and drag it to the canvas.

6. Click the icon on the TDH Inceptor input component card to open the TDH Inceptor Input Configuration dialog box.

7. In the TDH Inceptor Input Configuration dialog box, set the parameters.

   Parameter	Description
   Step Name	This is the name of the TDH Inceptor input component. Dataphin automatically generates the step name, and you can also modify it according to the business scenario. The name must meet the following requirements: Can only contain Chinese characters, letters, underscores (_), and numbers. Cannot exceed 64 characters.
   Datasource	The data source drop-down list displays all TDH Inceptor type data sources in the current Dataphin, including data sources for which you have read-through permission and those for which you do not. Click the icon to copy the current data source name. For data sources without read-through permission, you can click Request after the data source to request read-through permission. For more information, see Request Data Source Permission. If you do not have a TDH Inceptor type data source, click Create Data Source to create a data source. For more information, see Create TDH Inceptor Data Source.
   Table	Select the source table for data synchronization. Click the icon to copy the name of the currently selected table.
   Partition	If the table you selected is a partitioned table, you need to enter partition information, such as ds=${bizdate}.
   When Partition Does Not Exist	You can select the following policies to handle scenarios where the specified partition does not exist: Fail The Task: Terminate the task and mark it as failed. Succeed The Task Without Writing Data: The task runs successfully without writing data.
   File Encoding	Select the file encoding method. The system supports UTF-8 and GBK methods.
   Compression Format	Select the compression format of the file. The system supports compression formats such as zlib and hadoop-snappy.
   Field Separator	Enter the separator for file storage. If you do not enter it, the system defaults the field separator to a comma (,).
   Output Fields	The output fields area displays all field information of the selected table. If you do not need to output certain fields to downstream components, you can delete the corresponding fields: Note When the compute engine is TDH Inceptor, the output fields of the TDH Inceptor input component support viewing the classification of fields. Non-TDH Inceptor compute engines do not support this. Single Field Deletion Scenario: If you need to delete a small number of fields, you can click the icon in the operation column to delete the extra fields. Batch Field Deletion Scenario: If you need to delete many fields, you can click Field Management, select multiple fields in the Field Management dialog box, click the left shift icon to move the selected input fields to the unselected input fields, and click Confirm to complete the batch deletion of fields.

8. Click Confirm to complete the configuration of the TDH Inceptor input component properties.