Dataphin:Configure ArgoDB input widget

Limits

The ArgoDB input widget supports ArgoDB tables in data formats such as orc, parquet, text, holodesk, rc, and seq. However, it does not support the integration of transactional tables in ORC format.

Prerequisites

• An ArgoDB data source has been created. For more information, see Create an ArgoDB data source.

• To configure the properties of the ArgoDB input widget, the account must have read-through permission for the data source. If you lack the necessary permission, you must obtain it for the data source. For more information, see Request data source permission.

Procedure

1. On the Dataphin home page, select Development > Data Integration from the top menu bar.

2. In the top menu bar of the integration page, select Project (Dev-Prod mode requires selecting an environment).

3. In the left-side navigation pane, click Batch Pipeline. Then, in the Batch Pipeline list, click the offline pipeline you want to develop to open its configuration page.

4. Click the Component Library in the upper right corner of the page to open the Component Library panel.

5. In the Component Library panel's left-side navigation pane, select Input. Find the ArgoDB widget in the input widget list on the right and drag it to the canvas.

6. Click the icon on the ArgoDB input widget card to open the ArgoDB Input Configuration dialog box.

7. In the Argodb Input Configuration dialog box, configure the parameters.

   Parameter	Description
   Step Name	This is the name of the ArgoDB input widget. Dataphin automatically generates the step name, and you can also modify it according to the business scenario. The naming convention is as follows: It can only contain Chinese characters, letters, underscores (_), and numbers. It cannot exceed 64 characters.
   Datasource	The data source drop-down list displays all ArgoDB-type data sources currently in Dataphin, including data sources for which you have read-through permission and those for which you do not. Click the icon to copy the current data source name. For data sources without read-through permission, you can click Request after the data source to request read-through permission for the data source. For more information, see Request data source permission. If you do not have an ArgoDB-type data source, click Create Data Source to create a data source. For more information, see Create an ArgoDB data source.
   Table	Select the data table to be integrated according to the actual scenario. Click the icon to copy the name of the currently selected table.
   Partition	If the selected source table is a partitioned table, you need to fill in the partition information, such as state_date=‘20190101’. It also supports parameter methods to obtain data incrementally every day, such as state_date=${bizdate}.
   File Encoding	Select the encoding format according to the actual data situation. Supports UTF-8 and GBK.
   Compression Format	This is optional. If the file is compressed, select the corresponding compression format to allow Dataphin to decompress it.
   Field Separator	Use the set field separator to write to the target data source. If not filled in, the default is \u0001.
   Output Fields	The output fields area displays all fields of the selected table. If certain fields do not need to be output to downstream widgets, you can delete the corresponding fields: Note When the compute engine is ArgoDB, the output fields of the ArgoDB input widget support viewing the classification and grading of fields. Non-ArgoDB compute engines do not support this. If you need to delete a small number of fields, you can click the icon in the operation column to delete the extra fields. If you need to delete many fields, you can click Field Management, select multiple fields on the field management page, and then click the icon to move the selected input fields to the unselected input fields.

8. Click Confirm to complete the property configuration for the ArgoDB input widget.