After configuring the Kudu input widget, you can read data from the Kudu data source into Dataphin for data integration and development. This topic explains how to configure the Kudu input widget.
Prerequisites
The Kudu data source is created. For more information, see Create a Kudu data source.
The account configuring the Kudu input widget property must have read-through permission for the data source. If you lack the necessary permissions, you must obtain them from the data source. For more information, see Request data source permission.
Procedure
On the top menu bar of the Dataphin home page, select Development > Data Integration.
On the top menu bar of the integration page, select Project (Dev-Prod mode requires selecting the environment).
In the left-side navigation pane, click Batch Pipeline. In the Batch Pipeline list, click the offline pipeline that needs development to open its configuration page.
Click Component Library in the upper right corner of the page to open the Component Library panel.
In the left-side navigation pane of the Component Library panel, select Input, find the Kudu component in the input widget list on the right, and drag the component to the canvas.
Click the
icon in the Kudu input widget card to open the Kudu Input Configuration dialog box.In the Kudu Input Configuration dialog box, configure the parameters.
Parameter
Description
Basic configuration
Step Name
This is the name of the Kudu input widget. Dataphin automatically generates the step name, and you can also modify it according to the business scenario. The name must meet the following requirements:
Can only contain Chinese characters, letters, underscores (_), and numbers.
Cannot exceed 64 characters.
Datasource
In the data source drop-down list, all Kudu-type data sources are displayed, including data sources for which you have read-through permission and those for which you do not. Click the
icon to copy the current data source name.For data sources without read-through permission, you can request read-through permission for the data source by clicking Request after the data source. For more information, see Request, renew, and return data source permission.
If you do not have a Kudu-type data source, click Create to create a data source. For more information, see Create a Kudu data source.
Table
Select the table you need to read. Click the
icon to copy the name of the currently selected table.Advanced configuration
Batch Read Data Volume
The number of bytes for batch reading data, default is 1 MB. Must be greater than 0 (one decimal place).
Input Filter
Input filter conditions support
=, >, <, >=, <=, is not null, is null. Each expression must be enclosed in double quotes"". Each expression must be separated by spaces, such as:"id > 10", "name = "dataphin".Connection Timeout
The AdminOperationTimeout parameter is used to set the timeout for operations such as createTable and deleteTable. The default is 30 s. Setting it to 0 disables the timeout.
Read Timeout
The defaultOperationTimeout parameter is used to set the timeout for sessions and scans. The default is 30 s. Setting it to 0 disables the timeout.
Output Fields
The output fields area displays all fields hit by the selected table and filter conditions. If you do not need to output certain fields to downstream widgets, you can delete the corresponding fields:
Single field deletion scenario: If you need to delete a small number of fields, you can click the
icon in the operation column to delete the extra fields.Batch field deletion scenario: If you need to delete many fields, you can click Field Management, select multiple fields in the Field Management dialog box, then click the
left shift icon to move the selected input fields to the unselected input fields and click Confirm to complete the batch deletion of fields.
Click Confirm to complete the property configuration of the Kudu input widget.