Use the field selection transform component to selectively retain a subset of fields from upstream inputs.
Procedure
-
In the top menu bar of the Dataphin homepage, choose Developer > Data Integration.
-
In the top menu bar of the integration page, select Project. (In Dev-Prod mode, you need to select Environment).
-
In the left-side navigation pane, click Batch Pipeline. In the Batch Pipeline list, click the offline pipeline that you want to develop to open its configuration page.
-
Click Component Library in the upper-right corner of the page to open the Component Library panel.
-
In the left-side navigation pane of the Component Library panel, select Transform. In the transform component list on the right, find the Field Selection component and drag it to the canvas.
-
Click and drag the
icon of the target upstream and downstream components to connect them to the current field selection component. -
Click the
icon in the field selection component card to open the Field Selection Configuration dialog box. -
In the Field Selection Configuration dialog box, configure the parameters.
Parameter
Description
Step Name
The name of the field selection transform component. Dataphin generates a default step name that you can modify. The name must meet the following requirements:
-
It can contain only Chinese characters, letters, underscores (_), and numbers.
-
It can be up to 64 characters in length.
Input Fields
Lists the input fields from the upstream component output.
Output Fields
Lists the output fields. Click Field Management:
-
Click the
icon to move Selected Input Fields to Unselected Input Fields. -
Click the
icon to move Unselected Input Fields to Selected Input Fields.
After you select the input fields, the Output Fields area shows the following information:
-
Serial Number: The sequence number of the input field.
-
Input Field Name, Input Field Type: The name and type of the input field. If the name includes a table name, the table name is also displayed. These values are read-only.
-
Output Field Name: Defaults to the input field name and can be edited. If the name includes a table name, it is displayed as
table_name.field_name. If left empty, the input field name is used automatically. -
Output Field Type: Can be converted to a different data type based on the input field type. See the page for supported conversions.
-
Actions: Click the delete icon
in the Actions column to remove a field.
-
-
Click Confirm to save the Field Selection component configuration.