Dataphin:Configure the filter transform widget

Procedure

1. On the Dataphin home page, select Development > Data Integration from the top menu bar.

2. At the top menu bar of the integration page, select Project (choose an Environment for the Dev-Prod pattern).

3. In the left-side navigation pane, click Batch Pipeline. Then, in the Batch Pipeline list, click the Offline Pipeline you want to develop to open its configuration page.

4. Click Component Library in the upper right corner of the page to open the Component Library panel.

5. In the Component Library panel's left-side navigation pane, select Transform. Locate the Filter component in the list of transform widgets on the right and drag it onto the canvas.

6. Click and drag the icon from the target upstream and downstream components to connect them to the current filter component.

7. Click the icon on the filter component card to open the Transform-filter Configuration dialog box.

8. In the Transform-filter Configuration dialog box, set the necessary parameters.

   Parameter	Description
   Step Name	This is the name of the filter transform widget. Dataphin automatically generates the step name, but you can modify it based on the business scenario. The name must meet the following requirements: Can only contain Chinese characters, letters, underscores (_), and numbers. Cannot exceed 64 characters.
   Input Fields	Displays input fields based on the upstream output. If the field name contains a table name, the table name is also displayed.
   Filter Conditions	Enter filter conditions. Supports code editor mode and configuration mode. Code Editor Mode: Set filter conditions using scripts. Do not include WHERE; directly enter the content following the WHERE statement. For example, user_id is not null or and name != 'xxx'. Configuration Mode: Set filter conditions using the interface (configure the field, operator, and content in sequence. If the field name contains a table name, the table name is also displayed). Supported Filter Conditions include =, !=, >, >=, <, <=, LIKE, NOT LIKE, IS NULL, IS NOT NULL, START WITH, END WITH. Multiple conditions support AND and OR logical operators. For example: If the input fields include id, name, and ds, and you need to filter data where id<123 and name is not equal to abc. If using code editor mode, enter the statement id < 123 AND name != 'abc'; if configuring directly through the interface, the filter conditions should be as shown in the following image. Important The LIKE operator does not support using the (%) symbol to specify starting or ending characters; it only supports filtering data that contains the specified characters. If you need to specify starting or ending characters, you can use the START WITH and END WITH statements. For example, name START WITH 'Zhang' AND name END WITH 'San' will filter data where the starting character of the name field is Zhang and the ending character is San.

9. Click Confirm to finalize the property settings for the Filter Transform Widget.