Configure the data filtering node - DataWorks - Alibaba Cloud Documentation Center

The data filtering plug-in is used to filter data based on specific rules, such as the field size. Only data that meets the rules is retained.

Prerequisites

An input node is configured. For more information, see Data source types that support real-time synchronization.

Procedure

Go to the DataStudio page.
Log on to the DataWorks console. In the top navigation bar, select the desired region. In the left-side navigation pane, choose Data Development and O&M > Data Development. On the page that appears, select the desired workspace from the drop-down list and click Go to Data Development.
In the Scheduled Workflow pane of the DataStudio page, move the pointer over the icon and choose Create Node > Data Integration > Real-time Synchronization.
Alternatively, find the desired workflow in the Scheduled Workflow pane, right-click the workflow name, and then choose Create Node > Data Integration > Real-time Synchronization.
In the Create Node dialog box, set the Sync Method parameter to End-to-end ETL and configure the Name and Path parameters.
Click Confirm.
On the configuration tab of the real-time synchronization node, drag Data Filtering in the Conversion section to the canvas on the right. Draw a line to connect the data filtering node to the input node.
Click the data filtering node. In the configuration panel that appears, configure the parameters.
- Node configuration
  Rules: the rules based on which you want to filter data in a data source. Only data that meets the rules is retained. For example, in the preceding figure, a rule is configured to retain data for which the values of the "hello" field are less than the values of the "this" field.
  Note
  When you configure a rule, you can configure either the Customize parameter or the second Field parameter.
- Output field
  The names and types of output fields after filtering.
Click the icon in the top toolbar to save the node.