After configuring the source node for data parsing, you can define fields and expressions using the parsed data and apply filters for data filtering, enabling further analysis or output.
Prerequisites
Ensure the source node is configured.
Configure data computing
Configure expressions to preprocess output fields from the preceding node.
In the middle canvas, click the add
icon following the current node.From the node list that appears, click the Data Computing node.
Click the Data Computing node in the canvas to display the original field list in the right-side configuration panel.
Adjust the Field Name, Data Type, Precision, Decimal Place, and the Output To Next Node option as needed.
NoteOnly fields with the Output To Next Node option selected can be used in subsequent node configurations.
(Optional) Click Add Field, and in the dialog box that appears, configure the Field Name, Data Type, Precision, Decimal Places, Output To Next Node, and Expression following the page prompts.
ImportantExpressions support functions such as Date functions, Mathematical functions, and String functions. Nested use of different function types is permitted.
Use the Calculation Formula column's Original field for expressions.
JSONPath can be utilized in expressions to retrieve property values from JSON data, which may be structured as an array or a nested JSON.
For instance,
{a:{key1:v1, key2:v2}}allows retrieval of the valuev2by accessinga.key2.For additional details on JSONPath, refer to LanguageManual UDF.
Distinguish between single and double quotes when using variables in expressions:
Single quotes denote constants.
Double quotes or no quotes indicate variables.
(Optional) Repeat the previous step to add additional fields as needed.
To use newly added fields in the Data Computing node for further data generation, configure multiple expressions by following the steps outlined above.
Click Save in the upper right corner of the data parsing workspace to finalize the Data Computing node configuration.
Configure data filtering
Configure data filtering to refine the output fields further.
In the middle canvas, click the add
icon following the current node.From the node list that appears, click the Data Filtering node.
Click the Data Filtering node in the canvas. In the right-side configuration panel, use the table below to add filter conditions.
Item
Description
Field
Select the field you wish to filter.
Data Type
The data type is automatically determined upon field selection and is not editable.
Operator
Choose from operators such as ≠ (Not Equal To), = (Equal To), > (Greater Than), < (Less Than), >= (Greater Than Or Equal To), <= (Less Than Or Equal To), Between, Like, and isNotNull.
Filter Value
Input the value for the filter.
(Optional) Click Add AND Condition to include multiple AND conditions, following the steps above.
Click Save in the upper right corner of the data parsing workspace to finalize the Data Filtering node configuration.
What to do next
Once the data computing and filtering nodes are configured, proceed to set up other processing nodes or configure the target node to complete the data parsing task.