In log cleansing, operations such as splitting and static join are performed on logs to convert logs into the standard Key-Value (KV) format. Currently, two log cleansing modes are supported: intelligent splitting and custom splitting.
In this mode, log-based data capturing results are intelligently split and converted to the standard KV format. In addition, several optional schemes are provided. You can adjust the selected scheme, including deleting and modifying the scheme.
Select the Intelligent Split tab, and enter the intelligent split mode.
Click Obtain Scheme. Several schemes recommended by the system are displayed. Select one scheme as needed.
Move the pointer to the line where the field to be modified is located. The Edit and Delete icons are displayed. You can fine-tune the field as needed, for example, by modifying the field name and split type, as shown in the following figure.
Tip: After a scheme is modified, you can click Reset Scheme to restore the recommended scheme.
You can click the Custom Splitting tab to enter the custom splitting mode. For instructions on how to use this mode, see Use custom splitting.
Here is a log example.
2016-07-27 23:37:23|c0a895e114526786450161001d1ed9|9|EADS|BIZ-MONITOR|0|Category=Women's wear&Region=Shanghai&eventTeyp=3&Gender=0&Price=20|iZ28ql4lx29Z
2016-07-27 23:37:23|c0a895e114526786450161001d1ed9|9|EADS|BIZ-MONITOR|0|Category=Children's wear&Region=Shenzhen&eventTeyp=3&Gender=1&Price=30|iZ28ql4lx29Z
Build blocks by drag and drop.
Click Log Splitting Preview.
After data cleansing, click Save. Click Next to perform other configurations, such as dataset configuration.