You can filter logs by using the processor_filter_regex plug-in. This topic describes the parameters that you must configure for the processor_filter_regex plug-in. This topic also provides examples on how to configure the parameters.

Parameters

The following table describes the parameters that you can configure in the detail parameter when you set the type parameter to processor_filter_regex.
Note A log entry is collected only when it exactly matches the regular expressions that are specified in the Include parameter and does not match the regular expressions that are specified in the Exclude parameter.
Parameter Type Required Description
Include JSON Object No An array of key-value pairs. In each key-value pair, the key specifies a field, and the value specifies a regular expression that the value of the same field in each log entry must match. The specified keys are in AND relations. If the values of all fields in a log entry match the regular expressions that are specified in the Include parameter, the log entry is collected.
Exclude JSON Object No An array of key-value pairs. In each key-value pair, the key specifies a field, and the value specifies a regular expression that the value of the same field in each log entry must match. The specified keys are in OR relations. If the value of any field in a log entry matches a regular expression that are specified in the Include parameter, the log entry is not collected.

Configuration example

In this example, only log entries in which the value of the ip field is prefixed by 10, the value of the method field is POST, and the value of the browser field is not aliyun.* are collected.

  • Raw logs
    • Log entry 1
      "ip" : "10.**.**.**"
      "method" : "POST"
      "browser" : "aliyun-sdk-java"
    • Log entry 2
      "ip" : "10.**.**.**"
      "method" : "POST"
      "browser" : "chrome"
    • Log entry 3
      "ip" : "192.168.*.*"
      "method" : "POST"
      "browser" : "ali-sls-ilogtail"
  • Logtail plug-in configuration for processing
    {
       "type" : "processor_filter_regex",
        "detail" : {
             "Include" : {
                "ip" : "10\\..*",
                "method" : "POST"
             },
             "Exclude" : {
                "browser" : "aliyun.*"
             }
        }
    }
  • Output data
    Log entry Collected Reason
    Log entry 1 No The value of the browser parameter matches a regular expression that is specified in the Exclude parameter.
    Log entry 2 Yes The values of all fields match the regular expressions that are specified in the Include parameter.
    Log entry 3 No The value of the ip parameter does not match the regular expression that is specified in the Include parameter.