This topic describes the syntax, parameters, and examples of event check functions.

Functions

Type Function Description
Basic functions v Extracts the value of a field in an event. If two or more field names are passed to the function, the value of the first field that exists is returned.
e_has Checks whether a field exists.
e_not_has Checks whether a field does not exist.
Expression functions e_search Searches for an event with Lucene-like query syntax.
e_match* Checks whether the value of a field in an event meets conditions specified in an expression. This type of function include e_match, e_match_all, and e_match_any.
The following table lists the expression functions you can combine with event check functions.
Type Function Description
Logical functions op_and Invokes the AND operation among expressions.
op_or Invokes the OR operation among expressions.
op_not Invokes the NOT operation among expressions.
op_nullif Checks whether the values of two expressions are equal.
op_ifnull Returns the value of the first expression whose value is not None.
op_coalesce Returns the value of the first expression whose value is not None.

v

  • Syntax
    v(Field name, ..., default=None)
  • Parameters
    Parameter Data type Required Description
    Field name String Yes The name of the field in an event from which you need to extract a value.
    default Arbitrary No The value of this parameter is returned when none of the fields passed to the function are located. Default value: None.
  • Response

    The value of the first field that exists in the specified event is returned. If none of the fields passed to the function are located, the value of the default parameter is returned.

  • Example
    Assigns the value of the content field to the test_content field.
    • Raw log entry:
      content: hello
    • Transformation rule:
      e_set("test_content", v("content"))
    • Result:
      content: hello
      test_content: hello

e_has

  • Syntax
    e_has("Field name")
  • Parameters
    Parameter Data type Required Description
    Field name String Yes The name of a field in an event.
  • Response

    If the specified field is located, the value True is returned. Otherwise, the value False is returned.

  • Example
    Checks whether an event contains the content field. If the content field is located, the event is retained. Otherwise, the event is discarded.
    • Raw log entry:
      content: 123
    • Transformation rule:
      e_keep(e_has("content"))
    • Result:
      content: 123

e_not_has

  • Syntax
    e_not_has("Field name")
  • Parameters
    Parameter Data type Required Description
    Field name String Yes The name of a field in an event.
  • Response

    If the specified field is not located, the value True is returned. Otherwise, the value False is returned.

  • Example

    Checks whether the event does not contain the content field. If the event does not exist, the event is retained. Otherwise, the event is discarded.

    • Raw log entry:
      content: 123
    • Transformation rule:
      e_if(e_not_has("content"),KEEP,DROP)
    • Result:
      # The event is discarded.

e_search

  • Syntax
    e_search(Query string)
  • Parameters
    Parameter Data type Required Description
    Query string String Yes The syntax that is used to filter log data and simplify data processing.
  • Response

    If the specified conditions are satisfied, the value True is returned. Otherwise, the value False is returned.

  • Examples
    Full-text search
    e_search("active error")     # Searches for multiple substrings in full text. The substrings are associated with each other through the OR logical operator.
    e_search('"active error"')   # Searches for a substring in full text.
    
    Field search
    e_search("status: active")         # Searches for a substring in a specified field.
    e_search('author: "john smith"')   # Searches for a substring that contains a space character in a specified field.
    e_search('field: active error')   # Searches the specified field for the substring "active" or searches all logs for the substring "error". The query string in this example is equivalent to field:active OR "error".
    
    Exact match
    e_search('author== "john smith"')  
    
    Search with wildcard characters An asterisk (*) is used to match zero or more characters. A question mark (?) is used to match zero or more characters.
    e_search("status: active*test")    # The query string does not need to be enclosed with double quotation marks (") because it contains only an asterisk (*).
    e_search("status: active? good")    # The query string contains only a question mark (?), and therefore it does not need to be enclosed with double quotation marks (").
    e_search("status== ac*tive? good")  # The query string is used for exact match. 
    
    Escape special characters in a field value Asterisks (*) and question marks (?) in a field value must be escaped.
    e_search('status: "\*\?()[]:="')  # The field value in this example is valid. The value contains special characters and is enclosed with double quotation marks ("). The * and \ characters in the value are escaped.
    e_search("status: active\*test")  # The field value in this example is valid. The value contains an asterisk (*). Therefore, it does not need to be enclosed with double quotation marks (").
    e_search("status: active\? test")  # The field value in this example is valid. The value contains a question mark (?). Therefore, it does not need to be enclosed with double quotation marks (").
    
    Escape special characters in a field name
    e_search("\*\(\1+1\)\?: abc")                 # The format of the field name in this example is invalid. The field name cannot be enclosed with double quotation marks ("). Special characters must be escaped by using backslashes (\).
    e_search("__tag__\:__container_name__: abc")  # The format of the field name in this example is valid. The field name is not enclosed with double quotation marks ("). Special characters are escaped by using backslashes (\).
    [DO NOT TRANSLATE]
    
    Search with regular expressions
    e_search('content~="regular expression"')   # Search for a string that matches the regular expression.
    
    Numeric value comparison
    e_search('count: [100, 200]')   # >=100 and <=200
    e_search('count: [*, 200]')     # <=200
    e_search('count: [200, *]')     # >=200
    e_search('age >= 18')           # >= 18
    e_search('age > 18')            # > 18
    
    Logical relationships
    e_search("abc OR xyz")    # The logical operator is case insensitive.
    e_search("abc and (xyz or zzz)")
    e_search("abc and not (xyz and not zzz)")
    e_search("abc && xyz")    # and
    e_search("abc || xyz")    # or
    e_search("abc || ! xyz")   # or not

e_match*

  • Syntax
    e_match(Field name, Regular expression, full=True)
    e_match_all(Field name 1, Regular expression 1, Field name 2, Regular expression 2, ..., full=True)
    e_match_any(Field name 1, Regular expression 1, Field name 2, Regular expression 2, ..., full=True)
    Note
    • The field name and regular expression parameters in the function must appear in pairs.
    • This e_match function is often used together with the op_not, op_and, or op_or function.
  • Parameters
    Parameter Data type Required Description
    Field name String Yes The name of a field. If the specified field is not located, the condition specified by the field is not met.

    For example, if the f1 field is not located, the value returned for the e_match("f1", ...) clause is False.

    Regular expression String Yes The regular expression used to match strings. To match occurrences by using strings, you can use the str_regex_escape function to covert regular expressions.
    full Boolean No Indicates whether to implement exact match. By default, the parameter is set to True, which indicates exact match. For more information, see Regular expressions.
  • Response
    The value True is returned if the specified fields are located. Otherwise, the value False is returned.
    Note
    • e_match_any: If one or more fields are located, the value True is returned. Otherwise, the value False is returned.
    • e_match_all: If all specified fields are located, the value True is returned. Otherwise, the value False is returned.
  • Examples
    • Example 1
      Implements exact match by using the e_match function.
      • Raw low entry:
        k1: 123
      • Transformation rule:
        e_set("match",e_match("k1",r'\d+'))
      • Result:
        k1: 123
        match: True
    • Example 2
      Example 2: Implements exact match by using the e_match_all function.
      • Raw log entry:
        k1: 123
        k2: abc
        k3: abc123
      • Transformation rule:
        e_set("match",e_match_all('k1', r'\d+', 'k4', '. +'))
      • Result:
        k1: 123
        k2: abc
        k3: abc123
        match: False
      Note The e_match_any function works in a similar manner to the e_match_all function.