This topic describes the syntax and parameters of field processing functions. This topic also provides examples on how to use the functions.
Functions
Function | Description |
---|---|
v | Extracts the value of a field from an event. If multiple field names are passed in the function, the value of the first field that exists is returned. |
e_set | Adds a field or specifies a new value for an existing field. |
e_drop_fields | Deletes the log fields that meet a specified condition. |
e_keep_fields | Retains the log fields that meet a specified condition. |
e_pack_fields | Encapsulates specified log fields, and then assigns the log fields as a value to a new field. |
e_rename | Renames the log fields that meet a specified condition. |
v
You can use the v function to extract the value of a field from an event.
- Syntax
v(Field name, ..., default=None)
- Parameters
Parameter Type Required Description Field name String Yes The name of the field whose value you want to extract from an event. default Arbitrary No The value of this parameter is returned if none of the specified fields exist. Default value: None. - Response
The value of the first field that exists in an event is returned. If none of the specified fields exist, the value of the
default
parameter is returned. - Example
Assign the value of the content field to the test_content field.
- Raw log entry:
content: hello
- Transformation rule:
e_set("test_content", v("content"))
- Result:
content: hello test_content: hello
- Raw log entry:
e_set
- Syntax
e_set(key1, value1, key2, value2, mode="overwrite")
Notice- The key1 and value1 parameters must be specified in pairs.
- If you use the e_set function to specify a value for a time field, such as F_TIME or __time__, the value must be a numeric string.
e_set(F_TIME, "abc") # Invalid syntax. e_set(F_TIME, "12345678") # Valid syntax.
- Parameters
Parameter Type Required Description key String Yes The name of a log field. You can set this parameter to an expression that is used to return a string. For more information about how to specify special field names, see Event structure and fields. value Arbitrary Yes The new value of a specified field. If the value of this parameter is not a string, the function automatically converts the value to a string. For example, if you set this parameter to a value of the tuple, list, or dictionary type, the function automatically converts the value to a JSON string. For more information about the conversion rule of strings, see Automatic type conversion during assignment. Note If you set this parameter to None, the function does not update the original value of the specified field.mode String No The overwrite mode of fields. Default value: overwrite. For more information, see Field check and overwrite modes. - Response
The updated log entry is returned.
- Examples
- Example 1: Assign a fixed value to a field.
Add a new field named city and set the value to Shanghai.
e_set("city", "Shanghai")
- Example 2: Extract the value of an existing field, and then assign the value to another
field.
Call an expression function to extract the value of an existing field named ret, and then assign the value to a new field named result.
e_set("result", v("ret"))
- Example 3: Assign a dynamic value to a field.
Call multiple expression functions in sequence to obtain the value in lowercase of the first field from specified existing fields and specify the value for the result field.
e_set("result", str_lower(v("ret", "return")))
- Example 4: Specify a value for a field multiple times.
- Specify a fixed value for the event_type field.
e_set("event_type", "login event", "event_info", "login host")
- If the value of the ret field is fail, set the event_type field to login failed event.
e_if(e_search('ret==fail'), e_set("event_type", "login failed event" ))
- Specify a fixed value for the event_type field.
- Example 1: Assign a fixed value to a field.
e_drop_fields
You can use the e_drop_fields function to delete the log fields that meet a specified condition.
- Syntax
e_drop_fields (field 1, field 2, ....,regex=False)
- Parameters
Parameter Type Required Description field String Yes The name of a log field. The value of this parameter can be a regular expression. If the field name meets the specified condition, the field is deleted. Otherwise, the field is retained. For more information about regular expressions, see Regular expressions. You must specify at least one log field.
regex Boolean No If you set this parameter to False, regular expressions are not used to match log fields. Default value: True. - Example
If the value of the content field is 123, the content and age fields are deleted.
- Raw log entry:
age: 18 content: 123 name: twiss
- Transformation rule:
e_if(e_search("content==123"), e_drop_fields("content", "age",regex=True))
- Result:
name: twiss
- Raw log entry:
e_keep_fields
- Syntax
e_keep_fields (field 1, field 2, ....,regex=False)
- Parameters
Parameter Type Required Description field String Yes The name of a log field. The value of this parameter can be a regular expression. If the field name meets the specified condition, the field is retained. Otherwise, the field is deleted. You must specify at least one log field.
regex Boolean No If you set this parameter to False, regular expressions are not used to match log fields. Default value: True. - Example: If the value of the content field is 123, the content and age fields are retained.
- Raw log entry:
age: 18 content: 123 name: twiss
- Transformation rule:
e_if(e_search("content==123"), e_keep_fields("content", "age"))
- Result:
age: 18 content: 123
- Raw log entry:
e_pack_fields
- Syntax
e_pack_fields(output_fields,include=".*",exclude=None,drop_packed=True)
- Parameters
Parameter Type Required Description output_field String Yes The name of the output field. The value of the field is log data in the JSON format. include String No The whitelist. Fields that match the specified regular expression are encapsulated. Default value: ".*". This value indicates that all fields are encapsulated. For more information, see Regular expressions. exclude String No The blacklist. Fields that match the specified regular expression are not encapsulated. Default value: None. This value indicates that all fields are encapsulated. For more information, see Regular expressions. drop_packed Boolean No Specifies whether to delete raw fields after the fields are encapsulated. Default value: True. - True: The raw fields that are encapsulated are deleted in the output. This is the default value.
- False: The raw fields that are encapsulated are not deleted in the output.
- Response
The log entry in which specified fields are encapsulated is returned.
- Examples
- Example 1: Encapsulate all log fields into a value, and then assign the value to the
test field. By default, the raw fields that are encapsulated are deleted.
- Raw log entry:
test1:123 test2:456 test3:789
- Transformation rule:
e_pack_fields("test")
- Result:
test:{"test1": "123", "test2": "456", "test3": "789"}
- Raw log entry:
- Example 2: Encapsulate all log fields into a value, and then assign the value to the
test field. The raw fields that are encapsulated are not deleted.
- Raw log entry:
test1:123 test2:456 test3:789
- Transformation rule:
e_pack_fields("test",drop_packed=False)
- Result:
test:{"test1": "123", "test2": "456", "test3": "789"} test1:123 test2:456 test3:789
- Raw log entry:
- Example 3: Encapsulate the test and abcd fields into a value, and then assign the value to the content field. The raw fields that are encapsulated are not deleted.
- Raw log entry:
abcd@#%:123 test:456 abcd:789
- Transformation rule:
e_pack_fields("content", include="\w+", drop_packed=False)
- Result:
abcd:789 abcd@#%:123 content:{"test": "456", "abcd": "789"} test:456
- Raw log entry:
- Example 4: Encapsulate log fields that exclude the test and abcd fields into a value and assign the value to the content field. The raw fields that are encapsulated are deleted.
- Raw log entry:
abcd@#%:123 test:456 abcd:789
- Transformation rule:
e_pack_fields("content", exclude="\w+", drop_packed=True)
- Result:
abcd:789 content:{"abcd@#%": "123"} test:456
- Raw log entry:
- Example 1: Encapsulate all log fields into a value, and then assign the value to the
test field. By default, the raw fields that are encapsulated are deleted.
e_rename
You can use the e_rename function to rename the log fields that meet a specified condition.
- Syntax
e_rename("field 1", "renamed field 1", "field 2", "renamed field 2", ..., regex=False)
Note The field and renamed field parameters must be specified in pairs. - Parameters
Parameter Type Required Description field String Yes The name of a log field. The value of this parameter can be a regular expression. If the field name meets the specified condition, the field is renamed. For more information about regular expressions, see Regular expressions. You must specify at least one log field.
renamed field String Yes The new name of the field. regex Boolean No If you set this parameter to False, regular expressions are not used to match log fields. Default value: True. - Response
The field with the new name is returned.
- Examples
- Example 1
- Raw log entry:
host: 1006
- Transformation rule:
e_rename("host","client_host")
- Result:
client_host: 1006
- Raw log entry:
- Example 2
- Raw log entry:
host: 1006
- Transformation rule:
e_rename("url","rename_url")
- Result:
host: 1006
- Raw log entry:
- Example 1