This topic describes the mode parameter that specifies the field extraction mode, including the functions that use this parameter, options of the parameter, and constraints on field name extraction.

Related functions

The following table lists the functions that use the mode parameter.

Category Function Default value of mode
Value assignment function e_set overwrite
Value extraction functions e_regex fill-auto
e_json fill-auto
e_kv fill-auto
e_kv_delimit fill-auto
e_kv_delimit fill-auto
e_syslogrfc overwrite
Mapping and enrichment functions e_dict_map fill-auto
e_table_map fill-auto
e_search_dict_map overwrite
e_search_table_map fill-auto

Field check and overwrite modes

The following table describes the options available to the mode parameter.
Value Description
fill Sets the target field if the target field does not exist or its value is an empty string.
fill-auto Sets the target field if the new value is not an empty string and the target field does not exist or its value is an empty string.
add Sets the target field if the target field does not exist.
add-auto Sets the target field if the new value is not an empty string and the target field does not exist.
overwrite Always sets the target field.
overwrite-auto Sets the target field if the new value is not an empty string.
The following example shows how these modes work.
Raw log:
  a:         # Empty string.
  b: 100
Mode Example Result
add e_set("c", "123", mode='add') The c field is added as c: 123.
e_set("c", "", mode='add') The c field is added as c:.
e_set("a", "123", mode='add') The a field is not modified and remains a:.
add-auto e_set("c", "", mode='add-auto') The c field is not added.
fill e_set("c", "123", mode='fill') The c field is added as c: 123.
e_set("c", "", mode='fill') The c field is added as c:.
e_set("a", "123", mode='fill') The a field is modified to a: 123.
e_set("b", "123", mode='fill') The b field is not modified and remains b: 100.
fill-auto e_set("c", "", mode='fill-auto') The c field is not added.
overwrite e_set("c", "123", mode='overwrite') The c field is added as c: 123.
e_set("c", "", mode='overwrite') The c field is added as c:.
e_set("b", "200", mode='overwrite') The b field is modified to b: 200.
e_set("b", "", mode='overwrite') The b field is modified to b:.
overwrite-auto e_set("b", "", mode='overwrite-auto') The b field is not modified and remains b: 100.

Constraints on field name extraction

The constraints apply to functions such as e_json, e_kv, e_kv_delimit, and e_regex.

Only the fields whose names meet the constraints can be extracted. The fields that do not meet the constraints are discarded. The regular expression u'_*[\u4e00-\u9fa5\u0800-\u4e00a-zA-Z][\u4e00-\u9fa5\u0800-\u4e00\\w\\.\\-]*' is not supported. For example, the fields 123=abc __1__:100 1k=200 {"123": "456"} will be discarded.

The default constraints are used in the following example.
  • Raw log:
    data: {"k1": 100, "k2": {"k3": 200, "k4": {"k5": 300} } }
  • Processing rule:
    e_json("data", fmt='parent', sep="@", prefix="__", suffix="__",include_node=r"[\u4e00-\u9fa5\u0800-\u4e00a-zA-Z][\w\-\.]*", mode='fill-auto' )
  • Processing result:
    data: {"k1": 100, "k2": {"k3": 200, "k4": {"k5": 300} } }
    data@__k1__: 100
    k2@__k3__: 200
    k4@__k5__: 300