This topic describes the values of the mode parameter in different functions. The mode parameter specifies a field extraction mode.

Related functions

The following table describes the functions that use the mode parameter and the default value that is used for the mode parameter in each function.

Category Function Default value of mode
Value assignment functions e_set overwrite
Value extraction functions e_regex fill-auto
e_json fill-auto
e_kv fill-auto
e_csv, e_psv, and e_tsv fill-auto
e_kv_delimit fill-auto
e_anchor overwrite
e_syslogrfc overwrite
Mapping and enrichment functions e_dict_map fill-auto
e_table_map fill-auto
e_search_dict_map overwrite
e_search_table_map fill-auto

Field extraction check and overwrite modes

The following table describes the values of the mode parameter.
Value Description
fill Sets a new field if the field does not exist or if the field already exists but the value of the field is an empty string.
fill-auto Sets a new field if the new value is not an empty string and one of the following conditions is met: The field does not exist. The field already exists but the value of the field is an empty string.
add Sets a new field if the field does not exist.
add-auto Sets a new field if the new value is not an empty string and the field does not exist.
overwrite Always sets a new field.
overwrite-auto Sets a new field if the new value is not an empty string.
The following table provides examples on how the functions work in different modes.
  • Raw log
    a:         # An empty string
    b: 100
  • Transformation examples
    Mode Transformation rule Result
    add e_set("c", "123", mode='add')
    a:# An empty string
    b: 100
    c: 123
    e_set("c", "", mode='add')
    a:# An empty string
    b: 100
    c:
    e_set("a", "123", mode='add')
    a:# An empty string
    b: 100
    add-auto e_set("c", "", mode='add-auto') The c field is not added, and the raw log remains unchanged.
    fill e_set("c", "123", mode='fill')
    a:# An empty string
    b: 100
    c: 123
    e_set("c", "", mode='fill')
    a:# An empty string
    b: 100
    c:
    e_set("a", "123", mode='fill')
    a: 123
    b: 100
    e_set("b", "123", mode='fill') The b field remains b: 100.
    fill-auto e_set("c", "", mode='fill-auto') The c field is not added, and the raw log remains unchanged.
    overwrite e_set("c", "123", mode='overwrite')
    a:# An empty string
    b: 100
    c: 123
    e_set("c", "", mode='overwrite')
    a:# An empty string
    b: 100
    c: 
    e_set("b", "200", mode='overwrite')
    a:# An empty string
    b: 200
    e_set("b", "", mode='overwrite')
    a:# An empty string
    b: 
    overwrite-auto e_set("b", "", mode='overwrite-auto') The b field remains b: 100.

Limits on field names for extraction

Functions such as e_json, e_kv, e_kv_delimit, and e_regex are supported.

Only the fields whose names abide by the limits can be extracted. The fields whose names do not abide by the limits are discarded. The regular expression u'_*[\u4e00-\u9fa5\u0800-\u4e00a-zA-Z][\u4e00-\u9fa5\u0800-\u4e00\\w\\.\\-]*' is not supported. For example, the fields whose names match 123=abc __1__:100 1k=200 {"123": "456"} are discarded.

The following example shows how to use the default limits of a function on field names:
  • Raw log
    data: {"k1": 100, "k2": {"k3": 200, "k4": {"k5": 300} } }
  • Transformation rule
    e_json(
        "data",
        fmt="parent",
        sep="@",
        prefix="__",
        suffix="__",
        include_node=r"[\u4e00-\u9fa5\u0800-\u4e00a-zA-Z][\w\-\.]*",
        mode="fill-auto",
    )
  • Result
    data: {"k1": 100, "k2": {"k3": 200, "k4": {"k5": 300} } }
    data@__k1__:100
    k2@__k3__:200
    k4@__k5__:300