This topic describes the values of the mode parameter in different functions. The mode parameter specifies a field extraction mode.
Related functions
The following table describes the functions that use the mode parameter and the default value that is used for the mode parameter in each function.
Category | Function | Default value of mode |
---|---|---|
Value assignment functions | e_set | overwrite |
Value extraction functions | e_regex | fill-auto |
e_json | fill-auto | |
e_kv | fill-auto | |
e_csv, e_psv, and e_tsv | fill-auto | |
e_kv_delimit | fill-auto | |
e_anchor | overwrite | |
e_syslogrfc | overwrite | |
Mapping and enrichment functions | e_dict_map | fill-auto |
e_table_map | fill-auto | |
e_search_dict_map | overwrite | |
e_search_table_map | fill-auto |
Field extraction check and overwrite modes
Value | Description |
---|---|
fill | Sets a new field if the field does not exist or if the field already exists but the value of the field is an empty string. |
fill-auto | Sets a new field if the new value is not an empty string and one of the following conditions is met: The field does not exist. The field already exists but the value of the field is an empty string. |
add | Sets a new field if the field does not exist. |
add-auto | Sets a new field if the new value is not an empty string and the field does not exist. |
overwrite | Always sets a new field. |
overwrite-auto | Sets a new field if the new value is not an empty string. |
- Raw log
a: # An empty string b: 100
- Transformation examples
Mode Transformation rule Result add e_set("c", "123", mode='add')
a:# An empty string b: 100 c: 123
e_set("c", "", mode='add')
a:# An empty string b: 100 c:
e_set("a", "123", mode='add')
a:# An empty string b: 100
add-auto e_set("c", "", mode='add-auto')
The c
field is not added, and the raw log remains unchanged.fill e_set("c", "123", mode='fill')
a:# An empty string b: 100 c: 123
e_set("c", "", mode='fill')
a:# An empty string b: 100 c:
e_set("a", "123", mode='fill')
a: 123 b: 100
e_set("b", "123", mode='fill')
The b field remains b: 100
.fill-auto e_set("c", "", mode='fill-auto')
The c
field is not added, and the raw log remains unchanged.overwrite e_set("c", "123", mode='overwrite')
a:# An empty string b: 100 c: 123
e_set("c", "", mode='overwrite')
a:# An empty string b: 100 c:
e_set("b", "200", mode='overwrite')
a:# An empty string b: 200
e_set("b", "", mode='overwrite')
a:# An empty string b:
overwrite-auto e_set("b", "", mode='overwrite-auto')
The b field remains b: 100
.
Limits on field names for extraction
Functions such as e_json, e_kv, e_kv_delimit, and e_regex are supported.
Only the fields whose names abide by the limits can be extracted. The fields whose
names do not abide by the limits are discarded. The regular expression u'_*[\u4e00-\u9fa5\u0800-\u4e00a-zA-Z][\u4e00-\u9fa5\u0800-\u4e00\\w\\.\\-]*'
is not supported. For example, the fields whose names match 123=abc __1__:100 1k=200 {"123": "456"}
are discarded.
- Raw log
data: {"k1": 100, "k2": {"k3": 200, "k4": {"k5": 300} } }
- Transformation rule
e_json( "data", fmt="parent", sep="@", prefix="__", suffix="__", include_node=r"[\u4e00-\u9fa5\u0800-\u4e00a-zA-Z][\w\-\.]*", mode="fill-auto", )
- Result
data: {"k1": 100, "k2": {"k3": 200, "k4": {"k5": 300} } } data@__k1__:100 k2@__k3__:200 k4@__k5__:300