This topic describes the syntax of the functions that enrich data through mapping, and provides parameter descriptions and function examples.

Functions

Type Function Description
Field-based mapping e_dict_map Maps a specified field value to a field in the dictionary and returns the value of the matched field.
e_table_map Maps a specified field value to a row in the table and returns the value of the field in this row.
Search-based mapping e_search_dict_map Maps a search string to a key in a dictionary and returns the value of the matched key.
e_search_table_map Maps a search string to a column in a table and returns the value of the field in this column.

e_dict_map

  • Syntax 
    e_dict_map(data, field, output_field, case_insensitive=True, missing=None, mode="overwrite")
  • Parameters
    Parameter Type Required Description
    data Dictionary Yes The dictionary used for mapping. A dictionary consists of a collection of key-value pairs. The key must be a string.
    field String or string list Yes One or more field names. When the value of this parameter contains more than one field name:
    • The system maps the field names in sequence.
    • If more than one log entry matches your search condition and the value of the mode parameter is overwrite, the last matched log entry overwrites the previous ones.
    • If no log entries match your search condition, the system returns the value of the missing parameter.
    output_field String Yes The field name to return.
    case_insensitive Boolean No Specifies whether data is case-sensitive during mapping. The default value is True, which indicates that data is not case-sensitive.
    Note If more than one field in the dictionary matches the search condition and the value of the case_insensitive parameter is True, the system selects the field that uses the same case as the key. If no such key is found, the system randomly selects a field.
    missing String No The value returned when no matched fields are found. The default value is None, which indicates that no actions are performed.
    Note If the specified dictionary contains a mapping rule that returns the * wildcard when no matched fields are found, the missing parameter becomes invalid because the * wildcard has a higher mapping priority than the missing parameter.
    mode String No The overwrite mode for a field. Default value: overwrite. For more information, see Field check and overwrite modes.
  • Response

    An event that carries new field values is returned.

  • Examples
    • Example 1
      Raw log: 
      data: 123
pro: 1
      Processing rule: 
      e_dict_map({"1": "TCP", "2": "UDP", "3": "HTTP", "*": "Unknown"}, "pro", "protocol")
      Processing result: 
      data: 123
pro: 1
protocol: TCP
    • Example 2
      Raw log (three log entries): 
      status: 500
      status: 400
      status: 200
      Processing rule: 
      e_dict_map({"400": "Error", "200": "Success", "*": "Other"}, "status", "message")
      Processing result: 
      status: 500
message: Other
      status: 400
message: Error
      status: 200
message: Success

e_table_map

  • Syntax 
    e_table_map(data, field, output_fields, missing=None, mode="fill-auto")
  • Parameters
    Parameter Type Required Description
    data Table Yes The table to map values in multiple columns.
    field String, string list, or tuple list Yes The source fields that are mapped to the specified table in an event. If no such field is found in the event, no actions are performed on the source field. For more information about how to set special field names, see Event structure and fields.
    output_fields String, string list, or tuple list Yes The matched fields that are found in the specified table.
    missing String No The value returned when no matched fields are found. The default value is None, which indicates that no actions are performed. If the source field is mapped to more than one column, the value of the missing parameter can be a list of default values with a list length identical to that of the source field.
    Note If the specified dictionary contains a mapping rule that returns the * wildcard when no matched fields are found, the missing parameter becomes invalid because the * wildcard has a higher mapping priority than the missing parameter.
    mode String No The overwrite mode for a field. Default value: fill-auto. For more information, see Field check and overwrite modes.
  • Response

    An event that carries new field values is returned.

  • Default mapping and the mapping based on the missing parameter

    In a dictionary, the * wildcard has a higher mapping priority than the missing parameter. The missing parameter does not take effect if the * wildcard exists.

    If you set a value in a column to the * wildcard, the value can always match the search condition. Example: 
    c1,c2,d1,d2
c1,*,1,1
c2,*,2,2,
*,*,0,0
  • Examples
    • Example 1: One field is returned.
      Raw log: 
      data: 123
city: nj
      Processing rule: 
      e_table_map(tab_parse_csv("city,pop,province\nnj,800,js\nsh,2000,sh"), "city", "province")
      Processing result: 
      data: 123
city: nj
province: js
    • Example 2: Two fields are returned.
      Raw log: 
      data: 123
city: nj
      Processing rule: 
      e_table_map(tab_parse_csv("city,pop,province\nnj,800,js\nsh,2000,sh"), "city", ["province", "pop"])
      Processing result: 
      data: 123
city: nj
province: js
pop: 800
    • Example 3: The sep parameter is used in the tab_parse_csv function.
      Raw log: 
      data: 123
city: nj
      Processing rule: 
      e_table_map(tab_parse_csv("city#pop#province\nnj#800#js\nsh#2000#sh", sep='#'), "city", ["province", "pop"])
      Processing result: 
      data: 123
city: nj
province: js
pop: 800
    • Example 4: The quote parameter is used in the tab_parse_csv function.
      Raw log: 
      data: 123
city: nj
      Processing rule: 
      e_table_map(tab_parse_csv('city,pop,province\n|nj|,|800|,|js|\n|shang hai|,2000,|SHANG,HAI|', quote='|'), "city", ["province", "pop"])
      Processing result: 
      data: 123
city: nj
province: js
pop: 800
    • Example 5: The source fields differ from the fields in the specified table.
      Raw log: 
      data: 123
cty: nj
      Processing rule: 
      e_table_map(tab_parse_csv("city,pop,province\nnj,800,js\nsh,2000,sh"), [("cty","city")], "province")
      Processing result: 
      data: 123
cty: nj
province: js
    • Example 6: The source fields differ from the fields in the specified table, and the output fields are renamed.
      Raw log: 
      data: 123
cty: nj
      Processing rule: 
      e_table_map(tab_parse_csv("city,pop,province\nnj,800,js\nsh,2000,sh"), [("cty","city")], [("province","pro")])
      Processing result: 
      data: 123
cty: nj
pro: js

e_search_dict_map

  • Description

    This function maps a search string to a key in a dictionary and returns the value of the matched key.

  • Syntax 
    e_search_dict_map(data, output_field, multi_match=False, multi_join=" ", missing=None, mode="overwrite")
  • Parameters
    Parameter Type Required Description
    data Dictionary Yes The dictionary used for mapping. A dictionary consists of a collection of key-value pairs. The key must be a string. For more information, see dct_get.
    output_field String Yes The field name to return.
    multi_match Boolean No Specifies whether to allow more than one matched field. The default value is False, which indicates that the system returns only the value of the first matched field. If the value of this parameter is True, the system can combine the values of multiple matched fields by using the character specified by the multi_join parameter.
    multi_join String No The character to combine the values of multiple matched fields. The default value is a space. This parameter is valid when the value of the multi_match parameter is True.
    missing String No The value returned when no matched fields are found. The default value is None, which indicates that no actions are performed.
    Note If the specified dictionary contains a mapping rule that returns the * wildcard when no matched fields are found, the missing parameter becomes invalid because the * wildcard has a higher mapping priority than the missing parameter.
    mode String No The overwrite mode for a field. Default value: overwrite. For more information, see Field check and overwrite modes.
  • Response

    The matched results are returned.

  • Examples
    • Example 1: Data mapping in matching mode
      Raw log: 
      data: 123
pro: 1
      Processing rule: 
      e_search_dict_map ({"pro==1": "TCP", "pro==2": "UDP", "pro==3": "HTTP"}, "protocol")
      Processing result: 
      data: 123
pro: 1
protocol: TCP
    • Example 2: Data mapping based on the first character of each field
      Raw log: 
      status: 200,300
      Processing rule: 
      e_search_dict_map ({"status:2??": "ok", "status:3??": "redirect", "status:4??": "auth", "status:5??": "server_error"}, "status_desc", multi_match=True, multi_join="test")
      Processing result: 
      status: 200,300
status_desc: ok test redirect

e_search_table_map

  • Description

    This function maps search strings to the fields in a column of a table and returns the values housed in the other columns for the fields.

  • Syntax 
    e_search_table_map(data, inpt, output_fields, multi_match=False, multi_join=" ", missing=None, mode="fill-auto")
  • Parameters
    Parameter Type Required Description
    data Table Yes The table from which to obtain data. The name of one of the table columns must match the search string.
    inpt String Yes The field names that match the search string in the specified table.
    output_fields String, string list, or tuple list Yes The matched fields that were found in the specified table. The value of this parameter is a string, string list, or tuple list.
    multi_match Boolean No Specifies whether to allow more than one matched field. The default value is False, which indicates that the system returns only the value of the first matched field. If the value of this parameter is True, the system can combine the values of multiple matched fields by using the character specified by the multi_join parameter.
    multi_join String No The character to combine the values of multiple matched fields. The default value is a space. This parameter is valid when the value of the multi_match parameter is True.
    missing String No The value returned when no matched fields are found. The default value is None, which indicates that no actions are performed.
    Note If the specified dictionary contains a mapping rule that returns the * wildcard when no matched fields are found, the missing parameter becomes invalid because the * wildcard has a higher priority than the missing parameter.
    mode String No The overwrite mode for a field. Default value: fill-auto. For more information, see Field check and overwrite modes.
  • Table to search
    Name of column to search Type 1 Type 2
    content: guide and title:~"\w guide"' guide user
    content: city and title:~"\w shanghai" food home
  • Response

    The matched results are returned.

  • Examples
    • Example 1: Data mapping in simple mode
      Raw log: 
        data: 123
  city: sh
      Processing rule: 
      e_search_table_map(tab_parse_csv("search,pop,province\ncity==nj,800,js\ncity==sh,2000,sh"), "search", ["pop", "province"])
      Processing result: 
      data: 123
city: sh
province: sh
pop: 2000
    • Example 2: Data mapping in overwrite mode
      Raw log: 
      data: 123
city: nj
province:
      Processing rule: 
      e_search_table_map(tab_parse_csv("search,pop,province\ncity==nj,800,js\ncity==sh,2000,sh"), "search", "province",mode="overwrite")
      Processing result: 
      data: 123
city: nj
province: js
    • Example 3: Data mapping in missing mode where the table does not contain the province field
      Raw log: 
      data: 123
city: wh
province:
      Processing rule: 
      e_search_table_map(tab_parse_csv("search,pop,province\ncity==nj,800,\ncity==sh,2000,sh"), "search", "province",missing="Unknown")
      Processing result: 
      data: 123
city: wh
province: Unknown
    • Example 4: Data mapping in multi_match mode
      Raw log: 
      data: 123
city: nj,sh
province:
      Processing rule: 
      e_search_table_map(tab_parse_csv("search,pop,province\ncity:nj,800,js\ncity:sh,2000,sh"), "search", "province",multi_match=True, multi_join=",")
      Processing result: 
      data: 123
city: nj,sh
province: js,sh