Expand JSON fields - Simple Log Service - Alibaba Cloud Documentation Center

If you use Logtail to collect logs, you can use the processor_json plug-in to expand JSON fields. This topic describes the parameters of the processor_json plug-in and provides examples on how to configure the plug-in.

Important

Form configuration: You can use form configuration to add the plug-in when you collect text logs or container stdout and stderr.
Editor configuration in JSON: You cannot use editor configuration in JSON to add the plug-in when you collect text logs.

Entry point

If you want to use a Logtail plug-in to process logs, you can add a Logtail plug-in configuration when you create or modify a Logtail configuration. For more information, see Overview of Logtail plug-ins for data processing.

Configuration description

Important

Only Logtail V0.16.28 and later support the processor_json plug-in.

Form configuration

Parameters

Set the Processor Type parameter to Expand JSON Field. Then, configure other parameters based on the following table.

Parameter	Description
Original Field	The name of the original field to expand.
JSON Expansion Depth	The depth of JSON expansion. Default value: 0. This value indicates that the depth of JSON expansion is unlimited. If the value is n, the depth of JSON expansion is n.
Character to Concatenate Expanded Keys	The character that is used to connect expanded keys. The default value is an underscore (_).
Name Prefix of Expanded Keys	The prefix that is added to the names of expanded keys.
Expand Array	Specifies whether to expand the array. Only Logtail V1.8.0 and later support this parameter.
Retain Original Field	Specifies whether to retain the original field in the new log that is obtained after parsing.
Report Original Field Missing Error	Specifies whether to report an error if the raw log does not contain the original field.
Use Name of Original Field as Name Prefix of Expanded Keys	Specifies whether to add the name of the original field as the prefix to all expanded keys.
Retain Raw Logs If Parsing Fails	Specifies whether to retain the raw log if the raw log fails to be parsed.

Examples
The following example shows how to expand the JSON field s_key, add j as the prefix to the expanded keys, and then specify the name of the original field s_key as the names of the expanded keys:
- Raw log (path to the file from which Logtail reads logs)
```
{"s_key":"{\"k1\":{\"k2\":{\"k3\":{\"k4\":{\"k51\":\"51\",\"k52\":\"52\"},\"k41\":\"41\"}}}}"}
```
- Logtail plug-in configuration for data processing
- Processing result

Editor configuration in JSON

Parameters

Set the type parameter to processor_json. Then, configure other parameters in the detail section based on the following table.

Parameter	Type	Required	Description
SourceKey	String	Yes	The name of the original field to expand.
NoKeyError	Boolean	No	Specifies whether to report an error if the raw log does not contain the original field. Valid values: true (default) false
ExpandDepth	Int	No	The depth of JSON expansion. Default value: 0. This value indicates that the depth of JSON expansion is unlimited. If the value is n, the depth of JSON expansion is n.
ExpandConnector	String	No	The character that is used to connect expanded keys. The default value is an underscore (_).
Prefix	String	No	The prefix that is added to the names of expanded keys.
KeepSource	Boolean	No	Specifies whether to retain the original field in the new log that is obtained after parsing. Valid values: true (default) false
UseSourceKeyAsPrefix	Boolean	No	Specifies whether to add the name of the original field as a prefix to all expanded keys.
KeepSourceIfParseError	Boolean	No	Specifies whether to retain the raw log if the raw log fails to be parsed. Valid values: true (default) false
ExpandArray	Boolean	No	Specifies whether to expand the array. Only Logtail V1.8.0 and later support this parameter. false (default) true: For example, `{"k":["1","2"]}` is expanded to `{"k[0]":"1","k[1]":"2"}`.

Examples

The following example shows how to expand the JSON field s_key, add j as the prefix to the expanded keys, and then specify the name of the original field s_key as the names of the expanded keys:

Raw log (path to the file from which Logtail reads logs)

{"s_key":"{\"k1\":{\"k2\":{\"k3\":{\"k4\":{\"k51\":\"51\",\"k52\":\"52\"},\"k41\":\"41\"}}}}"}

Logtail plug-in configuration for data processing

{
  "processors":[
    {
      "type":"processor_json",
      "detail": {
        "SourceKey": "content",
        "NoKeyError":true,
        "ExpandDepth":0,
        "ExpandConnector":"-",
        "Prefix":"j",
        "KeepSource": false,
        "UseSourceKeyAsPrefix": true
      }
    }
  ]
}

Processing result