You can use the processor_json plug-in to expand JSON fields. This topic describes the parameters of the processor_json plug-in. This topic also provides an example on how to configure the plug-in.

Important You can use form configuration to add Logtail plug-ins only if you select Kubernetes - Standard Output in the Import Data section.

Entry point

If you want to use a Logtail plug-in to process logs, you can add a Logtail plug-in configuration when you create or modify a Logtail configuration. For more information, see Entry point.

Usage notes

Important Only Logtail V0.16.28 and later support the processor_json plug-in.

Form configuration

  • Parameters
    Set Processor Type to Expand JSON Field. Then, configure other parameters based on the following table.
    ParameterDescription
    Original FieldThe name of the original field to expand.
    JSON Expansion DepthThe depth of JSON expansion. Default value: 0, which indicates that the depth of JSON expansion is unlimited. If the value is n, the depth of JSON expansion is n.
    Character to Concatenate Expanded KeysThe character that is used to connect expanded keys. The default value is an underscore (_).
    Name Prefix of Expanded KeysThe prefix that is added to the names of expanded keys.
    Retain Original FieldSpecifies whether to retain the original field in the new log that is obtained after parsing.
    Report Original Field Missing ErrorSpecifies whether to report an error if the raw log does not contain the original field.
    Use Name of Original Field as Name Prefix of Expanded KeysSpecifies whether to add the name of the original field as a prefix to all expanded keys.
    Retain Raw Logs If Parsing FailsSpecifies whether to retain the raw log if the raw log fails to be parsed.
  • Configuration example
    The following example shows how to expand the JSON field s_key and add j and the name of the original field s_key as a prefix to the expanded keys:
    • Raw log
      "s_key":"{\"k1\":{\"k2\":{\"k3\":{\"k4\":{\"k51\":\"51\",\"k52\":\"52\"},\"k41\":\"41\"}}})"
    • Logtail plug-in configuration for data processing Expand JSON fields
    • Result
      "js_key-k1-k2-k3-k4-k51":"51"
      "js_key-k1-k2-k3-k4-k52":"52"
      "js_key-k1-k2-k3-k41":"41"

Editor configuration in JSON

  • Parameters
    Set type to processor_json. Then, configure other parameters in detail based on the following table.
    ParameterTypeRequiredDescription
    SourceKeyStringYesThe name of the original field to expand.
    NoKeyErrorBooleanNoSpecifies whether to report an error if the raw log does not contain the original field. Valid values:
    • true: reports an error. This is the default value.
    • false: does not report an error.
    ExpandDepthIntNoThe depth of JSON expansion. Default value: 0, which indicates that the depth of JSON expansion is unlimited. If the value is n, the depth of JSON expansion is n.
    ExpandConnectorStringNoThe character that is used to connect expanded keys. The default value is an underscore (_).
    PrefixStringNoThe prefix that is added to the names of expanded keys.
    KeepSourceBooleanNoSpecifies whether to retain the original field in the new log that is obtained after parsing. Valid values:
    • true: retains the original field. This is the default value.
    • false: does not retain the original field.
    UseSourceKeyAsPrefixBooleanNoSpecifies whether to add the name of the original field as a prefix to all expanded keys.
    KeepSourceIfParseErrorBooleanNoSpecifies whether to retain the raw log if the raw log fails to be parsed. Valid values:
    • true: retains the raw log. This is the default value.
    • false: does not retain the raw log.
  • Configuration example
    The following example shows how to expand the JSON field s_key and add j and the name of the original field s_key as a prefix to the expanded keys:
    • Raw log
      "s_key":"{\"k1\":{\"k2\":{\"k3\":{\"k4\":{\"k51\":\"51\",\"k52\":\"52\"},\"k41\":\"41\"}}})"
    • Logtail plug-in configuration for data processing
      {
        "processors":[
          {
            "type":"processor_json",
            "detail": {
              "SourceKey": "s_key",
              "NoKeyError":true,
              "ExpandDepth":0,
              "ExpandConnector":"-",
              "Prefix":"j",
              "KeepSource": false,
              "UseSourceKeyAsPrefix": true
            }
          }
        ]
      }
    • Result
      "js_key-k1-k2-k3-k4-k51":"51"
      "js_key-k1-k2-k3-k4-k52":"52"
      "js_key-k1-k2-k3-k41":"41"