ECI收集的用户标准输出、错误日志是原生K8S落盘的日志,K8S会在每行日志前增加时间戳、来源等信息,这就破坏了用户原生的日志格式,比如用户的标准输出原生记录的是JSON格式,K8S添加前缀后JSON解析就会失败,如下:
2020-04-02T15:40:05.440500764+08:00 stdout F {"key1":"val1","key2":"val2"}
2020-04-02T15:40:07.442412564+08:00 stdout F {"key1":"val1","key2":"val2"}
2020-04-02T15:40:09.442774495+08:00 stdout F {"key1":"val1","key2":"val2"}
2020-04-02T15:40:11.443799303+08:00 stdout F {"key1":"val1","key2":"val2"}
2020-04-02T15:40:13.445099622+08:00 stdout F {"key1":"val1","key2":"val2"}
2020-04-02T15:40:15.445934358+08:00 stdout F {"key1":"val1","key2":"val2"}
2020-04-02T15:40:17.447064707+08:00 stdout F {"key1":"val1","key2":"val2"}
2020-04-02T15:40:19.448112987+08:00 stdout F {"key1":"val1","key2":"val2"}
2020-04-02T15:40:21.449393263+08:00 stdout F {"key1":"val1","key2":"val2"}

本文采用SLS Processor能力解决用户日志JSON解析的问题。

ECI的用户日志会收集到用账户下的日志仓库内,找到相应的logstore后修改配置,采用极简模式并启用插件处理能力,如下图:

插件配置内容如下: 参见 sls-json-processor
{
    "processors": [
        {
            "type": "processor_anchor",
            "detail": {
                "SourceKey": "content",
                "Anchors": [
                    {
                        "Start": "stdout F ",
                        "Stop": "",
                        "FieldName": "json_content",
                        "FieldType": "string",
                        "ExpondJson": false
                    }
                ]
            }
        },
        {
            "type": "processor_json",
            "detail": {
                "SourceKey": "json_content",
                "KeepSource": false,
                "ExpandConnector": ""
            }
        }
    ]
}

保存配置后隔几秒,就可以查看到正常解析的日志内容,如下所示:

可见,SLS已经正确解析了JSON日志。