log4j logs

Last Updated: Sep 30, 2017

Log Service supports collection of log4j logs in two ways:

  • By using LogHub log4j Appender
  • By using Logtail to collect log4j log files

Collect Log4j logs using Loghub Log4j Appender

For detailed introductions and operations, refer to log4j appender.

Collect Log4j logs using Logtail

Log4j log fall into two types, the first generation and the second generation. This document takes default configuration of the first generation for example, and shows how to configure the regular expression. If you use Log4j 2, you need to modify the default configuration, and print the date completely.

  1. <Configuration status="WARN">
  2. <Appenders>
  3. <Console name="Console" target="SYSTEM_OUT">
  4. <PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss:SSS zzz} [%t] %-5level %logger{36} - %msg%n"/>
  5. </Console>
  6. </Appenders>
  7. <Loggers>
  8. <Logger name="com.foo.Bar" level="trace">
  9. <AppenderRef ref="Console"/>
  10. </Logger>
  11. <Root level="error">
  12. <AppenderRef ref="Console"/>
  13. </Root>
  14. </Loggers>
  15. </Configuration>

For details about the standard process, refer to Quick start and Apache. Choose the corresponding configuration based on your network deployment and actual situation.

The automatically generated results are for reference only. You need to manually edit the generated regular expression to fit all the logs that might be selected by Logtail.

The following shows the sample log of log4j default format printed to a file.

  1. 2013-12-25 19:57:06,954 [10.207.37.161] WARN impl.PermanentTairDaoImpl - Fail to Read Permanent Tair,key:e:470217319319741_1,result:com.example.tair.Result@172e3ebc[rc=code=-1, msg=connection error or timeout,value=,flag=0]

Matching of the beginning of a line in multiline logs (the beginning of a line is expressed by IP information):

  1. \d+-\d+-\d+\s.*

Regular expression used to extract log information:

  1. (\d+-\d+-\d+\s\d+:\d+:\d+,\d+)\s\[([^\]]*)\]\s(\S+)\s+(\S+)\s-\s(.*)

Time conversion format:

  1. %Y-%m-%d %H:%M:%S

Sample log extraction result:

Key Value
time 2013-12-25 19:57:06,954
ip 10.207.37.161
level WARN
class impl.PermanentTairDaoImpl
message Fail to Read Permanent Tair,key:e:470217319319741_1,result:com.example.tair.Result@172e3ebc[rc=code=-1, msg=connection error or timeout,value=,flag=0]
Thank you! We've received your feedback.