Collection methods

Log Service can collect Log4j logs by using:
  • LogHub Log4j Appender
  • Logtail

Collect Log4j logs by using LogHub Log4j Appender

For more information, see Log4j Appender.

Collect Log4j logs by using Logtail

Log4j logs are classified into Log4j 1 logs and Log4j 2 logs. This topic describes how to collect Log4j 1 logs by configuring regular expressions based on the default configuration of Log4j 1 logs. If you want to collect Log4j 2 logs, you must modify the default configuration to record complete date information.
<Configuration status="WARN">
  <Appenders>
    <Console name="Console" target="SYSTEM_OUT">
      <PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss:SSS zzz} [%t] %-5level %logger{36} - %msg%n"/>
    </Console>
  </Appenders>
  <Loggers>
    <Logger name="com.foo.Bar" level="trace">
      <AppenderRef ref="Console"/>
    </Logger>
    <Root level="error">
      <AppenderRef ref="Console"/>
    </Root>
  </Loggers>
</Configuration>

For more information about how to configure Logtail to collect Log4j logs, see Python logs. Select the corresponding configuration based on your network deployment and actual situation.

The automatically generated regular expression is only based on the sample log and may not be suitable for other logs. Therefore, you must make minor adjustments to the regular expression after it is automatically generated.

The following sample is a Log4j log that is written to a file in the default log format:
2013-12-25 19:57:06,954 [10.207.37.161] WARN impl.PermanentTairDaoImpl - Fail to Read Permanent Tair,key:e:470217319319741_1,result:com.example.tair.Result@172e3ebc[rc=code=-1, msg=connection error or timeout,value=,flag=0]
Regular expression that matches the first line of log entries (with an IP address that indicates the beginning of log entries):
\d+-\d+-\d+\s. *
Regular expression used to extract log information:
(\d+-\d+-\d+\s\d+:\d+:\d+,\d+)\s\[([^\]]*)\]\s(\S+)\s+(\S+)\s-\s(. *)
Time conversion format:
%Y-%m-%d %H:%M:%S
The following table describes the extraction results of the sample log.
Key Value
time 2013-12-25 19:57:06,954
ip 10.207.37.161
level WARN
class impl.PermanentTairDaoImpl
message Fail to Read Permanent Tair,key:e:470217319319741_1,result:com.example.tair.Result@172e3ebc[rc=code=-1, msg=connection error or timeout,value=,flag=0]