This topic describes how to use LogHub Log4j Appenders or Logtail to collect Log4j logs.

Background information

Log4j is an open source project of Apache. Log4j allows you to specify the level, output destination, and output format of logs. The log levels are ERROR, WARN, INFO, and DEBUG. The output destination determines whether logs are displayed in consoles or files. The output format determines the format of logs that are displayed. In this topic, the default configuration of Log4j is used. Default configuration: 
<Configuration status="WARN">
  <Appenders>
    <Console name="Console" target="SYSTEM_OUT">
      <PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss:SSS zzz} [%t] %-5level %logger{36} - %msg%n"/>
    </Console>
  </Appenders>
  <Loggers>
    <Logger name="com.foo.Bar" level="trace">
      <AppenderRef ref="Console"/>
    </Logger>
    <Root level="error">
      <AppenderRef ref="Console"/>
    </Root>
  </Loggers>
</Configuration>
The following example shows a Log4j log: 
2013-12-25 19:57:06,954 [10.10.10.10] WARN impl.PermanentTairDaoImpl - Fail to Read Permanent Tair,key:e:470217319319741_1,result:com.example.tair.Result@172e3ebc[rc=code=-1, msg=connection error or timeout,value=,flag=0]

Use LogHub Log4j Appenders to collect Log4j logs

For more information about how to use LogHub Log4j Appenders to collect Log4j logs, see Log4j Appender.

Use Logtail to collect Log4j logs

Log Service provides configuration wizards to help you create Logtail configurations. The following example describes how to collect Log4j logs in full regex mode.

  1. Log on to the Log Service console.
  2. In the Import Data section, click RegEx - Text Log.
  3. Select the project and Logstore. Then, click Next.
  4. Create a machine group.
    • If a machine group is available, click Use Existing Machine Groups.
    • If no machine groups are available, perform the following steps to create a machine group. In this example, an Elastic Compute Service (ECS) instance is used.
      1. On the ECS Instances tab, select Manually Select Instances. Then, select the ECS instance that you want to use and click Execute Now.

        For more information, see Install Logtail on ECS instances.

        Note If you want to collect logs from an ECS instance that belongs to a different Alibaba Cloud account, a server in an on-premises data center, or a server of a third-party cloud service provider, you must manually install Logtail. For more information, see Install Logtail on a Linux server or Install Logtail on a Windows server. After you manually install Logtail, you must configure a user identifier on the server. For more information, see Configure a user identifier.
      2. After Logtail is installed, click Complete Installation.
      3. In the Create Machine Group step, configure Name and click Next.

        Log Service allows you to create IP address-based machine groups and custom identifier-based machine groups. For more information, see Create an IP address-based machine group and Create a custom ID-based machine group.

  5. Select the new machine group from the Source Server Groups section and move the machine group to the Applied Server Groups section. Then, click Next.
    Notice If you apply a machine group immediately after you create the machine group, the heartbeat status of the machine group may be FAIL. This issue occurs because the machine group is not connected to Log Service. To resolve this issue, you can click Automatic Retry. If the issue persists, see What do I do if a Logtail machine group has no heartbeats?
  6. In the Logtail Config step, create a Logtail configuration.
    The following list describes the key parameters that are used to create the Logtail configuration. For more information about other parameters, see Collect logs in full regex mode.
    • Log Sample: 2021-12-25 19:57:06,954 [10.10.10.10] WARN impl.PermanentTairDaoImpl - Fail to Read Permanent Tair,key:e:470217319319741_1,result:com.example.tair.Result@172e3ebc[rc=code=-1, msg=connection error or timeout,value=,flag=0]
    • RegEx: (\d+-\d+-\d+\s\d+:\d+:\d+,\d+)\s\[([^\]]*)\]\s(\S+)\s+(\S+)\s-\s(.*)
    • Extracted Content: Log Service automatically extracts content from logs based on the sample log and regular expression that you specify. You must specify keys for the extracted content.
    Log4j logs
  7. Preview data, configure indexes, and then click Next.
    By default, full-text indexing is enabled for Log Service. You can also configure field indexes based on collected logs in manual or automatic mode. For more information, see Configure indexes.
    Note If you want to query and analyze logs, you must enable full-text indexing or field indexing. If you enable both full-text indexing and field indexing, the system uses only field indexes.