Analyze Log4j logs - Simple Log Service - Alibaba Cloud Documentation Center

This topic describes how to analyze Log4j logs. In this example, the logs of an e-commerce company are analyzed.

Prerequisites

Log4j logs are collected. For more information, see Collect Log4j logs.
Indexes are created. For more information, see Create indexes.
The following figure shows the indexes that are used in this example.

Background information

Log4j is an open source project of Apache. Log4j allows you to specify the level, output destination, and output format of logs. The log levels are ERROR, WARN, INFO, and DEBUG in descending order. The output destination specifies whether logs are displayed in consoles or files. The output format determines the format of logs that are displayed.

In this example, the company wants to obtain the best solution for its e-commerce platform. The company needs to analyze information such as platform stability, system errors, data security, and behavioral data. The behavioral data includes logon methods, logon time, logon duration, browsed pages, dwell time for pages, average order time, and consumption levels. Simple Log Service provides the log collection and log analysis features to help store and analyze logs. The following examples show sample logs that are collected by Simple Log Service.

The following log records logon information:

level:  INFO  
location:  com.aliyun.log4jappendertest.Log4jAppenderBizDemo.login(Log4jAppenderBizDemo.java:38)
message:  User login successfully. requestID=id4 userID=user8  
thread:  main  
time:  2022-01-26T15:31+0000

The following log records purchase information:

level:  INFO  
location:  com.aliyun.log4jappendertest.Log4jAppenderBizDemo.order(Log4jAppenderBizDemo.java:46)
message:  Place an order successfully. requestID=id44 userID=user8 itemID=item3 amount=9  
thread:  main  
time:  2022-01-26T15:31+0000

Procedure

Log on to the Simple Log Service console.
In the Projects section, click the project that you want to manage.
In the left-side navigation pane, click Log Storage. In the Logstores list, click the Logstore that you want to manage.

Enter a query statement in the search box, and click Last 15 Minutes to specify a query time range.

For more information, see Step 1: Enter a query statement.

Query the three positions at which the most errors occur within the previous hour.

level: ERROR | select location ,count(*) as count GROUP BY  location  ORDER BY count DESC LIMIT 3

Query the numbers of logs at each log level that are generated within the previous 15 minutes.
```
| select level ,count(*) as count GROUP BY level ORDER BY count DESC
```

Query the three users that log on to the platform most frequently within the previous hour.

login | SELECT regexp_extract(message, 'userID=(?<userID>[a-zA-Z\d]+)', 1) AS userID, count(*) as count GROUP BY userID ORDER BY count DESC LIMIT 3

Query the total payment of each user within the previous 15 minutes.

order | SELECT regexp_extract(message, 'userID=(?<userID>[a-zA-Z\d]+)', 1) AS userID, sum(cast(regexp_extract(message, 'amount=(?<amount>[a-zA-Z\d]+)', 1) AS double)) AS amount GROUP BY userID