Analyze logs collected by ARMS - Application Real-Time Monitoring Service

If an exception occurs in your application, you can use logs to accurately identify the exception. The log analysis feature allows you to analyze logs that are collected by Log Service or Application Real-Time Monitoring Service (ARMS). This topic describes how to enable the log analysis feature and analyze logs that are collected by ARMS.

Prerequisites

The version of the ARMS agent is V2.7.1.4 or later. Log on to the ARMS console. In the left-side navigation pane, choose Application Monitoring > Agents. Click the Java Release Notes tab to obtain the agent installation package of V2.7.1.4.

Log sources

The log analysis feature supports logs that are collected by Log Service or ARMS. This topic describes how to query and analyze logs that are collected by ARMS. For more information about how to analyze logs that are collected by Log Service, see Analyze logs collected by Log Service.

Analyze logs collected by ARMS: You can use the ARMS agent to collect the output of the log framework and push the collected logs to the log analysis center of ARMS. After you enable the log analysis feature, you can query and analyze the logs of your application in the ARMS console without the need to perform other operations.
Note If you analyze logs collected by ARMS, you must upgrade the ARMS agent to V2.7.1.4 or later.
Analyze logs collected by Log Service:
You must make sure that the logs of your application are collected to Log Service. Then, you must specify the project and Logstore that store the logs on the Application Settings page of the ARMS console. To help you analyze logs, the console pages of Log Service are embedded in ARMS.

Enable the log analysis feature

Log on to the ARMS console. In the left-side navigation pane, choose Application Monitoring > Applications.
On the Applications page, select a region in the top navigation bar and click the name of the application that you want to manage.
Note If the icon is displayed in the Language column, the application is connected to Application Monitoring. If a hyphen (-) is displayed, the application is connected to Tracing Analysis.
In the left-side navigation pane, click Application Settings. On the page that appears, click the Custom Configuration tab.
In the Business Log Association Settings section of the Custom Configuration tab, set the log source to Default Log Source, turn on Automatic Log Collection, select the severity levels of logs, set the maximum size of a single log, and set the limits on reported logs.
Note If the severity level of a log entry is lower than the log output level configured in your application, the log entry is not collected. For example, if the log output level configured in your application is Warn, logs of the Debug level is not collected even if you select Debug when you configure the severity levels of logs in the ARMS console. If you want to modify the log output level of an application without restarting the application, you can use the Arthas Shell feature of ARMS to dynamically modify the log output level of the application. For more information, see Arthas Shell.
On the Custom Configuration tab, click Save in the lower-left corner.
In the left-side navigation pane, choose Application Diagnosis > Log Analysis.

Log collection

Use the ARMS agent to intercept the log printing method of the log framework and obtain the log content.
Add some contexts (for example, traceId, spanName, and threadName) to logs.
Push logs to the ARMS log analysis center for storage.
Query logs on the Log Analysis page of the ARMS console.

Note The latency of log collection is about 10 seconds.

Extra overhead

CPU overhead: 0.01 cores
Memory overhead: less than 20 MB
Bandwidth overhead: less than or equal to the configured upper limit of log entries reported per second multiplied by the maximum number of characters in a log entry

Structure of a log entry

A log entry contains the label fields attached to the log and the fields parsed from the log.

Table 1. Label fields attached to logs
Field	Description
majorVersion	The major version number of the agent.
minorVersion	The minor version number of the agent.
serverIp	The IP address of the host where the agent resides.
userId	The ID of the user who owns the application.
appId	The ID of the application.
job	The source of the log entry. Valid value: arms-agent.
logType	The type of the log entry. Valid value: userLog.

Table 2. Fields parsed from logs
Field	Description
level	The severity level of the log entry.
log	The formatted time and content of the log entry. Use a hyphen (-) to join the time and log content.
loggerName	The name of the logger.
parentAppId	The ID of the upstream application in the trace that is associated with the log entry.
spanName	The name of the span that is associated with the log entry.
parentSpanName	The name of the parent span that is associated with the log entry.
threadName	The name of the thread.
traceId	The ID of the trace that is associated with the log entry.
ts	The time when the log entry was reported.

Common query statements

The following examples show how to query and analyze the logs of an application whose appId is a2n80plglh@89f2dd21b561bdc.

Keyword-based query

To query logs that contain the keyword error, execute the following statement:

{job="arms-agent", appId="a2n80plglh@89f2dd21b561bdc"} |= "error"

To query logs whose traceId is eaac105afb16540713955671006d0009, execute the following statement:

{job="arms-agent", appId="a2n80plglh@89f2dd21b561bdc"} |= "eaac105afb16540713955671006d0009"

Multi-condition query

To query logs that contain both error and Exception, execute the following statement:

{job="arms-agent", appId="a2n80plglh@89f2dd21b561bdc"} |= "error" |= "Exception"

Analyze logs

To analyze the change trend of the number of error logs based on spanName, execute the following statement:

sum(count_over_time({job="arms-agent", appId="a2n80plglh@89f2dd21b561bdc"} | json [1m])) by (spanName)

The following figure shows the query result. Log analysis (latest)

You can also query the change trend of log output based on loggerName, threadName, and levels.