Configure log analysis to associate SLS logs with trace IDs - Application Real-Time Monitoring Service

When an application throws an exception or behaves unexpectedly, logs are the fastest path to the root cause. Application Real-Time Monitoring Service (ARMS) connects application logs stored in Simple Log Service (SLS) with distributed trace IDs. After setup, jump from a trace directly to the log entries recorded during that request, or search logs by trace ID to locate the exact execution path.

Important

Application Monitoring provides a new application details page for users who have enabled the new billing mode. For more information, see Billing (new). If you have not enabled the new billing mode, click Switch to New Version on the Application List page to access the new interface.

How it works

After you bind an SLS Logstore to your ARMS application, ARMS injects trace IDs into your application logs through the logging framework. Each log entry is tagged with the trace ID of the request that produced it. This enables two workflows:

Trace to logs: Open a trace in Trace Explorer and view all log entries produced during that request.
Logs to trace: Search logs by trace ID to locate the distributed trace for a specific log entry.

Note

ARMS does not collect logs into the Logstore. Configure log collection in SLS separately. For more information, see Data collection overview.

Supported logging frameworks

Automatic trace ID injection works with the following Java logging frameworks:

log4j
log4j2
Logback

If your application uses a different logging framework, automatic injection is not supported. See Manually retrieve trace IDs for an alternative approach.

When trace IDs are available

A trace context only exists at request entry points. Logs printed outside these contexts cannot be associated with trace IDs.

Scenario	Trace ID available?	Reason
Application receives an HTTP request, runs a database query, and prints a log	Yes	HTTP entry point creates trace context
HTTP request triggers an asynchronous task via a thread pool to execute a database query and print logs (agent v4.x)	Yes	v4.x propagates trace context across asynchronous threads
HTTP request triggers an asynchronous task via a thread pool to execute a database query and print logs (agent v3.x)	No	v3.x does not propagate trace context across asynchronous threads
Application uses a JDK thread pool to run periodic database queries at startup	No	No request entry point exists

Note

Check the thread name in the log entry to determine which scenario applies. Tomcat threads start with http-nio- and Dubbo threads start with DubboServerHandler-. Logs from these threads should have trace IDs.

Prerequisites

Before you begin, make sure that you have:

An application monitored by ARMS. For more information, see Application Monitoring overview
Simple Log Service activated. Log on to the Simple Log Service console and follow the on-screen instructions
An SLS project
A Logstore
Trace data collected and imported to SLS. For more information, see Data collection overview

Step 1: Associate business logs with trace IDs

Log on to the ARMS console. In the left-side navigation pane, choose Application Monitoring > Application List.
On the Application List page, select a region in the top navigation bar and click the application name.
Note
The Language column shows which language each application uses:
- : Java
- : Go
- : Python
- Hyphen (-): Application monitored in Managed Service for OpenTelemetry
In the top navigation bar, choose Configuration > Custom Configurations.
In the Application log Association configuration section, configure the following settings:
- Set Log Source to Log service SLS.
- Turn on Automatically associate business logs with TraceId.
- Select the region, project, and Logstore where your application logs are stored.
Click Save.

Step 2: Query and analyze logs

In the top navigation bar, choose Scenario-based Analysis > Log Analysis.
Enter a query statement in the search box. A query statement consists of a search statement and an analytic statement in the format Search statement|Analytic statement. For syntax details, see Search syntax and Aggregate functions.
Set the time range. Specify a start time and end time in hours or down to the second. You can also set a relative time range without specifying the start time or end time.
Note
Query results may include logs generated up to 1 minute before or after the specified time range.
Click Search & Analyze to view the results.

Manually retrieve trace IDs

If automatic trace ID injection does not work with your logging framework, retrieve trace IDs programmatically with the ARMS SDK for Java and include them in your log messages:

Span span = Tracer.builder().getSpan();   // No new span is created.
String traceId = span.getTraceId();

logger.warn("traceId={} this is your log message", traceId);

FAQ

Why are trace IDs missing from my logs after I updated the XML configuration?

First, verify that logs are reaching your SLS Logstore and that the correct project and Logstore are selected under Application log Association configuration in the ARMS console.

If logs exist in SLS but trace IDs are missing, the root cause depends on the scope of the problem.

All logs are missing trace IDs

Check these causes in order (most common first):

The ARMS agent is not mounted. Confirm that application metrics display normally in the ARMS console.
The agent or a plug-in is disabled. Go to the application details page and check the Probe switch settings section on the Custom Configurations tab. Make sure the main switch and relevant plug-ins (for example, the Tomcat plug-in) are turned on.
Your server framework is not supported. Unsupported web containers, RPC frameworks, scheduled task frameworks, or message frameworks prevent trace ID injection. See Java components and frameworks supported by ARMS for the full compatibility list.
Your logging framework is not supported. Only log4j, log4j2, and Logback are supported. Deep modifications to these frameworks may also break injection.
The log XML configuration has errors. To isolate this:
- On agent v4.1.6 or later, enable automatic trace ID filling. If trace IDs appear, the issue is in your XML configuration.
- Use the ARMS SDK to manually retrieve trace IDs (see Manually retrieve trace IDs). If trace IDs are returned, the XML configuration is the problem.
- If neither approach works, submit a ticket.

Some logs are missing trace IDs

A trace context only exists at request entry points -- HTTP interfaces, RPC interfaces, scheduled tasks, and message consumption. Logs printed outside these contexts (for example, during general background operations) cannot be associated with trace IDs. See When trace IDs are available for a detailed breakdown.

If none of these scenarios explain the issue, submit a ticket.

Why are logs empty when I view them from Trace Explorer?

Check whether all logs for the application are missing trace IDs. If so, see the previous FAQ.
If only specific interfaces have no logs, those interfaces likely do not produce log output internally.
If neither explanation applies, submit a ticket.

Why does the Logstore have no logs after I bind it to a project?

ARMS does not collect logs into the Logstore on your behalf. Configure application log collection in SLS separately by following the SLS documentation.

Why do application logs fail to associate with trace IDs while business logs succeed?

The agent v3.x has known issues instrumenting Spring Cloud Gateway. Upgrade to agent v4.x to resolve this.

Why do logs in an asynchronous task fail to associate with trace IDs after I wrap it with TraceCallable.wrap()?

Upgrade to agent v4.x. The v4.x agent propagates trace context across asynchronous threads automatically, with no code changes required.

Why does trace ID association fail after I enable automatic trace ID filling?

All logs are affected: The configuration failed to reach the agent. Check the connectivity of the Application Configuration Management (ACM) server. For more information, see Network configurations required by the ARMS agent for Java.
Some logs are affected: See Why are trace IDs missing from my logs after I updated the XML configuration?.