Application Real-Time Monitoring Service:Write trace IDs and span IDs to logs

Supported logging frameworks

For details on how MDC auto-instrumentation works, see Logger MDC auto-instrumentation.

Configure Log4j 2

1. Add the OpenTelemetry Log4j 2 dependency to your pom.xml. Replace OPENTELEMETRY_VERSION with the latest version.

      <dependencies>
        <dependency>
          <groupId>io.opentelemetry.instrumentation</groupId>
          <artifactId>opentelemetry-log4j-context-data-2.17-autoconfigure</artifactId>
          <version>OPENTELEMETRY_VERSION</version>
          <scope>runtime</scope>
        </dependency>
      </dependencies>

2. Add %X{trace_id} and %X{span_id} to the pattern in log4j2.xml.

      <?xml version="1.0" encoding="UTF-8"?>
      <Configuration>
        <Appenders>
          <Console name="Console" target="SYSTEM_OUT">
            <PatternLayout
                pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} trace_id=%X{trace_id} span_id=%X{span_id} trace_flags=%X{trace_flags} - %msg%n"/>
          </Console>
        </Appenders>
        <Loggers>
          <Root>
            <AppenderRef ref="Console" level="All"/>
          </Root>
        </Loggers>
      </Configuration>

Configure Logback

1. Add the OpenTelemetry Logback dependency to your pom.xml. Replace OPENTELEMETRY_VERSION with the latest version.

      <dependencies>
        <dependency>
          <groupId>io.opentelemetry.instrumentation</groupId>
          <artifactId>opentelemetry-logback-mdc-1.0</artifactId>
          <version>OPENTELEMETRY_VERSION</version>
        </dependency>
      </dependencies>

2. Update logback.xml to include %X{trace_id} and %X{span_id} in the pattern, and wrap your appender with OpenTelemetryAppender.

      <?xml version="1.0" encoding="UTF-8"?>
      <configuration>
        <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
          <encoder>
            <pattern>%d{HH:mm:ss.SSS} trace_id=%X{trace_id} span_id=%X{span_id} trace_flags=%X{trace_flags} %msg%n</pattern>
          </encoder>
        </appender>
   
        <!-- Wrap your logging appender with OpenTelemetryAppender -->
        <appender name="OTEL" class="io.opentelemetry.instrumentation.logback.mdc.v1_0.OpenTelemetryAppender">
          <appender-ref ref="CONSOLE"/>
        </appender>
   
        <!-- Use the wrapped "OTEL" appender instead of the original "CONSOLE" one -->
        <root level="INFO">
          <appender-ref ref="OTEL"/>
        </root>
      </configuration>

Configure Logback in Spring Boot

Spring Boot uses Logback by default. Instead of editing XML, add a single line to application.properties:

logging.pattern.level=trace_id=%mdc{trace_id} span_id=%mdc{span_id} %5p

Example output:

2024-06-26 10:56:31.200 trace_id=8f7ebd8a73f9a8f50e6a00a87a20952a span_id=1b08f18b8858bb9a  INFO 53724 --- [nio-8081-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring DispatcherServlet 'dispatcherServlet'
2024-06-26 10:56:31.201 trace_id=8f7ebd8a73f9a8f50e6a00a87a20952a span_id=1b08f18b8858bb9a  INFO 53724 --- [nio-8081-exec-1] o.s.web.servlet.DispatcherServlet        : Initializing Servlet 'dispatcherServlet'
2024-06-26 10:56:31.209 trace_id=8f7ebd8a73f9a8f50e6a00a87a20952a span_id=1b08f18b8858bb9a  INFO 53724 --- [nio-8081-exec-1] o.s.web.servlet.DispatcherServlet        : Completed initialization in 8 ms
2024-06-26 10:56:31.296 trace_id=8f7ebd8a73f9a8f50e6a00a87a20952a span_id=5743699405074f4e  INFO 53724 --- [nio-8081-exec-1] com.example.httpserver.ot.OTServer       : hello world

Configure logrus

1. Import the required packages.

      import (
          oteltrace "go.opentelemetry.io/otel/trace"
          "github.com/sirupsen/logrus"
      )

2. Create a helper function that extracts trace IDs and span IDs from a span and returns them as logrus fields.

      func LogrusFields(span oteltrace.Span) logrus.Fields {
          return logrus.Fields{
              "trace_id": span.SpanContext().TraceID().String(),
              "span_id":  span.SpanContext().SpanID().String(),
          }
      }

3. Use the helper function after creating a span to log with trace context.

      _, span := tracer.Start(ctx, "spanName")
      defer span.End()
   
      logEntry := logrus.WithFields(LogrusFields(span))
      logEntry.Info("This is an info message with trace and span ID")

Configure the logging module

1. Enable automatic injection by setting the following environment variable:

      export OTEL_PYTHON_LOG_CORRELATION=true

2. Add %(otelTraceID)s and %(otelSpanID)s to your log format. OpenTelemetry replaces these placeholders with the actual trace ID and span ID at runtime.

   Log format example:

      formatter = logging.Formatter(
          '%(asctime)s %(levelname)s [%(name)s] [%(filename)s:%(lineno)d] [trace_id=%(otelTraceID)s span_id=%(otelSpanID)s resource.service.name=%(otelServiceName)s] - %(message)s'
      )

   Example output:

      2024-06-25 10:00:55,494 INFO [app] [test.py:80] [trace_id=70561de6a164ac991bfff2281b7f**** span_id=4b7f3c798460**** resource.service.name=ot-python-demo] - hello world

End-to-end example with Flask

This example walks through a complete setup: create a Flask app, instrument it with OpenTelemetry, and verify that trace IDs appear in the logs.

Prerequisites:

• Python and pip installed (this example uses Python 3.12.4 and pip 24.1.1)

pip3 install flask

pip3 install opentelemetry-distro \
        opentelemetry-exporter-otlp
opentelemetry-bootstrap -a install

import logging
from logging.handlers import RotatingFileHandler
from flask import Flask

app = Flask(__name__)

# Create a formatter that includes OpenTelemetry trace context
formatter = logging.Formatter(
    '%(asctime)s %(levelname)s [%(name)s] [%(filename)s:%(lineno)d] [trace_id=%(otelTraceID)s span_id=%(otelSpanID)s resource.service.name=%(otelServiceName)s] - %(message)s'
)

# Write logs to a file with rotation
# Replace <log-file-path> with the actual path, for example: /var/log/app.log
file_handler = RotatingFileHandler('<log-file-path>', maxBytes=10000, backupCount=1)
file_handler.setLevel(logging.INFO)
file_handler.setFormatter(formatter)
app.logger.addHandler(file_handler)

@app.route('/')
def hello_world():
    app.logger.info('Hello World!!!')
    return 'Hello, World!'

if __name__ == '__main__':
    app.run()

export OTEL_PYTHON_LOG_CORRELATION=true
export OTEL_SERVICE_NAME=ot-python-demo
export OTEL_TRACES_EXPORTER=otlp
export OTEL_METRICS_EXPORTER=none

# Replace test.py with your actual filename if different
opentelemetry-instrument python3 test.py

* Running on http://127.0.**.**:5000

2024-06-25 10:00:55,494 INFO [app] [test.py:80] [trace_id=70561de6a164ac991bfff2281b7f**** span_id=4b7f3c798460**** resource.service.name=ot-python-demo] - hello world