Application Real-Time Monitoring Service:Control conversation history collection for LLM applications

Collection behavior and data format

This is the default behavior. The probe records input messages, output messages, system instructions, and tool definitions in JSON format within the Span Attributes.

Configuration

• You can configure this behavior using the following environment variables:

  Environment variable	Value
  OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT	True
  OTEL_INSTRUMENTATION_GENAI_MESSAGE_CONTENT_CAPTURE_STRATEGY	"span-attributes"

• For Java applications, you can also add System Properties to the startup command. For example:

  -Dotel.instrumentation.genai.capture-message-content=true \
  -Dotel.instrumentation.genai.message-content.capture-strategy=span-attributes

Example

[
  {
    "role": "user",
    "parts": [
      {
        "type": "text",
        "content": "Weather in Paris?"
      }
    ]
  }
]

[
  {
    "role": "assistant",
    "parts": [
      {
        "type": "tool_call",
        "id": "call_VSPygqKTWdrhaFErNvMV18Yl",
        "name": "get_weather",
        "arguments": {
          "location": "Paris"
        }
      }
    ],
    "finish_reason": "tool_call"
  }
]

[
  {
    "type": "function",
    "name": "get_weather",
    "description": "Get the current temperature for a specific location."
  }
]

[
  {
    "role": "user",
    "parts": [
      {
        "type": "text",
        "content": "Weather in Paris?"
      }
    ]
  },
  {
    "role": "assistant",
    "parts": [
      {
        "type": "tool_call",
        "id": "call_VSPygqKTWdrhaFErNvMV18Yl",
        "name": "get_weather",
        "arguments": {
          "location": "Paris"
        }
      }
    ]
  },
  {
    "role": "tool",
    "parts": [
      {
        "type": "tool_call_response",
        "id": " call_VSPygqKTWdrhaFErNvMV18Yl",
        "response": "rainy, 57°F"
      }
    ]
  }
]

[
  {
    "role": "assistant",
    "parts": [
      {
        "type": "text",
        "content": "The weather in Paris is currently rainy with a temperature of 57°F."
      }
    ],
    "finish_reason": "stop"
  }
]

Collection behavior and data format

In this mode, the probe does not record details such as input messages, output messages, or system instructions. For tool definitions, only basic information is recorded in JSON format.

Configuration

• You can configure this behavior using the following environment variable:

  Environment variable	Value
  OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT	False

• For Java applications, you can also add a System Property to the startup command. For example:

  -Dotel.instrumentation.genai.capture-message-content=false

Example

[
  {
    "type": "function",
    "name": "get_weather"
  }
]

Collection behavior and data format

In this mode, only basic information is kept in the Span Attributes. The probe records input messages, output messages, system instructions, and tool definitions as a single-line log in JSON format. The log is saved to a local log file.

By default, when the probe starts, it searches for the first available folder in the following order of priority to use as the log path.

1. If a folder is specified using the APSARA_APM_AGENT_WORKSPACE_DIR environment variable, logs are recorded in the .apsara-apm/{language}/logs subfolder of the specified folder.

2. Probe log folder: /home/admin/.opt/.apsara-apm/{language}/logs

3. Home folder: ~/.apsara-apm/{language}/{agent_version}_{agent_commit_id}/logs

When the application starts, the probe prints a log message to stdout that indicates the log storage folder. To simplify folder management, you can specify a folder using the APSARA_APM_AGENT_WORKSPACE_DIR environment variable.

The conversation history log file is named in the genai_messages_{ip}_{pid}.log format. The maximum log file size is 256 MB. When this size is exceeded, the file is rotated. Only the two most recent log files are kept in the file system. Older files are deleted.

Configuration

• You can configure this behavior using the following environment variables:

  Environment variable	Value
  OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT	True
  OTEL_INSTRUMENTATION_GENAI_MESSAGE_CONTENT_CAPTURE_STRATEGY	"event"

• For Java applications, you can also add System Properties to the startup command. For example:

  -Dotel.instrumentation.genai.capture-message-content=true \
  -Dotel.instrumentation.genai.message-content.capture-strategy=event

Example

{
  "scope": {
    "name": "aliyun.instrumentation.openai",
    "version": "1.0.1"
  },
  "timeUnixNano": 1760080084146812928,
  "severity": "UNSPECIFIED",
  "attributes": {
    "event.name": "gen_ai.client.inference.operation.details",
    "gen_ai.provider.name": "openai",
    "gen_ai.operation.name": "chat",
    "gen_ai.request.model": "gpt-4",
    "gen_ai.request.max_tokens": 200,
    "gen_ai.request.top_p": 1.0,
    "gen_ai.response.id": "chatcmpl-9J3uIL87gldCFtiIbyaOvTeYBRA3l",
    "gen_ai.response.model": "gpt-4-0613",
    "gen_ai.usage.output_tokens": 17,
    "gen_ai.usage.input_tokens": 47,
    "gen_ai.response.finish_reasons": ["tool_calls"],
    "gen_ai.input.messages": "[{\"role\":\"user\",\"parts\":[{\"type\":\"text\",\"content\":\"Weather in Paris?\"}]}]",
    "gen_ai.output.messages": "[{\"role\":\"assistant\",\"parts\":[{\"type\":\"tool_call\",\"id\":\"call_VSPygqKTWdrhaFErNvMV18Yl\",\"name\":\"get_weather\",\"arguments\":{\"location\":\"Paris\"}}],\"finish_reason\":\"tool_call\"}]",
    "gen_ai.tool.definitions": "[{\"type\":\"function\",\"name\":\"get_weather\",\"description\":\"Get the current temperature for a specific location.\"}]"
  },
  "traceId": "0b46a347592ac487ed092ebe802c6818",
  "spanId": "b3c40af8cd1a522c"
}

{
  "scope": {
    "name": "aliyun.instrumentation.openai",
    "version": "1.0.1"
  },
  "timeUnixNano": 1760080084176812928,
  "severity": "UNSPECIFIED",
  "attributes": {
    "event.name": "gen_ai.client.inference.operation.details",
    "gen_ai.provider.name": "openai",
    "gen_ai.operation.name": "chat",
    "gen_ai.request.model": "gpt-4",
    "gen_ai.request.max_tokens": 200,
    "gen_ai.request.top_p": 1.0,
    "gen_ai.response.id": "chatcmpl-VSPygqKTWdrhaFErNvMV18Yl",
    "gen_ai.response.model": "gpt-4-0613",
    "gen_ai.usage.output_tokens": 52,
    "gen_ai.usage.input_tokens": 97,
    "gen_ai.response.finish_reasons": ["stop"],
    "gen_ai.input.messages": "[{\"role\":\"user\",\"parts\":[{\"type\":\"text\",\"content\":\"Weather in Paris?\"}]},{\"role\":\"assistant\",\"parts\":[{\"type\":\"tool_call\",\"id\":\"call_VSPygqKTWdrhaFErNvMV18Yl\",\"name\":\"get_weather\",\"arguments\":{\"location\": \"Paris\"}}]},{\"role\":\"tool\",\"parts\":[{\"type\":\"tool_call_response\",\"id\":\"call_VSPygqKTWdrhaFErNvMV18Yl\",\"response\":\"rainy, 57°F\"}]}]",
    "gen_ai.output.messages": "[{\"role\":\"assistant\",\"parts\":[{\"type\":\"text\",\"content\":\"The weather in Paris is currently rainy with a temperature of 57°F.\"}],\"finish_reason\":\"sto\"}]"
  },
  "traceId": "0b46a347592ac487ed092ebe802c6818",
  "spanId": "0a706a178bd746c5"
}

Step 1: Install LoongCollector

If LoongCollector is already installed in your environment, skip this step.

Step 2: Create a collection configuration

1. Log on to the Simple Log Service console, click the project, and expand the desired Logstore. Then, click the icon next to Data Collection and click Integrate Now in the JSON - Text Log area.

2. Select an existing machine group or create a machine group for the host where the logs are located.

   

3. Create a collection configuration. Under Input Configurations, set File Path to the log folder path. Check the standard output from the application startup to confirm the folder path. For Processing Configurations, select JSON parsing.

   

   

4. Add at least the following indexes to the logs to facilitate data retrieval and analysis.
   

Step 3: View collected logs in SLS

After you complete the initial configuration, logs are collected in SLS within a few minutes:

Configuration

• You can configure this behavior using the following environment variable:

  Environment variable	Value
  OTEL_INSTRUMENTATION_GENAI_MESSAGE_CONTENT_MAX_LENGTH	8192

• For Java applications, you can also add a System Property to the startup command. For example:

  -Dotel.instrumentation.genai.message-content.max-length=8192

Message bodies that can be truncated