After you install the Python agent for your Large Language Model (LLM) application, Application Real-Time Monitoring Service (ARMS) starts monitoring it. The Conversation Analysis page lists sessions from a specified time period and lets you examine the details of each one.
Prerequisites
An agent is installed for your LLM application. For more information, see Integrate an LLM application or inference service with ARMS.
Procedure
-
Log on to the ARMS console. In the left-side navigation pane, choose .
-
On the Application List page, select a region at the top of the page, and then click the name of your application.
-
In the top navigation bar, click Conversation Analysis.
-
Click the search box. From the drop-down list, filter sessions by criteria such as session ID, user, session duration, number of traces, total tokens, input tokens, and output tokens.
-
The session list displays information for each session, including Duration, Tokens (including input tokens and output tokens), Number of Traces, User, and Start Time.
Click the session name or Details in the Actions column to view the details of the session.
Click a trace ID to view the trace details. For more information, see LLM Trace Analysis.
The top of the session details page displays summary metrics, including Session ID, total duration, total tokens, and number of traces. The Basic Information section shows fields such as Start Time, user ID, and model name. A Gantt chart in the middle of the page visualizes the execution time distribution for each trace. Below the chart, detail cards provide the trace ID, token count, and a breakdown of token usage by model for each trace.