After you install the Python agent for your large language model (LLM) application, Application Real-Time Monitoring Service (ARMS) starts monitoring the application. The Scenario-based analysis page shows a list of sessions and model usage for your LLM application within a specified time range.
Prerequisites
The agent for your LLM application must be installed. For instructions, see Connect an LLM application or inference service to ARMS.
Session analysis
-
Click the search box to filter sessions by criteria such as Session ID, User, Session duration, Number of traces, Total tokens, Input tokens, and Output tokens.
-
Click a session name or Details in the Actions column to view session details.
Click a Trace ID to view trace details. For more information, see LLM trace analysis.
The session details panel displays basic information, including the Session ID, Session duration, Total tokens, Number of traces, Start time, User ID, and Model. The main area features a Gantt chart visualizing the timeline of all traces. Below the chart, a list of individual traces shows their input/output content, Trace ID, token consumption, and the model used.
Model analysis
Select a model at the top of the tab. For more information about the metrics, see LLM metric reference.
