Application Real-Time Monitoring Service:LLM application details

Large Language Model (LLM) applications are built on models trained with vast amounts of data and parameters to understand and generate human-like natural language. They are widely used in natural language processing, text generation, and intelligent dialogue.

LLM outputs are often unpredictable due to challenges such as discrepancies between training and production results, performance degradation from data distribution shifts, stale data quality, and unreliable external data sources. Monitoring model output quality is therefore essential to promptly identify any degradation.

Application Real-Time Monitoring Service (ARMS) supports automatic instrumentation of LLM applications through the ARMS agent for Python. After you integrate your LLM application with ARMS, you can view trace diagrams to analyze input and output across different operation types and token consumption.

Topics related to monitoring views:

• Overview

• Performance analysis

• Token analysis

• LLM operations

• Provided Services

• Dependencies

• LLM Trace Explorer

• Exception analysis

• Session analysis

• Scenario-based analysis

Topics related to metrics and fields:

• LLM metrics

• LLM Trace field definitions