Application Real-Time Monitoring Service:LLM application details

Large Language Model (LLM) applications refer to various applications developed based on LLMs. These models, trained with vast amounts of data and parameters, are capable of answering questions posed in human-like natural language. Consequently, they find extensive applications in fields such as natural language processing, text generation, and intelligent dialogue.

Given that the output results of LLMs are often unpredictable, alongside challenges like discrepancies between training and production outcomes, performance degradation due to data distribution shifts, stale data quality, and reliance on unreliable external data sources, these uncontrollable factors can significantly impact the overall performance of LLM applications. Therefore, it becomes crucial to promptly identify any decline in model output quality.

Application Real-Time Monitoring Service (ARMS) supports automatic instrumentation of LLM applications through the ARMS agent for Python. Once your LLM application is integrated with ARMS, you can view the trace diagrams of the application, facilitating a more intuitive analysis of relevant information, such as input and output across different operation types and token consumption.

Topics related to monitoring views:

• Overview

• Performance analysis

• Token analysis

• LLM operations

• Services provided

• Dependencies

• LLM Trace Explorer

• Exception analysis

• Session analysis

• Scenario-based analysis

Topics related to metrics and fields:

• LLM metrics

• Trace fields for LLM applications