Application Monitoring in the Cloud Monitor 2.0 console lets you adjust vLLM and SGLang collection scope, trace granularity, and logging behavior at any time after an inference service goes live. Most changes take effect without restarting the inference engine. Collect finer detail while you isolate a problem, and tighten collection when requests carry sensitive data or log volume grows too large.
Each section in this topic describes one console panel and every switch in it: what the switch does, its default value, and when a change takes effect. The appendix lists the matching environment variables, which you can use to set collection behavior statically before the application starts.
Open the collection configuration page
Log on to the Cloud Monitor 2.0 console.
Choose Application Observability > Application Monitoring, and then click the name of the target inference service.
Click the Application Configuration tab and adjust the switches in the panels described in this topic.
Inference engine settings are spread across four panels on the Application Configuration tab: Agent Switch Settings, AI Application Collection Configuration, vLLM Inference Engine Configuration, and SGLang Inference Engine Configuration.
In the tables in this topic, Default is the value that is used when the console does not apply a value for that item. The agent then reads the environment variable. If the environment variable is also unset, the built-in default of the agent or the engine applies. A default marked "in the agent" is a built-in value of the agent, and a default marked "in the engine" is a built-in value of the engine.
Agent Switch Settings
The LLM (large language model) services group in the Agent Switch Settings panel contains two check boxes, vllm-plugin and sglang-plugin, which control whether the agent loads the corresponding plugin at startup. After you clear a check box, the agent no longer instruments that engine, so neither traces nor metrics are produced. The agent reads these check boxes while it loads, so a change takes effect only after you restart the application. To pause collection temporarily without a restart, use the collection switches in the panels described in the following sections.
AI Application Collection Configuration
Switch | Default | Description |
Collect LLM input and output content | Enabled | Specifies whether request prompts and model outputs are recorded in observable data. Turn this switch off when requests carry sensitive data. |
Maximum field length for LLM input and output content | 8192 | The maximum number of characters collected for a single input or output. Longer content is truncated. |
Collection policy for LLM input and output content | Record in span attributes | Where collected content is written. Select Record in span attributes to view the content with the trace, or Record in log events when the content is large. |
Collect multi-modal data | Disabled | Applies only to client framework instrumentation, such as LangChain, LiteLLM, DashScope, and Google ADK. The vLLM and SGLang engine instrumentation does not read this switch. |
After you turn off Collect LLM input and output content, the vLLM entry spans chat {model} and text_completion {model} are no longer produced. The request-level llm_request span and its child spans are unaffected, and the embeddings {model} and rerank {model} entry spans are unaffected as well. Whether instrumentation is registered for this content is decided when the agent starts.
vLLM Inference Engine Configuration
Switch | Default | Description |
Collect metrics for vLLM | Enabled by default in the agent | Specifies whether the agent reads metrics from the inference engine. After you turn this switch off, the agent stops collecting metrics. Whether collected metrics are reported also depends on the Report metrics switch on the Custom Configuration page, which this topic does not cover. |
vLLM collection granularity | 1 by default in the agent | The level of detail in collected traces. The vLLM collection granularity accepts the following values:
|
vLLM request logs | Disabled by default in the engine | Prints request reception and lifecycle logs, such as Received request, Added request, aborted, and failed. |
vLLM iteration detail logs | Disabled by default in the engine | For each EngineCore iteration, prints the number of requests, the number of tokens, and the duration of the context stage and the generation stage. Use these logs to investigate scheduling problems such as batches that are not full and starved requests. |
Associate vLLM logs with TraceId | Disabled by default in the agent | Injects the TraceId and the SpanId into the vLLM log prefix so that you can look up logs and traces from each other. |
vLLM log level | INFO by default in the engine | Adjusts the vLLM log level dynamically. The agent changes both the logger and its handler so that the handler does not filter out DEBUG logs. |
At granularity 2, the agent forces logprobs computation for requests so that it can collect token-level details. This changes the actual computation behavior of the engine and adds noticeable performance and data volume overhead. Set vLLM collection granularity to 2 only while you troubleshoot a problem. At granularity 2, the number of events in a single span is proportional to the number of iterations and can exceed the OpenTelemetry default limit of 128. Events beyond that limit are dropped from the span. OTEL_SPAN_EVENT_COUNT_LIMIT controls that limit.
The vLLM iteration detail logs print every iteration in full and are not sampled, so log volume is high in high-throughput scenarios. Turn them off as soon as troubleshooting is done.
SGLang Inference Engine Configuration
Switch | Default | Description |
Collect metrics for SGLang | Enabled | Specifies whether the agent reads metrics from the inference engine. When this switch is on, the agent computes request-level metrics for the engine: time to first token (TTFT), inter-token latency, total token count, and end-to-end duration. You do not need to restart the engine or add the |
SGLang collection granularity | 3 + composite stages | The depth of the span tree produced by native SGLang OpenTelemetry instrumentation. A larger value produces more detail. It takes effect only when native tracing of the engine is on. The agent turns on native tracing when the engine starts, so you do not need to add the |
SGLang request logs | Disabled by default in the engine | Turns the request logs of TokenizerManager on and off dynamically. |
SGLang request log verbosity | 2 by default in the engine | The level of detail printed in request logs. A larger value prints more content. |
SGLang log level | INFO by default in the engine | Adjusts the log level of the |
For the switches in this table that override engine values, the agent records the original value of the engine the first time it applies an override. After you remove the configuration item from the console, the agent restores the recorded value. Turning off Collect metrics for SGLang stops only the agent from reporting metrics. It does not disable the native Prometheus metrics of the engine.
Appendix: environment variables
To set collection behavior statically before the application starts, use the environment variables in the following table. Console configuration takes precedence over environment variables, so an environment variable is used only when the console does not apply a value for that item.
Console switch | Environment variable | Console configuration key |
|
|
|
|
|
|
Collect LLM input and output content |
|
|
Maximum field length for LLM input and output content |
|
|
Collection policy for LLM input and output content |
|
|
Collect metrics for vLLM |
|
|
vLLM collection granularity |
|
|
vLLM request logs | — |
|
vLLM iteration detail logs | — |
|
Associate vLLM logs with TraceId | — |
|
vLLM log level | — |
|
Collect metrics for SGLang |
|
|
SGLang collection granularity | — |
|
SGLang request logs | — |
|
SGLang request log verbosity | — |
|
SGLang log level | — |
|