Deploy OpenTelemetry Collector as a sidecar in your function instance to automatically collect and report Prometheus metrics exposed by the LLM inference engine—such as token throughput and request latency. You do not need to modify your business code. Metrics flow seamlessly into Alibaba Cloud ARMS Managed Prometheus Service, enabling production-grade, visualized, and alertable real-time monitoring of your LLM service.
Preparations
Before you begin, complete the following cloud service and resource preparations. Also obtain key information and permissions required for later configuration. If all prerequisites are met, proceed directly to the next steps.
Activate Object Storage Service (OSS) and create a bucket
Store model files, the OpenTelemetry Collector binary, and its configuration file. Your Function Compute instances access these files by mounting the bucket. If you already have an available bucket, skip to the next step.
-
Activate Object Storage Service (OSS). In the navigation pane on the left, go to the Buckets page. Click Create Bucket.
-
Configure the following key parameters. Keep all other settings at their defaults:
-
Bucket Name: Enter a globally unique name. To ensure uniqueness, use a combination such as project-name-region-random-string—for example,
llmfiles-hangzhou-a1b2c3. -
Region: Choose the same region where you plan to deploy your model service in Function Compute.
-
-
Click Create.
Deploy an LLM inference model in Function Compute
If you have not yet deployed an LLM inference model service in Function Compute, follow the steps below to deploy a Qwen3-0.6B model powered by the vLLM inference engine for learning and testing. Skip to the next step if you already have one ready.
-
Download the model files and upload them to OSS. Download URL: Qwen3-0.6B.
ImportantThe model file
model.safetensorsis large. Wait until all files show an upload success status. -
Go to the Function Compute console > Functions. At the top of the page, select your region. Click Create Function.
-
Select GPU Function. Click Create {title}.
-
In the Create Function page, configure the following core settings. Keep all other settings at their defaults or configure them as needed by referring to Create a GPU function:
-
Function Name: Use only letters, numbers, underscores (_), and hyphens (-). Do not start with a number or hyphen. Length must be 1–64 characters. We recommend using a combination such as model-random-string—for example, qwen3-06b-a1b2c3.
-
Instance type: Select Elastic instance. Keep all other settings at their defaults.
-
Sample code:: Select Use Custom Repository Image.
-
Container Image:
serverless-registry.cn-hangzhou.cr.aliyuncs.com/functionai/vllm-openai:v0.10.1 -
Startup Command:
vllm serve /mnt/qwen3 --port 9000 --served-model-name Qwen/Qwen3-0.6B --trust-remote-code -
Listening Port: 9000.
-
Execution Timeout Period: Set to 600.
-
Permissions: For Function Role, select AliyunFcDefaultRole.
-
Storage: Enable Mount OSS. For Bucket/Bucket subdirectory, select the subdirectory containing your model files. For Function Local Directory, enter
/mnt/qwen3—matching the directory used in the Startup Command.
-
-
Click Create. You are redirected to the Function Details page.
-
In the Function Topology, click Trigger. On the Modify Trigger {name} page, set Authentication Method to No Authentication.
NoteYou change Authentication Method to No Authentication only for testing. In production, configure authentication as described in Authentication and Authorization.
Activate ARMS and create a Prometheus instance
This instance receives and persistently stores performance metrics from your LLM model service, pushed by OpenTelemetry Collector. It also provides query capabilities. If you do not have an available instance, follow the steps below to create one. Record the Remote Write URL. Skip to the next step if you already have one ready.
-
Activate Application Real-Time Monitoring Service. Then go to the ARMS console.
-
In Prometheus Monitoring > Instances, click Create Prometheus Instance.
-
In the Create Instance page, select General Instance for Instance Type. Enter a custom Instance Name. Keep all other settings at their defaults—or configure them as needed by referring to Manage Prometheus Instances.
-
Click Create Now. You are redirected to the instance Settings page. Copy the Remote Write URL now. You need it when configuring OpenTelemetry Collector.
Procedure
Step 1: Download and configure OpenTelemetry Collector
OpenTelemetry Collector collects Prometheus metrics exposed by your LLM inference service and pushes them to a remote Prometheus instance using the Remote Write protocol.
-
Download and extract otelcol-contrib_0.136.0_linux_amd64.zip. After successful extraction, you get a binary file named
otelcol-contriband a configuration file namedotel_config.yaml.This topic uses version v0.136.0. For other versions, see opentelemetry-collector-releases
-
Edit the
otel_config.yamlconfiguration file with the following settings. For full parameter descriptions, see opentelemetry/configuration:-
job_name:: The job name. Use the same name as your function for easier metric querying and analysis. -
targets: Replace{port}with your function’s listening port. For example, the function created in Deploy an LLM inference model in Function Compute uses port 9000. -
To find the port: Go to Function Management > Functions. Click your Function Name. On the Function Details page, go to Configuration > Basic Configurations > Listening Port
-
username/password: Set to your Alibaba Cloud account’s AccessKey ID and AccessKey Secret. If you use a RAM user, ensure that the RAM user has the AliyunPrometheusMetricWriteAccess or AliyunCloudMonitorFullAccess permission. See Write open-source Prometheus metrics to Alibaba Cloud Prometheus instances via Remote Write URLs. -
endpoint: Enter the public Remote Write URL of your Prometheus instance.To find the URL: Go to Prometheus Instances. Click your Instance Name/Instance ID. On the Settings page, copy the URL.
-
scrape_interval(optional): The interval between metric scrapes.
A complete example of the
otel_config.yamlfile follows:receivers: prometheus: config: scrape_configs: - job_name: "{uner-vllm}" scrape_interval: 15s metrics_path: /metrics static_configs: - targets: ["0.0.0.0:9000"] processors: batch: extensions: basicauth/prw: client_auth: username: "LT******S2K" password: "t6Z7o******B55" exporters: prometheusremotewrite: endpoint: "https://workspace-default-******.cn-hangzhou.******/api/v1/write" auth: authenticator: basicauth/prw service: extensions: [basicauth/prw] pipelines: metrics: receivers: [prometheus] processors: [batch] exporters: [prometheusremotewrite] -
Step 2: Upload the binary and configuration file to OSS
-
In the Buckets page, click your bucket name.
-
In the navigation pane on the left, go to Object Management > Objects. Click Upload Object.
-
In the upload panel, drag and drop the
otelcol-contribbinary and the editedotel_config.yamlfile from Step 1 into the upload area. -
Keep all other parameters at their defaults. Click Upload Object.
Step 3: Update your function instance configuration
-
Go to the Function Compute console. Select Function Management > Functions. Click your Function Name.
-
Start Parameter Configuration: By default, LLM metrics are enabled. For example, the vLLM engine shown here requires no action. Some engines require explicit enablement via a start parameter.
For SGLang, click Configuration > the Edit button for Basic Configuration. Add
--enable-metricsto the Startup Command. -
Edit Configuration > Advanced Settings > Storage: Enable Mount OSS. Configure the OSS Mount Target:
Bucket/Bucket Subdirectory: Select the bucket and subdirectory where you uploaded the
otelcol-contribbinary and the editedotel_config.yamlfile.NoteIf you previously mounted OSS in Deploy an LLM inference model in Function Compute, click Add Mount Target to add a new bucket.
-
Function Local Directory: Enter the local directory path to mount into your function instance—for example,
/mnt1/. -
Edit Configuration > Instance Configuration:
-
Enable Instance preheating.
-
Set Warmup Timeout to 10.
This is the timeout for executing the Initializer hook. Unit is seconds. Valid range is 1–300. Adjust as needed.
-
Select Warm-up Procedure Type as Command. For Command, use:
NoteReplace /mnt1 in the command below with your actual function local directory.
nohup /mnt1/otelcol-contrib --config=/mnt1/otel_config.yaml >/dev/null 2>&1 & -
Click Deploy to save the configuration.
-
Verification and Testing
Verify that the model service is running
Call the model service using curl. Replace https://{fc_public_endpoint} with your function’s HTTP trigger public endpoint—for example, https://*******.cn-****.fcapp.run.
To find the endpoint: Go to Function Management > Functions. Click your Function Name. On the Function Details page, go to Trigger > Configurations > Internet Endpoint
curl --request POST \
--url https://{fc_public_endpoint}/v1/chat/completions \
--header 'Authorization: Bearer REPLACE_BEARER_TOKEN' \
--header 'content-type: application/json' \
--data '{"messages":[{"content":"Hello! Who are you?","role":"user"}],"model":"Qwen/Qwen3-0.6B","stream":false}'Sample response:
{
"id":"chatcmpl-b27876a60f9749859638********b",
"object":"chat.completion",
"created":1762760556,
"model":"Qwen/Qwen3-0.6B",
"choices":[
{
"index":0,
"message":{
"role":"assistant",
"content":"I'm happy to meet you, everyone! Are you enjoying our chat?\t\t\t\t\t\t\t\nNow we can talk about study and life!",
"refusal":null,
"annotations":null,
"audio":null,
"function_call":null,
"tool_calls":[],
"reasoning_content":null
},
"logprobs":null,
"finish_reason":"stop",
"stop_reason":null
}
],
"service_tier":null,
"system_fingerprint":null,
"usage":{
"prompt_tokens":14,
"total_tokens":42,
"completion_tokens":28,
"prompt_tokens_details":null
},
"prompt_logprobs":null,
"kv_transfer_params":null
} Verify that metrics are reported
-
Go to the ARMS console > Prometheus Monitoring > Instances. Click your Prometheus instance name.
-
On the Prometheus instance details page, click Metrics Management > Metrics Statistics. This page shows metrics count, data volume, and more.
-
Click Metrics Explorer. Enter the PromQL query:
vllm:prompt_tokens_created{}.
What to do next
Configure Grafana dashboards
-
In the Application Real-Time Monitoring Service (ARMS) console, select Grafana Service > Overview. Click Create Workspace and configure as follows:
Configuration Item
Description
Workspace Name
Enter a custom Grafana workspace name.
Admin Password
Set the Admin password.
Region
Select the region where the workspace will be located.
Version
Billing version of Managed Service for Grafana. For pricing details, see Billing Rules.
Grafana Version
Select a Grafana version.
User Account Count
Select the number of accounts. For pricing details, see Billing Rules.
Subscription Duration
Select the subscription duration.
Auto-Renewal on Expiration
If enabled, billing starts automatically after the trial ends. For pricing details, see Billing Rules.
Resource Group
Use resource groups to classify and manage resources under your Alibaba Cloud account. Apply permissions, deployments, and monitoring at the group level instead of per resource.
Tag
Set a tag key and tag value for the workspace to support grouped management.
-
In the Workspaces list, select the workspace you just created.
-
In the Cloud Service Integration section, select Prometheus Monitoring Service and integrate your target Prometheus instance.
-
In Basic Information, click View Public Endpoint and open it in your browser. Sign in with Sign in with Alibaba Cloud.
-
Configure visualization panels. You can use sample Grafana configurations provided by vLLM, SGLang, and others. The examples below are for reference only:
Configure alerting
In the Alibaba Cloud Managed Prometheus Service console, configure alerting rules as described in Create Prometheus Alert Rules.
Billing Information
The main Alibaba Cloud products involved in this solution and their billing details are listed below:
-
Application Real-Time Monitoring Service (ARMS): The Prometheus and Grafana services used in this solution are provided by Application Real-Time Monitoring Service (ARMS). For billing details, see Product Billing (New).
-
Object Storage Service (OSS): Storing the
OpenTelemetry Collectorbinary and configuration file incurs minimal storage fees. For details, see Billing Overview. -
Function Compute (FC): Running the Collector process consumes small amounts of CPU and memory. You may need to adjust your function instance type based on actual load. For details, see Billing Overview.