Alibaba Cloud Model Studio:Model monitoring

Connect to a custom application

The following example shows how to retrieve monitoring data using the Prometheus HTTP API. For complete API usage, see the Prometheus HTTP API documentation.

• Example 1: Query the token consumption for all models across all workspaces in an Alibaba Cloud account on November 20, 2025 (UTC), with the query set to model_usage and the step size set to step=60 s.

  Example

  Parameter description

  GET {HTTP API}/api/v1/query_range?query=model_usage&start=2025-11-20T00:00:00Z&end=2025-11-20T23:59:59Z&step=60s Accept: application/json Content-Type: application/json Authorization: Basic base64Encode(AccessKey:AccessKeySecret)

  query: The value of query can be replaced with any metric name from the Monitoring metrics list below. Expand to view monitoring metrics Type Metric name Description Call count model_call_count Total number of model calls Call duration model_call_duration_total Total model call duration model_call_duration Average model call duration model_call_duration_p50 p50 model call duration model_call_duration_p99 p99 model call duration model_first_token_duration_total Total time to first token model_first_token_duration Average time to first token model_first_token_duration_p50 p50 time to first token model_first_token_duration_p99 p99 time to first token Time per non-first token model_generation_duration_per_token_total Total time per non-first token model_generation_duration_per_token Average time per non-first token model_generation_duration_per_token_p50 p50 time per non-first token model_generation_duration_per_token_p99 p99 time per non-first token Usage model_usage Total model usage HTTP API: Replace {HTTP API} with the HTTP API address you got in Step 1. Authorization: Concatenate the AccessKey and AccessKey secret of your Alibaba Cloud account in the format AccessKey:AccessKeySecret, encode the string in Base64, and provide it in the format Basic encoded_string. Example value: Basic TFRBSTV3OWlid0U4XXXXU0xb1dZMFVodmRsNw== Note:AccessKey and AccessKey secret must belong to the same Alibaba Cloud account as the Prometheus instance from Step 1.

• Example 2: Building on Example 1, add filters to retrieve token consumption only for a specific model (qwen-plus) in a specific workspace (llm-nymssti2mzww****).

  Example

  Description

  GET {HTTP API}/api/v1/query_range?query=model_usage{workspace_id="llm-nymssti2mzww****",model="qwen-plus"}&start=2025-11-20T00:00:00Z&end=2025-11-20T23:59:59Z&step=60s Accept: application/json Content-Type: application/json Authorization: Basic base64Encode(AccessKey:AccessKeySecret)

  query: Enclose multiple filter conditions in {} and separate them with commas. For example: {workspace_id="value1",model="value2"}. The following is a list of supported filter conditions (LabelKey). Expand to view supported filter conditions LabelKey Description user_id Alibaba Cloud account ID. For a RAM user, this is the UID. For more information, see the referenced document apikey_id Get the API Key ID (not the API key) on the Key Management (Singapore | US | Beijing) page. Note An apikey_id value of -1 indicates that the call originated from the Alibaba Cloud Model Studio console, not through an API. workspace_id Workspace ID. For more information, see the referenced document. model Model. protocol Protocol type. Possible values: HTTP: Non-streaming HTTP SSE: Streaming HTTP WS: WebSocket protocol sub_protocol Sub-protocol. Possible values: DEFAULT: Synchronous call ASYNC: Asynchronous call Commonly used for image generation models. Text-to-image generation status_code HTTP status code. Only the model_call_count metric supports this LabelKey. error_code Error code. Only the model_call_count metric supports this LabelKey. usage_type Usage type. Only the model_usage metric supports this LabelKey. Possible values: total_tokens input_tokens output_tokens cache_tokens image_tokens audio_tokens video_tokens image_count audio_count video_count duration characters audio_tts times

Connect to Grafana

Add a monitoring data source in Grafana (self-managed or Alibaba Cloud Grafana service). This example uses Grafana 10.x (English version). The procedure for other versions is similar. For more information, see the official Grafana documentation.

1. Add a data source:

   1. Log on to Grafana with an administrator account. Click the icon in the upper-left corner and select Administration > Data sources. Click + Add new data source and select Prometheus as the data source type.

   2. On the Settings tab, configure the data source information:

      • Name: Enter a custom name.

      • Prometheus server URL: Enter the HTTP API address you got in Step 1.

      • Auth: Enable Basic auth and set User to the AccessKey of your Alibaba Cloud account and Password to the AccessKey secret of your Alibaba Cloud account.

        The AccessKey and AccessKey secret must belong to the same Alibaba Cloud account as the Prometheus instance from Step 1.

      

   3. Click Save & Test at the bottom of the tab.

2. Query metrics:

   1. Click the icon in the upper-left corner of the Grafana page, then click Dashboards in the navigation pane on the left.

   2. On the Dashboards page, click New > New dashboard on the right to create a new dashboard.

   3. Click + Add visualization and select the data source that you just created.

   4. On the Edit Panel page, click the Query tab. In the A section, select _name_ and the metric name from the Label filters field. The following example shows how to query model token consumption using the model_usage metric:

      Example

      Description

      In the figure, replace the value for _name_ (model_usage) with any metric name from the Monitoring metrics list below. Expand to view monitoring metrics Type Metric name Description Call count model_call_count Total number of model calls Call duration model_call_duration_total Total model call duration model_call_duration Average model call duration model_call_duration_p50 p50 model call duration model_call_duration_p99 p99 model call duration model_first_token_duration_total Total time to first token model_first_token_duration Average time to first token model_first_token_duration_p50 p50 time to first token model_first_token_duration_p99 p99 time to first token Time per non-first token model_generation_duration_per_token_total Total time per non-first token model_generation_duration_per_token Average time per non-first token model_generation_duration_per_token_p50 p50 time per non-first token model_generation_duration_per_token_p99 p99 time per non-first token Usage model_usage Total model usage Add the following label filters to further refine your query: Expand to view supported filter conditions LabelKey Description user_id Alibaba Cloud account ID. For a RAM user, this is the UID. For more information, see the referenced document apikey_id Get the API Key ID (not the API key) on the Key Management (Singapore | US | Beijing) page. Note An apikey_id value of -1 indicates that the call originated from the Alibaba Cloud Model Studio console, not through an API. workspace_id Workspace ID. For more information, see the referenced document. model Model. protocol Protocol type. Possible values: HTTP: Non-streaming HTTP SSE: Streaming HTTP WS: WebSocket protocol sub_protocol Sub-protocol. Possible values: DEFAULT: Synchronous call ASYNC: Asynchronous call Commonly used for image generation models. Text-to-image generation status_code HTTP status code. Only the model_call_count metric supports this LabelKey. error_code Error code. Only the model_call_count metric supports this LabelKey. usage_type Usage type. Only the model_usage metric supports this LabelKey. Possible values: total_tokens input_tokens output_tokens cache_tokens image_tokens audio_tokens video_tokens image_count audio_count video_count duration characters audio_tts times

   5. Click Run queries to execute the query.

      If the chart displays data successfully, the configuration is correct. Otherwise, verify the following: 1. The HTTP API address, AccessKey, and AccessKey secret are correct. 2. Monitoring data exists in the Prometheus instance from Step 1.