Use an address to access data in Managed Service for Prometheus from an open-source Prometheus instance - Cloud Monitor

Managed Service for Prometheus provides a standard remote read interface. You can use this interface to remotely access monitoring data in Prometheus. This topic describes how to use a remote read address to allow an Prometheus instance to access Managed Service for Prometheus monitoring data.

Important

The Prometheus Remote Read mode reads a large amount of raw time series data to a local Prometheus engine for computation and has many usage limitations. Therefore, we recommend that you directly use the HTTP API query protocol provided by Managed Service for Prometheus.

Managed Service for Prometheus provides a complete Prometheus query API. It processes queries on the server and returns only the final results, delivering performance many times faster than using Remote Read. The API allows you to send queries directly over HTTP and connect tools like Grafana without a local Prometheus engine. For configuration details, see Connect Grafana or self-managed applications to Prometheus data by using an HTTP API endpoint.

Limitations of the Remote Read approach include:

High infrastructure costs of maintaining a local Prometheus compute engine.
High end-to-end latency because each query requires transferring large volumes of raw data over the network.
Strict service limits that are easy to exceed when querying large datasets.

Limitations

The remote read interface does not support HTTP/2.

Prerequisites

You have created a Managed Service for Prometheus instance. For more information, see Create an instance.

(Optional) Step 1: Grant CMS read and write permissions to a RAM user

If you created your Prometheus instance with your primary Alibaba Cloud account, and you need to use a RAM user's credentials (AccessKey ID and AccessKey Secret) for remote read and write, you must first grant the RAM user read and write permissions on CloudMonitor.

Log on to the RAM console by using your primary Alibaba Cloud account or as a RAM administrator. In the left-side navigation pane, choose Permission Management > Authorization.

On the Authorization page, click Create Authorization. On the Create Authorization page, configure the parameters as described in the following table.

Parameter	Description
Resource Scope	Select a resource scope based on your requirements.
Principal	Specify the principal. This is the RAM user to whom you want to grant permissions.
Policy	Add one of the following permissions to the RAM user as needed: `AliyunCloudMonitorFullAccess`: Grants full permissions on CloudMonitor. After you grant this permission, the RAM user can view, edit, delete, and perform all other operations on instances of all sub-products. `AliyunCloudMonitorReadOnlyAccess`: Grants read-only permissions on CloudMonitor. After you grant this permission, the RAM user can only view information about instances of all sub-products but cannot modify or delete them.

Click OK, and then click Close.

Step 2: Obtain the remote read address

Log on to the Cloud Monitor console. In the left navigation pane, choose Managed Service for Prometheus > Instances. The Instances page appears.
In the top navigation bar, select the region where your instance is located. Find your target instance and click Actions in the Settings column.
On the Settings tab, copy the remote read address for the public network or private network based on your requirements.

The public network address is in the format http://cn-{region}.arms.aliyuncs.com:9090/api/v1/prometheus/{token}. The private network address is in the format http://cn-{region}-intranet.arms.aliyuncs.com:9090/api/v1/prometheus/{token}. The URL includes an authentication token at the end.

Step 3: Configure open-source Prometheus

Install Prometheus. For installation instructions, see the open-source Prometheus download page.

Edit the Prometheus.yml configuration file. Add the following content to the end of the file. Replace the remote_read URL with the address that you obtained in Step 2. Then, save the file.

global:
  scrape_interval: 15s
  evaluation_interval: 15s
scrape_configs:
  - job_name: 'prometheus'
    static_configs:
    - targets: ['localhost:9090']
remote_read:
  # Replace with your remote read address.
  - url: "http://cn-{region}.arms.aliyuncs.com:9090/api/v1/prometheus/{token}"
    read_recent: true

Restart the open-source Prometheus service.

Step 4: View data in self-managed Prometheus

In a web browser, open the following address to access your self-managed Prometheus:
```
http://localhost:9090
```
On the Prometheus page, click Graph in the top navigation bar.
On the Graph tab, enter a query in the expression box and click Execute.
For example, to query the CPU utilization of a specific IP address:
```
100 - (avg by (instance) (rate(node_cpu_seconds_total{instance=":9100", mode="idle"}[5m])) * 100)
```

FAQ

Query returns no data and no error

For Prometheus V1 instances, remote read does not require an AccessKey. If no data is returned after you configure remote read, check whether external_labels is configured in your local Prometheus configuration. This setting can prevent the remote read from returning the expected data.

[xxx prometheus]# head prometheus.yml
global:
  scrape_interval: 60s
  evaluation_interval: 15s
  external_labels:
    cloud: xxx
    cluster: xxx