Enable Remote Read from Open-Source Prometheus for Cross-System Monitoring - Managed Service for Prometheus

Managed Service for Prometheus supports the standard Prometheus remote read protocol. After you add a remote_read section to your self-hosted Prometheus configuration, your local Prometheus server can query time series data stored in the cloud alongside local data.

Important

Remote read transfers raw time series data over the network to your local Prometheus engine. This increases operational overhead, adds network latency, and is subject to service limits that are easily exceeded at scale.

For most use cases, the HTTP API query protocol is a better choice. The HTTP API:

Runs queries server-side, delivering performance gains of tens of times over local computation
Works with standard HTTP requests -- no additional infrastructure required
Integrates natively with Grafana

Protocol limitations

The remote read endpoint does not support HTTP/2.

Prerequisites

A Managed Service for Prometheus instance is created
Open-source Prometheus is installed on your server

Step 1: Grant RAM user permissions (required only for RAM user credentials)

Skip this step if you use your Alibaba Cloud account directly.

If the Prometheus instance belongs to an Alibaba Cloud account and you plan to authenticate with a Resource Access Management (RAM) user's AccessKey pair, grant Cloud Monitor permissions to that RAM user first.

Log on to the RAM console with your Alibaba Cloud account or as a RAM administrator. In the left navigation pane, choose Permissions > Grants.

On the Permission page, click Grant Permission and configure the following parameters.

Parameter	Description
Resource Scope	Select a scope based on your requirements.
Principal	Select the target RAM user.
Policy	Attach one of the following policies: - `AliyunCloudMonitorFullAccess` -- Full permissions on Cloud Monitor, including the ability to view, modify, and delete instances. - `AliyunCloudMonitorReadOnlyAccess` -- Read-only permissions on Cloud Monitor. The RAM user can view instances but cannot modify or delete them.

Click Grant permissions, then click Close.

Step 2: Get the remote read URL

Log on to the Cloud Monitor console. In the left navigation pane, choose Managed Service for Prometheus > Instances.
In the top navigation bar, select the region of your Prometheus instance. Find the target instance and click Settings in the Actions column.
On the Settings tab, copy the remote read URL. Two URLs are available -- one for internet access and one for internal network access. Use the internal network URL when your Prometheus server runs within the same Alibaba Cloud region for lower latency.

Step 3: Configure open-source Prometheus

Open the prometheus.yml configuration file on your Prometheus server.

Add a remote_read section at the end of the file.

Parameter	Description
`url`	Replace `<your-remote-read-url>` with the URL you copied in Step 2. Example: `http://ts-xxxxxxxxxxxx.hitsdb.rds.aliyuncs.com:3242/api/prom_read`
`read_recent`	Set to `true` so that Prometheus reads remote data for all time ranges, including those within the local retention period. If set to `false` or omitted, Prometheus only reads remote data for time ranges outside local retention.

   global:
     scrape_interval: 15s
     evaluation_interval: 15s
   scrape_configs:
     - job_name: 'prometheus'
       static_configs:
       - targets: ['localhost:9090']
   remote_read:
     - url: "<your-remote-read-url>"
       read_recent: true

Restart Prometheus to apply the configuration.

Verify the setup

Open the Prometheus web UI at http://localhost:9090.
Click Graph in the top navigation bar.
Enter a PromQL query in the expression field and click Execute. For example, query the CPU usage of a specific node: Replace <ip-address> with the IP address of the target node. If remote read is configured correctly, the Graph tab displays query results with data from the managed service.
```
   100 - (avg by (instance) (rate(node_cpu_seconds_total{instance="<ip-address>:9100", mode="idle"}[5m])) * 100)
```

What's next

Query metrics using the HTTP API -- For production workloads, the HTTP API provides faster query performance without the overhead of a local Prometheus engine.
Connect Grafana to Managed Service for Prometheus -- Visualize cloud metrics in Grafana dashboards.
Service limits -- Review remote read and API query quotas.