Migrate self-managed Prometheus to Alibaba Cloud Managed Service for Prometheus - Managed Service for Prometheus

Migrating from a self-managed open source Prometheus to Managed Service for Prometheus involves three stages: metric collection, visual analysis, and alert configuration. This document describes the migration methods for each stage across different deployment scenarios.

Step 1: Migrate metric collection

During metric collection, Prometheus periodically scrapes and stores metric data from target monitoring components or their corresponding exporters based on scrape job configurations.

Scenario 1: Container Service for Kubernetes (ACK)

Procedure

Connect your Container Service for Kubernetes (ACK) cluster to Managed Service for Prometheus. For instructions, see Container Observability.

Note
When you connect an ACK cluster, the Managed Service for Prometheus collection agent (the ack-arms-prometheus component) automatically installs node-exporter. This may cause a port conflict and prevent node-exporter from starting. To resolve this, change the default port of node-exporter. In your container cluster, navigate to Applications and then Helm. Find the arms-prometheus application, click Update, locate the nodeExporter - args - port configuration parameter, and change the default port.
Configure collection tasks. Managed Service for Prometheus supports multiple configuration methods, including ServiceMonitor, PodMonitor, and custom collection rules. For more information, see Manage custom collection rules for container environments.
To verify that the collection tasks are working correctly, log on to the Cloud Monitor console and go to the Integration Center. Find your container cluster and click its details. Then, click the Self-Monitoring tab to view the environment's scrape targets. You can expand a specific scrape job to view the collected metric information. Select Targets from the left navigation pane and confirm that all scrape targets have a status of 'up', which indicates a successful configuration.

Scenario 2: Self-managed Kubernetes

This scenario is similar to Scenario 1: Container Service for Kubernetes (ACK). The only difference is that you must first register your self-managed Kubernetes cluster as a registered cluster in Container Service for Kubernetes (ACK). After registration, follow the configuration steps in Scenario 1.

Scenario 3: Alibaba Cloud ECS

Procedure

Log on to the Cloud Monitor console, select Host Monitoring in the Integration Center, and follow the on-screen wizard to create an ECS environment instance and deploy the exporter and collection configurations for ECS host monitoring. On the ARMS Integration Center page, select Infrastructure > Host Monitoring. On the configuration page on the right, select ECS(VPC) as the environment type and select the corresponding VPC instance. In the configuration information, for NodeExporter installation method, select Automatic installation, and for Host service discovery method, select Taint label selection. Set the ECS taint label (such as ack.alibabacloud.com/nodepool with a value of *), enable Collect process status metrics, and set the Node-Exporter service port to 9100.

Note
The host monitoring component installs node-exporter and process-exporter by default. To avoid port conflicts, you can set custom ports in the integration configuration. You can also choose to install these exporters as needed.
Configure custom collection tasks. If you have other exporters deployed on your ECS instances, go to the Integration Center in the Cloud Monitor console, find Custom Metric Collection, select the ECS(VPC) environment, and configure the exporter's service port, metric collection path, and scrape interval. Click OK. Repeat this process for each additional exporter.
To verify that the collection tasks are running correctly, go to the Integration Center in the Cloud Monitor console. Find your ECS environment and click its details. Then, click Self-Monitoring to view the scrape targets for the environment. You can expand a specific scrape job to view the collected metric information.

Scenario 4: Self-managed data center

Prerequisites

You must connect your self-managed data center to one of your Alibaba Cloud VPCs by using a method such as Express Connect.
You must manually install the required exporters on each host in your self-managed data center.

Procedure

Log on to the Cloud Monitor console. In the Integration Center, select "Host Monitor (self-service installation + IP domain selection)". Follow the instructions to connect your hosts, which creates an ECS environment instance and generates the collection configuration for host monitoring.
In the Integration Center, connect the monitoring components within the VPC. This enables Managed Service for Prometheus to generate scrape jobs, scrape monitoring data, and create default dashboards and alert rules.
(Optional) For custom collection tasks, you can configure custom collection rules for the corresponding environment instance in the Integration Center.
Similar to Scenario 3: Alibaba Cloud ECS, you can go to the Integration Center to view the scrape targets.

Step 2: Migrate Grafana visualization

Scenario 1: Use self-managed Grafana

After your metrics are sent to Managed Service for Prometheus, you can continue to use your existing self-managed Grafana. Managed Service for Prometheus supports integration with self-managed Grafana instances. For more information, see V2 instances.

Scenario 2: Use Managed Service for Grafana

If you use Managed Service for Grafana, you need to import your existing Grafana dashboards into the managed service.

Prerequisites

Activate Managed Service for Grafana and create a Grafana workspace.

Procedure

Connect your Managed Service for Prometheus instance to Managed Service for Grafana. On the workspace details page for Managed Service for Grafana, find the Cloud Service Integration section and select Prometheus Monitoring Service. In the instance list on the right, find the target Prometheus instance and click Integrate in the Actions column. Follow the prompts to complete the integration. A data source for the Prometheus instance is automatically created in your Grafana workspace with a name in the format "prom-${PrometheusInstanceId}".
After the integration is complete, click the public URL of your managed Grafana workspace to log in. When creating a new dashboard, import your existing dashboard JSON file, select the data source you integrated in the previous step, and follow the instructions to finish.

Step 3: Migrate alert configurations

Similar to visual analysis migration, Managed Service for Prometheus provides out-of-the-box alert rules for the components involved in the migration, such as container clusters, MySQL, and Redis.

For container, Kubernetes, and ECS integration scenarios, go to the Integrated Components page in the Integration Center of the Cloud Monitor console. Click Alerts for a specific component to view its default alert rules.

You can create and manage custom alert rules from the Prometheus Alert Rules page in the Cloud Monitor console for Prometheus. You can also import existing Prometheus alert rules by using a Prometheus alert rule template. For example, the default P1 alert rules generated for the ECS component include ECS Classic Network Outbound Bandwidth Usage Alert (threshold > 80%), ECS Outbound Packet Drop Rate Alert (threshold > 0), ECS Inbound Packet Drop Rate Alert (threshold > 0), and ECS Disk Usage Alert (threshold > 80%). All these rules use a custom PromQL detection type, have a duration of 5 minutes, and are active by default.

Migrate historical data from a self-managed Prometheus

If you want to migrate historical data from a self-managed open source Prometheus to Managed Service for Prometheus, see Migrate historical data from a self-managed open source Prometheus to Managed Service for Prometheus.