Monitor ASM instances using a self-managed Prometheus instance - Alibaba Cloud Service Mesh

Prometheus is an open source monitoring tool for cloud-native applications. This topic describes how to integrate a self-managed Prometheus instance with Alibaba Cloud Service Mesh (ASM) to monitor a mesh. The open source ack-prometheus-operator from Alibaba Cloud Container Service for Kubernetes (ACK) is used as an example.

Prerequisites

An ACK cluster is created. For more information, see Create an ACK managed cluster.
An ASM instance is created. For more information, see Create an ASM instance. A Prometheus instance and a Grafana instance are created in the ACK cluster. For more information, see Use open source Prometheus for monitoring.

Install ack-prometheus-operator

This topic uses the open source ack-prometheus-operator from ACK as a configuration example. If you install a self-managed Prometheus instance in the cluster using another method, add the content of the additionalScrapeConfigs field to the scrape configs of the Prometheus instance. For more information about scrape configs, see prometheus_scrape_config.

Log on to the ACK console. In the left navigation pane, click Clusters.
On the Clusters page, find the cluster you want and click its name. In the left-side navigation pane, choose Applications > Helm.
On the Helm page, find ack-prometheus-operator and click Update in the Actions column.

In the Update Release panel, modify the additionalScrapeConfigs field in the YAML text box, and then click OK.

The YAML file is long. You can press Ctrl+F to search for the additionalScrapeConfigs field. The modified additionalScrapeConfigs field is as follows:

additionalScrapeConfigs:
# Scrape config for envoy stats
- job_name: 'envoy-stats'
  metrics_path: /stats/prometheus
  kubernetes_sd_configs:
  - role: pod
  relabel_configs:
  - source_labels: [__meta_kubernetes_pod_container_port_name]
    action: keep
    regex: '.*-envoy-prom'
  - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
    action: replace
    regex: ([^:]+)(?::\d+)?;(\d+)
    replacement: $1:15090
    target_label: __address__
  - action: labeldrop
    regex: __meta_kubernetes_pod_label_(.+)
  - source_labels: [__meta_kubernetes_namespace]
    action: replace
    target_label: namespace
  - source_labels: [__meta_kubernetes_pod_name]
    action: replace
    target_label: pod_name

Generate monitoring data

Send requests to services on the data plane. In this example, the Bookinfo application is used. For more information, see Getting started. Go to the Productpage page of Bookinfo. Refresh the page multiple times. 监控数据

Verify the result

Check whether Envoy proxies collect the specified metrics

Run the following command. If the output contains monitoring data, the Envoy proxies are collecting the specified metrics. If the output is empty, the Envoy proxies are not collecting the specified metrics.

details=$(kubectl get pod -l app=details -o jsonpath={.items..metadata.name})
kubectl exec $details -c istio-proxy -- curl -s localhost:15090/stats/prometheus |grep istio

View the metrics that are monitored by Prometheus

Log on to the ACK console. In the left navigation pane, click Clusters.
On the Clusters page, find the cluster you want and click its name. In the left navigation pane, choose Network > Services.
On the Services page, find Prometheus and click its IP address in the External IP column.
On the Prometheus page, enter istio_requests_total in the search box and click Execute. The metrics that are monitored by Prometheus are displayed.

View the Grafana dashboard of the metric data

Log on to the ACK console. In the left navigation pane, click Clusters.
On the Clusters page, find the cluster you want and click its name. In the left navigation pane, choose Network > Services.
On the Services page, find Grafana, and then click its IP address in the External IP column.
On the Grafana page, select Istio Workload Dashboard in the upper-left corner. The following figure shows the Grafana dashboard that is displayed.