:Use Managed Service for Prometheus to install and configure other exporters

Quick access for other types of exporters is under development and will be released in the future. If your exporter is not supported by Managed Service for Prometheus, install the exporter, configure service discovery, and then create a dashboard. This topic provides an example on how to install and configure open source exporters that are not supported by Managed Service for Prometheus. In this example, MySQL is used.

For more information about the exporters of open source Prometheus, see Exporters and integrations.

Prerequisites

• A Container Service for Kubernetes (ACK) cluster is created. For more information, see Create an ACK dedicated cluster.
• The ACK cluster is monitored by Managed Service for Prometheus. For more information, see Create a Prometheus instance to monitor an ACK cluster.
• An Elastic Compute Service (ECS) instance is monitored by Managed Service for Prometheus. For more information, see Create a Prometheus instance to monitor an ECS instance.

Procedure

The following figure describes how to use Managed Service for Prometheus to monitor a MySQL database.

Step 1: Deploy an application

To capture MySQL data in an ACK cluster, you must deploy the mysqld-exporter image that is provided by open source Prometheus to the ACK cluster. To deploy the mysqld-exporter image, perform the following operations:

1. Log on to the ACK console.
2. In the left-side navigation pane, click Clusters.
3. On the Clusters page, find the cluster to which you want to deploy the application and click Applications in the Actions column.
4. Create a container group.
   1. In the left-side navigation pane, choose Workloads > Deployments.
   2. On the Deployments page, click Create from YAML in the upper-right corner.
   3. On the Create page, enter the following code in the Template code editor and click Create:
      Note Replace the values of <yourMySQLUsername>, <yourMySQLPassword>, <IP>, and <port> with the actual values of the MySQL database.

      apiVersion: apps/v1 # for versions before 1.8.0 use apps/v1beta1
      kind: Deployment
      metadata:
        name: mysqld-exporter
        labels:
          app: mysqld-exporter
      spec:
        replicas: 1
        selector:
          matchLabels:
            app: mysqld-exporter
        template:
          metadata:
            labels:
              app: mysqld-exporter
          spec:
            containers:
            - name: mysqld-exporter
              imagePullPolicy: Always
              env:
                - name: DATA_SOURCE_NAME
                  value: "<yourMySQLUsername>:<yourMySQLPassword>@(<IP>:<port>)/"
              image: prom/mysqld-exporter
              ports:
              - containerPort: 9104
                name: mysqld-exporter

   The newly created container group is displayed on the Deployments page.
5. Create a Service.
   1. In the left-side navigation pane, choose Network > Services.
   2. On the Services page, click Create Resources in YAML.
   3. On the Create page, enter the following code in the Template code editor and click Create:

      apiVersion: v1
      kind: Service
      metadata:
        labels:
          app: mysqld-exporter
        name: mysqld-exporter
      spec:
        ports:
        - name: mysqld-exporter
          port: 9104
          protocol: TCP
          targetPort: 9104
        type: NodePort
        selector:
          app: mysqld-exporter

   The newly created Service is displayed on the Services page.

Step 2: Configure service discovery

To configure service discovery of Managed Service for Prometheus to receive data from the MySQL database, perform the following steps:

1. Log on to the Prometheus console.
2. In the top navigation bar of the Prometheus Service page, select the region where the ACK cluster resides. Find the cluster and click Settings in the Actions column.
3. On the Settings page, click the Service Discovery tab. On the Service Discovery tab, click the ServiceMonitor tab.
4. On the ServiceMonitor tab, click Add ServiceMonitor.
5. In the Add ServiceMonitor dialog box, enter the following code and click OK:

   apiVersion: monitoring.coreos.com/v1
   kind: ServiceMonitor
   metadata:
     # Enter a unique name.
     name: tomcat-demo
     # Enter a namespace.
     namespace: default
   spec:
     endpoints:
     - interval: 30s
       # Enter the value of the Name field of the Port parameter of the Prometheus exporter in the service.yaml file.
       port: tomcat-monitor
       # Enter the path of the Prometheus exporter.
       path: /metrics
     namespaceSelector:
       any: true
       # The namespace of the NGINX demo application.
     selector:
       matchLabels:
         # Enter the value of the Label field in the service.yaml file to find the service.yaml file.
         app: tomcat

   The configured service discovery task is displayed on the ServiceMonitor tab.

Step 3: Configure a Grafana dashboard

To configure a Grafana dashboard to display data, perform the following steps:

1. Go to the homepage of Grafana dashboards.
2. In the left-side navigation pane, choose + > Import.
3. On the Import page, enter 7362 in the Import via grafna.com field and click Load next to the field. 7362 is the ID of the MySQL Grafana template provided by Prometheus.
   
4. On the Import page, perform the following operations and click Import:
   
   1. Enter a dashboard name in the Name field.
   2. Select your ACK cluster from the Folder drop-down list.
   3. Select your ACK cluster from the prometheus drop-down list.
   The following figure shows the configured Grafana dashboard.

Step 4: Create an alert rule

To create an alert rule to monitor a metric, perform the following steps:

1. Log on to the Prometheus console.
2. In the top navigation bar of the Prometheus Monitoring page, select the region where the monitored Kubernetes cluster resides. Then, click the name of the Kubernetes cluster.
3. In the left-side navigation pane, click Alarm configuration.
4. On the Alarm configuration page, click Create Alert in the upper-right corner.
5. In the Create Alert panel, set the parameters.
   1. Optional:Select a template from the Alarm template drop-down list.
   2. Enter a rule name in the Rule Name field. Example: Alert for inbound traffic.
   3. Enter a PromQL statement as the expression in the Alarm expression field. Example: (sum(rate(kube_state_metrics_list_total{job="kube-state-metrics",result="error"}[5m])) / sum(rate(kube_state_metrics_list_total{job="kube-state-metrics"}[5m]))) > 0.01.
      Important If a PromQL statement contains a dollar sign ($), an error is returned. You must delete the equal sign (=) and the parameters on both sides of the equal sign (=) from the statement that contains the dollar sign ($). For example, change sum (rate (container_network_receive_bytes_total{instance=~"^$HostIp.*"}[1m])) to sum (rate (container_network_receive_bytes_total[1m])).
   4. Enter a number N in the duration field. After the alert rule is created, an alert notification is sent only when the alert condition is met for N consecutive minutes. For example, you can enter 1. In this case, an alert notification is sent only when the alert condition is met for one consecutive minute.
      Note The alert condition refers to the condition specified by the PromQL statement. By default, Prometheus collects data at intervals of 15 seconds. If less than 4N consecutively collected data records meet the alert condition, no alert notification is sent. The threshold 4N is calculated by using the following formula: N × 60/15 = 4N. In this example, N is set to 1. Therefore, an alert notification is sent only when four consecutively collected data records meet the alert condition. To ensure that an alert notification can be sent each time a collected data record meets the alert condition, set N to 0.
   5. Enter the notification content in the Alarm message field.
   6. Optional:In the Labels section of Advanced Configuration, click Create Tag to add one or more tags to the alert rule. The specified tags can be used as options for a notification rule.
   7. Optional:In the Annotations section of Advanced Configuration, click Create Annotation. Then, enter message in the Key field and {{variable name}} alert message in the Value field. The specified annotation is in the format of message:{{variable name}} alert message. Example: message:{{$labels.pod_name}} restart.

      You can customize a variable name or select an existing tag as the variable name. The following content describes the existing tags:

      • The tags that are carried in the metrics of an alert rule expression.
      • The tags that are created when you create an alert rule. For more information, see Create an alert rule.
      • The default tags provided by ARMS. The following table describes the default tags.

        Tag	Description
        alertname	The name of the alert. The format is <Alert name>_<Cluster name>.
        _aliyun_arms_alert_level	The level of the alert.
        _aliyun_arms_alert_type	The type of the alert.
        _aliyun_arms_alert_rule_id	The ID of the alert rule.
        _aliyun_arms_region_id	The ID of the region.
        _aliyun_arms_userid	The ID of the user.
        _aliyun_arms_involvedObject_type	The subtype of the associated object, for example, ManagedKubernetes or ServerlessKubernetes.
        _aliyun_arms_involvedObject_kind	The type of the associated object, for example, app or cluster.
        _aliyun_arms_involvedObject_id	The ID of the associated object.
        _aliyun_arms_involvedObject_name	The name of the associated object.

   8. Click OK.
   After you create the alert rule, the Alarm configuration page displays the rule, as shown in the following figure.