The Prometheus monitoring feature allows you to add metric trend charts to the alert cards of a DingTalk group. You can obtain more information about alerts from metric trend charts, which helps troubleshoot problems. This topic describes how to configure Prometheus alert policies to add metric trend charts to alert cards.

Prerequisites

Configure a metric trend chart

  1. Log on to the ARMS console.
  2. In the left-side navigation pane, choose Prometheus Service > Prometheus Alert Rules.
  3. On the Prometheus Alert Rules page, perform one of the following operations based on your needs:
    • To configure a metric trend chart for an existing alert, find the alert and click Edit next to the alert.
    • To configure a metric trend chart for a new alert, click Create Prometheus Alert Rule.
  4. On the Edit Prometheus Alert Rules or Create Prometheus Alert Rule page, perform one of the following operations based on your needs:
    1. In the Advanced Configuration section, click Create Annotation next to Annotations to add the following annotations.
      Key Value Example
      _aliyun_display_promql The PromQL statement used to display the metric trend chart. Typically, it is the alert PromQL statement with the threshold judgment removed. For example, change cpu_usage > 80 to cpu_usage. _aliyun_display_promql:cpu_usage
      _aliyun_display_name The name of the metric in the trend chart. _aliyun_display_name: container CPU utilization
    2. Select the notification policy where the DingTalk group chat is specified as the alert contact.
    3. Modify other parameters. For more information about other parameters, see Create an alert rule for a Prometheus instance.
    4. Click Save.

View the metric trend chart

After the configuration is complete and an alert is triggered, you can view the metric trend chart in the alert card for the DingTalk group.

For more information about alert cards, see Manage alerts in a DingTalk group.

FAQ

  • Why do I still need to manually configure the alert PromQL statement even if I do not use the statement?

    The alert PromQL statement is not exactly the same as the PromQL statement to display the metric trend chart in terms of threshold judgment and the AND/OR logic. You must remove the threshold judgment from the alert PromQL statement.

  • Why is the metric trend chart not displayed after the _aliyun_display_promql is specified?

    Only when the _aliyun_display_promql parameter and the time series obtained by using the PromQL statement have the exact same label, the metric trend chart can be rendered. Typically, multiple time series can be obtained after you execute a PromQL statement. A time series is selected based on the label to render the metric trend chart. For example, if the metric obtained by executing the cpu_usage > 80 statement contains the pod_name and container labels, the query result by using the _aliyun_display_promql parameter must also contain the labels.