All Products
Search
Document Center

Cloud Monitor:Configure Prometheus alerts

Last Updated:Dec 04, 2025

The new version of Managed Service for Prometheus is integrated with Cloud Monitor, which lets you create Prometheus alert rules directly in Cloud Monitor. By default, alert events generated by these rules are integrated into the ARMS alert management system. This topic describes how to create Prometheus alert rules and use the ARMS alert management system to send alert notifications to channels such as DingTalk and text messages.

Prerequisites

Create an alert rule

  1. Log on to the Cloud Monitor console.

  2. In the navigation pane on the left, choose Managed Service for Prometheus > Alert Rule.

  3. On the Alert Rule page, select a destination region, and then click Create Alert Rule.

  4. In the Create Alert Rule panel, set the alert parameters.

    Parameter

    Description

    Example

    Rule Name

    The name of the alert rule.

    Production cluster - container CPU utilization alert

    Data Source

    Select the Prometheus instance for which you want to create an alert rule.

    Production cluster

    Detection Condition Definition Method

    Configure based on a preset metric: Prometheus provides a set of preset alert metrics. You can select a preset metric and configure an alert rule for the metric.

    Custom PromQL: Enter a Prometheus Query Language (PromQL) statement directly.

    Configure based on a preset metric

    Based on Predefined Metrics - Metric Group

    Select a metric group.

    Kubernetes Load

    Based on Predefined Metrics - Metric

    Select the metric for which you want to configure an alert rule. Different metric groups correspond to different metrics.

    Container CPU utilization

    Based on Predefined Metrics - Filter Condition

    Specify the scope of the alert rule based on the alert metric. An alert event is generated for any resource that meets the filter condition and the alert condition.

    The following filter conditions are available:

    • Traverse: The alert rule applies to all resources in the current Prometheus instance. This is the default filter condition.

    • Equal To: After you select this condition, enter a specific resource name. The created alert rule applies only to the specified resource. You cannot specify multiple resources.

    • Not Equal To: After you select this condition, enter a specific resource name. The created alert rule applies to all resources except the specified one. You cannot specify multiple resources.

    • Regular Match: After you select this condition, enter a regular expression to match resource names. The alert rule applies to all resources that match the regular expression.

    • Regular Not Match: After you select this condition, enter a regular expression to match resource names. The alert rule filters out all resources that match the regular expression.

    Traverse

    Based on Predefined Metrics - Detection Condition

    Set the conditions for generating alert events based on the preset content of the alert metric.

    The alert condition is met when the container CPU utilization is greater than 80%.

    Based on Predefined Metrics - PromQL Preview

    The PromQL statement that corresponds to the alert condition. You can use the PromQL statement to directly create an alert using the custom PromQL method, or create a dashboard using this PromQL statement.

    -

    Custom PromQL - Custom PromQL

    Use a PromQL statement to set the alert rule expression.

    100 - avg by(instance) (rate(node_cpu_seconds_total{mode="idle"}[2m])) * 100 > 80

    Query Result Preview

    Displays the values of the metric configured in the current alert rule as a time series curve.

    None

    Severity

    The default severity is P4. The severity levels in ascending order are P4, P3, P2, and P1.

    P4

    Duration

    An alert event is generated only when the threshold is met for N minutes or longer.

    1

    Alert Check Cycle

    The interval at which the alert rule checks whether the data meets the alert condition. The unit is minutes.

    1

    Detect After Data Completion

    Specifies whether to determine whether the alert condition is triggered based on complete data after all monitoring data in the current epoch has been collected and processed.

    Yes

    Alert Content

    Use Go template syntax to customize alert parameter variables in the alert content.

    For the Configure based on a preset metric method, the system provides preset templates for different metrics. If a preset template does not meet your needs, you can modify it.

    Example:

    Node {{ $labels.instance }} CPU utilization{{$labels.metrics_params_opt}} {{$labels.metrics_params_value}}%, current CPU utilization {{ printf "%.2f" $value }}%

    You can also add custom tags to the alert content. Example:

    image.png

    • When Go Template renders a template, it fills the template based on data from the context. The namespace {{$labels.namespace}} and pod {{$labels.pod_name}} are taken from the results of the PromQL query:

      image.png

    • To add custom tags and annotations, introduce them using {{$labels.custom_label_key}}.

    Namespace: {{$labels.namespace}} / Pod: {{$labels.pod_name}} / Container: {{$labels.container}} CPU utilization{{$labels.metrics_params_opt_label_value}} {{$labels.metrics_params_value}}%, current value{{ printf "%.2f" $value }}%

    Tags

    Set tags for the alert. The tags can be used as options for notification policy matching rules.

    None

    Annotations

    Set annotations for the alert.

    None

    Alert Notification

    Alert events are sent to the ARMS alert management system by default. You can go to the ARMS alert management platform to configure notification policies to send alert notifications.

    Yes

  5. After you are finished, click OK. You can view the status of the new alert rule on the Alert Rules page.

Configure alert notifications

  1. Log on to the ARMS console. In the left-side navigation pane, choose Alert Management > Notification Policies.

  2. On the Notification Policies page, click Create Notification Policy. On the Create Notification Policy page, enter a name for the policy.

  3. In the Match Rules section, set the match rules for alert events.

    1. Set Data Source to ARMS-Prometheus Monitoring.

      image

    2. Set the match rule expression. You can define custom tags or select existing tags, and then select the corresponding values. If a required value is not in the drop-down list, you can enter it directly.

      Existing tags include system tags and tags from the metrics in the alert rule.

      Expand to view configuration examples

      Specify a Prometheus instance

      To specify a Prometheus instance, set clustername to the name of the Prometheus instance.

      image

      Configure based on metric tags

      Notification rules in ARMS can route notifications based on the tags of alert events. By default, the labels used for grouping in the PromQL expression of a Prometheus alert rule become the tags for the resulting alert events.

      • For example, if the alert expression is:

        AliyunEcs_CPUUtilization{}>90

        Then all labels of the AliyunEcs_CPUUtilization metric are used as tags for the alert event.

      • If the alert expression is:

        avg(AliyunEcs_CPUUtilization) by (instanceId,instanceName) > 90

        Then only the instanceId and instanceName labels are used as tags for the alert event.

      Based on the tags of alert events, you can configure different notification policies. For example, to send alert events for which instanceId is abcdexxxxxx to a specific DingTalk group, configure the rule as instanceId equals abcdexxxxxx.

      image

    3. Click Next.

  4. In the Event Group step, select an option to specify whether to group alert events, and click Next.

    • If you specify that alert events are not grouped, all alert events are sent to the contacts in an alert notification.

    • If you specify that alert events are grouped, you must also specify a field. Alert events that have the same field value are grouped. Each group of alert events is sent to contacts in one alert notification.

  5. In the Notification Objects step, configure the following parameters.

    1. Click + Add Notification Object to select contacts.

      Contact types:

      通知策略-当告警生成时

      • Contact: When you select a contact, select a notification method, such as by phone call, text message, or email.

      • Contact group: When you select a contact group, select a notification method, such as by phone call, text message, or email.

      • Shift: When you select a shift, select a notification method, such as by phone call, text message, or email.

      • Instant messaging (IM) applications: You can send alert notifications to contacts from IM applications, including DingTalk, Lark, and WeCom.

      • Webhook: You can use a webhook to send alert notifications.

    2. Specify whether to send clearing notifications after an alert is cleared.

      If you choose to send clearing notifications after an alert is cleared, the status of an alert automatically changes to Resolved if all events related to the alert are handled. When an alert is cleared, the system sends clearing notifications to the contacts.

    3. Specify a notification template. For more information, see Configure a notification template and a webhook template.

    4. Specify the period of time during which alert notifications are sent.

    5. (Optional)Select the ticket system to which alert notifications are sent. For more information, see Integration.

    6. Click Next.

  6. In the Policy to Repeat/Escalate/Clear Alerts step, specify whether to send repeated notifications or use an escalation policy, and click Next.

    • Whether to Resend Notifications: To send repeated notifications, specify the interval at which alert notifications are sent. Alert notifications are continuously sent at the specified interval until the alert is cleared.

    • Whether to configure an escalation policy for alerts:

      • If you do not use an escalation policy, an alert notification is sent only once before an alert is cleared.

      • If you use an escalation policy, an alert notification is sent to other notification objects based on the escalation policy until the alert is cleared.

    • Whether to enable manual recovery of alerts: After you enable manual recovery, an alert is not automatically cleared if an alert event is not triggered within the automatic recovery time that is set on the Alert Integrations tab of the Integrations page.

  7. In the Action Integration step, specify the automatic action after an alert is triggered or cleared. For more information, see Create an action integration to execute an alerting plan.