By setting up alert rules in Grafana, you can monitor various metrics for your services. When an alert is triggered, it notifies your operations team to help you quickly identify and resolve issues, preventing service disruptions. This topic shows you how to configure Grafana native alert rules.
Grafana 8.2.x and earlier
Step 1: Log on to Grafana
-
Log on to the Managed Service for Grafana console. In the left-side navigation pane, click Workspace Management.
-
On the Workspace Management page, find the workspace that you want to manage and click the URL in the URL column to go to Grafana.
NoteYou can log on to Grafana with the administrator account of Grafana and the password that you configured when you created the workspace. You can also click Sign in with Alibaba Cloud to log on to Grafana with the current Alibaba Cloud account.
Step 2: Create a notification channel
In the left-side navigation pane, click the
icon and choose Notification channels. Then, click New channel. On the Notification channels page, configure the parameters for the new channel.
|
Parameter |
Description |
|
Name |
A custom name for the channel. |
|
Type |
The notification type, such as DingTalk, Webhook, or Email. |
|
Addresses |
The destination address for notifications. In this example, use the webhook URL of a DingTalk chatbot. For information about how to obtain the webhook URL, see Obtain the webhook URL of a DingTalk chatbot. |
Step 3: Create an alert rule
-
In the left-side navigation pane, choose
> Dashboard, and then click Add an empty panel. -
On the Query tab, select a data source and configure a query. Then, on the Alert tab, click Create Alert.
Parameter
Description
Rule
Name: The name of the alert rule.
Evaluate every: How often to evaluate the alert rule. For example, 1m, 2m, or 5m specifies that the alert is evaluated every one, two, or five minutes, respectively.
For: The duration a condition must persist before an alert is sent.
Conditions
The conditions that trigger the alert. For example, an alert is triggered when the average value (
avg()) of query A exceeds 600.No data and error handling
Specifies whether to send an alert when no data is returned or a query times out.
Notifications
-
Send to: The destination notification channel. In this example, it is the DingTalk channel created in Step 2: Create a notification channel.
-
Message: The content of the alert notification. For information about supported template variables, see Message templating.
-
Tags: Custom tags.
-
(Optional) Step 4: View alert rules
In the left-side navigation pane, choose
> Alert Rules. The Alert Rules page lists your alert rules.
Grafana 8.2.x and later
The user interface differs slightly between Grafana 9.0.x and Grafana 10.0.x, but the overall functionality is similar. This section uses Grafana 9.0.x as an example. You can also create an alert rule on a panel page as described in Step 3: Create an alert rule.
Step 1: Disable integrated ARMS alert plug-in
-
Log on to the Managed Service for Grafana console. In the left-side navigation pane, click Workspace Management.
-
On the Workspace Management page, click the ID of the workspace that you want to manage.
-
In the left-side navigation pane, click Alert Management, disable the integrated ARMS alert plug-in, and then enable Grafana native alerting.
Step 2: Create an alert rule
-
On the Workspace Management page, find the workspace that you want to manage and click the URL in the URL column to go to Grafana.
NoteYou can log on to Grafana with the administrator account of Grafana and the password that you configured when you created the workspace. You can also click Sign in with Alibaba Cloud to log on to Grafana with the current Alibaba Cloud account.
-
In the left-side navigation pane, choose
> Alert rules, and then click New alert rule or Create alert rule. -
On the Create alert rule page, configure the rule parameters and then click Save. The page includes sections for query and alert conditions, and rule name and group configurations. The following table describes the parameters.
Parameter
Description
Set a query and alert condition
Set the query and the alert expression.
-
Query patterns: The query statements to evaluate.
-
Operation: The type of alert expression. Classic condition is a legacy expression type that is compatible with alert rules created in Grafana 8.2.x and earlier.
-
Conditions: The conditions that trigger the alert. For example, an alert is triggered when the average value (
avg()) of query A exceeds 600.
Alert evaluation behavior
Configure the alert evaluation interval and behavior.
-
Evaluate every: How often to evaluate the alert rule. For example, 1m, 2m, or 5m specifies that the alert is evaluated every one, two, or five minutes, respectively.
-
For: The duration a condition must be met before an alert is fired.
-
Configure no data and error handling: Specifies whether to send an alert when no data is returned or a query times out.
Add details for your alert
Set the alert details.
-
Rule name: The name of the alert rule.
-
Folder: The folder where the alert rule is stored.
-
Group: The group that contains the alert rule. Alerts in the same group are evaluated at the same interval.
-
Summary and annotations: Defines the alert content and lets you add links to related dashboards, folders, and other information.
Notifications
Set Custom labels. These labels link the alert rule to a notification policy.
-
-
The new alert appears on the Alert rules page.
Step 3: Create a contact point
-
In the left-side navigation pane, choose
> Contact points. -
On the Contact points tab, click New contact point or Add contact point.
-
On the New contact point page, configure the following parameters.
Parameter
Description
Name
The name of the contact point.
Contact point type
The notification type, such as DingTalk, Webhook, or Email.
Addresses
The destination address for notifications. In this example, use the webhook URL of a DingTalk chatbot. For information about how to obtain the webhook URL, see Obtain the webhook URL of a DingTalk chatbot.
Step 4: Create a notification policy
A notification policy links an alert rule to a contact point.
-
In the left-side navigation pane, choose
> Notification policies. -
On the Notification policies tab, click New nested policy or New specific policy. After configuring the parameters, you will receive alert notifications from Grafana native alerting.
For example, in the Matching labels section, set Label to
app, Operator to=, and Value totest. For Contact point, select ARMS_GRAFANA. Then, click Save policy.Parameter
Description
Matching labels
The labels that match an alert rule to this notification policy.
Contact point
The contact point to which this policy routes alert events.
Other parameters control alert grouping and silences.