ITSM, part of Application Real-Time Monitoring Service (ARMS), is an out-of-the-box intelligent alerting service that receives alert information from various clients, such as Prometheus. You can send native Grafana alert events to ITSM, where they are automatically processed through deduplication, grouping, and algorithmic noise reduction. ITSM then sends alerts to different notification objects based on label content, which helps O&M teams reduce project risks. ITSM also provides a variety of notification methods, including phone calls, text messages, emails, DingTalk, WeCom, Lark, and webhooks, to handle diverse alert scenarios.
Prerequisites
On the Parameter Settings page of the target workspace, ensure the following alert parameters are set:
The unified_alerting parameter is set to
true.The alerting parameter is set to
false.
For Grafana 8.2.x and earlier
Step 1: Log on to Grafana
-
Log on to the Managed Service for Grafana console. In the left-side navigation pane, click Workspace Management.
-
On the Workspace Management page, find the workspace that you want to manage and click the URL in the URL column to go to Grafana.
NoteYou can log on to Grafana with the administrator account of Grafana and the password that you configured when you created the workspace. You can also click Sign in with Alibaba Cloud to log on to Grafana with the current Alibaba Cloud account.
Step 2: Create a notification channel
Create a notification channel named ARMS_GRAFANA if one does not already exist.
In the left-side navigation pane of Grafana, choose
> Notification channels.On the Notification channels tab, click New channel to create a notification channel.
Parameter
Description
Name
Enter
ARMS_GRAFANA.Type
Select webhook.
Url
The integration URL used to send Grafana alerts to ARMS. For more information about how to create a Grafana integration and obtain the URL, see Integrate Grafana alerts.
Step 3: Create a Grafana alert rule
In the left-side navigation pane of Grafana, choose
> Manage.On the Manage tab, click the dashboard for which you want to create an alert.
On the dashboard page, find the panel where you want to create an alert, click the menu icon
next to the panel name, and then click Edit.On the Edit Panel page, click the Alert tab, and then click Create Alert.
Configure other alert parameters as needed. For more information, see the official Grafana documentation or Configure Grafana-native alert rules.
In the Notifications section, click the
icon next to Send to, and then select the notification channel named ARMS_GRAFANA.After creating the alert rule, click Save in the upper-right corner of the Edit Panel page.
When Grafana triggers an alert, it sends the alert event to the page in the ARMS console. For more information, see View alert event history.
Step 4: Create an ARMS notification policy
Managed Service for Grafana creates a corresponding notification policy by default. To receive Grafana alert notifications, configure the contact information in the policy.
-
Log on to the ARMS console. In the left-side navigation pane, choose .
In the Notification Policies list, find the notification policy named ARMS_GRAFANA and click Edit in the Operation column. If the ARMS_GRAFANA notification policy does not exist, create it. For more information, see Create a notification policy.
In the Edit Notification Policy panel, click the Notification Receiver tab. On the Notification Receiver tab, complete the following configurations. For more information, see Notification Policy.
In the When an alert is triggered, section, click Add Notification Object, and select the notification recipients and methods.
Parameter
Description
Notification Receiver
Notification recipients include contacts, contact groups, schedules, DingTalk, Lark, WeCom, or webhooks. To create a notification object, see Overview of notification objects.
NoteA contact with an unverified phone number cannot receive alert notifications by phone. To verify a phone number, see Manage contacts.
Notification Methods
Supported notification methods include phone call, text message, and email. You can select multiple methods.
In the Notification Template section, set the notification message format for Email, SMS, Phone, and DingTalk/Lark/WeCom.
After you complete the configuration, click Save.
For Grafana 8.2.x and later
The alerting UI may differ slightly between Grafana 9.0.x and 10.0.x, but the overall process is the same. This section uses Grafana 9.0.x as an example.
Step 1: Disable the ARMS alert plug-in
-
Log on to the Managed Service for Grafana console. In the left-side navigation pane, click Workspace Management.
-
On the Workspace Management page, click the ID of the workspace that you want to manage.
In the left-side navigation pane, click Alert Management, disable the integrated ARMS alert plug-in, and enable native Grafana alerting.
Step 2: Create a Grafana alert rule
-
On the Workspace Management page, find the workspace that you want to manage and click the URL in the URL column to go to Grafana.
NoteYou can log on to Grafana with the administrator account of Grafana and the password that you configured when you created the workspace. You can also click Sign in with Alibaba Cloud to log on to Grafana with the current Alibaba Cloud account.
In the left-side navigation pane of Grafana, choose
> Browse.On the Browse tab, click the dashboard for which you want to create an alert.
On the dashboard page, select the target panel. Then, click
on the right of the panel name, and click Edit.On the Edit Panel page, click the Alert tab, and then click Create alert rule from this panel.
Set other alerting parameters. For more information, see Grafana documentation or Configure Grafana native alert rules.
Click Save in the upper-right corner.
When a Grafana alert is triggered, the event is reported to in the ARMS console. For more information, see View alert event history.
Step 3: Create a contact point
Create a contact point named ARMS_GRAFANA if one does not already exist.
In the left-side navigation pane, choose
> Contact points.On the Contact points page, click New contact point or Add contact point, and configure the following parameters.
Parameter
Description
Name
Enter
ARMS_GRAFANA.Type
For Type, select Webhook.
Addresses
The integration URL used to send Grafana alerts to ARMS. For more information about how to create a Grafana integration and obtain the URL, see Integrate Grafana alerts.
Step 4: Create a Grafana notification policy
A notification policy links an alert rule to a contact point.
In the left-side navigation pane, choose
> Notification policies.On the Notification policies tab, click New specific policy or New nested policy, and configure the following parameters.
For example, in the Matching labels section, set Label to app, Operator to =, and Value to test. For Contact point, select ARMS_GRAFANA. After finishing the configuration, click Save policy.
Parameter
Description
Matching labels
Matches alert rules to this policy based on labels.
Contact point
Specifies the contact point to which alert events received by this notification policy are sent.
Other parameters are used for configuring alert grouping and silencing.
Step 5: Create an ARMS notification policy
Managed Service for Grafana creates a corresponding notification policy by default. To receive Grafana alert notifications, configure the contact information in the policy.
-
Log on to the ARMS console. In the left-side navigation pane, choose .
In the Notification Policies list, find the notification policy named ARMS_GRAFANA and click Edit in the Operation column. If the ARMS_GRAFANA notification policy does not exist, create it. For more information, see Create a notification policy.
In the Edit Notification Policy panel, click the Notification Receiver tab. On the Notification Receiver tab, complete the following configurations. For more information, see Notification Policy.
In the When an alert is triggered, section, click Add Notification Object, and select the notification recipients and methods.
Parameter
Description
Notification Receiver
Notification recipients include contacts, contact groups, schedules, DingTalk, Lark, WeCom, or webhooks. To create a notification object, see Overview of notification objects.
NoteA contact with an unverified phone number cannot receive alert notifications by phone. To verify a phone number, see Manage contacts.
Notification Methods
Supported notification methods include phone call, text message, and email. You can select multiple methods.
In the Notification Template section, set the notification message format for Email, SMS, Phone, and DingTalk/Lark/WeCom.
After you complete the configuration, click Save.
View alert notifications
When a notification policy is triggered, you receive an alert notification in one of the following ways:
Text messages
Email
DingTalk group: After you receive an alert notification in a DingTalk group, you can handle the alert directly within the group chat. For more information, see Handle alerts in an alert notification group chat.
Related topics
To use native Grafana alerting, see Configure Grafana-native alert rules.
To upgrade your Grafana version, see View workspace information.
You can enable or disable Grafana alerts on the Alert Rules tab in Grafana. For more information, see the official Grafana documentation.