This topic describes the terms of the alerting feature in Log Service.

Term Description
Logstore Log Service provides Logstores to store log data. You can use the SQL-92 syntax to query log data. Monitoring tasks are based on the log search and analysis feature.
Metricstore Log Service provides Metricstores to store time series data. You can use the PromQL syntax and SQL-92 syntax to analyze time series data. Monitoring tasks depend on the log search and analysis feature.
Alert If an alert is triggered by the specified an alert monitoring rule, the event is passed to the notification management system.

The alerting module of Log Service provides subsystems, features, entities, and submodules such as the alert monitoring system and alert monitoring rules.

Alert monitoring system The alert monitoring system is a subsystem that is used to trigger alerts. The alert monitoring system consists of monitoring rules and resource data.

You can configure alert monitoring rules to periodically check query results. Then, the system evaluates the check result based on the rules and triggers alerts and sends recovery notifications based on action policies.

Alert management system The alert management system is a subsystem that is used to manage denoising policies and alert statuses. The alert management system consists of alert policies, alert incidents, and alert dashboards.

The alert management system dispatches, suppresses, deduplicates, and merges alerts based on alert policies. Then, the processed alerts are sent to the notification management system. The alert management system also allows you to set alert statuses and alert handlers.

Notification management system The notification management system is a subsystem that is used to manage notification methods and recipients. The notification management system consists of action policies, alert templates, calendars, users, user groups, on-duty groups, and notification method quotas.

The notification management system sends notifications to specified recipients by using specified notification methods. Recipients can be users, user groups, or on-duty groups. The notification management system also allows you to escalate alerts and customize alert templates.

Alert ingestion system The alert ingestion system is used to ingest external alerts. The alert ingestion system consists of alert ingestion services and alert ingestion applications.

Each alert ingestion application provides an API to ingest external alerts, including recovery notifications from Zabbix and Prometheus. After an external alert is ingested, the alert is processed and sent to the alert management system for further processing.

Alert monitoring system

The alert monitoring system is used to trigger alerts. The alert monitoring system consists of alert monitoring rules and resource data. The following figure shows the architecture of the alert monitoring system.

The alert monitoring system
Term Description
Alert monitoring rule Alert monitoring rules include the configurations that are specified to monitor data, such as query statements and monitored objects. You can monitor log data, time series data, and resource data. For more information, see Create an alert monitoring rule for logs.
Resource data Log Service provides an independent, modifiable, and tabular storage structure to store various resource configurations and customized data. When you perform union queries, you can use resource data. For example, you can use resource data to monitor blacklists and whitelists.

For more information, see Create resource data.

Alert severity Alert severity is a non-identifying attribute of an alert. Alert severity indicates the severity level of an alert. Alert severities include critical, high, medium, low, and report. For more information, see Specify alert severities.

If an external alert is ingested to Log Service, the alert severity is determined based on the specified protocol. If the alert severity fails to be determined by the protocol, the alert severity is medium by default.

Group evaluation Group evaluation is a parameter of alert monitoring rules. If you use query statements, you can group query results by field. The alert monitoring system can monitor data in different groups and trigger separate alerts if the specified conditions are met. This way, you can use an alert monitoring rule to monitor multiple objects and independently manage each group. For more information, see Group evaluation.
Evaluate expressions An evaluate expression provides specific syntax that is used to configure conditions to trigger alerts or dynamically evaluate the severity of alerts.

You can use evaluate expressions to perform logical comparisons and calculations based on the fields of query results. If the specified conditions are matched, true is returned. For more information, see Syntax of evaluate expressions.

Alert label An alert label is an identifier of an alert in the format of key-value pairs. For example, you can add a label when you configure an alert monitoring rule. If an alert is triggered, the label is added to the alert as an alert attribute. Labels can be quoted in alert templates. You can also use labels as alert attributes when you manage alerts and configure action policies.
  • When you monitor data from Metricstores and group query results by label, Log Service uses the labels as alert attributes.
  • When you monitor data from Logstores and group query results based on fields, Log Service uses the key-value pairs of the fields as alert attributes.

For more information, see Labels.

Alert annotation An alert annotation is a non-identifying attribute of an alert in the format of key-value pairs. For example, you can add an annotation when you configure an alert monitoring rule. If an alert is triggered, the annotation is used as an alert attribute. Annotations can be quoted in alert templates. You can also set annotations as an alert attribute when you manage alerts and configure action policies. For more information, see Annotations.
Recovery notification If an alert is triggered, the alert is in the triggered state. If the triggered alert is cleared and the alert is in the resolved state, a recovery notification is sent. If you enable the recovery notification feature, recovery notifications are sent based on the check results obtained by the alert monitoring system. If the alert conditions are not met during the current check period but an alert was triggered during the last check period, an alert notification is sent. If you configure multiple monitoring tasks, we recommend that you enable this feature. This way, you can receive notifications at the first opportunity after alerts are cleared. For more information, see Recovery notifications.

Alert management

The alert management system is used to manage denoising policies and alert statuses. The alert management system consists of alert policies, alert incidents, and alert dashboards. The following figure shows the architecture of the alert management system.

Alert management system
Term Description
Alert policy Alert policies are configuration entities of the alert management system. You can configure alert policies when you ingest external alerts and configure alert monitoring rules. If the alert management system receives alerts (including recovery notifications), the system denoises or merges alerts based on alert policies. Then, the alert is sent to the notification management system to send notifications.
Alert fingerprint The alert management system calculates a fingerprint for each alert. Alerts with the same fingerprint are considered as the same alert. Fingerprints are calculated based on alert identifiers. The alert identifiers include the Alibaba Cloud accounts to which alerts belong, projects in which alert data resides, IDs of monitoring rules, and labels of alerts. For more information, see Deduplicate alerts based on fingerprints.
Alert silence When you configure alert policies, you can configure silence policies. If an alert is triggered during the silence period and the alert matches the specified conditions, no alert notification is sent. For more information, see Silence policies.
Alert suppression You can configure suppression policies when you configure alert policies. If the alert management system receives an alert that matches the conditions in the suppression policy, the alert is suppressed. For example, if a Kubernetes cluster triggers a critical alert and you want to silence alerts whose severity is not critical, you can create a suppression policy. For more information, see Suppress alerts.
Route consolidation policy When you configure alert policies, you can configure route consolidation policies to merge alerts. If the alert management system receives an alert that matches the conditions of a route consolidation policy, the alert is merged and grouped into a merge set. After the alert is delayed or deduplicated based on the route consolidation policies, the alert is sent to the notification management system. For more information, see Merge alerts.
Merge set A merge set stores alerts that are merged and grouped. Each merge set can have one or more fingerprints. After a merge set is delayed and deduplicated based on a route consolidation policy, the merge set is sent to the notification management system.
Alert incident If an alert in a merge set is sent to the alert monitoring system and processed based on a specified alert policy, an alert incident is automatically created. You can set incident statuses and incident handlers in the console. Incident statuses include confirmed, resolved, ignored, and pending evaluation. For more information, see Switch an incident phase.

Log Service can automatically update incident statuses and escalate alerts. For example, if an alert incident is in the resolved state and the alert is triggered again, the status is set to pending evaluation. If an incident is in the confirmed state and a recovery notification is received, the status of the incident is automatically set to resolved.

Note Alert incidents are different from alert events. An alert incident indicates the operations that you perform to handle an alert. An alert event indicates an alert.

Notification management system

The notification management system is used to manage the notification methods and recipients. The notification management system consists of action policies, alert templates, calendars, users, user groups, on-duty groups, and notification method quotas. The following figure shows the architecture of the notification management system.

Action policies
Term Description
Action policy Alert policies are configuration entities of the alert management system. After alerts are grouped into one or more merge sets based on specified route consolidation policies the alerts are sent to the notification management system. Then, notifications are sent to specified recipients by using specified notification methods. The recipients can be users, user groups, or on-duty groups. Alerts can be escalated if they are not resolved within a specified time range.

You can specify conditions to escalate alerts when you configure action policies.

For information about how to configure action policies, see Create an action policy.

Alert escalation If an alert is not confirmed or resolved within a specified time range, the alert is processed based on the action policies that are specified on the Second Action Policy tab. This way, the alert can be handled at the earliest opportunity. For more information, see Escalate alert incidents.
Alert template Log Service sends alert notifications based on the content specified in alert templates. You can specify text content in alert templates as alert notifications. You can also use alert variables in alert templates. If you send alert notifications by using webhooks, you can specify notification formats based on specific protocols. For example, you can set a content format to meet the requirements of Enterprise WeChat. For more information, see Create an alert template.
Calendar The notification management system provides calendars. You can use the global default calendar or customize a calendar.
  • The global default calendar provides information such as time zones, holidays, business days, and business hours.

    The time to send notifications is based on the settings of the global default calendar, such as business days, non-business days, business hours, and non-business hours.

  • Only on-duty groups can customize the calendar. For example, they can customize business days and holidays.
Users Users are recipients who receive notifications. User information contains the user ID, username, phone number, and email address. When you configure action policies, you can specify users as recipients to receive notifications. You can specify users to handle incidents when you manage incidents.

For information about how to create a user, see Create a user.

User group A user group contains the information of the user group ID, user group name, and user information. Each user group contains one or more users. When you configure action policies, you can specify user groups to receive notifications.

For information about how to create a user group, see Create a user group.

On-duty group An on-duty group contains the information of on-duty users and on-duty groups. The information includes on-duty group ID, on-duty group name, and calendar settings, such as rotating shifts and substitute shifts. Each on-duty group contains one or more users or user groups. When you configure action policies, you can specify on-duty groups to receive notifications.

For information about how to create an on-duty group, see Create an on-duty group.

Rotating shift A rotating shift is a configuration item of an on-duty group when you configure shift plans for users or user groups. You can add multiple rotating shifts to an on-duty group. You can configure a rotating shift based on the days that you select on the calendar. You can also configure a rotating shift for non-consecutive hours.

For more information, see Rotating shifts and substitute shifts.

Substitute shift A substitute shift is a configuration item of an on-duty group when you set substitute plans for users or user groups. You can add multiple substitute shifts to an on-duty group.

For more information, see Rotating shifts and substitute shifts.

Notification method quota Log Service allows you to specify a quota for notifications that are sent by using SMS messages, voice calls, or emails. If the number of notifications sent to a recipient by using a specified notification method reaches the quota, the recipient can no longer receive notifications by using the same method. You can specify a quota for notifications that a recipient can receive every day.

For more information, see Configure notification quotas.

Alert ingestion

The alert ingestion system is used to ingest external alerts. The alert ingestion system consists of alert ingestion services and alert ingestion applications.

Term Description
Alert ingestion applications Alert ingestion applications provide the protocol and API information of ingested alerts, such as Zabbix and Prometheus. By default, alerts are ingested over a LAN or the Internet and use HTTP and HTTPS protocols. If an external alert is ingested to Log Service, the alert is passed to the alert management system based on the specified alert policies.
Alert ingestion service The container or namespace of the ingested alert. Alert ingestion services are used for alerts that have the same category. For example, if the alerts from Zabbix and Prometheus belong to the same category, the alerts can be processed by the same service.