Frequently asked questions about alert monitoring rules in Simple Log Service, including alert severity, no-data alerts, recovery, and quota management.
Can I dynamically set different alert severities?
Yes. When you configure an alert monitoring rule, you can define different severity levels based on specific conditions. For more information, see Set alert severity.
Does Simple Log Service support no-data alerts?
Yes. When you configure an alert monitoring rule, you can enable the no-data alert feature and specify a separate alert severity. For more information, see No-data alert.
Does Simple Log Service support automatic alert recovery?
Yes. When you configure an alert monitoring rule, you can enable alert recovery. For more information, see Set up recovery notifications.
Sending alerts to multiple targets
Yes. You can enable group evaluation when you configure an alert monitoring rule. For more information, see Set up group evaluation.
Temporarily disable an alert rule
Yes. On the Alert Monitoring Rule page, click Disable in the Actions column.
Grant permissions to a RAM user
To allow a RAM user to manage alerts, you must grant the required permissions. For more information, see Grant a RAM user permissions to manage alerts.
"Alert count exceeds the maximum limit" error
If you receive the Alert count exceeds the maximum limit error when you create an alert monitoring rule, the number of alert monitoring rules in the project has reached the maximum limit (100 by default). You can increase the quota by following the instructions in Manage resource quotas.
The quota can be increased to a maximum of 200 rules per project. If you need more alert monitoring rules, consider the following strategies:
-
Delete unnecessary alert monitoring rules from the project.
-
Collect logs into different projects to reduce the number of alert monitoring rules in a single project.
For example, you can collect logs from Service A into Project1 and logs from Service B into Project2. Then, you can create alert monitoring rules in different projects.
-
Consolidate similar alert monitoring rules.
For example, to monitor data in the same Logstore, you can create a single alert monitoring rule and use group evaluation to monitor multiple targets. For more information, see Set up group evaluation.
-
Use data transformation or Scheduled SQL to aggregate data into a single Logstore before you create an alert monitoring rule.
For example, if you need to monitor error logs from multiple Logstores, you can consolidate all error logs into one Logstore and then create a single alert monitoring rule for it.
Configure alerts based on log keywords
After logs are collected into Simple Log Service, you can configure alert monitoring rules that trigger based on specific keywords. For more information, see Configure alerts based on log keywords.
Monitor different field values
If you do not know all possible values of a field in advance but need to monitor whether any value meets the trigger condition, you can use group evaluation. Group evaluation groups data by the specified field and evaluates the trigger condition for each group separately. For more information, see Set up group evaluation.
For example, if you store metric data from multiple servers in one Metricstore and want to receive a separate alert for each server when its CPU utilization (cpu_util) exceeds 95%, you can use group evaluation.
Only one trigger condition takes effect
Query and analysis results are evaluated against trigger conditions in order. Once a result matches a condition, subsequent conditions are skipped. Therefore, order your trigger conditions from the highest severity to the lowest. For more information, see Set alert severity.
Missing alerts or false positives
-
Missing alerts: An alert is configured to trigger if the error log count exceeds 10. However, when you query the logs on the Search & Analysis page of a Logstore, you find that the error count was greater than 10 during a specific period, but no alert was triggered.
-
False positives: An alert is configured to trigger if the queries per second (QPS) drops below 100. However, an alert is triggered even when the QPS is greater than 100 during a specific period.
Missing alerts or false positives usually occur because of the latency between when data is written to a Logstore and when it becomes available for queries. If the time range in your alert monitoring rule uses a relative time, the query may exclude the latest data, leading to inaccurate results. To prevent this, extend the query time range or align the query with a fixed time window. For more information, see Monitoring timeliness.
Cause: Notify threshold not reached
On the Alert History chart of the Alert History Statistics dashboard, if Trigger Alert is 'true' and Cause is 'Notify threshold not reached', the trigger condition was met but the number of consecutive triggers has not reached the configured Threshold of Continuous Triggers. For example, if you set Threshold of Continuous Triggers to 3, an alert is sent only after the trigger condition is met three consecutive times.