All Products
Search
Document Center

Simple Log Service:FAQ about alert monitoring rules

Last Updated:Aug 29, 2023

This topic provides answers to frequently asked questions (FAQ) about alert monitoring rules in Simple Log Service.

How do I grant the required permissions to a RAM user if I want to use the RAM user to manage alerts?

Before you can use a RAM user to manage alerts, you must grant the required permissions to the RAM user. For more information, see Authorize a RAM user to manage alerts.

What do I do if the "Alert count exceeds the maximum limit" error occurs when I create an alert monitoring rule?

If the Alert count exceeds the maximum limit error occurs when you create an alert monitoring rule, the maximum number of alert monitoring rules in the current project is reached. By default, you can create a maximum of 100 alert monitoring rules for each project. You can submit a ticket to increase the maximum limit in your project.

The maximum number of alert monitoring rules in a single project can be increased to 200. If you want to create more alert monitoring rules, use the following methods:

  • Delete the alert monitoring rules that you no longer need from the project.

  • Save logs to different projects to reduce the number of alert monitoring rules in a single project.

    For example, you can save the logs that are collected from Service A to Project1 and save the logs that are collected from Service B to Project2. Then, you can create alert monitoring rules in the projects.

  • Merge similar alert monitoring rules.

    For example, you can create only one alert monitoring rule to monitor the data in a Logstore. You can configure the Group Evaluation parameter in the alert monitoring rule to monitor the data of multiple groups at the same time. For more information, see Use the group evaluation feature.

  • Use the data transformation feature or Scheduled SQL feature to save data to a Logstore, and then create alert monitoring rules. For more information, see Data transformation overview or How Scheduled SQL works.

    For example, if you want to monitor the error logs that are stored in multiple Logstores, you can save all error logs to one Logstore, and then create an alert monitoring rule for the Logstore.

How do I configure an alert monitoring rule based on keywords?

After Simple Log Service collects logs, you can use the alerting system of Simple Log Service to configure alert monitoring rules based on specific keywords in the logs. For more information, see Configure alerts based on log keywords.

How do I monitor different values of a field?

You can use the group evaluation feature to check whether a value of a specific field meets the specified trigger condition of an alert monitoring rule. You can specify the field as a label to group data. The data in each group is evaluated based on the alert trigger condition. For more information, see Use the group evaluation feature.

In this example, the metric data of multiple servers is stored in a Metricstore. If the CPU utilization (cpu_util) of a server exceeds 95%, an alert is triggered and Simple Log Service sends an alert notification for each server. In this case, you can use the group evaluation feature.

Why does only one of the specified trigger conditions take effect?

Query and analysis results are evaluated in sequence based on the specified trigger condition. If one of the query and analysis results meets the first evaluation condition, the other evaluation conditions that you specify are skipped. We recommend that you specify the highest severity level as the first evaluation condition when you configure the Severity parameter in the Trigger Condition field. For more information, see Specify severity levels for alerts.

Why do missing alerts or false positives occur?

  • Missing alerts: For example, if the number of error logs is greater than 10, an alert is triggered. However, no alert is triggered when you query logs on the Search & Analysis page of a Logstore even if the number of error logs is greater than 10 within the specified time range.

  • False positives: For example, if the queries per second (QPS) is less than 100, an alert is triggered. However, an alert is triggered when you query logs on the Search & Analysis page of a Logstore even if the QPS is greater than 100 within the specified time range.

In most cases, missing alerts or false positives occur because latency exists between the time when data is written to a Logstore and the time when you can query the data. If you set the time range of a query statement in an alert monitoring rule to a relative time, the query result may be inaccurate. To prevent missing alerts and false positives, we recommend that you extend the time range of the specified query statement in the alert monitoring rule or set the time range to a time frame. For more information, see Monitoring timeliness.

What do I do if "true" is displayed in the Trigger Alert column and "Notify threshold not reached" is displayed in the Cause column on the Alert History chart?

If "true" is displayed in the Trigger Alert column and "Notify threshold not reached" is displayed in the Cause column on the Alert History chart of the Alert History Statistics dashboard, the Threshold of Continuous Triggers parameter is specified but the number of continuous triggers does not reach the specified threshold. For example, if you set the Threshold of Continuous Triggers parameter to 3, an alert is triggered only if the specified trigger condition is met in three consecutive check periods.

Alert History Statistics