The intelligent inspection feature inspects service data and identifies anomalies in an automated, intelligent, and adaptive manner. This topic describes how to use SQL statements to aggregate metrics for intelligent inspection.
Prerequisites
Data is collected and stored in a Logstore, which is referred to as the source Logstore. For more information, see Data collection overview.
Indexes are configured for the source Logstore. For more information, see Create indexes.
An Intelligent Anomaly Analysis instance is created. For more information, see Create an instance.
Step 1: Create an intelligent inspection job
Log on to the Simple Log Service console.
Go to the Create Job page.
In the Log Application section, click Intelligent Anomaly Analysis.
In the instance list, click the ID of the instance that you want to manage.
In the left-side navigation pane, click Intelligent Inspection.
In the Inspection Job section, click Create Now.
In the Basic Information step of the Create Intelligent Inspection Job wizard, configure the following parameters and click Next.
Parameter
Description
Job Name
The name of the intelligent inspection job. You can enter a custom name.
Project
The project to which the source Logstore or Metricstore belongs.
Region
The region where the project resides.
Logstore Type
The storage unit in which your data is stored.
If your data is stored in a Logstore, select Logstores.
If your data is stored in a Metricstore, select Metricstores.
Source Logstore
If you set Logstore Type to Logstores, you must set Source Logstore to the Logstore in which your source data is stored.
Metricstores
If you set Logstore Type to Metricstores, you must set Metricstores to the Metricstore in which your source data is stored.
Role
The Alibaba Cloud Resource Name (ARN) of AliyunLogETLRole. If you have completed authorization when you create the instance, the ARN is automatically displayed.
Target Store
The destination Logstore. The value is fixed as internal-ml-log.
In the Data Feature Settings step of the Create Intelligent Inspection Job wizard, set Data Type to SQL Aggregation, enter a query statement, and then configure the following parameters.
The following example shows a query statement. For more information, see Log search overview and Log analysis overview.
* | select __time__ - __time__ % 60 as time, eip, avg(inpps) as inpps, avg(outpps) as outpps from log group by time, eip order by time limit 10000
Parameter
Description
Time
The field that specifies time in source data. By default, Simple Log Service uses the
__time__
field in the source Logstore.Granularity
The interval of data observation. Unit: seconds. Valid values: 5 to 3600.
Entity
The field that specifies an entity in source data. The intelligent inspection job aggregates data to generate time series for the entity.
Feature
The field that specifies a feature in source data. How do I configure the Minimum Value and Maximum Value parameters for data features?
In the Algorithm Configurations step of the Create Intelligent Inspection Job wizard, perform the following operations:
In the Algorithm Configurations section, configure the following parameters. Then, in the Data Sampling section, select an entity and click Sample Data Preview to check whether the parameter settings are suitable for the source data and whether expected results can be obtained.
Parameter
Description
Algorithm
The algorithm that is used to identify anomalies. Default value: Stream Graph Algorithm. For more information, see Algorithms.
Time Series Segments
The number of segments into which the time series of the specified metric is discretized. The discretization helps you construct metric charts and reduce the impact of alert notifications.
Default value: 8.
We recommend that you set this parameter to a value within the range of 5 to 20.
The sensitivity of alert notifications linearly decreases with the value of this parameter.
Observation Length
The number of historical samples that you want to inspect.
Default value: 2880.
We recommend that you set this parameter to a value within the range of 200 to 4000.
We recommend that you configure this parameter based on the number of samples that you want to inspect within two observation cycles. For example, if the observation granularity is 1 minute and the observation cycle is 1 day, Simple Log Service can inspect 2,880 samples for the metric within two days. We recommend that you set this parameter to a value that is greater than or equal to 2880.
Sensitivity
The sensitivity level based on which Simple Log Service generates scores for anomalies.
Samples whose scores are greater than 0.5 are abnormal. If the score of a sample is greater than 0.75, an alert is triggered.
A higher sensitivity level indicates that a higher score is required to trigger an alert.
In the Scheduling Settings section, configure the following parameters.
Parameter
Description
Start At
The scheduled start time.
NoteAfter you create an intelligent inspection job, the job starts at the date and point in time that you specify.
Data Latency
The number of seconds for which the job is delayed after the scheduled start time. Valid values: 0 to 120. Unit: seconds.
You can configure this parameter to ensure data integrity if data is written to a Logstore or a Metricstore with a latency.
Click Next.
In the Alert Configuration step of the Create Intelligent Inspection Job, configure the following parameters and click Complete.
Parameter
Description
Alert Policy
Alert policies are used to merge, silence, and suppress alerts.
If you set Alert Policy to Simple Mode or Standard Mode, you do not need to configure an alert policy. By default, Simple Log Service uses the built-in alert policy sls.builtin.dynamic to manage alerts.
If you set Alert Policy to Advanced Mode, you can select a built-in or custom alert policy to manage alerts. For more information about how to create an alert policy, see Create an alert policy.
Action Policy
Action policies are used to manage alert notification methods and the frequency at which alert notifications are sent.
If you set Alert Policy to Simple Mode, you need to only configure an action group for this parameter.
After you configure an action group, Simple Log Service automatically creates an action policy named
Rule name-Action policy
. Alert notifications are sent based on the action policy for all alerts that are triggered based on your alert rule. For more information, see Notification methods.ImportantYou can modify an action policy on the Action Policy tab. For more information, see Create an action policy. If you add conditions when you modify an action policy, the value of Alert Policy automatically changes to Standard Mode.
If you set Alert Policy to Standard Mode or Advanced Mode, you can select a built-in or custom action policy to send alert notifications. For more information about how to create an action policy, see Create an action policy.
If you set Alert Policy to Advanced Mode, you can turn on or turn off Custom Action Policy. For more information, see Dynamic action policy mechanism.
Inspection Results
After you create an intelligent inspection job, you can click the job in the job list to view details.
Related operations
After you create an intelligent inspection job, you can modify or delete the job on the Intelligent Inspection page.
After you delete an intelligent inspection job, the job cannot be restored. Proceed with caution.