The intelligent inspection feature of Simple Log Service helps you inspect data such as logs and metrics and identify data anomalies in an automated, intelligent, and adaptive manner. This feature uses the stream graph algorithm or the stream decomposition algorithm to inspect data. This topic describes the scenarios and parameters of these algorithms. This topic also provides examples on how to preview the inspection results that are generated by these algorithms.
Stream graph algorithm
The stream graph algorithm is developed based on Time2Graph. This algorithm can reduce noises in data and calculate the offset of each abnormal sample. This algorithm is suitable if you want to inspect a large volume of time series data that includes significant noise and insignificant cyclic changes. For more information, see Time-Series Event Prediction with Evolutionary State Graph.
Description
The stream graph algorithm uses online machine learning methods to analyze each sample and learn from the sample data in real time. You can use this algorithm to identify anomalies in the following types of time series data:
Machine-level metrics, such as CPU utilization, memory usage, and disk read and write speeds
Performance metrics, such as queries per second (QPS), traffic volume, success rate, and latency
Golden metrics.
Parameters
You can configure the parameters of the stream graph algorithm in the Algorithm Configurations step of the Create Intelligent Inspection Job wizard. For more information, see Use consumer groups to pull metric data for intelligent inspection and Use SQL statements to aggregate metric data for intelligent inspection.
The following table describes the parameters.
Parameter | Description |
Time Series Segments | The number of sets into which the time series of the specified metric is discretized. This can help you generate metric charts and reduce the impact of alert notifications. We recommend that you set this parameter to a suitable value based on the preview of the inspection results that are generated.
|
Observation Length | The number of historical samples that you want to inspect.
|
Sensitivity | The sensitivity based on which Simple Log Service generates scores for anomalies.
|
Preview
The following figure shows a sample preview.
Stream decomposition algorithm
The stream decomposition algorithm is developed based on RobustSTL. This algorithm supports batch processing and incurs higher costs than the stream graph algorithm. The stream decomposition algorithm is suitable for scenarios in which you want to inspect a small volume of metric data in a precise manner. If you want to analyze a large volume of data, we recommend that you split the data into batches or use the stream graph algorithm. For more information, see RobustSTL: A Robust Seasonal-Trend Decomposition Algorithm for Long Time Series.
Description
You can use the stream decomposition algorithm to inspect data that includes major cyclic changes. For example, you can use this algorithm to inspect the data of your business metrics.
Common data samples include cyclic changes. For example, the data that is collected for the number of visits to a game or the number of orders that are placed by customers includes cyclic changes.
Parameter configuration
You can configure the parameters of the stream graph algorithm in the Algorithm Configurations step of the Create Intelligent Inspection Job wizard. For more information, see Use consumer groups to pull metric data for intelligent inspection and Use SQL statements to aggregate metric data for intelligent inspection.
The following table describes the parameters.
Parameter | Description |
Time Duration | The number of samples that you want to inspect within an observation cycle. The default observation cycle is one day. For example, if the observation granularity is 120 seconds and the observation cycle is one day, the value of this parameter is calculated based on the following formula: 24 × 60 × 60/120 = 720. Important You must calculate the value of this parameter based on the preceding formula. If you do not calculate the value based on the preceding formula, the inspection results are affected. |
Sensitivity | The sensitivity level based on which Simple Log Service generates scores for anomalies.
|
Preview
By default, Simple Log Service inspects the samples that are generated within the most recent four observation cycles to generate a preview of the inspection results. The following figure shows a sample preview.
If your data includes significant noise and major cyclic changes, you must change the value of the Time Duration parameter until an appropriate preview is displayed. Some anomalies may not be identified or trigger alerts due to noises in data. The following figure shows a sample preview. 