The intelligent inspection feature of Log Service allows you to inspect data such as logs and metrics and identify exceptions in the data in an automated, intelligent, and adaptive manner. This feature uses the stream graph algorithm or the stream decomposition algorithm to inspect data. This topic describes the use scenarios and parameter settings of these algorithms. This topic also provides previews of the inspection results generated by these algorithms.

Stream graph algorithm

The stream graph algorithm is developed based on Time2Graph. This algorithm can reduce data noises and calculate the offset of each abnormal sample. This algorithm is suitable for scenarios in which you want to inspect a large amount of time series data that shows significant noises and insignificant cyclic changes. For more information, see Time-Series Event Prediction with Evolutionary State Graph.

Use scenarios

The stream graph algorithm uses online machine learning technologies to analyze each sample and learn from the sample data in real time. You can use this algorithm to identify exceptions in the following types of time series data:
  • Machine-level metrics, such as CPU utilization, memory usage, and disk read and write speeds
  • Business metrics, such as queries per second (QPS), traffic volume, success rate, and latency
  • Golden metrics

Parameter settings

You can configure the parameters of the stream graph algorithm in the Algorithm Configurations step of the Create Intelligent Inspection Task wizard. For more information, see Create an intelligent inspection task to inspect a metric and Create an intelligent inspection task to inspect log data.

Stream graph algorithm

The following table describes the parameters.

Parameter Description
Time Series Segments The number of segments into which the time series of the specified metric is discretized. The discretization helps you construct metric charts and reduce the impact of alert noises. We recommend that you set this parameter to different values and preview the inspection results that are generated. This way, you can find the most suitable value of this parameter.
  • The default value of this parameter is 8.
  • We recommend that you set this parameter to a value within the range of 5 to 20.
  • A smaller value of this parameter indicates a higher degree of noise reduction and a larger number of missed alerts.
  • A larger value of this parameter indicates a lower degree of noise reduction and a larger number of identified exceptions.
Observation Length The number of historical samples that you want to inspect.
  • We recommend that you set this parameter to a value within the range of 200 to 4000.
  • A larger value of this parameter indicates a larger number of historical samples that need to be inspected, a higher accuracy of exception identification, and higher costs.
  • A smaller value of this parameter indicates a smaller number of historical samples that need to be inspected, a higher impact of noises on exception identification, and lower costs.
Sensitivity The sensitivity based on which Log Service generates scores for exceptions.
  • The valid values of this parameter are Low, Medium, and High.
  • A higher sensitivity indicates that a higher score is required to trigger an alert.
  • Samples whose scores are greater than 0.5 are abnormal. If the score of a sample is greater than 0.75, an alert is triggered.

Preview

The following figure shows an example preview.

Stream graph algorithm

Stream decomposition algorithm

The stream decomposition algorithm is developed based on RobustSTL. This algorithm supports batch processing, but at higher costs than the stream graph algorithm. The stream decomposition algorithm is suitable for scenarios in which you want to precisely inspect a small amount of metric data. If you need to analyze a large amount of data, we recommend that you split the data into batches or use the stream graph algorithm. For more information, see RobustSTL: A Robust Seasonal-Trend Decomposition Algorithm for Long Time Series.

Use scenarios

You can use the stream decomposition algorithm to inspect data that shows prominent cyclic changes. For example, you can use this algorithm to inspect the data of your business metrics.
Note Data that shows cyclic changes includes the number of visits to a game and the number of orders placed by customers.

Parameter settings

You can configure the parameters of the stream decomposition algorithm in the Algorithm Configurations step of the Create Intelligent Inspection Task wizard. For more information, see Create an intelligent inspection task to inspect a metric and Create an intelligent inspection task to inspect log data.

Stream decomposition algorithm

The following table describes the parameters.

Parameter Description
Time Duration The number of samples that you want to inspect within an observation cycle. The default observation cycle is one day. For example, if the observation granularity is 120 seconds and the observation cycle is one day, the value of this parameter is calculated based on the following formula: 24 × 60 × 60/120 = 720.
Notice You must determine the value of this parameter based on the preceding formula. Otherwise, the inspection effect is affected.
Sensitivity The sensitivity based on which Log Service generates scores for exceptions.
  • The valid values of this parameter are Low, Medium, and High.
  • A higher sensitivity indicates that a higher score is required to trigger an alert.
  • Samples whose scores are greater than 0.5 are abnormal. If the score of a sample is greater than 0.75, an alert is triggered.

Preview

By default, Log Service inspects the samples that are generated within the most recent four observation cycles to produce a preview of the inspection results. The following figure shows an example preview.

Stream decomposition algorithm
If your data shows significant noises and prominent cyclic changes, you must adjust the value of the Time Duration parameter until you obtain an expected preview. Some exceptions may be missed or falsely trigger alerts due to data noises. The following figure shows an example preview. Stream decomposition algorithm