The intelligent inspection feature of Log Service allows you to inspect log data and identify exceptions in the log data in an automated, intelligent, and adaptive manner. This topic walks you through the process of creating an intelligent inspection task that monitors the network access logs of an enterprise and generates alerts in the event of exceptions.

Prerequisites

Procedure

  1. Log on to the Log Service console.
  2. In the Projects section, click the project in which you want to create an intelligent inspection task.
  3. In the left navigation sidebar, choose Jobs > Intelligent Inspection.
  4. Associate AliyunLogETLRole with Log Service. You need to perform this step only if this is the first time that you create an intelligent inspection task.
    Note
    • Log Service writes inspection events to a Logstore named internal-ml-log. Log Service can write inspection events to the internal-ml-log Logstore only after you associate AliyunLogETLRole with Log Service.
    • You must associate AliyunLogETLRole with Log Service by using the Alibaba Cloud account to which the project belongs.
  5. In the Intelligent Inspection pane, click the plus icon.
  6. In the Basic Information step of the Create Intelligent Inspection Task wizard, configure the following parameters and click Next.
    Parameter Description
    Task Name The name of the intelligent inspection task.
    Source Logstore The name of the Logstore that is used to store the source data.
  7. In the Algorithm Configurations step of the Create Intelligent Inspection Task wizard, complete the following operations:
    1. In the Data Feature Settings section, set the Data Type parameter to Non-indexed Data, enter the SQL query statement that is used to aggregate the time series data of the specified entity, and then configure the following parameters.
      sql
      Parameter Description
      Time The field that is used to identify the time information in the source data.
      Granularity The time interval at which Log Service observes the metric you specify. Unit: seconds. Valid values: 5 to 3600.
      Entity The field that is used to identify the entity you want to observe. The intelligent inspection task produces the time series data of the entity based on the specified identifier field.
      Feature The metric that you want to observe. You can specify a minimum value and a maximum value to dictate the value range of the metric. If you are not sure about the value range of the metric, you can leave the value range unspecified.
    2. In the Algorithm Configurations section, configure the following parameters, select an entity from the Data Sampling drop-down list, and then click Sample Data Preview to check whether the parameter settings are suitable for the source data and whether expected results can be obtained.
      algorithm_log
      Parameter Description
      Algorithm The algorithm that is used to detect exceptions. Default value: Stream Graph Algorithm.
      Time Series Segments The number of segments into which the time series of the specified metric is discretized. The discretization helps you construct metric charts and reduce alert noises.
      • This parameter defaults to 8.
      • We recommend that you set this parameter to a value within the range of 5 to 20.
      • A smaller value of this parameter indicates less sensitivity to alert noises.
      Observation Length The number of historical samples that you want to observe.
      • The default value of this parameter is 2880.
      • We recommend that you set this parameter to a value within the range of 200 to 4000.
      • We recommend that you set this parameter based on the number of samples observed within two observation cycles. For example, if the observation granularity is 1 minute and the observation cycle is 1 day, the specified metric has 2880 samples within two observation cycles. In this case, we recommend that you set this parameter to a value that is greater than or equal to 2880.
      Sensitivity The sensitivity based on which Log Service generates scores for exceptions.
      • Samples whose scores are greater than 0.5 are abnormal. If the score of a sample is greater than 0.75, an alert is triggered.
      • A higher sensitivity indicates that a higher score is required to trigger an alert.
    3. In the Scheduling Settings section, specify the date and time at which you want to start the intelligent inspection task.
      Note After an intelligent inspection task is created, the task starts at the date and time that you specify.
    4. Click Next.
  8. In the Alert Configuration step of the Create Intelligent Inspection Task wizard, configure the following parameters and click Complete.
    Parameter Description
    Alert Policy The policy that is used to merge, silence, and denoise alerts.
    • If you select Simple Mode or Standard Mode, you do not need to configure an alert policy. By default, Log Service uses the sls.builtin.dynamic alert policy.
    • If you select Advanced Mode, you can select a built-in alert policy or a custom alert policy. For more information about how to create an alert policy, see Create an alert policy.
    Action Policy The policy that is used to manage the notification channel and the frequency at which Log Service sends alerts.
    • If you set the Alert Policy parameter to Simple Mode, you need only to configure an action group.
      After you configure an action group, Log Service creates an action policy named Rule name-Action policy. Log Service uses the action policy to send all alerts that are triggered based on the specified alert rule. For more information, see Notification methods.
      Notice You can modify the action policy on the Action Policy tab. For more information, see Create an action policy. If you add evaluation criteria when you modify the action policy, the value of the Alert Policy parameter is automatically changed to Standard Mode.
    • If you set the Alert Policy parameter to Standard Mode or Advanced Mode, you can select a built-in action policy or a custom action policy. For information about how to create an action policy, see Create an action policy.

      If you set the Alert Policy parameter to Advanced Mode, you can enable or disable Custom Action Policy. For more information, see Dynamic action policy mechanism.

    Cycle The cycle based on which Log Service observes the specified metric. Log Service triggers the action policy only once and sends only one alert even if duplicate alerts are triggered within the specified observation cycle.

Inspection results

After an intelligent inspection task is created, you can view the task on the Intelligent Inspection page.

basicinfo
After a period of time, you can view the inspection results that are generated by the task on the Intelligent Inspection page. result

What to do next

After an intelligent inspection task is created, you can modify or delete the task on the Intelligent Inspection page.

Notice An intelligent inspection task that is deleted cannot be recovered. Proceed with caution when you delete an intelligent inspection task.