After you enable the log collection feature for protected objects in Web Application Firewall (WAF), you can query and analyze the logs of the protected objects. Then, you can generate charts and configure alerts based on the query and analysis results.

Prerequisites

Query and analyze logs

  1. Log on to the WAF 3.0 console.
  2. In the top navigation bar, select the resource group and region to which the WAF instance belongs. You can select the Chinese Mainland or Outside Chinese Mainland region.
  3. In the left-side navigation pane, choose Logs > Log Service.
  4. In the upper part of the Log Reports page, select the protected object whose logs you want to query.
    Notice Make sure that the log collection feature is enabled for the protected object. If the log collection feature is disabled for the protected object, WAF does not collect the logs of the protected object, and you cannot query or analyze the logs of the protected object. To enable the log collection feature for a protected object, turn on Status for the protected object on the Log Reports page. You can also go to the Log Configuration page and turn on the switch in the Log Collection column for the protected object on the Log Collection tab. For more information, see Log collection.
    Log reports
  5. On the Log Query tab, execute a query statement to query and analyze the logs of the protected object that you selected.
    Log query and analysis

    1. Enter a query statement in the search box that is shown in Red Box 1.
      Query statements use the syntax that is specific to Log Service. For more information about the syntax, see Search syntax. The log fields that are included in WAF logs are used as query fields in the query statements. For more information about the log fields that are supported by WAF, see Fields in logs.
      If you are not familiar with the query syntax, we recommend that you use the Advanced Search feature. To use the Advanced Search feature, you need only to expand Advanced Search above the search box, specify search conditions, and then click Search. A query statement is automatically generated in the search box based on the search conditions. Advanced searchThe following table describes the search conditions that are supported by the Advanced Search feature.
      Search condition Description
      IP Address The IP address of the client that sends the request.
      Request ID The unique ID that is generated by WAF for the client request. The request ID is provided when WAF returns a block page or a response page to the client. The response page prompts the client to complete slider CAPTCHA verification. You can use the request ID to analyze and troubleshoot the error.
      Rule ID The ID of the WAF protection rule that matches the request. To query the rule ID, go to the Protection Rules page. To query the rule ID, you can also go to the Security Reports page and view the record of matched rules or statistics. For more information, see Security reports.
      Status Code Returned from Origin Server The HTTP status code that the origin server sends in response to the request from WAF.
      Status Code Returned by WAF The HTTP status code that WAF sends in response to the request from the client.
      Protection Module The WAF protection module that matches the request. For more information about WAF protection modules and how to configure the modules, see Protection configuration overview.
    2. If you want to perform calculation and statistical operations on the query results, append an analytic statement to the search statement in the search box that is shown in Red Box 1. If you do not want to analyze the query results, skip this step.
      Analytic statements and search statements are separated by vertical bars (|). Analytic statements use the standard SQL-92 syntax. For more information about analytic statements, see Log analysis overview.
    3. Specify the time range to query logs by using the time picker that is shown in Red Box 2.
    4. Click Search & Analyze that is shown in Red Box 3.
      In the lower part of the page, you can view the query and analysis results in a log distribution histogram and on the Raw Logs, Graph, and LogReduce tabs. You can perform various operations based on the query and analysis results. For example, you can perform quick analysis, generate charts, and configure alerts. For more information, see Description of query and analysis results.

Description of query and analysis results

Log Service displays query and analysis results in a log distribution histogram, on the Raw Logs tab, and on the Graph tab. Log Service allows you to perform operations on the results. For example, you can configure alerts and create saved searches.
Note When you execute a query statement, only 100 lines of data is returned by default. You can use a LIMIT clause to specify the number of lines that can be returned. For more information, see LIMIT clause.
  • Log distribution histogram
    The log distribution histogram shows the distribution of returned logs in different periods of time. Log distribution histogram
    • When you move the pointer over a green rectangle, you can view the period of time that is represented by the rectangle and the number of returned logs within the period.
    • If you click a green rectangle, you can view log distribution at a finer-grained level. In addition, you can view the returned logs within the period of time on the Raw Logs tab.
  • Raw Logs tab
    The Raw Logs tab displays the logs that are queried. You can click the Table or Raw Data tab to view the logs and perform the following operations: Raw Logs tab
    • Quick Analysis: You can analyze the distribution of a field within a period of time. For more information, see Quick analysis.
      You can click the Alias icon to specify whether to show the names or aliases of fields. You can create aliases when you configure indexes. For example, if the alias of host_name is host, host is displayed in the Quick Analysis list after you select Show Field Aliases.
      Note If a field does not have an alias, the name of the field is displayed in the Quick Analysis list even if you select Show Field Aliases.
    • Context query: On the Raw Data tab, you can find a log and click the Query Logs - 004 icon to query the context information about the log in the raw log file. For more information, see Context query.
      Note You can perform context query only on the logs that are collected by Logtail.
    • LiveTail: On the Raw Data tab, you can find a log and click the LiveTail icon to monitor logs in real time and extract important information from the logs. For more information, see LiveTail.
      Note You can use LiveTail only on the logs that are collected by Logtail.
    • Tag Configurations: On the Raw Data tab, you can click the Settings icon and select Tag Configurations to hide less important fields. Tag
    • Column Settings: On the Table tab, you can click the Settings icon and select Column Settings to specify the columns that you want to display in the table. The column names are field names, and the column content is field values. Column Settings
    • JSON Configurations: On the Table or Raw Data tab, you can click the Settings icon and select JSON Configurations to specify the level for JSON expansion.
    • Event Settings: On the Table or Raw Data tab, you can click the Settings icon and select Event Settings to configure events for raw logs. For more information, see Configure events.
    • Log Download: On the Table or Raw Data tab, you can click the Log Download icon to download logs. You can specify the tool that is used to download logs and the range of logs to download. For more information, see Download logs.
  • Graph tab
    After you execute a query statement, you can view the query and analysis results on the Graph tab.
    • View query and analysis results: Log Service renders the results of the query statement to charts. Log Service provides various types of charts, such as tables, line charts, and column charts. For more information, see Chart overview.
    • Add a chart to a dashboard: Log Service provides dashboards on which you can analyze data in real time. You can click Add to New Dashboard to save the query and analysis results as a chart to a dashboard. For more information, see Visualization overview.
    • Configure interactive events: Interactive events are important for data analysis. You can use interactive events to switch between the levels of data dimensions and the analysis granularities to obtain more detailed information. Interactive events include events to open a Logstore, open quick analysis, open a dashboard, open trace analysis, open trace details, and customize an HTTP link. For more information, see Configure a drill-down event.
  • LogReduce tab

    On the LogReduce tab, you can click Enable LogReduce to cluster similar logs during log collection. For more information, see LogReduce.

  • Alerting

    On the query and analysis page, you can choose Save as Alert > New Alert to configure alerts based on the query and analysis results. For more information, see Configure an alert in Log Service.

  • Saved search

    On the query and analysis page, you can click Save Search to save a query statement as a saved search. For more information, see Saved search.