All Products
Search
Document Center

Web Application Firewall:Query logs

Last Updated:Oct 20, 2023

After you enable the log collection feature for the protected objects of Web Application Firewall (WAF), you can query and analyze the logs of the protected objects. Then, you can generate charts and configure alerts based on the query and analysis results.

Prerequisites

Query and analyze logs

  1. Log on to the WAF 3.0 console. In the top navigation bar, select the resource group and the region in which the WAF instance is deployed. You can select Chinese Mainland or Outside Chinese Mainland.

  2. In the left-side navigation pane, choose Security Operations > Log Service.

  3. In the upper part of the Log Service page, select the protected object whose logs you want to query.

    Important

    Make sure that the log collection feature is enabled for the protected object. If the log collection feature is disabled for the protected object, WAF does not collect the logs of the protected object, and you cannot query or analyze the logs of the protected object. To enable the log collection feature for a protected object, turn on Status for the protected object on the Log Service page. You can also go to the Log Configuration page and turn on the switch in the Log Collection column for the protected object on the Log Collection tab. For more information, see Configure log settings and manage log storage capacity.

    日志报表
  4. On the Log Query tab, enter a query statement. Then, execute the statement to query and analyze the logs of the protected object that you selected.

    日志查询与分析

    To query and analyze the logs of the protected object, perform the following steps:

    1. Enter a query statement in the search box that is labeled 1 in the preceding figure.

      The query statement that you enter must use the syntax that is specific to Alibaba Cloud Simple Log Service. For more information about the syntax, see Search syntax. The log fields that are included in WAF logs are used as query fields in the query statements. For more information about the log fields that are supported by WAF, see Fields in logs.

      If you are not familiar with the query syntax, we recommend that you use the Advanced Search feature. To use the Advanced Search feature, you need to only expand Advanced Search above the search box, specify search conditions, and then click Search. A query statement is automatically generated in the search box based on the search conditions. 高级搜索The following table describes the search conditions that are supported by the Advanced Search feature.

      Search condition

      Description

      IP

      The IP address of the client that sends the request.

      Request ID

      The unique ID that is generated by WAF for the client request. The request ID is provided when WAF returns a block page or a response page to the client. The response page prompts the client to complete slider CAPTCHA verification. You can use the request ID to analyze and troubleshoot the error.

      Rule ID

      The ID of the WAF protection rule that matches the request. To query the rule ID, go to the Protection Rules page. To query the rule ID, you can also go to the Security Reports page and view the record of matched rules or statistics. For more information, see Security reports.

      Status Code Returned from Origin Server

      The HTTP status code that is sent by the origin server in response to the request from WAF.

      Status Code Returned by WAF

      The HTTP status code that WAF sends in response to the request from the client.

      Protection Module

      The WAF protection module that matches the request. For more information about WAF protection modules and how to configure the modules, see Protection configuration overview.

    2. If you want to perform calculation and statistical operations on the query results, append an analytic statement to the search statement in the search box that is labeled 1 in the preceding figure. If you do not want to analyze the query results, skip this step.

      Separate analytic statements and search statements with vertical bars (|). Analytic statements must use the standard SQL-92 syntax. For more information about analytic statements, see Log analysis overview.

    3. Specify the time range to query logs by using the time picker that is labeled 2 in the preceding figure.

    4. Click Search & Analyze that is labeled 3 in the preceding figure.

      In the lower part of the page, you can view the query and analysis results in a log distribution histogram and on the Raw Logs, Graph, and LogReduce tabs. You can perform various operations based on the query and analysis results. For example, you can perform quick analysis, generate charts, and configure alerts. For more information, see Description of query and analysis results.

Description of query and analysis results

You can view the query and analysis results in a log distribution histogram, on the Raw Logs tab, or by using a chart. You can also configure alert rules and saved searches.

Important

By default, only 100 rows of data are returned after you execute a query statement. You can use a LIMIT clause to change the number of returned rows. For more information, see LIMIT clause.

  • Query result histogram

    The query result histogram displays the distribution of query and analysis results in different time ranges. Query result histogram

    • If you move the pointer over a green rectangle, you can view the time range that is represented by the rectangle and the number of logs that are obtained within the time range.

    • If you double-click the green rectangle, you can view a more fine-grained log distribution. You can also view the query and analysis results on the Raw Logs tab.

  • Raw Logs tab

    On the Raw Logs tab, you can view the logs that match your search conditions. You can click Table or Raw Data to view the logs and perform the following operations: Raw Logs tab

    • Quick analysis: You can use this feature to analyze the distribution of a specific field within a specific period of time. For more information, see Quick analysis.

      You can click the Alias icon to specify whether to show the names or aliases of fields. You can specify aliases when you create indexes. For example, if you set the alias of the host_name field to host, host is displayed in the Quick Analysis list after you select Show Field Aliases.

      Note

      If you do not specify an alias for a field, the field name is displayed after you select Show Field Aliases.

    • Contextual query: On the Raw Data tab, you can find a log and click the Query logs - 004 icon to view the context of the log in the raw log file. For more information, see Contextual query.

      Important

      The contextual query feature is supported only for log data that is collected by Logtail.

    • LiveTail: On the Raw Data tab, you can find a log and click the LiveTail icon to monitor logs in real time and extract important information from the logs. For more information, see LiveTail.

      Important

      The LiveTail feature is supported only for log data that is collected by Logtail.

    • Tag configurations: On the Raw Data tab, you can click the Settings icon and select Tag Configurations to hide fields that are less important. Tag

    • Column settings: On the Table tab, you can click the Settings icon and select Column Settings to specify the columns that you want to display in the table. The column names are field names, and the column content is used as field values. Column settings

    • JSON configurations: On the Table or Raw Data tab, you can click the Settings icon and select JSON Configurations to specify the level for JSON expansion.

    • Event settings: On the Table or Raw Data tab, you can click the Settings icon and select Event Settings to configure events for raw logs. For more information, see Configure events.

    • Log download: On the Table or Raw Data tab, you can click the Download Log icon to download logs. You can specify the method that is used to download logs and the range of logs to download. For more information, see Download logs.

  • Graph tab

    After you execute a query statement, you can view the query and analysis results on the Graph tab.

    • View query and analysis results: Simple Log Service renders the results of query statements into charts. Simple Log Service provides multiple chart types, such as tables, line charts, and bar charts. Simple Log Service provides the following versions of charts: Pro and Standard. For more information, see Overview of charts (Pro) and Chart overview.

    • Simple Log Service allows you to create dashboards to perform real-time data analysis. You can click Add to New Dashboard to save query and analysis results as charts to a dashboard. For more information, see Overview of visualization.

    • Configure interaction occurrences: Interaction occurrences are important for data analysis. You can use interaction occurrences to switch between the levels of data dimensions and the analysis granularities to obtain more detailed information. For more information, see Drill-down events.

  • LogReduce tab

    On the LogReduce tab, you can click Enable LogReduce to cluster similar logs. For more information, see LogReduce.

  • Alerting

    On the Search & Analysis page, you can click the Save as Alert icon to configure an alert rule based on the query and analysis results. For more information, see Configure an alert monitoring rule in Log Service.

  • Saved searches

    On the Search & Analysis page, you can click the Save Search icon to save a query statement as a saved search. For more information, see Saved search.