All Products
Search
Document Center

Simple Log Service:Scan logs

Last Updated:Jan 18, 2024

Simple Log Service provides the scan feature that scans logs based on specified fields to return query and analysis results. This feature allows you to query and analyze logs without the need to configure indexes for the logs. The scan feature includes scan-based query and scan-based analysis. This topic describes how to use the scan feature and provides examples.

Prerequisites

  • Logs are collected. For more information, see Data collection overview.

  • Indexes are created. For more information, see Create indexes.

    Important

    You do not need to create indexes for the fields that are used for scanning. However, search statements still rely on indexes.

Procedure

  1. Log on to the Simple Log Service console.

  2. In the Projects section, click the project that you want to manage.

  3. On the Log Storage > Logstores tab, click the Logstore that you want to manage.

  4. Click the 扫描 icon to enable the scan feature.

    Note

    You do not need to enable the scan feature for the regions in which the Simple Log Service Processing Language (SPL) syntax is supported. For more information about the regions in which the SPL syntax is supported, see Supported regions.

    扫描

  5. In the query statement field, enter a scan statement.

    Enter a scan statement in one of the following formats: Search statement | SPL statement or Search statement | set session mode=scan; Analytic statement based on the standard SQL syntax.

  6. Click 15 Minutes(Relative) and specify a time range for the query.

    You can select a relative time, select a time frame, or specify a custom time range. The time range that you can specify supports minute-precision at most.

Automatic scanning

Important

Automatic scanning and page turning are supported only for scan-based query.

If you perform a scan-based query and the system returns a message that indicates no data for the scan, you can click the 扫描 icon to trigger a new automatic scan until the system returns logs that meet the scan condition or the number of scans reaches 20.

扫描日志

In the Automatic Scanning dialog box, you can view the progress of each automatic scan.

自动扫描

Page turning

When you perform a page turn, Simple Log Service performs a scan-based query to ensure the continuity of scan results.

By default, Simple Log Service can scan up to 100,000 logs at a time. During page turning, the number of logs that are displayed on a page may be less than the number that is specified by the Items per page parameter. However, you can still move forward one page. In this case, the number of logs that meet the scan condition among the 100,000 logs that are scanned is less than the number that is specified by the Items per page parameter.

For example, the total number of logs is 200,000, and the number that is specified by the Items per page parameter is 20. After a scan is complete, only 15 logs are returned, and you can move forward one page. In this case, only 15 logs among the first 100,000 logs that are scanned meet the scan condition. You can perform a page turn. Then, Simple Log Service scans the remaining 100,000 logs and returns the logs that meet the scan condition.

翻页

Examples

  • Use the scan feature to query the logs of requests whose duration exceeds 100 milliseconds.

    • Scan statement

       * | where cast(request_time as bigint) >= 100
    • Scan results 扫描结果

  • Use the scan feature to query the logs of requests that are initiated by the S3FDy2GigffB0ceG user.

    • Scan statement

      *| WHERE id='S3FDy2GigffB0ceG'
    • Scan results 扫描

  • Use the scan feature to calculate the numbers of requests by request method.

    • Scan statement

      Note

      When you use the scan feature, you must add set session mode=scan; before the SQL statement.

      * | set session mode=scan; SELECT request_method, count(*) AS PV GROUP BY request_method
    • Scan results 扫描分析

API calls

You can call the GetLogs operation to use the scan feature. For more information, see GetLogs.

Important

If you want to use the scan feature, you must add set session mode=scan; before the analytic statement in the query parameter. Example: * | set session mode=scan; select count(*) as pv.