This topic describes how to query and analyze JSON logs in the Log Service console. In the following example, JSON website logs are used.

Prerequisites

JSON logs are collected. For more information, see Collect logs in simple mode.

Usage notes

When you query and analyze JSON logs, take note of the following information:
  • A query statement is in the Search statement|Analytic statement format. In an analytic statement, you must enclose a field name in double quotation marks ("") and enclose a string in single quotation marks ('').
  • You must add all parent paths to a field that you want to use in the KEY1.KEY2.KEY3 format. Example: content.request.request_length.
  • Log Service allows you to query and analyze leaf nodes in JSON objects. However, you cannot query or analyze child nodes that contain leaf nodes.
  • You cannot query or analyze fields whose values are JSON arrays. In addition, you cannot query or analyze the fields in a JSON array.

Step 1: Configure indexes

  1. Log on to the Log Service console.
  2. In the Projects section, click the project that you want to view.
  3. Choose Log Storage > Logstores. On the Logstores tab, click the Logstore that you want to view.
  4. In the upper-right corner of the Search & Analysis page of the Logstore, choose Index Attributes > Attributes.
    If the indexing feature is not enabled, click Enable.
  5. Configure field indexes.
    You can configure one index at a time until all indexes are configured. You can also click Automatic Index Generation. If you click Automatic Index Generation, Log Service automatically configures indexes based on the first log in the previewed data.
    Important
    • If you want to use the analysis feature, you must turn on Enable Analytics for the fields that you want to analyze when you configure indexes. For more information, see Create indexes.
    • Indexes are automatically configured for some reserved fields in Log Service. For more information, see Reserved fields.
    • Log Service allows you to configure indexes for leaf nodes in JSON objects. However, you cannot configure indexes for child nodes that contain leaf nodes. For example, you can configure an index for the request_time field, but you cannot configure an index for the time field.
    • You cannot configure indexes for fields whose values are JSON arrays. In addition, you cannot configure indexes for the fields in a JSON array. For example, the value of the body_bytes_sent field is a JSON array. In this case, you cannot configure an index for the field.
    • When you configure indexes for a field in a JSON object, you must add the parent path to the field in the KEY1.KEY2 format. Example: time.request_time.
    Configure indexes
  6. Click OK.
    Important After you configure indexes, the indexing feature takes effect only for newly collected data. If you want to query historical data, you must use the reindexing feature. For more information, see Reindex logs for a Logstore.

Step 2: Query logs

On the search and analysis page of a Logstore, enter a search statement in the search box, and specify a time range to query logs.

  • Query the logs of requests for which status code 200 is returned. 
    content.status:200
  • Query the logs of requests whose length is greater than 70. 
    content.request.request_length > 70
  • Query the logs of GET requests. 
    content.request.request_method:GET

Step 3: Analyze logs

On the search and analysis page of a Logstore, enter a query statement in the search box, and specify a time range to analyze logs.

  • Calculate the number of logs of requests by status code. 
    * | SELECT "content.status", COUNT(*) AS PV GROUP BY "content.status"
    PV
  • Calculate the number of requests by request duration and sort analysis results by request duration in ascending order. 
    * | SELECT "content.time.request_time", COUNT(*) AS count GROUP BY "content.time.request_time" ORDER BY "content.time.request_time"
    Request duration
  • Calculate the average request duration by request method. 
    * | SELECT avg("content.time.request_time") AS avg_time,"content.request.request_method"  GROUP BY "content.request.request_method"
    Average request duration

Sample log

The following figure shows a sample JSON log.

Sample log