All Products
Search

This Product

    Log Service:Query and analyze logs

    Document Center

    Log Service:Query and analyze logs

    Last Updated:Aug 03, 2023

    After you enable the indexing feature and create indexes for a Logstore, you can query and analyze the logs that are stored in the Logstore in real time.

    Prerequisites

    • Logs are collected. For more information, see Data collection overview.

      Important Before you can analyze logs, you must store the logs in a Standard Logstore. For more information, see Manage a Logstore.

    • Indexes are created. For more information, see Create indexes.

      If you want to analyze logs, you must create field indexes and turn on Enable Analytics.

    Procedure

    1. Log on to the Log Service console.
    2. In the Projects section, click the project that you want to manage.
    3. On the Log Storage > Logstores tab, click the Logstore that you want to manage.

    4. Enter a query statement in the search box.

      A query statement consists of a search statement and an analytic statement in the Search statement|Analytic statement format. For more information, see Search syntax and SQL syntax and functions.

    5. Click 15 Minutes(Relative) to specify a time range for the query.

      You can also specify a closed time interval in an analytic statement by using the __time__ field. Example: * | SELECT * FROM log WHERE __time__>1558013658 AND __time__< 1558013660. The time range that you specify by using one of the preceding methods is accurate to minutes. The query and analysis results may contain logs that are generated 1 minute earlier or later than the specified time range.

      Important

      • To ensure that full data can be queried, we recommend that you specify a query time range that is accurate to minutes.

      • If you specify a time range in an analytic statement, the time range is used for query and analysis.

      If you want to specify a time range that is accurate to seconds in an analytic statement, you must use the from_unixtime function or to_unixtime function to convert the time format. Examples:

      • * | SELECT * FROM log WHERE from_unixtime(__time__) > from_unixtime(1664186624) AND from_unixtime(__time__) < now()

      • * | SELECT * FROM log WHERE __time__ > to_unixtime(date_parse('2022-10-19 15:46:05', '%Y-%m-%d %H:%i:%s')) AND __time__ < to_unixtime(now())

    Manage query and analysis results

    You can view the query and analysis results in a log distribution histogram, on the Raw Logs tab, or by using a chart. You can also configure alert rules and saved searches.

    Important

    By default, only 100 rows of data are returned after you execute a query statement. You can use a LIMIT clause to change the number of returned rows. For more information, see LIMIT clause.

    • Query result histogram

      The query result histogram displays the distribution of query and analysis results in different time ranges. Query result histogram

      • If you move the pointer over a green rectangle, you can view the time range that is represented by the rectangle and the number of logs that are obtained within the time range.

      • If you double-click the green rectangle, you can view a more fine-grained log distribution. You can also view the query and analysis results on the Raw Logs tab.

    • Raw Logs tab

      On the Raw Logs tab, you can view the logs that match your search conditions. You can click Table or Raw Data to view the logs and perform the following operations: Raw Logs tab

      • Quick analysis: You can use this feature to analyze the distribution of a specific field within a specific period of time. For more information, see Quick analysis.

        You can click the Alias icon to specify whether to show the names or aliases of fields. You can specify aliases when you create indexes. For example, if you set the alias of the host_name field to host, host is displayed in the Quick Analysis list after you select Show Field Aliases.

        Note

        If you do not specify an alias for a field, the field name is displayed after you select Show Field Aliases.

      • Contextual query: On the Raw Data tab, you can find a log and click the Query logs - 004 icon to view the context of the log in the raw log file. For more information, see Contextual query.

        Important

        The contextual query feature is supported only for log data that is collected by Logtail.

      • LiveTail: On the Raw Data tab, you can find a log and click the LiveTail icon to monitor logs in real time and extract important information from the logs. For more information, see LiveTail.

        Important

        The LiveTail feature is supported only for log data that is collected by Logtail.

      • Tag configurations: On the Raw Data tab, you can click the Settings icon and select Tag Configurations to hide fields that are less important. Tag

      • Column settings: On the Table tab, you can click the Settings icon and select Column Settings to specify the columns that you want to display in the table. The column names are field names, and the column content is used as field values. Column settings

      • JSON configurations: On the Table or Raw Data tab, you can click the Settings icon and select JSON Configurations to specify the level for JSON expansion.

      • Event settings: On the Table or Raw Data tab, you can click the Settings icon and select Event Settings to configure events for raw logs. For more information, see Configure events.

      • Log download: On the Table or Raw Data tab, you can click the Download Log icon to download logs. You can specify the method that is used to download logs and the range of logs to download. For more information, see Download logs.

    • Graph tab

      After you execute a query statement, you can view the query and analysis results on the Graph tab.

      • View query and analysis results: Simple Log Service renders the results of query statements into charts. Simple Log Service provides multiple chart types, such as tables, line charts, and bar charts. Simple Log Service provides the following versions of charts: Pro and Standard. For more information, see Overview of charts (Pro) and Chart overview.

      • Simple Log Service allows you to create dashboards to perform real-time data analysis. You can click Add to New Dashboard to save query and analysis results as charts to a dashboard. For more information, see Overview of visualization.

      • Configure interaction occurrences: Interaction occurrences are important for data analysis. You can use interaction occurrences to switch between the levels of data dimensions and the analysis granularities to obtain more detailed information. For more information, see Drill-down events.

    • LogReduce tab

      On the LogReduce tab, you can click Enable LogReduce to cluster similar logs. For more information, see LogReduce.

    • Alerting

      On the Search & Analysis page, you can click the Save as Alert icon to configure an alert rule based on the query and analysis results. For more information, see Configure an alert monitoring rule in Log Service.

    • Saved searches

      On the Search & Analysis page, you can click the Save Search icon to save a query statement as a saved search. For more information, see Saved search.