To help you efficiently query logs, Log Service provides a set of query syntax for setting query conditions.

Query methods

Enable and set indexes Then, enter a query analysis statement on the log query page to query logs.

Log query statements are the first half of query analysis statements. Log query statements are used to specify the filtering rules for log queries and return the log data that meets the condition. Log query statements support full-text queries and field-specific queries.
  • Full-text queries

    In full-text queries, a log entry is considered as a special key-value pair, in which the value indicates the entire log content. You can specify keywords for a full-text query. Specifically, you can specify the keywords which must be included in or excluded from the query condition. The log data that meets the specified query condition is returned.

    In addition to common full-text queries, Log Service also supports phrase queries and fuzzy queries.
    • Common full-text queries: You need to specify a keyword and rule. The log data that contains the keyword and conforms to the rule is returned.

      Example: a and bindicates that the query results must contain the keywords a and b.

    • Phrase queries: If the target phrase contains a space, you can enclose the phrase with a pair of double quotation marks (""). In this case, the phrase is regarded as a complete keyword for a log query.

      Example: "http error" indicates that the query results must contain the http error keyword.

    • Fuzzy queries: You can specify a keyword that is a maximum of 64 characters in length, and use an asterisk (*) or a question mark (?) at the end or in the middle of the keyword. Log Service searches for a maximum of 100 words that meet the specified query condition and returns the log entries that contain these words.

      Example: addr? indicates that Log Service searches for a maximum of 100 words that start with addr and returns the log entries that contain these words.

  • Field-specific queries

    After configuring indexes for fields, you can specify the name or value of a specific field to query log data. For fields of the Double or Long data type, you can also specify the value range for queries. Example: Latency>5000 and Method:Get* and not Status:200 indicates the log entries that meet the following conditions are queried: The value of the Latency field is more than 5000, the value of the Method field starts with Get, and the value of the Status field value is not 200.

    You can perform a single query or use operators to perform combined queries based on the data types specified for field indexes. For more information, see Indexes and data types.

Precautions

  • If full-text queries and field-specific queries are both configured, and the delimiters specified for the two query methods are different, the delimiter specified for field-specific queries is used. In this case, no data can be found by using full-text queries.
  • You can only query fields with a specified value range after specifying the data type of the fields to DOUBLE or LONG. If the field data type is not specified or the syntax for querying value ranges is incorrect, Log Service determines that the query condition takes effect for full-text queries. In this case, unexpected query results may be returned.
  • If you change the field data type from TEXT to NUMERIC, only the = operator can be used for the data collected before the change.

Operators

The following table lists the operators supported by log query statements.
Operator Description
and Binary operator. The format is query1 and query2, indicating the intersection of the query results of query1 and query2. If no syntax keyword exists among words, the relation among words is and by default.
or Binary operator. The format is query1 or query2, indicating the union of the query results of query1 and query2.
not Binary operator. The format is query1 not query2, indicating a result that meets query1 and does not meet query2. This format is equivalent to query1–query2. not query1 indicates that the log data that does not contain the query results of query1 is returned.
( , ) The operator used to merge one or more sub-query conditions into one query condition. You must enclose the sub-query conditions in the parentheses (). The sub-queries are performed first.
: The operator used to query key-value pairs. term1:term2 is a key-value pair. If the key or value contains reserved characters such as spaces and colons (:), use a pair of double quotation marks ("") to enclose the entire key or value.
" The operator used to convert a keyword into a common character. All the terms enclosed in a pair of double quotation marks are queried and are not considered as syntax keywords. When used in queries on key-value pairs, the terms inside a pair of double quotation marks are considered as a whole.
\ Escape character. It is used to escape quotation marks. The escaped quotation marks indicate the symbols themselves and are not considered as escape characters. Example: "\"".
| The pipeline operator used to indicate that another calculation is performed based on the previous calculation. Example: query1 | select count (1).
count The count operator used to indicate the number of log entries.
* The wildcard character used to perform fuzzy matching. It can replace zero or more characters. Example: que* indicates that the log entries with words starting with que are returned.
Note A maximum of 100 log entries that meet the query condition are returned.
? The wildcard character used to perform fuzzy matching. It can only replace one character. Example: qu? ry indicates that the log entries with words starting with qu, ending with ry, and containing a character in between are returned.
__topic__ Queries the log data of zero or more topics. Example: _ Topic __: mytopicname.
__tag__ Queries a tag value of a tag key. Example: __tag__:tagkey:tagvalue.
source Queries the log data based on an IP address. Example: source:127.0.0.1.
> Queries the log data with the value of a field greater than a specific number. Example: latency > 100.
>= Queries the log data with the value of a field greater than or equal to a specific number. Example: latency >= 100.
< Queries the log data with the value of a field less than a specific number. Example: latency < 100.
<= Queries the log data with the value of a field less than or equal to a specific number. Example: latency <= 100.
= Queries the log data with the value of a field equal to a specific number. Example: latency = 100.
in Queries the log data with a field falling in a specific value range. Brackets [] are used to indicate closed intervals and parentheses () are used to indicate open intervals. Two numbers are enclosed in brackets or parentheses and separated by multiple spaces. "in" must be in lower-case. Example: latency in [100 200] or latency in (100 200].
Note
  • Operators are case-insensitive.
  • Operator priorities are sorted in the descending order as follows::>">()>and>not>or.
  • Log Service reserves the right to use the following operators. If you need to use the following operators as query keywords, enclose them with a pair of double quotation marks (""): sort, asc, desc, group, avgsum, min, max, and limit.

Query examples

Expected log entries Example
Log entries that contain a and b a and b or a b
Log entries that contain a or b a or b
Log entries that contain a but do not contain b a not b
Log entries that do not contain a not a
Log entries that contain a and b, but do not contain c a and b not c
Log entries that contain a or b and must contain c (a or b ) and c
Log entries that contain a or b, but do not contain c (a or b ) not c
Log entries that contain a and b and may contain c a and b or c
Log entries whose FILE field contains apsara FILE:apsara
Log entries whose FILE field contains apsara and shennong FILE:"apsara shennong", FILE:apsara FILE: shennong, or FILE:apsara and FILE:shennong
Log entries that contain and and
Log entries whose FILE field contains apsara or shennong FILE:apsara or FILE:shennong
Log entries whose file info field contains apsara "file info":apsara
Log entries that contain quotation marks \"
Log entries that start with shen shen*
Log entries whose FILE field starts with shen FILE:shen*
Log entries whose value of the FILE field is shen* FILE: "shen*"
Log entries that contain words starting with shen, ending with ong, and containing a single character in between shen? ong
Log entries that contain words starting with shen and aps shen* and aps*
Log entries of topic1 and topic2 __topic__:topic1 or __topic__ : topic2
Log entries with a tag key-value pair being tagkey1 and tagvalue2 __tag__ : tagkey1 : tagvalue2
Log entries with a latency greater than or equal to 100 and less than 200 latency> = 100 and latency <200 or latency in [100 200)
Log entries with a latency greater than 100 latency > 100
Log entries that do not contain spider and whose http_referer field does not contain opx not spider not bot not http_referer:opx
Log entries whose cdnIP field is not left empty not cdnIP:""
Log entries whose cdnIP field does not exist not cdnIP:*
Log entries whose cdnIP field exists cdnIP:*

Topic-specific log queries

Each Logstore can be divided into one or more topics. You can specify topics when querying logs, which helps increase query performance. We recommend that you classify the log data in Logstores based on topics if you need a level 2 category.

If you specify one or more topics, you only query the log data from the specified topics. By default, the log data from all topics is queried if no topic is specified.

For example, you can classify log entries with the different domain names based on topics.
Figure 1. Log topics
Log Topic
Query syntax
  • The log data from all topics can be queried. If no topic is specified in the query syntax, the log data from all topics is queried.
  • You can query a single topic by using the following syntax: __topic__:topicName. You can also specify a topic in the URL parameter.
  • You can also query multiple topics. Example: __topic__:topic1 or __topic__:topic2 indicates the union of log data from topic1 and topic2 .

Fuzzy queries

Log Service supports fuzzy queries. You can specify a keyword that is a maximum of 64 characters in length, and use an asterisk (*) or a question mark (?) at the end or in the middle of the keyword. Log Service searches for a maximum of 100 words that meet the specified query condition and returns the log entries that contain these words.

Limits
  • You must specify an asterisk (*) or a question mark (?) for the prefix. The prefix cannot be used at the beginning of a keyword.
  • You can retrieve accurate results by specifying keywords precisely.
  • You cannot use fuzzy matching to search for a keyword that exceeds 64 characters in length. We recommend you specify a keyword with less than 64 characters in length.
  • A maximum of 100 log entries that meet the query condition are returned.