To help you query logs more effectively, Log Service provides a set of query syntax to express query conditions. You can specify query conditions by using the GetLogs and GetHistograms interfaces in Log Service API or on the query page of the Log Service console. This document introduces the syntax of query conditions in details.
Log Service supports creating an index for the LogStore in the following methods:
- Full text index: Query the entire line of logs as a whole without differentiating key and value.
- Key/value index: Query logs after specifying a key. For example,
Type:action. All the strings under the specified key will be queried.
LogSearch query conditions support the following keywords.
|and||Binary operator. Format:
|or||Binary operator. Format:
|not||Binary operator. Format:
|:||Used to query the key-value pairs.
|“||Converts a keyword to a common query character. Each term enclosed in quotation marks (“) can be queried and will not be considered as a syntax keyword. Or all the terms enclosed in quotation marks (“) are regarded as a whole in the key-value query.|
|\||Escape character. Used to escape quotation marks (“). After being escaped, the quotation marks (“) indicate the symbols themselves and will not be considered as escape characters. For example,
||||Pipeline operator. Indicates to perform more computing based on the previous computing. For example, query1 | timeslice 1h | count.|
|timeslice||Time slice operator. Indicates the length of time during which the data is regarded as a whole for computing. The usage modes are timeslice 1h, timeslice 1m, and timeslice 1s, respectively indicating to regard 1 hour, 1 minute, and 1 second as a whole. For example, query1 | timeslice 1h | count indicates to query the query condition and return the total number of times with 1 hour as the time slice.|
|count||Count operator. Indicates the number of log entries.|
|*||Fuzzy query keyword. Used to replace zero or multiple characters. For example, the query results of
Note: At most 100 query results can be returned.
|?||Fuzzy query keyword. Used to replace one character. For example, the query results of
||Query the data under a topic. With the new syntax, you can query the data of zero or multiple topics in the query. For example,
||Query a tag value under a tag key. For example,
|source||Query the data of an IP. For example,
|>||Query the logs with a field value greater than a specific number. For example,
|>=||Query the logs with a field value greater than or equal to a specific number. For example,
|<||Query the logs with a field value less than a specific number. For example,
|<=||Query the logs with a field value less than or equal to a specific number. For example,
|=||Query the logs with a field value equal to a specific number. For example,
|in||Query the logs with a field staying within a specific range. Brackets () are used to indicate closed intervals and parentheses (
- Syntax keywords are case-insensitive.
- Priorities of syntax keywords are sorted in descending order as follows:
: > " > ( ) > and not > or.
- Log Service reserves the right to use the following keywords:
sort asc desc group by avg sum min max limit. To use these keywords, enclose them in quotation marks (“).
- If both the full text index and key/value index are configured and have different word segmentation characters, data cannot be queried using the full text query method.
- Set the column type as double or long before performing a numeric query. If the column type is not set or the syntax used for the numeric range query is incorrect, Log Service translates the query condition as a full text index, which might lead to an unexpected result.
- If you change the column type from text to numeric, only the = query is supported for the data before this change.
- Logs that contain a and b at the same time:
a and bor
- Logs that contain a or b:
a or b.
- Logs that contain a but do not contain b:
a not b.
- All the logs that do not contain a:
- Query the logs that contain a and b, but do not contain c:
a and b not c.
- Logs that contain a or b and must contain c:
(a or b ) and c.
- Logs that contain a or b, but do not contain c:
(a or b ) not c.
- Logs that contain a and b and might contain c:
a and b or c.
- Logs whose FILE field contains apsara:
- Logs whose FILE field contains apsara and shennong:
FILE:apsara FILE: shennong, or
FILE:apsara and FILE:shennong.
- Logs containing and:
- Logs with the FILE field containing apsara or shennong:
FILE:apsara or FILE:shennong.
- Logs with the file info field containing apsara:
- Logs that contain quotation marks (“):
- Query all the logs starting with shen:
- Query all the logs starting with shen under the FILE field:
- Query all the logs starting with shen, ending with ong, and having a character in the middle:
- Query the logs starting with shen and aps:
shen* and aps*.
- Query the distribution of logs starting with shen, with 20 minutes as the time slice:
shen*| timeslice 20m | count.
- Query all the data under topic1 and topic2:
__topic__:topic1 or __topic__ : topic2.
- Query all the data of tagvalue2 under tagkey1:
__tag__ : tagkey1 : tagvalue2.
- Query all the data with a latency greater than or equal to 100 and less than 200, which can be written in the following two ways:
latency >=100 and latency < 200or
latency in [100 200).
- Query all the requests with a latency greater than 100, which must be written in the following way:
latency > 100.
- Query the logs that do not contain spider and do not contain opx in http_referer:
not spider not bot not http_referer:opx.
Each LogStore can be divided into one or more subspaces by the topic. During query, specifying topics can limit the query range so as to increase the speed. Therefore, we recommend that you use topic to divide the LogStore if you have a secondary classification requirement for the LogStore.
With one or more topics specified, the query is only performed in the topics that meet the conditions. However, data under all the topics is queried by default if no topic is specified.
For example, use topic to classify logs under different domain names:
Topic query syntax:
- Data under all the topics can be queried. Data of all the topics will be queried if no topic is specified in the query syntax and parameter.
- Topic can be queried in the query. The query syntax is
__topic__:topicName. The old mode (specify the topic in the URL parameter) is still supported at the same time.
- Multiple topics can be queried. For example,
__topic__:topic1 or __topic__:topic2indicates to query the data union under Topic1 and Topic2.