Log Service provides a set of query syntaxes used to express query conditions to help you easily query logs. You can specify query conditions through the GetLogs and GetHistograms interfaces in the Log Service API or on the query page of the Log Service console. This section details query condition syntax.
You can create an index for the LogStore in two modes:
- Full text index: The entire line of log is queried as a whole, without differentiating the key and value (Key, Value).
- Key/value index: Query is performed when Key is specified, for example, FILE:app, Type:action. All the contained strings under this key will be hit.
Log Service query conditions support the following keywords:
|and||Binary operator, in the format of
|or||Binary operator, in the format of
|not||Binary operator, in the format of
|(,)||Left and right brackets are used to merge one or multiple sub-queries into one query to improve the query priority in the brackets.|
|:||Used to query the key-value pair.
|“||Converts a keyword into a common query character. Any term in the left and right quotation marks will be queried and will not be used as a syntax keyword. Or all the terms in the left and right quotation marks are regarded as a whole in the key-value query.|
|\||Escape character. Used to escape quotation marks; the quotation marks after escaping indicate the symbols themselves and will not be considered as escape characters, for example,
||||Pipeline operator, indicating more computing based on the previous computing, for example, query1 | timeslice 1h | count.|
|timeslice||Time slice operator indicates the length of time during which the data is regarded as a whole for computing.The methods of use are timeslice 1h, timeslice 1m, and timeslice 1s, which respectively indicate 1 hour, 1 minute, and 1 second as a whole. For example, query1 | timeslice 1h | count indicates querying the query condition, and the total times with 1 hour as the time spice are returned.|
|count||Count operator, indicating the number of logs.|
|*||Fuzzy query keyword, used to replace 0 or multiple characters. For example:
|?||Fuzzy query keyword, used to replace one character. For example,
||Query the data under a certain topic. Under the new syntax, the data of 0 or more topics can be queried in the query, for example,
||Query a tag value under a tag key, for example,
|source||Query the data of an IP, for example,
|source||Query the data of an IP, for example,
|>||Query the logs with the value of a field greater than a specific number, for example,
|>=||Query the logs with the value of a field greater than or equal to a specific number, for example,
|<||Query the logs with the value of a field less than a specific number, for example,
|<=||Query the logs with the value of a field less than or equal to a specific number, for example,
|=||Query the logs with the value of a field equal to a specific number, for example,
|in||Query the logs with a field falling within a specific range. Brackets () are used to indicate closed intervals and parentheses (()) indicate open intervals, with two numbers enclosed and separated by spaces. For example,
- Syntax keywords are case-insensitive.
- Priorities of syntax keywords are sorted in descending order as follows
: > " > ( ) > and not > or.
- Log Service reserves the right to use the following keywords:
sort asc desc group by avg sum min max limit. If you need to use the following keywords, use quotation marks to contain them.
- If the full text index and the key/value index have different word segmentation characters when they are configured, data cannot be queried using the full text query method.
- To perform a numeric query, the data type of the queried column must be set as double or long. If no data type is set or the syntax used for the numeric range query is incorrect, Log Service translates the query condition as a full text index, which can lead to an unexpected result.
- If you change the data type of a column from text to numeric, only the = query is supported for the data prior to this change.
- Logs that contains a and b at the same time:
a and bor
- Logs that contain a or b:
a or b.
- Logs that contain a but no b:
a not b.
- Those in all the logs that contain no a:
- Query the logs that contain a and b, but no c:
a and b not c.
- Logs that contain a or b and must contain c:
(a or b ) and c.
- Logs that contain a or b, but no c:
(a or b ) not c.
- Logs that contain a and b and may contain c:
a and b or c.
- Logs whose FILE field contains apsara:
- Logs whose FILE field contains apsara and shennong:
FILE:apsara FILE: shennongor
FILE:apsara and FILE:shennong.
- Logs containing and:
- Logs with the FILE field containing apsara or shennong:
FILE:apsara or FILE:shennong.
- Logs with the file info field containing apsara:
- Logs that contain quotation marks:
- Logs starting with shen:
- Query all the logs starting with shen under the FILE field:
- Query the logs starting with shen, ending with ong end and with a character in the middle:
- Query all the logs starting with shen and aps:
shen* and aps*.
- Query the distribution of logs starting with shen, with the time slice as 20 minutes:
shen*| timeslice 20m | count.
- Query all the data under topic1 and topic2:
__topic__:topic1 or __topic__ : topic2.
- Query all the data of tagvalue2 under tagkey1:
__tag__ : tagkey1 : tagvalue2.
- Query for all the data with a latency greater than or equal to 100 and less than 200 can be written in either of the following ways:
latency >=100 and latency < 200or
latency in [100 200).
- Query for all the requests with a latency greater than 100 must be written in the following way:
latency > 100.
Each LogStore can be divided into one or more subspaces according to the topic. During query, the query range can be limited for the specified topic to increase the speed. This means that users with a level-2 classification requirement for the LogStore are recommended to use topic to divide the LogStore.
When one or more topics are specified to perform query, query is only implemented in the topic that meets the conditions. However, if no topic is entered, the data under all the topics is queried by default.
Example in which topics are used to classify logs under different domain names:
Topic query example:
- The data under all the topics can be queried. The data of all the topics will be queried if no topic is specified in the query syntax and parameter.
- Topic can be queried in the query. The query syntax is
__topic__:topicName. The old mode is still supported at the same time. The topic is specified in the URL parameter.
- Multiple topics can be queried, for example,
__topic__:topic1 or __topic__:topic2indicates the union of data under topic1 and topic2.
The added syntax of Log Service provides the custom interval function. The query syntax is as follows:
where_condition | timeslice 1[hms] |count
h stands for the unit of hour
m stands for the unit of minute
s stands for the unit of second
The interval size can be adjusted by changing the timeslice parameter. For example, to query the data of 2 hours, the corresponding results of different timeslice parameters are as follows:
|timeslice parameter||Number of Histogram intervals||Size of each interval|