This topic describes how to use the search syntax that is provided by Simple Log Service to specify search conditions. You can efficiently query logs based on the search syntax.
Search types
A search statement specifies one or more search conditions and returns the logs that meet the specified conditions. Searches are classified by indexing method into full-text searches and field-specific searches, or classified by precision into exact searches and fuzzy searches.
Before you can specify a numeric range to query logs based on a field, you must set the data type of the field to double or long. If you do not set the data type of a field to double or long, or the syntax of the numeric range is invalid, Simple Log Service performs a full-text search and the search result that is returned may be different from the expected result. For example, if you execute the
owner_id>100
search statement and the data type of the owner_id field is not double or long, logs that contain owner_id, >, and 100 are returned. In this example, the greater sign (>) is not a delimiter.If you change the data type of a field from text to double or long, you can use only the equal sign (=) to query the logs that are collected before the change.
Query and analysis results vary based on index configurations. You must configure indexes based on your business requirements. If you create both full-text indexes and field indexes, the configurations of the field indexes take precedence.
Full-text searches and field-specific searches
Search type
Description
Example
Full-text search
After you configure full-text indexes, Simple Log Service splits a log into multiple strings by using the delimiters that you specify. You can specify keywords and rules in a search statement to query logs. The keywords can be field names or field values.
PUT and cn-shanghai
: returns the logs that contain the keywords PUT and cn-shanghai.Field-specific search
After you configure field indexes, you can query logs. To query logs, specify field names and field values in the key:value format. You can perform basic searches or combined searches based on the data types of the fields in the field indexes. For more information, see Data types.
request_time>60 and request_method:Ge*
: returns the logs in which the value of the request_time field is greater than 60 and the value of the request_method field starts with Ge.Exact searches and fuzzy searches
Search type
Description
Example
Exact search
Complete strings are used for queries.
Simple Log Service uses word segmentation to query logs. Phrases cannot be completely matched by based on an exact search. For example, the search statement
abc def
returns the logs that containabc
ordef
. The phrase abc def cannot be completely matched. If you want the phraseabc def
to be completely matched, you can perform a phrase search or use a LIKE clause. For more information, see Phrase search and How do I query logs by using exact match?host:example.com
: returns the logs in which the value of the host field contains example.com.PUT
: returns the logs that contain the keyword PUT.* | Select * where http_user_agent like '%like Gecko%'
: returns the logs in which the value of the http_user_agent field contains the phraselike Gecko
.#"redo_index/1"
: returns the logs that contain the phraseredo_index/1
.
Fuzzy search
You can add an asterisk (*) or a question mark (?) as a wildcard character in the middle or at the end of a keyword to perform a fuzzy search. Each keyword must be 1 to 64 characters in length. If a keyword contains a wildcard character, Simple Log Service searches all logs and obtains up to 100 strings that match the keyword. Then, Simple Log Service returns the logs that contain one or more of these strings. The more accurate a keyword is, the more accurate the search results are.
ImportantA keyword cannot start with an asterisk (*) or a question mark (?).
The long and double data types do not support asterisks (*) or question marks (?) in fuzzy searches. You can specify a numeric range when you perform a fuzzy search. Example: status in [200 299].
A fuzzy search is performed based on samples by using the following mechanism:
If you enable the field indexing feature and specify a field to query logs, Simple Log Service randomly obtains samples from the indexed data of the field and returns part of the search results.
If you enable the full-text indexing feature and do not specify a field to query logs, Simple Log Service randomly obtains samples from the full-text indexed data and returns part of the search results.
addr*
: searches for 100 strings that start with addr from logs, and returns the logs that contain one or more of these strings.host:www.yl*
: searches for 100 strings that start with www.yl from the value of the host field. Then, Simple Log Service returns the logs in which the value of the host field contains one or more of these strings.
For more information, see How do I query logs by using fuzzy match?.
Operators
The following table describes the operators that are supported by search statements.
The in operator is case-sensitive. Other operators are not case-sensitive.
Simple Log Service supports the following operators: sort, asc, desc, group by, avg, sum, min, max, and limit. If you want to use the preceding operators as keywords, you must enclose the operators in double quotation marks ("").
The following list shows the priorities of the operators in descending order:
Colons (:)
Double quotation marks ("")
Parentheses ()
and and not
or
Operator | Description |
and | The and operator. Example: If no syntax keyword exists among multiple keywords, the keywords are joined by using the and operator by default. For example, |
or | The or operator. Example: |
not | The not operator. Example: |
( ) | This operator is used to increase the priority of the search conditions that are enclosed in parentheses (). Example: |
: | This operator is used for field-specific searches based on the key:value format. Example: If a field name or a field value contains reserved characters such as space characters and colons (:), you must enclose the field name or field value in double quotation marks (""). Example: |
"" | This operator is used to enclose a syntax keyword. If a syntax keyword is enclosed in double quotation marks (""), the keyword is converted to an ordinary character. For example, In a field-specific search, the strings that are enclosed in double quotation marks ("") are considered as a whole string. |
\ | The escape character. This character is used to escape double quotation marks (""). Double quotation marks ("") can indicate themselves only after they are escaped. For example, if the content of a log is |
* | The wildcard character. This character is used to match zero, one, or multiple characters. Example: Note Simple Log Service searches all logs and obtains up to 100 strings that meet the specified conditions. Then, Simple Log Service returns the logs that contain one or more of the 100 strings and meet the search conditions. |
? | The wildcard character. This character is used to match a single character. Example: |
> | This operator is used to query the logs in which the value of a specified field is greater than a specified numeric value. Example: |
>= | This operator is used to query the logs in which the value of a specified field is greater than or equal to a specified numeric value. Example: |
< | This operator is used to query the logs in which the value of a specified field is smaller than a specified numeric value. Example: |
<= | This operator is used to query the logs in which the value of a specified field is smaller than or equal to a specified numeric value. Example: |
= | This operator is used to query the logs in which the value of a specified field is equal to a specified numeric value. Equal signs (=) and colons (:) have the same effect on fields of the double or long data type. For example, |
in | This operator is used to query the logs in which the value of a specified field is within a specified numeric range. Brackets [] indicate a closed interval, and parentheses () indicate an open interval. A space character is used to separate two numbers in a numeric range. Examples: Important The characters of in must be in lowercase. |
__source__ | This operator is used to query the logs of a specified log source. Wildcard characters are supported. Example: Important The __source__ field is a reserved field in Simple Log Service. This field can be abbreviated to source. If you configure a custom field in the source format, the custom field conflicts with the reserved source field in Simple Log Service. If you want to search for the custom field, you must use Source or SOURCE in a search statement. |
__tag__ | This operator is used to query logs based on metadata. Example: |
__topic__ | This operator is used to query the logs of a specified log topic. Example: |
Examples of search statements
If you execute a search statement on different logs based on different index configurations, the statement returns different results. The examples in this section are provided based on the following sample log and index configurations.
Sample log
An NGINX access log is used as the sample log.

Index configurations
Before you can execute a search statement, make sure that indexes are configured. For more information, see Create indexes.
Common search examples
Expected search result | Search statement |
Logs that contain successful GET requests (status codes: 200 to 299) |
|
Logs that contain GET requests but do not contain the China (Shanghai) region |
|
Logs that contain GET requests or POST requests |
|
Logs that do not contain GET requests |
|
Logs that contain successful GET requests or successful POST requests |
|
Logs that contain failed GET requests or failed POST requests |
|
Logs that contain successful GET requests (status codes: 200 to 299) and in which the request duration is less than 60 seconds |
|
Logs in which the request duration is equal to 60 seconds |
|
Logs in which the request duration is greater than or equal to 60 seconds and is less than 200 seconds |
|
Logs in which the request_time field is empty or the value of the field is an invalid number |
Note
|
Logs that contain the request_time field and in which the value of the field is a number |
|
Logs that contain and |
Note In this search statement, and is a common string but not an operator. |
Logs in which the value of the request method field contains PUT |
Important The name of the request method field contains a space character. You must enclose the field name in double quotation marks ("") in a search statement. |
Logs whose topic is HTTPS or HTTP |
|
Logs that are collected from the 192.0.2.1 host |
The Important If a log is processed by using the data transformation feature or a Logtail plug-in, the key in a tag field is converted to a common key. If you want to search for the log, you must enclose the name of the tag field in double quotation marks ("") in the search statement. Example: |
Logs that contain |
For more information, see Use the LIKE clause to implement fuzzy match. |
Logs in which the remote_user field is not empty |
|
Logs in which the remote_user field is empty |
|
Logs in which the value of the remote_user field is not null |
|
Logs that do not contain the remote_user field |
|
Logs that contain the remote_user field |
|
Logs in which the value of the city field is not Shanghai |
Note If you want to query Chinese strings, you must turn on Include Chinese when you configure indexes. For more information, see Create indexes. |
Advanced search examples
Fuzzy search
Expected search result
Search statement
Logs that contain specific strings. The strings start with cn.
cn*
Logs in which the value of the region field starts with cn.
region:cn*
Logs in which the value of the region field contains cn*.
region:"cn*"
NoteIn this search statement,
cn*
is a complete string. Examples:If the content of a log is
region:cn*,en
and the delimiter is a comma (,), Simple Log Service splits the log content intoregion
,cn*
, anden
. You can use the search statement to search for the log.If the content of a log is
region:cn*hangzhou
, Simple Log Service considerscn*hangzhou
as a whole. Therefore, you cannot use the search statement to search for the log.
Logs that contain specific strings. The strings start with mozi, end with la, and include one character between mozi and la.
mozi?la
Logs that contain specific strings. The strings start with mo, end with la, and include zero, one, or more characters between mo and la.
mo*la
Logs that contain specific strings. The strings start with moz or sa.
moz* and sa*
Logs in which the value of the region field ends with hai.
You cannot use a search statement to search for the logs. You can use the LIKE clause in an SQL statement to search for the logs. For more information, see Use the LIKE clause to implement fuzzy match.
* | select * from log where region like '%hai'
Delimiter-based searches
Simple Log Service splits the content of a log into multiple strings based on the delimiters that you specify. The default delimiters are
, '";=()[]{}?@&<>/:\n\t\r
. If you leave the Delimiter parameter empty, Simple Log Service considers the value of each field as a whole. In this case, you can search for a log only by using a complete string or a fuzzy search. For more information about how to specify delimiters, see Create indexes.For example, the value of the http_user_agent field is
Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.2 (KHTML, like Gecko) Chrome/192.0.2.0 Safari/537.2
.If you leave the Delimiter parameter empty, Simple Log Service considers the field value as a whole. In this case, you cannot search for logs by using the
http_user_agent:Chrome
search statement.If you set the Delimiter parameter to
, '";=()[]{}?@&<>/:\n\t\r
, Simple Log Service splits the field value intoMozilla
,5.0
,Windows
,NT
,6.1
,AppleWebKit
,537.2
,KHTML
,like
,Gecko
,Chrome
,192.0.2.0
,Safari
, and537.2
. In this case, you can search for logs by using thehttp_user_agent:Chrome
search statement.
NoteIf a search keyword contains delimiters, you can perform a phrase search or use a LIKE clause. Examples:
Phrase search:
#"redo_index/1"
. For more information, see Phrase search.LIKE clause:
* | select * from log where key like 'redo_index/1'
.
Expected search result
Search statement
Logs in which the value of the http_user_agent field contains Chrome
http_user_agent:Chrome
Logs in which the value of the http_user_agent field contains Linux and Chrome
http_user_agent:"Linux Chrome"
http_user_agent:Linux and http_user_agent:Chrome
Logs in which the value of the http_user_agent field contains Firefox or Chrome
http_user_agent:Firefox or http_user_agent:Chrome
Logs in which the value of the request_uri field contains /request/path-2
request_uri:/request/path-2
Logs in which the value of the request_uri field starts with /request and does not contain /file-0
request_uri:/request* not request_uri:/file-0
Logs in which the phrase
redo_index/1
is completely matched#"redo_index/1"
* | select * from log where key like 'redo_index/1'
NoteYou can perform a phrase search or use a LIKE clause to completely match the phrase. If you perform an exact search, strings such as
redo_index
and1
are matched.Searches for the JSON logs whose field values are JSON objects or JSON arrays
If the value of a field is in the JSON format, you can set the data type of the field to JSON or use JSON functions to query and analyze the field. For more information, see FAQ about the query and analysis of JSON logs.