General syntax rules for query strings - Simple Log Service

The query string syntax is a feature of the Simple Log Service (SLS) domain-specific language (DSL). You can use this syntax to quickly filter data and simplify conditional statements. This topic describes the general syntax rules for query strings.

Function list

You can use the query string syntax with the following functions.

Type	Function	Scenario
Event check function	e_search	Use a query string to check if a field value in an event meets specific conditions.
Resource function	res_log_logstore_pull	Get data from a Logstore and return it as a table. Use a query string to configure blacklists and whitelists for filtering.
Resource function	res_rds_mysql	Get data from an RDS for MySQL database and return it as a table. Use a query string to configure blacklists and whitelists for filtering.

Feature overview

The following table lists the search features that support field search and full-text search.

Feature	Field search	Full-text search
Substring search	Supported	Supported
Wildcard search (`*?`)	Supported	Supported
Exact match search	Supported	Not supported
Regular expression search	Supported	Not supported
Numeric range search	Supported	Not supported
Numeric comparison	Supported	Not supported
Logical operators (and, or, not) and combinations	Supported	Supported

Character escaping

In the query string syntax, you must escape special characters, such as the asterisk (*) and backslash (\).

Escape a field name
Do not enclose field names in double quotation marks (""). If a field name contains special characters, you must escape them with a backslash (\). For example:
- \*$1+1$\?: abc: Escape the special characters with a backslash (\).
- __tag__\:__container_name__: abc: Escape the special character with a backslash (\).
- Chinese_Field: abc: You do not need to escape Chinese characters.
- "content": abc: Invalid definition. Do not enclose field names in double quotation marks ("").
Escape a search value
- If a value contains double quotation marks ("") or a backslash (\), you must escape it with a backslash (\). For example, content: "abc\"xy\\z".
  Note
  You can only enclose search values in double quotation marks (""). Use single quotation marks ('') to enclose the outer string and double quotation marks ("") to enclose the inner search value. For example, e_search("domain: '/url/test.jsp'") is incorrect. The correct format is e_search('domain: "/url/test.jsp"').
- When you search for an asterisk (*) or a question mark (?), you must escape it with a backslash (\). Otherwise, the character is treated as a wildcard character.
- If a value contains only Chinese characters, letters, digits, underscores (_), hyphens (-), asterisks (*), or question marks (?), you do not need to enclose it in double quotation marks (""). In all other cases, you must enclose the value in double quotation marks (""). For example:
  - status: "\*\?()[]:=": Enclose the value in double quotation marks ("") and escape the asterisk (*) and question mark (?) with a backslash (\). The other characters do not need to be escaped.
  - content: ()[]:=: Invalid definition. The value must be enclosed in double quotation marks ("").
  - status: active\*test, status: active\?test: The field value contains only letters, an asterisk (*), and a question mark (?). In this case, you only need to escape the asterisk (*) or question mark (?). You do not need to enclose the value in double quotation marks ("").

Substring search

Full-text search
Search for a substring in all fields.
- Syntax
```
e_search('substring')
```
- Examples
  - e_search('"fault"'): Searches for a Chinese substring.
  - e_search('"active error"'): Searches for a complete substring that contains a space.
  - e_search('active error'): Searches for multiple substrings. The OR operator is used by default between substrings.
Field search
Search for a substring in a specific field.
- Syntax
```
e_search('...')
```
- Examples
  - e_search('status: active'): Performs a substring search.
  - e_search('author: "john smith"'): Searches for a substring that contains a space.
  Note
  e_search('field: active error') is equivalent to field:active OR "error". This query searches for "active" in the field field or performs a full-text search for "error".

Wildcard search

An asterisk (*) represents zero or more characters. A question mark (?) represents one character. It can also represent a wide character, such as a Chinese character.

Full-text search
Search for a substring in all fields.
- Syntax
```
e_search('substring')
```
- Examples
  - e_search('active*test'): Matches zero or more characters. Because the string contains an asterisk (*), you do not need to enclose it in double quotation marks ("").
  - e_search('occurs*fault'): Matches zero or more characters. It can match occurs fault and occurs critical fault.
  - e_search('active?good'): Matches one character. Because the string contains a question mark (?), you do not need to enclose it in double quotation marks ("").
  - e_search('ac*tive?good'): Performs a pattern match.
  - e_search('ac*tive??go*od'): You can use multiple wildcards together.
Field search
Search for a substring in a specific field.
- Syntax
```
e_search('field_name:substring')
```
- Examples
  - e_search('status: active*test'): Matches zero or more characters.
  - e_search('status: active?good'): Matches one character.

Exact match

An exact match requires that the field value is a complete match from beginning to end.

Syntax

e_search('field_name==exact_match_string')

Examples
- e_search('author== "john smith"'): The value of the author field is exactly equal to john smith.
- e_search('status== ac*tive?good'): You can use this operator with wildcard characters.

Regular expression match

Regular expression matching is a more powerful matching method than wildcard matching.

Syntax
```
e_search('field_name~=regular_expression_string')
```
Note
- Because a regular expression can contain a backslash (\), you can use r to prevent escaping.
- By default, a partial match is performed. For an exact match, you can add ^ at the beginning and $ at the end of the expression.
Examples
- e_search('status~= "\d+"'): The value of the status field contains digits.
- e_search('status~= "^\d+$"'): The value of the status field is a number.

Numeric comparison

You can compare numeric values.

Direct numeric comparison

You can use >, >=, =, <, and <= for comparison.

e_search('age >= 18')  #  >=18
e_search('age > 18')   #  > 18
e_search('age = 18')   #  = 18
e_search('age <= 18')  #  <=18
e_search('age < 18')   #  < 18

Numeric range comparison

You can use a closed interval. Use an asterisk (*) to indicate an open boundary.

e_search('count: [100, 200]') # >=100 and  <=200
e_search('count: [*, 200]')   # <=200
e_search('count: [200, *]')   # >=200

Logical relationships

Logical relationships between searches are supported. You can also use parentheses () for nesting.

Logical relationship	Keyword
AND	`and`, `AND`, `&&`, and others. The keywords are not case-sensitive.
OR	`or`, `OR`, and others. The keywords are not case-sensitive.
NOT	`not`, `NOT`, `!`, and others. The keywords are not case-sensitive.

Examples:

e_search('abc OR xyz')    # Relational operators are not case-sensitive.
e_search('abc and (xyz or zzz)')
e_search('abc and not (xyz and not zzz)')
e_search('abc && xyz')    # and
e_search('abc || xyz')    # or
e_search('abc || !xyz')   # or not

Substring matching also supports logical relationships:

e_search('field: (abc OR xyz)')      # The field contains abc or xyz.
e_search('field: (abc OR not xyz)')  # The field contains abc or does not contain xyz.
e_search('field: (abc && !xyz)')     # The field contains abc and does not contain xyz.

Field check

You can use a query string to check a field.

e_search('field: *'): The field exists.
e_search('not field:*'): The field does not exist.
e_search('not field:""'): The field does not exist.
e_search('field: "?"'): The field exists and its value is not empty.
e_search('field==""'): The field exists and its value is empty.
e_search('field~=".+"'): The field exists and its value is not empty.
e_search('not field~=".+"'): The field does not exist or its value is empty.
e_search('not field==""'): The field does not exist or its value is not empty.