General syntax rules for query strings - Simple Log Service

Quick reference

The following functions support query string syntax.

Type	Function	Scenario
Event check function	e_search	Use a query string to determine if the field value of an event meets specific conditions.
Resource function	res_log_LogStore_pull	Gets and returns tabular data from a Logstore. Supports filtering with blacklists and whitelists configured by a query string.
Resource function	res_rds_mysql	Gets and returns tabular data from RDS MySQL. Supports filtering with blacklists and whitelists configured by a query string.

Feature overview

The following table describes search feature support for field search and full-text search.

Feature	Field search	Full-text search
Substring search	Supported	Supported
Wildcard search using `*?`	Supported	Supported
Exact match search	Supported	Not supported
Regular expression search	Supported	Not supported
Numeric range search	Supported	Not supported
Numeric comparison	Supported	Not supported
Logical operators: and, or, not, and combinations	Supported	Supported

Character escaping

Escape special characters such as asterisks (*) and backslashes (\) in query strings.

Field name escaping
Do not enclose field names in double quotation marks (""). If a field name contains special characters, escape them with a backslash (\). For example:
- \*$1+1$\?: abc: Escape with a backslash (\).
- __tag__\:__container_name__: abc: Escape with a backslash (\).
- chinese_field: abc: Chinese characters do not need to be escaped.
- "content": abc: Invalid definition. Do not enclose field names in double quotation marks ("").
Search value escaping
- If a value contains double quotation marks ("") or a backslash (\), escape them with a backslash (\). For example, content: "abc\"xy\\z".
  
  Note
  Search values must be enclosed in double quotation marks (""). Enclose the outer string in single quotation marks ('') and the inner search value in double quotation marks (""). For example, e_search("domain: '/url/test.jsp'") is incorrect. The correct format is e_search('domain: "/url/test.jsp"').
- When you search for an asterisk (*) or a question mark (?), escape them with a backslash (\). Otherwise, they are treated as wildcard characters.
- A string that contains only Chinese characters, letters, digits, underscores (_), hyphens (-), asterisks (*), or question marks (?) does not require double quotation marks (""). Otherwise, you must enclose the string in double quotation marks (""). For example:
  - status: "\*\?()[]:=": Enclose the value in double quotation marks (""). Escape the asterisk (*) and question mark (?) with a backslash (\). Other characters do not need to be escaped.
  - content: ()[]:=: Invalid definition. The value must be enclosed in double quotation marks ("").
  - status: active\*test and status: active\?test: The field value contains only letters and an asterisk (*) or a question mark (?). You only need to escape the asterisk (*) or question mark (?). You do not need to enclose the value in double quotation marks ("").

Substring search

Full text search

Search for a substring across all fields.
- Syntax
```
e_search('substring')
```
- Examples
  - e_search('"error"'): Search for a substring.
  - e_search('"active error"'): Search for a complete substring that contains a space.
  - e_search('active error'): Search for multiple substrings. The substrings are combined with an OR operator by default.
Field search

Search for a substring in a specific field.
- Syntax
```
e_search('...')
```
- Examples
  - e_search('status: active'): Substring search.
  - e_search('author: "john smith"'): Search for a substring that contains a space.
  Note
  e_search('field: active error') is equivalent to field:active OR "error". This searches for `active` in the field field or performs a full text search for `error`.

Wildcard search

An asterisk (*) matches zero or more characters. A question mark (?) matches exactly one character, including wide characters such as Chinese characters.

Full text search
Search for a substring across all fields.
- Syntax
```
e_search('substring')
```
- Examples
  - e_search('active*test'): Matches zero or more characters. Because the string contains an asterisk (*), you do not need to enclose it in double quotation marks ("").
  - e_search('occurs*error'): Matches zero or more characters. For example, it can match occurs error and occurs critical error.
  - e_search('active?good'): Matches a single character. Because the string contains a question mark (?), you do not need to enclose it in double quotation marks ("").
  - e_search('ac*tive?good'): Matches a specific pattern.
  - e_search('ac*tive??go*od'): Supports multiple wildcards.
Field search
Search for a substring in a specific field.
- Syntax
```
e_search('field_name:substring')
```
- Examples
  - e_search('status: active*test'): Matches zero or more characters.
  - e_search('status: active?good'): Matches a single character.

Exact match

An exact match requires the entire field value to match from start to end.

Syntax

e_search('field_name==exact_match_string')

Examples
- e_search('author== "john smith"'): The author field exactly matches john smith.
- e_search('status== ac*tive?good'): Use with wildcard characters.

Regular expression match

Regular expression matching provides more powerful pattern matching than wildcards.

Syntax
```
e_search('field_name~=regular_expression_string')
```
Note
- Because regular expressions can contain backslashes (\), use r to prevent escaping.
- By default, this performs a partial match. To perform an exact match, add ^ at the beginning and $ at the end of the expression.
Examples
- e_search('status~= "\d+"'): The status field contains digits.
- e_search('status~= "^\d+$"'): The status field is a number.

Numeric comparison

Compare field values as numbers.

Direct numeric comparison

Use the operators >, >=, =, <, and <= for comparison.

e_search('age >= 18')  #  >=18
e_search('age > 18')   #  > 18
e_search('age = 18')   #  = 18
e_search('age <= 18')  #  <=18
e_search('age < 18')   #  < 18

Numeric range comparison:

Use a closed interval. Use an asterisk (*) to indicate an unbounded side.

e_search('count: [100, 200]') # >=100 and  <=200
e_search('count: [*, 200]')   # <=200
e_search('count: [200, *]')   # >=200

Logical operators

Combine any searches with logical operators.Alternatively, use parentheses () to nest conditions.

Logical Relationship	Keyword
AND	`and`, `AND`, and `&&`. The keywords are not case-sensitive.
OR	`or`, `OR`, etc., are case-insensitive.
NOT	`not`, `NOT`, and `!`. The keywords are not case-sensitive.

Examples:

e_search('abc OR xyz')    # Logical operators are case-insensitive
e_search('abc and (xyz or zzz)')
e_search('abc and not (xyz and not zzz)')
e_search('abc && xyz')    # and
e_search('abc || xyz')    # or
e_search('abc || !xyz')   # or not

Logical relationships are also supported for substring matching:

e_search('field: (abc OR xyz)')      # The field contains abc or xyz
e_search('field: (abc OR not xyz)')  # The field contains abc or does not contain xyz
e_search('field: (abc && !xyz)')     # The field contains abc and does not contain xyz

Field evaluation

Use a search string to verify whether a field exists or meets certain conditions.

e_search('field: *'): The field exists.
e_search('not field:*'): The field does not exist.
e_search('not field:""'): The field does not exist.
e_search('field: "?"'): The field exists and its value is not empty.
e_search('field==""'): The field exists and its value is empty.
e_search('field~=".+"'): The field exists and its value is not empty.
e_search('not field~=".+"'): The field does not exist or its value is empty.
e_search('not field==""'): The field does not exist or its value is not empty.