You can check and process data based on specific conditions by using the functions. This topic describes how to use functions to check data in various scenarios.
Scenario 1: Check whether a field exists
- Raw log entry
a: a_value b: // Empty string
- Domain-specific language (DSL) orchestration
- Solution 1: Use the
e_has
ande_not_has
functions.e_if(e_has("a"),e_set("has_a", true)) e_if(e_has("b"),e_set("has_b", true)) e_if(e_has("c"),e_set("has_c", true)) e_if(e_not_has("a"),e_set("not_has_a", true)) e_if(e_not_has("b"),e_set("not_has_b", true)) e_if(e_not_has("c"),e_set("not_has_c", true))
- Solution 2: Use the
e_search
function.e_if(e_search('a: *'),e_set("has_a", true)) e_if(e_search('b: *'), e_set("has_b", true)) e_if(e_search('c: *'), e_set("has_c", true)) e_if(e_search('not a: *'), e_set("not_has_a", true)) e_if(e_search('not b: *'), e_set("not_has_b", true)) e_if(e_search('not c: *'), e_set("not_has_c", true))
Note In the preceding example, ane_if
function is written for each condition to better illustrate the solution. You can simplify the function by including all conditions and the corresponding operations ase_if(condition 1, operation 1, condition 2, operation 2)
.
- Solution 1: Use the
- Result
a:a_value b: // Empty string has_a: true has_b: true has_c: false not_has_a: false not_has_b: false not_has_c: true
Scenario 2: Check whether a field value exists and is not empty
- Raw log entry
a: a_value b: // Empty string
- DSL orchestration
- Solution 1 (recommended): Use the
v
function that returns a field value.e_if(v("a"), e_set("not_empty_a", true)) e_if(v("b"), e_set("not_empty_b", true)) e_if(v("c"), e_set("not_empty_c", true))
Note If the field value extracted by thev
function exists and is not empty, theBool
value true is returned. Otherwise, false is returned. - Solution 2: Use the
e_search
function.# The field value contains at least one character. e_if(e_search('a: "?"'), e_set("not_empty_a", true)) e_if(e_search('b: "?"'), e_set("not_empty_b", true)) e_if(e_search('c: "?"'), e_set("not_empty_c", true)) # Regular expression e_if(e_search('a~=".+"'), e_set("not_empty_a", true)) e_if(e_search('b~=".+"'), e_set("not_empty_b", true)) e_if(e_search('c~=".+"'), e_set("not_empty_c", true)) # The field value exists and is not empty. e_if(e_search('a: * and not a==""'), e_set("not_empty_a", true)) e_if(e_search('b: * and not b==""'), e_set("not_empty_b", true)) e_if(e_search('c: * and not c==""'), e_set("not_empty_b", true))
- Solution 1 (recommended): Use the
- Result
a: a_value b: // Empty string not_empty_a: true not_empty_b: false not_empty_c: false
Scenario 3: Check whether a field value exists and is empty
- Raw log entry
a: a_value b: // Empty string
- DSL orchestration
- Solution 1 (recommended): Use the
v
function that returns a field value.e_if(op_and(e_has("a"), op_not(v("a"))), e_set("empty_a", true)) e_if(op_and(e_has("b"), op_not(v("b"))), e_set("empty_b", true)) e_if(op_and(e_has("c"), op_not(v("c"))), e_set("empty_c", true)) # Invalid syntax e_if(op_not(v("a")), e_set("empty_a", true)) e_if(op_not(v("b")), e_set("empty_b", true)) e_if(op_not(v("c")), e_set("empty_c", true))
Note If the field value extracted by thev
function exists and is not empty, theBool
value true is returned. Otherwise, false is returned. The true value is returned if the field value does not exist or if the field value isNone
. - Solution 2: Use the
e_search
function.e_if(e_search('a==""'), e_set("empty_a", true)) e_if(e_search('b==""'), e_set("empty_b", true)) e_if(e_search('c==""'), e_set("empty_c", true)) # Invalid syntax e_if(e_search('a:""'), e_set("empty_a", true)) e_if(e_search('b:""'), e_set("empty_b", true))
Note In the preceding example of the invalid syntax, thee_search
function is used for partial query. In this case, true is returned if the value of thea: ""
field exists, regardless of whether the value is empty.
- Solution 1 (recommended): Use the
- Result
a: a_value b: // Empty string empty_a: false empty_b: true empty_b: false
Scenario 4: Perform actions based on the logical relationships between field values
- Raw log entries
Log entry 1 http_host: example status: 200 request_method: GET scheme: https header_length: 700 body_length: 1200 Log entry 2 http_host: example.org status: 200 request_method: POST scheme: https header_length: 100 body_length: 800 Log entry 3 http_host: example.net status: 200 request_method: GET scheme: http header_length: 700 body_length: 800 Log entry 4 http_host: aliyundoc.com status: 404 request_method: GET scheme: https header_length: 100 body_length: 300
- Requirement 1
Add the
type
field to all log entries in which the value of thestatus
field is 200. The value of the type field is normal.- DSL orchestration
e_if(e_match("status", "200"), e_set("type", "normal")) Or e_if(e_search('status==200'), e_set("type", "normal"))
Note- You can use one of these solutions in scenarios where the requirements are simple.
- In this case,
status:200
can be used to check whether the value of the status field contains 200. To be more precise, we recommend that you usestatus==200
.
- Result
Log entry 1 type: normal http_host: example status: 200 request_method: GET scheme: https header_length: 700 body_length: 1200 Log entry 2 type: normal http_host: example.org status: 200 request_method: POST scheme: https header_length: 100 body_length: 800 Log entry 3 type: normal http_host: example.net status: 200 request_method: GET scheme: http header_length: 700 body_length: 800 Log entry 4 http_host: aliyundoc.com status: 404 request_method: GET scheme: https header_length: 100 body_length: 300
- DSL orchestration
- Requirement 2
Add the
type
field to all log entries that meet the following conditions: the value of thestatus
field is 200, the value of therequest_method
field is GET, and the value of thescheme
field is https. The value of the type field is normal.- DSL orchestration
e_if(e_search('status==200 and request_method==GET and scheme==https'), e_set("type", "normal")) Or e_if(e_match_all("status", "200", "request_method", "GET", "scheme", "https"), e_set("type", "normal"))
Note- You can use the
e_search
ore_match_all
function to match multiple fields. Thee_search
function is simpler. - In this case,
status:200
can be used to check whether the value of the status field contains 200. To be more precise, we recommend that you usestatus==200
.
- You can use the
- Result
Log entry 1 type: normal http_host: example status: 200 request_method: GET scheme: https header_length: 700 body_length: 1200 Log entry 2 http_host: example.org status: 200 request_method: POST scheme: https header_length: 100 body_length: 800 Log entry 3 http_host: example.net status: 200 request_method: GET scheme: http header_length: 700 body_length: 800 Log entry 4 http_host: aliyundoc.com status: 404 request_method: GET scheme: https header_length: 100 body_length: 300
- DSL orchestration
- Requirement 3
Add the
type
field to all log entries that meet one or more of the following conditions: the value of thestatus
field is 200, the value of therequest_method
field is GET, or the value of thescheme
field is https. The value of the type field is normal.- DSL orchestration
e_if(e_search('status==200 or request_method==GET or scheme==https'), e_set("type", "normal")) Or e_if(e_match_any("status", "200", "request_method", "GET", "scheme", "https"), e_set("type", "normal"))
- Result
Log entry 1 type: normal http_host: example status: 200 request_method: GET scheme: https header_length: 700 body_length: 100 Log entry 2 type: normal http_host: example.org status: 200 request_method: POST scheme: https header_length: 100 body_length: 800 Log entry 3 type: normal http_host: example.net status: 200 request_method: GET scheme: http header_length: 700 body_length: 800 Log entry 4 type: normal http_host: aliyundoc.com status: 404 request_method: GET scheme: https header_length: 100 body_length: 1300
- DSL orchestration
- Requirement 4
Add the
type
field to all log entries that meet the following conditions: the value of thestatus
field is 200, the value of therequest_method
field is GET, and the sum of the values of theheader_length
andbody_length
fields is less than or equal to 1000. The value of the type field is normal.- DSL orchestration
e_if(op_and(e_search('status: 200 and request_method: GET'), op_le(op_sum(v("header_length"), v("body_length")), 1000)), e_set("type", "normal"))
Note You can use thee_search
function and other expression functions for multiple logical operations. - Result
Log entry 1 type: normal http_host: example status: 200 request_method: GET scheme: https header_length: 700 body_length: 100 Log entry 2 http_host: example.org status: 200 request_method: POST scheme: https header_length: 100 body_length: 800 Log entry 3 http_host: example.net status: 200 request_method: GET scheme: http header_length: 700 body_length: 800 Log entry 4 http_host: aliyundoc.com status: 404 request_method: GET scheme: https header_length: 100 body_length: 1300
- DSL orchestration