This topic describes basic syntax of LOG domain specific language (DSL).


Start the comment of a step with a number sign (#). Example:
# Set the default topic. (This is a comment placed at the beginning of a step.)
e_set("__topic__", "access_log")   # Set the default topic. (This is a comment placed at the end of a step.)

Line wrapping

When the parameter list of a function does not fit on a single line, you may need to break the parameter list. If the parameter list contains a comma (,), break the parameter list after the comma (,). If a string is too long, break it with a backslash (\) to indicate that the string continues in the next line. Example:
e_set("__topic__", "v1",
        "type", "v2",       # Break after a comma.
        "length", 100)
e_set("__topic__", "this is a very long long long..." \
                            "...long text")

Function calls

  • Basic function call
    e_set("abc", "xyz")
    Note The number of parameter values that you pass to the function must be the same as the number of parameters defined for the function.
  • Named parameters
    A parameter with a default value is called a named parameter. For example, the mode parameter in the e_set("abc", "xyz", mode="fill") function is a named parameter.
    • The named parameters in some functions must be set under specific conditions. For more information, see the parameter description of each function.
    • Set a named parameter in the following way: mode=....
    • Multiple named parameters can be passed in any order. For example, e_csv("data", ["f1", "f2", "f3"], sep='#', quote="|") is equivalent to e_csv("data", ["f1", "f2", "f3"], quote="|", sep='#').
    Note In a function call, named parameters always come after non-named parameters.
  • Function call combination
    You can pass the return value of a function as a parameter to another function. In this case, make sure that the data types of the return value and the parameter are the same. Example:
    e_set("abc", v("xyz"))
    e_set("abc", str_lower(v("xyz")))
  • Variable parameters

    Some functions support variable parameters. v("f1", ...) indicates that more parameters can be passed, for example, v("f1", "f2", "f3").

    If you need to pass variable parameters and named parameters at the same time, place named parameters at the end of the parameter list, for example, v("f1", "f2", "f3", "f4", mode="fill").


LOG DSL in standard mode does not support operators directly. You can use functions to implement the corresponding operations.
Operation Function Example
Addition (+) op_add op_add(v("age"), 2)
Subtraction (-) op_sub op_sub(v("age"), 2)
Multiplication (*) op_mul op_mul(v("size"), 2)
Exponentiation (**) op_pow op_pow(v("size"), 2)
Floor division (//) op_div_floor op_div_floor(v("bytes"), 1024)
Modulus (%) op_mod op_mod(v("age"), 10)
Negation (-) op_neg op_neg(v("profit"))
Existence check (in) op_in op_in(["pass", "ok"], v("result"))
Nonexistence check (not in) op_not_in op_in(["pass", "ok"], v("result"))
Logical AND (and) op_and op_and(op_gt(v("age"), 18), op_lt(v("age"), 31))
Logical OR (or) op_or op_or(op_le(v("age"), 18), op_gt(v("age"), 65))
Logical NOT (not) op_not op_not(op_gt(v("age"), 18))
Equal (==) op_eq op_eq(v("name"), "xiao ming")
Not equal to (!=) op_ne op_ne(v("name"), "xiao ming")
Greater than (>) op_gt op_gt(ct_int(v("age")), )
Greater than or equal to (>=) op_ge op_ge(ct_int(v("age")), 18)
Less than (<) op_lt op_lt(ct_int(v("age")), 18)
Less than or equal to (<=) op_le op_le(ct_int(v("age")), 18)
String slicing[ ...] op_slice op_slice(v("message"), 0, 20)
# * 
e_set("a", 3600 * 6)    # Wrong
e_set("a", op_mul(3600, 6))    # Correct

# /
e_set("bytes_kb", v("bytes") / 1024)    # Wrong
e_set("bytes_kb", op_div_floor(v("bytes"), 1024))    # Correct

True or false judgment

Some functions determine the event processing logic by checking whether a condition is true or false. A condition can be a fixed value or a value returned by an expression function.

LOG DSL supports true or false judgment on all data types according to the rules described in the following table.
Data type True False
Boolean True or true False or false
None - Always False
Number Not 0 or 0.0 0 or 0.0
String Not empty Empty
Byte Not empty Empty
Tuple Not empty Empty
List Not empty Empty
Dictionary Not empty Empty
Table True if the object exists False if the value is NULL (None)
Datetime True if the object exists False if the value is NULL (None)
The following functions discard events based on the conditions:
e_if(True, DROP)      # Discard all events.
e_if(1, DROP)       # Discard all events.
e_if(v("abc"), DROP)      # Discard an event if the abc field exists and is not empty.
e_if(str_isdigit(v("abc")), DROP) # Discard an event if the abc field exists and consists of only digits.