This topic describes the basic syntax that is used in the domain-specific language (DSL) of Log Service.

Comments

Start the comment of a step with a number sign (#). Examples:
# Set the default topic. This is a comment placed at the beginning of a step.
e_set("__topic__", "access_log")   # Set the default topic. This is a comment placed at the end of a step.

Line wrapping

If the parameter list or string of a function does not fit on a single line, you can break the parameter list or the string. If the parameter list contains a comma (,), you can break the parameter list after the comma (,). If you want to break a string, use a backslash (\) to indicate that the string continues in the next line. Examples:
e_set("__topic__", "v1",
        "type", "v2",       # Break the parameter list with a comma (,).
        "length", 100)
e_set("__topic__", "this is a very long long long .........." \
                            "......long text")

Function invoking

  • Invoke the basic functions
    For example:
    e_set("abc", "xyz")
    Note The number of parameter values that you pass to the function must be the same as the number of parameters defined for the function.
  • Pass the basic variable parameters
    For example:
    str_replace(value, old [,new [,count] ]) 
    Note The parameters in the square brackets are optional, for example, the new and count parameters in the preceding function. You cannot pass these parameters in the same way as the named parameters.
    # The following functions are invalid.
    str_replace("a-b-c", "-", new='%')
    str_replace("a-b-c", "-", new='%', count=1)
    You must pass these parameters in sequence.
    str_replace("a-b-c", "-", '%')
    str_replace("a-b-c", "-", '%', 2)
  • Pass the named parameters
    A parameter with a default value is called a named parameter. For example, the mode parameter in the e_set("abc", "xyz", mode="fill") function is a named parameter.
    • You must pass the values for the named parameters in some functions under specific conditions. For more information, see the parameter description of each function.
    • You can pass the value of a named parameter by setting a parameter in the format of mode=....
    • Multiple named parameters can be passed in random order. For example, e_csv("data", ["f1", "f2", "f3"], sep='#', quote="|") is equivalent to e_csv("data", ["f1", "f2", "f3"], quote="|", sep='#').
    Note The named parameters always come after the non-named parameters.
  • Invoke a combination of functions
    You can pass the returned value of a function as the value of a parameter to another function. In this case, you must make sure that the returned value is of the same data type as the value of the parameter. Examples:
    e_set("abc", v("xyz"))
    e_set("abc", str_lower(v("xyz")))
  • Variable parameters

    You can pass variable parameters to some functions. The v("f1", ....) function indicates that multiple parameters can be passed, for example, v("f1", "f2", "f3").

    If you need to pass both variable parameters and named parameters, you must place the named parameters after the variable parameters, for example, v("f1", "f2", "f3", "f4", mode="fill").

Operator

You cannot use operators in DSL of the standard mode. Instead, you can use functions to perform the operations.
Operation Function Example
Addition (+) op_add op_add(v("age"), 2)
Subtraction (-) op_sub op_sub(v("age"), 2)
Multiplication (*) op_mul op_mul(v("size"), 2)
Exponentiation (**) op_pow op_pow(v("size"), 2)
Floor division (//) op_div_floor op_div_floor(v("bytes"), 1024)
Modulus (%) op_mod op_mod(v("age"), 10)
Negation (-) op_neg op_neg(v("profit"))
Existence check (in) op_in op_in(["pass", "ok"], v("result"))
Nonexistence check (not in) op_not_in op_not_in(["pass", "ok"], v("result"))
Logical operator AND (and) op_and op_and(op_gt(v("age"), 18), op_lt(v("age"), 31))
Logical operator OR (or) op_or op_or(op_le(v("age"), 18), op_gt(v("age"), 65))
Logical operator NOT (not) op_not op_not(op_gt(v("age"), 18))
Equal to (==) op_eq op_eq(v("name"), "xiao ming")
Not equal to (! =) op_ne op_ne(v("name"), "xiao ming")
Greater than (>) op_gt op_gt(ct_int(v("age")), )
Greater than or equal to (>=) op_ge op_ge(ct_int(v("age")), 18)
Less than (<) op_lt op_lt(ct_int(v("age")), 18)
Less than or equal to (<=) op_le op_le(ct_int(v("age")), 18)
String slicing ([ ...]) op_slice op_slice(v("message"), 0, 20)
Assume that the value of the a field is 3600 * 6. The following examples show an invalid function and a valid function to set the value for the field.
# * 
e_set("a", 3600 * 6)    # Invalid
e_set("a", op_mul(3600, 6))    # Valid

# /
e_set("bytes_kb", v("bytes") / 1024)    # Invalid
e_set("bytes_kb", op_div_floor(v("bytes"), 1024))    # Valid

True or false evaluation

Some functions determine the event processing logic by checking whether a condition is true or false. A condition can be a fixed value or a value returned by an expression function.

You can perform a true or false evaluation of all data types in DSL orchestration. The following table lists the rules for true or false evaluation of various data types.
Data type True False
Boolean True, true False, false
None - Always false
Numeric Not 0 or 0.0 0 or 0.0
String Not empty Empty string
Bytes Not empty Empty bytes
Tuple Not empty Empty tuple
List Not empty Empty list
Dictionary Not empty Empty dictionary
Table One or more tables exist No table exists
Datetime One or more datetime objects exist No datetime object exists
The following functions can be used to discard log entries based on the conditions:
e_if(True, DROP)      # Discard log entries if the value of the first parameter is true.
e_if(1, DROP)       # Discard log entries if the value of the first parameter is 1.
e_if(v("abc"), DROP)      # Discard log entries if the abc field exists and is not empty.
e_if(str_isdigit(v("abc")), DROP) # Discard log entries if the abc field exists and contains only digits.