All Products
Search
Document Center

Simple Log Service:Basic syntax

Last Updated:Aug 11, 2023

This topic describes the basic syntax of the domain-specific language (DSL) for Simple Log Service.

Comments

Start the comment of a step with a number sign (#). Examples:

# Specify the default topic. This is a comment at the beginning of a step. 
e_set("__topic__", "access_log")   # Specify the default topic. This is a comment at the end of a step. 

Line wrapping

If the parameter list or string of a function does not fit on a single line, you can separate the parameter list or the string.

  • If the parameter list contains a comma (,), you can split the parameter list immediately after the comma (,).

  • If you want to split a string, use a backslash (\) to indicate that the string continues in the next line.

Examples:

e_set("__topic__", "v1",
        "type", "v2",       # Use a comma (,) as a line feed. 
        "length", 100)
e_set("__topic__", "this is a very long long long .........." \  # Use a backslash (\) as a line feed. 
                            "......long text")

Function invoking

  • Invoke the basic functions

    e_set("abc", "xyz")
    Note

    When you write a data transformation statement, the data types and the number of parameters that you pass to a function must meet the syntax of the function.

  • Pass the basic variable parameters

    str_replace(value, old [,new [,count] ]) 
    Note

    The parameters that are enclosed in the square brackets [] are optional. For example, the new and count parameters in the preceding function are optional. You cannot pass these parameters the same way you pass the named parameters. You must pass these parameters in sequence.

    # Invalid examples
    str_replace("a-b-c", "-", new='%')
    str_replace("a-b-c", "-", new='%', count=1)
    # Valid examples
    str_replace("a-b-c", "-", '%')
    str_replace("a-b-c", "-", '%', 2)
  • Pass the named parameters

    A parameter that has a default value is called a named parameter. For example, the mode parameter in the e_set("abc", "xyz", mode="fill") function is a named parameter.

    • You must pass the values of the named parameters in specific functions based on specific conditions. For more information, see the parameter description of each function.

    • You can pass the value of a named parameter when you configure a parameter in the format of mode=....

    • You can pass multiple named parameters in random order. For example, e_csv("data", ["f1", "f2", "f3"], sep='#', quote="|") is equivalent to e_csv("data", ["f1", "f2", "f3"], quote="|", sep='#').

    Note

    The named parameters follow the non-named parameters.

  • Invoke a combination of functions

    You can pass the returned value of a function as the value of a parameter to another function. In this case, you must make sure that the returned value is of the same data type as the value of the parameter. Examples:

    e_set("abc", v("xyz"))
    e_set("abc", str_lower(v("xyz")))
  • Variable parameters

    You can pass variable parameters to specific functions. The v("f1", ....) function specifies that multiple parameters can be passed. Example: v("f1", "f2", "f3").

    If you need to pass both variable parameters and named parameters, you must place the named parameters after the variable parameters. Example: v("f1", "f2", "f3", "f4", mode="fill").

Operators

  • Comparison operators

    The following comparison operators are supported in the DSL for Simple Log ServiceSimple Log Serviced mode: >, <,>=, <=,! =,==. You can also use the comparison functions that are provided by Log Service to perform the operations.

    • Use comparison operators

      # The following examples show how to use comparison operators. If the comparison condition is evaluated to True, the related log is discarded. 
      e_if(3 > 2, DROP)       # If 3 is greater than 2, the log is discarded. 
      e_if(3 < 2, DROP)       # If 3 is less than 2, the log is discarded. 
      e_if(3 >= 2, DROP)      # If 3 is greater than or equal to 2, the log is discarded. 
      e_if(3 <= 2, DROP)      # If 3 is less than or equal to 2, the log is discarded. 
      e_if(3 == 2, DROP)      # If 3 is equal to 2, the log is discarded. 
      e_if(3 != 2, DROP)      # If 3 is not equal to 2, the log is discarded. 
      e_if(1 < 2 < 3, DROP)   # If 2 is greater than 1 and 2 is less than 3, the log is discarded. 
      e_if(0 < ct_int(v('x')) < 100, DROP) # If the value of the x field is greater than 0 and less than 100, the log is discarded. 
    • Use the comparison functions that are provided by Simple Log Service

      Operation

      Function

      Example

      Equal to (==)

      op_eq

      op_eq(v("name"), "xiao ming")

      Not equal to (!

      op_ne

      op_ne(v("name"), "xiao ming")

      Greater than (>)

      op_gt

      op_gt(ct_int(v("age")), )

      Greater than or equal to (>=)

      op_ge

      op_ge(ct_int(v("age")), 18)

      Less than (<)

      op_lt

      op_lt(ct_int(v("age")), 18)

      Less than or equal to (<=)

      op_le

      op_le(ct_int(v("age")), 18)

  • Logical operators

    The following logical operators are supported in the DSL for Simple Log Service in standard mode: AND, OR, and NOT. You can also use the logical functions that are provided by Simple Log Service to perform the operations.

    • Use logical operators

      # The following examples show how to use logical operators. If the logical condition is evaluated to True, the related log is discarded. 
      e_if(True and False, DROP)     # False is returned.
      e_if(True or False, DROP)      # True is returned.
      e_if(True and not False, DROP) # True is returned.
      e_if(3 > 2 and 1 < 3, DROP)    # True is returned.
      e_if(ct_int(v('x')) > 100 or ct_int(v('y')) < 100, DROP) # If the value of the x field is greater than 100 or the value of the y field is less than 100, True is returned.
    • Use the logical functions that are provided by Simple Log Service

      Operation

      Function

      Example

      Logical operator AND (and)

      op_and

      op_and(op_gt(v("age"), 18), op_lt(v("age"), 31))

      Logical operator OR (or)

      op_or

      op_or(op_le(v("age"), 18), op_gt(v("age"), 65))

      Logical operator NOT (not)

      op_not

      op_not(op_gt(v("age"), 18))

  • Other operators

    You cannot directly use specific DSL operators in standard mode. Simple Log Service provides functions that you can use to perform the operations. The following table describes the operators and functions.

    Operation

    Function

    Example

    Addition (+)

    op_add

    op_add(v("age"), 2)

    Subtraction (-)

    op_sub

    op_sub(v("age"), 2)

    Multiplication (*)

    op_mul

    op_mul(v("size"), 2)

    Exponentiation (**)

    op_pow

    op_pow(v("size"), 2)

    Floor division (//)

    op_div_floor

    op_div_floor(v("bytes"), 1024)

    Modulus (%)

    op_mod

    op_mod(v("age"), 10)

    Negation (-)

    op_neg

    op_neg(v("profit"))

    Existence check (in)

    op_in

    op_in(["pass", "ok"], v("result"))

    Nonexistence check (not in)

    op_not_in

    op_not_in(["pass", "ok"], v("result"))

    String slicing ([ ...])

    op_slice

    op_slice(v("message"), 0, 20)

    In this example, the value of the a field is 3600 * 6. The following examples show an invalid function and a valid function to specify the value for the field.

    # * 
    e_set("a", 3600 * 6)           # Invalid
    e_set("a", op_mul(3600, 6))    # Valid
    
    # /
    e_set("bytes_kb", v("bytes") / 1024)                 # Invalid
    e_set("bytes_kb", op_div_floor(v("bytes"), 1024))    # Valid

True or false evaluation

Some functions check whether a condition is true or false to specify the event processing logic. A condition can be a fixed value or a value returned by an expression function.

You can perform true or false evaluation for all types of data in the DSL for Simple Log Service orchestration. The following table describes the rules for true or false evaluation.

Data type

True

False

Boolean

True, true

False, false

None

N/A

Always false

Numeric

Not 0 or 0.0

0 or 0.0

String

Not empty

Empty string

Bytes

Not empty

Empty bytes

Tuple

Not empty

Empty tuple

List

Not empty

Empty list

Dictionary

Not empty

Empty dictionary

Table

One or more tables exist

No table exists

Datetime

One or more datetime objects exist

No datetime object exists

The following functions can be used to discard logs based on the conditions:

e_if(True, DROP)                     # If the value of the first parameter is True, the log is discarded. 
e_if(1, DROP)                        # If the value of the first parameter is 1, the log is discarded. 
e_if(v("abc"), DROP)                 #If the abc field exists and the value of this field is not empty, the log is discarded. 
e_if(str_isdigit(v("abc")), DROP)    # If the abc field exists and the value of this field contains only digits, the log is discarded.