This topic describes the causes for startup errors of the data transformation engine and the troubleshooting methods.

Error impact
{
"errorMessage": "ETL config doesn't pass security check, detail: XXXXXX"
}
internal-etl-log
Logstore.
- If an error occurs during the transformation engine startup phase, the data transformation task will keep retrying until the engine startup succeeds or is manually stopped.
- If the retry succeeds after the transformation rules are modified, the data transformation task works properly without losing any logs or generating any duplicate logs.
Common errors and troubleshooting methods
- The basic syntax is incorrect.
The compiled transformation rules do not conform to the LOG DSL syntax. For example, the rules contain unpaired parentheses (( )), or commas (,) are incorrectly written as colons (:).
- Error log:
{ "errorMessage": "ETL config doesn't pass security check, detail: invalid syntax" } { "errorMessage": "ETL config doesn't pass security check, detail: unexpected EOF while parsing" } ...
- Troubleshooting method:
Locate the specific syntax error based on the
traceback
information in the error log. As shown in the following figure,e_set("test", v("status"))
is incorrectly written ase_set("test": v("status"))
.
- Error log:
- Operators are used, which is not allowed.
All operations in LOG DSL must be completed by using functions in LOG DSL. Operations such as numeric operations and size comparison must be completed by using the
op_*
function. Operators+ - * / > <
cannot be used for these operations.- Error log:
{ "errorMessage": "ETL config doesn't pass security check, detail: invalid type detected: <class `_ast.BinOp`> " }
- Troubleshooting method:
Check the LOG DSL rules to make sure that all operations, such as numeric operations and size comparison, are completed by using functions in LOG DSL instead of any invalid operators.
- Examples:
e_set("b", v("a") - 10) # Error e_set("b", op_sub(v("a"), 10)) # Correct e_set("b", v("a") >= v("c")) # Error e_set("b", op_ge(v("a"), v("c"))) # Correct
- Error log:
- An error occurred while passing the parameter type in a function or the called function
does not exist.
An error occurs if the parameter type passed to a function is different from the parameter type received by the function or the called function does not exist.
- Error log:
{ "errorMessage": "ETL config doesn't pass security check, detail: invalid call in detected: function_name" }
- Troubleshooting method:
- Check whether the called function exists and the function name is correct. If the function exists and has a correct name, check whether the parameter type passed to the function is correct.
- Locate the error function based on the
traceback
information in the error log. As shown in the following figure, thedt_totimestamp
function has an error. The function exists in LOG DSL, so you need to check whether the correct parameter type is passed to thedt_totimestamp
function.
- Examples:
# Error example e_set("time1", "2019-06-03 2:41:26") e_set("time2", dt_totimestamp(v("time1"))) # Correct example e_set("time1", "2019-06-03 2:41:26") e_set("time2", dt_totimestamp(dt_parse(v("time1")))) # Correct example e_set("time1", "2019-06-03 2:41:26") e_set("time2", dt_parsetimestamp(v("time1")))
- The parameter type received by the
dt_totimestamp
function is a datetime object, butv("time1")
in the code is a string. An error occurs because an incorrect parameter type is passed to the function. - To fix the error, use the
dt_parse
function to convert the string to a datetime object before passing the parameter type to thedt_totimestamp
function. You can also use thedt_parsetimestamp
function that can receive strings to replace thedt_totimestamp
function.
- The parameter type received by the
- Error log:
- Expression functions are called globally.
The LOG DSL syntax supports two types of functions: global operation functions and expression functions. Only global operation functions can be called globally in the data transformation process. If an expression function is called globally, an error occurs.
- Error log:
{ "errorMessage": "ETL config doesn't pass security check, detail: invalid type detected: <class '_ast.Expr'>" }
- Troubleshooting method:
Check whether an expression function is called globally in the data transformation process.
- Examples:
# Error example op_add(v("a"), v("b")) str_lower(v("name")) # Correct example e_set("add", op_add(v("a"), v("b"))) e_set("lower", str_lower(v("name")))
- Error log:
- Parameters are assigned with variable values.
The LOG DSL syntax does not support value assignment by using variables. Variable values can only be passed in stateless mode.
- Error log:
{ "errorMessage": "ETL config doesn't pass security check, detail: invalid assign detected: variable_name" }
- Troubleshooting method:
- Check whether the LOG DSL rule uses variables as values.
- Locate the error based on the
traceback
information in the error log.
- Examples:
# Error example sum_value = op_add(v("a"), v("b")) e_set("sum", sum_value) # Correct example e_set("sum", op_add(v("a"), v("b")))
- Error log: