Startup errors of the data transformation engine - Data Transformation| Alibaba Cloud Documentation Center

This topic describes the causes for startup errors of the data transformation engine and the troubleshooting methods.

The first step of a data transformation task is to start the data transformation engine. Errors may occur in this step if domain specific language (DSL) rules for log transformation fail in the internal security audit by the data transformation engine. Startup errors of the data transformation engine

Error impact

If incorrect LOG DSL rules are detected during the transformation engine startup process, the following error message is returned:

{
  "errorMessage": "ETL config doesn't pass security check, detail: XXXXXX"
}

Note You can find error logs in exception details of the data transformation diagnostic report or in the internal-etl-log Logstore.

If an error occurs during the transformation engine startup phase, the data transformation task will keep retrying until the engine startup succeeds or is manually stopped.
If the retry succeeds after the transformation rules are modified, the data transformation task works properly without losing any logs or generating any duplicate logs.

Common errors and troubleshooting methods

The basic syntax is incorrect.
The compiled transformation rules do not conform to the LOG DSL syntax. For example, the rules contain unpaired parentheses (( )), or commas (,) are incorrectly written as colons (:).
- Error log:
```
{
  "errorMessage": "ETL config doesn't pass security check, detail: invalid syntax"
}
{
  "errorMessage": "ETL config doesn't pass security check, detail: unexpected EOF while parsing"
}
...
```
- Troubleshooting method:
  Locate the specific syntax error based on the traceback information in the error log. As shown in the following figure, e_set("test", v("status")) is incorrectly written as e_set("test": v("status")).
Operators are used, which is not allowed.
All operations in LOG DSL must be completed by using functions in LOG DSL. Operations such as numeric operations and size comparison must be completed by using the op_* function. Operators + - * / > < cannot be used for these operations.
- Error log:
```
{
  "errorMessage": "ETL config doesn't pass security check, detail: invalid type detected: <class `_ast.BinOp`> "
}
```
- Troubleshooting method:
  Check the LOG DSL rules to make sure that all operations, such as numeric operations and size comparison, are completed by using functions in LOG DSL instead of any invalid operators.
- Examples:
```
e_set("b", v("a") - 10) # Error
e_set("b", op_sub(v("a"), 10)) # Correct

e_set("b", v("a") >= v("c")) # Error
e_set("b", op_ge(v("a"), v("c"))) # Correct
```
An error occurred while passing the parameter type in a function or the called function does not exist.
An error occurs if the parameter type passed to a function is different from the parameter type received by the function or the called function does not exist.
- Error log:
```
{
  "errorMessage": "ETL config doesn't pass security check, detail: invalid call in detected: function_name"
}
```
- Troubleshooting method:
  - Check whether the called function exists and the function name is correct. If the function exists and has a correct name, check whether the parameter type passed to the function is correct.
  - Locate the error function based on the traceback information in the error log. As shown in the following figure, the dt_totimestamp function has an error. The function exists in LOG DSL, so you need to check whether the correct parameter type is passed to the dt_totimestamp function.
- Examples:
```
# Error example
e_set("time1", "2019-06-03 2:41:26")
e_set("time2", dt_totimestamp(v("time1")))

# Correct example
e_set("time1", "2019-06-03 2:41:26")
e_set("time2", dt_totimestamp(dt_parse(v("time1"))))

# Correct example
e_set("time1", "2019-06-03 2:41:26")
e_set("time2", dt_parsetimestamp(v("time1")))
```
  - The parameter type received by the dt_totimestamp function is a datetime object, but v("time1") in the code is a string. An error occurs because an incorrect parameter type is passed to the function.
  - To fix the error, use the dt_parse function to convert the string to a datetime object before passing the parameter type to the dt_totimestamp function. You can also use the dt_parsetimestamp function that can receive strings to replace the dt_totimestamp function.
Expression functions are called globally.
The LOG DSL syntax supports two types of functions: global operation functions and expression functions. Only global operation functions can be called globally in the data transformation process. If an expression function is called globally, an error occurs.
- Error log:
```
{
  "errorMessage": "ETL config doesn't pass security check, detail:  invalid type detected: <class '_ast.Expr'>"
}
```
- Troubleshooting method:
  Check whether an expression function is called globally in the data transformation process.
- Examples:
```
# Error example
op_add(v("a"), v("b"))
str_lower(v("name"))

# Correct example
e_set("add", op_add(v("a"), v("b")))
e_set("lower", str_lower(v("name")))
```
Parameters are assigned with variable values.
The LOG DSL syntax does not support value assignment by using variables. Variable values can only be passed in stateless mode.
- Error log:
```
{
  "errorMessage": "ETL config doesn't pass security check, detail: invalid assign detected: variable_name"
}
```
- Troubleshooting method:
  - Check whether the LOG DSL rule uses variables as values.
  - Locate the error based on the traceback information in the error log.
- Examples:
```
# Error example
sum_value = op_add(v("a"), v("b"))
e_set("sum", sum_value)

# Correct example
e_set("sum", op_add(v("a"), v("b")))
```