Fix res_log_logstore_pull Errors in SLS Log Transformation - Simple Log Service

This topic describes the causes of resource data loading errors in data transformation rules and provides troubleshooting and resolution methods.

Error handling mechanism

For more information about the error handling mechanism for resource loading errors, see res_log_logstore_pull, res_rds_mysql, and res_oss_file.

General troubleshooting

Using a resource function alone causes resource loading errors.

Sample transformation rules

res_log_logstore_pull(endpoint="cn-shenzhen.log.aliyuncs.com",ak_id="xxx",
        ak_secret="xxx",project="etl-test-shenzhen",
        fields=["__source__"]),field="processid",output_fields=["xx"]

res_rds_mysql(address="xx",username="xx",password="xx",database="xx")

res_oss_file(endpoint='xx',ak_id="xx",ak_key="xx",bucket='xx', file='xx',format='xx',change_detect_interval=0)

Error log

aliyun.log.logexception.LogException: {"errorCode": "InvalidEtlConfig", "errorMessage": "ETL config doesn't pass security check, detail: invalid type detected: <class '_ast.Expr'>", "requestId": ""}

Troubleshooting method

The preceding error log indicates a syntax error in the transformation configuration. This occurs when you use only the res_log_logstore_pull, res_rds_mysql, or res_oss_file function. Resource functions cannot be used alone.
Solution

You can use the e_table_map function together with the resource function. For more information, see Resource functions.

References

For errors that occur when pulling data from another LogStore, see LogStore data (dimension table) loading errors.
For errors that occur when pulling data from OSS, see OSS data loading errors.
For syntax errors that occur when pulling data from RDS, see RDS MySQL data loading syntax errors.