Overview

On the logstores page of the log Service Console, select diagnose to view all log collection error messages of the current Logstore. This topic describes the types of common errors that are reported when Log service collects data and the corresponding solutions.

Detail

Error type Server error description Method
LOG_GROUP_WAIT_TOO_LONG_ALARM Data packets wait a long time from when they are generated to when they are sent. Check whether the data is sent properly, that the data volume exceeds the default settings, the quota is insufficient, or a network problem exists.
LOGFILE_PERMINSSION_ALARM Logtail has no permission to read the specified file. Check the Logtail startup account on the server. We recommend that you start Logtail as root.
SPLIT_LOG_FAIL_ALARM The regular expression fails to match with the first line of a log entry and cannot parse the log entry into multiple lines. Check the correctness of the regular expression to match the first line. For a single line log, you can set the regular expression to .*.
MULTI_CONFIG_MATCH_ALARM A file can be collected by only one Logtail configuration.
Note: Docker standard output can be collected by multiple Logtail configurations.
Check whether a file is collected in multiple configs. If yes, delete unnecessary configs.
REGEX_MATCH_ALARM The log content does not match the regular expression in regular expression mode. Copy the log sample from the error content for re-matching and generate a new regular expression for parsing.
PARSE_LOG_FAIL_ALARM Logtail fails to parse logs because the log format does not conform to the definition in the parsing modes such as JSON and delimiter. Click the error message to view the details.
CATEGORY_CONFIG_ALARM The Logtail configuration is invalid. A common reason is that a regular expression fails to extract the file path as a Topic. If this error occurs, submit a ticket.
LOGTAIL_CRASH_ALARM Logtail crashes because it has exceeded the upper limit of machine resource usage. Modify the upper limit of CPU and memory usage. For more information, see configure startup parameters. If you have any questions, open a ticket.
REGISTER_INOTIFY_FAIL_ALARM Failed to register the log listener in the Linux because the folder is not authorized or has been deleted. Check whether Logtail has the permission to access the folder and whether the folder has been deleted.
DISCARD_DATA_ALARM The CPU resources for configuring Logtail are insufficient or the incoming traffic to log service is restricted.

For more information about how to modify the upper limit of CPU usage or concurrent inbound traffic, see configure startup parameters. If you have any questions, open a ticket.

SEND_DATA_FAIL_ALARM
  • The Alibaba Cloud account has not created any AccessKey pair.
  • The Logtail client cannot connect to the log service server, or the network signal is bad.
  • The write quota in log service is insufficient.
  • Your Alibaba Cloud account creates an AccessKey pair.
  • In Linux. The check for a local best profile /usr/local/ilogtail/ilogtail_config.json, curl [$IP command checking to see if there was any back.
    Note: the IP address of the [$IP] server.
  • Increase the number of shards for the Logstore so that more data can be written to the Logstore.
SEND_QUOTA_EXCEED_ALARM The traffic of writing logs exceeds the limit. Resize partitions in the log service console.
READ_LOG_DELAY_ALARM Log collection lags behind log generation. Generally, this error is caused by the insufficient CPU resources configured for Logtail or network bandwidth throttling. For more information, see configure Logtail startup parameters and modify the CPU usage or limits on concurrent inbound traffic to the Internet. If you have any questions, open a ticket .
DROP_LOG_ALARM Log collection lags behind log generation, and there are more than 20 unprocessed log rotations. Generally, this error is caused by the insufficient CPU resources configured for Logtail or the network bandwidth throttling. Configure startup parameters and modify the CPU usage or inbound concurrency limit. If you have any questions, open a ticket .
LOGDIR_PERMINSSION_ALARM Logtail has no permission to read the log monitoring directory. Check whether the log monitoring directory exists. If the directory exists, check the directory permission setting.
ENCODING_CONVERT_ALARM Log transcoding fails. Check whether the configured log encoding method is consistent with the actual log encoding.
OUTDATED_LOG_ALARM Logs expire with a time lag of more than 12 hours. Cause:
  • Log parsing is more than 12 hours behind schedule.
  • The custom time field is incorrectly configured.
  • The time output of the log entry program is abnormal.
  • Check whether READ_LOG_DELAY_ALARM exists. If READ_LOG_DELAY_ALARM is found, check the time field configuration.
  • Check the time field configuration. If the time field settings are correct, check whether the time of the log entry program is correct. If you have any questions, please submit work orders.
STAT_LIMIT_ALARM The number of files in the log collection configuration directory exceeds the upper limit. Check whether the log collection configuration directory contains a large amount of files and subdirectories, and properly configure the monitored root directory and the maximum monitoring depth of the directory.
DROP_DATA_ALARM When the log collection process exits, writing logs to the local disk expires. In this case, the logs that have not been written to the local disk will be discarded. This error is usually caused by a serious blocking of log collection. See the Logtail configuration startup parameter to modify the upper limit of CPU usage or concurrent inbound traffic of the network. If you have any questions, open a ticket.
INPUT_COLLECT_ALARM An error occurred while collecting the input source. See error handling.
HTTP_LOAD_ADDRESS_ALARM The Address in the HTTP input source is invalid. Check the validity of the Address.
HTTP_COLLECT_ALARM An exception occurred during HTTP collection. Fix the error based on the error message. Timeout may occur.
FILTER_INIT_ALARM An exception occurred during filter initialization. Generally, the regular expression of the filter is invalid. Therefore, fix the error as prompted.
INPUT_CANAL_ALARM An error occurs when MySQL binlogs run. Fix the error according to the error message. The canal service may restart when logtail config is updated. Errors caused by service restart can be ignored.
CANAL_INVALID_ALARM The internal state of MySQL binlogs is abnormal. Check whether the table schema is being modified when the error occurs. This error generally occurs when meta data changes are caused by table schema modifications during binlog running. In other cases, submit a ticket.
MYSQL_INIT_ALARM An error occurs during MySQL initialization. See error handling.
MYSQL_CHECKPOING_ALARM The MySQL Checkpoint format is invalid. Determine whether to modify the Checkpoint configuration. Submit a ticket if you are still unable to resolve this problem.
MYSQL_TIMEOUT_ALARM The MySQL Query timed out. Check whether the MySQL server and network connection are normal.
MYSQL_PARSE_ALARM Parsing MySQL Query Results fails. Check whether the checkpoint format configured on MySQL is consistent with the format of the corresponding fields.
AGGREGATOR_ADD_ALARM Failed to add data to queue. Ignore the error if the actual data amount is large because the error is caused by excessive fast data sending.
ANCHOR_FIND_ALARM Possible error causes are anchor plug-in faults, config faults, or mismatch between the config and log. Click the error message to view details, which may contain the following error types. Check whether the corresponding config encounters faults accordingly.
  • anchor cannot find key: the SourceKey is specified in the config but its corresponding field cannot be found in the log.
  • anchor no start: The keywords specified by Start cannot be found in the value of SourceKey.
  • anchor no stop: The keywords specified by Stop cannot be found in the value of SourceKey.
ANCHOR_JSON_ALARM An error occurs when the anchor plug-in Performs JSON expansion on the keywords specified by Start and Stop. Click the error message to view details. Check the keywords and the related config. Check whether there is any config fault or invalid log.
CANAL_RUNTIME_ALARM An error occurs when the Binlog plug-in runs. Click the error message to view details, and then handle the error accordingly. The error is related to the connected MySQL master database.
CHECKPOINT_INVALID_ALARM The plug-in Fails to parse the Checkpoint. Click the error message to view details, and then handle the error according to the checkpoint key, checkpoint content (the first 1,024 bytes), and other information.
DIR_EXCEED_LIMIT_ALARM The number of directories for simultaneous monitoring exceeds the upper limit. Check whether the config of the current Logstore and other configs applied on Logtail contain excessive directories. If yes, configure the monitoring root directory and the maximum directory monitoring depth as needed.
DOCKER_FILE_MAPPING_ALARM Logtail fails to add Docker file mapping after running the Logtail command. Click the error message to view details, and then handle the error accordingly.
DOCKER_FILE_MATCH_ALARM The specified file cannot be found in Docker. Click the error message to view details, and then handle the error according to the container information and file path.
DOCKER_REGEX_COMPILE_ALARM The docker stdout plug-in Fails to construct a regular expression based on BeginLineRegex in the config. Click the error message to view details, and then check whether the regular expression is correct.
DOCKER_STDOUT_INIT_ALARM The docker stdout collection initialization fails. Click the error message to view details, which may contain the following error types:
  • Host..... version.... error: Check whether the Docker engine specified in the config is accessible.
  • load checkpoint error: ignore the error if there is no impact because the error is caused by checkpoint loading failure.
  • container...: the error message returned because the specified container has an invalid label value. Currently, only the stdout and stderr labels are allowed. Check for detailed errors.
DOCKER_STDOUT_START_ALARM The stdout file size exceeds the upper limit during stdout collection initialization. Ignore the error because, in normal cases, the stdout file already exists at the first collection.
DOCKER_STDOUT_STAT_ALARM The error message returned because the stdout file of Docker cannot be checked. This can be ignored because the container cannot be accessed when it exits.
FILE_READER_EXCEED_ALARM The number of objects opened by Logtail exceeds the upper limit. A large number of files are in the collection Status. Check whether the collection configuration is reasonable.
GEOIP_ALARM The geoip plug-in is incorrect. Click the error message to view details, which may contain the following error types:
  • Invalidp...: the error message returned because the specified IP address cannot be obtained. Check whether SourceKey in the config is correct or whether an invalid log exists.
  • parseip...: failed to resolve the city based on the IP address. View the detailed error information for troubleshooting.
  • cannot find key...: The plug-in cannot find the specified SourceKey from the log. Check whether the config is faulty or whether an invalid log exists.
HTTP_INIT_ALARM The HTTP plug-in incorrectly compiles the ResponseStringMatch regular expression specified in the config. Click the error message to view details, and then check whether the regular expression is correct.
HTTP_PARSE_ALARM The HTTP plug-in fails to obtain HTTP responses. Click the error message to view details, and then check the config or the requested HTTP server.
INIT_CHECKPOINT_ALARM The Binlog plug-in fails to load the checkpoint. In this case, the plug-in will ignore the checkpoint and recollect the log. Click the error message to view details, and then determine whether the error can be ignored.
LOAD_LOCAL_EVENT_ALARM Logtail handles a local event. This warning generally does not appear, and it is only necessary to troubleshoot if the warning is caused by humans. Click the error message to view details, and then handle the error according to the file name, config name, project, Logstore, and other information.
LOG_REGEX_FIND_ALARM The processor_split_log_regex or processor_split_log_string plug-in cannot obtain the SplitKey specified by the config from the log. Click the error message to view details, and then check whether the config is faulty.
LUMBER_CONNECTION_ALARM The service_lumberjack plug-in Fails. An error occurs when you stop the server plug-in. Click the error message to view details, and then handle the error accordingly. In normal cases, this error can be ignored.
LUMBER_LISTEN_ALARM An error occurs when the service_lumberjack plug-in initializes the listener. Click the error message to view details, which may contain the following error types:
  • init tls error...: Check whether the TLS configurations are correct.
  • listen init error...: Check whether the address-related settings are correct.
LZ4_COMPRESS_FAIL_ALARM An error occurs when Logtail executes LZ4 compression. Click the error message to view details, and then handle the error according to the values of log lines, project, category, and region.
MYSQL_CHECKPOINT_ALARM The MySQL plug encounters a checkpoint error. Click the error message to view details, which may contain the following error types:
  • init checkpoint error...: initializing the checkpoint fails. In this case, check whether the checkpoint column specified by the configuration and the corresponding values are correct.
  • not matched checkpoint...: the checkpoint information does not match. In this case, check whether the mismatch is caused by manual operations, for example, config updates. If yes, ignore the error.
NGINX_STATUS_COLLECT_ALARM An error occurs when the nginx_status plug-in obtains the server status. Click the error message to view details, and then handle the error according to the URL and other information.
NGINX_STATUS_INIT_ALARM The nginx_status plug-in Fails to initiate and parse the URL specified by the config. Click the error message to view details, and then check whether the address is correct according to the URL.
OPEN_FILE_LIMIT_ALARM Logtail cannot open new files because the number of opened files has exceeded the upper limit. Click the error message to view details, and then handle the error according to the log file path, Project, Logstore, and other information.
OPEN_LOGFILE_FAIL_ALARM An error occurs when Logtail opens a file. Click the error message to view details, and then handle the error according to the log file path, Project, Logstore, and other information.
PARSE_DOCKER_LINE_ALARM The service_docker_stdout plug-in Fails to parse the log. Click the error message to view details, which may contain the following error types:
  • parse docker line error:emptyline: the log is empty.
  • parse json docker line error...: parsing logs in JSON format fails. In this case, handle the error according to the error message and the first 512 bytes of the log.
  • parse cri docker line error...: parses the log in CRI format. In this case, handle the error according to the error message and the first 512 bytes of the log.
PLUGIN_ALARM Plug-In initialization and related call errors occurred. Click the error message to view details, which may contain the following error types. Handle the error accordingly.
  • init plugin error...: Failed to initialize the plug-in.
  • hold on error...: The Plug-In fails to be paused.
  • resumeerror...: resumes the plug-in failure.
  • start service error...: failed to start service input-type plug-ins.
  • stop service error...: stopping service input-type plug-ins fails.
PROCESSOR_INIT_ALARM The Regex plug-in Fails to compile the Regex regular expression specified by the config. Click the error message to view details, and then check whether the regular expression is correct.
PROCESS_TOO_SLOW_ALARM Logtail parses logs too slowly.
  1. Click the error message to view details, and then determine whether the slow parsing is normal according to the number of logs, buffer size, and parsing time.
  2. If the slow parsing is abnormal, check whether inappropriate parsing configurations exist. For example, the processes on the node where Logtail resides occupy excessive CPU resources, or an inefficient regular expression exists.
REDIS_PARSE_ADDRESS_ALARM The Redis plug-in Fails to parse the ServerUrls specified by the config. Click the error message to view details, and then check the URL.
REGEX_FIND_ALARM The Regex plug-in cannot find the fields specified by SourceKey in the config from the log. Click the error message to view details, and then check whether SourceKey settings are incorrect or an invalid log exists.
REGEX_UNMATCHED_ALARM The regex plug-in fails to match the regular expression. Click the error message to view details, which may contain the following error types. Handle the error accordingly, for example, checking whether the config is correct.
  • unmatch this log content...: the log cannot match the regular expression in the configuration
  • match result count less...: the number of matched logs is less than that of Keys specified in the config.
SAME_CONFIG_ALARM There are configs with the same name in a Logstore. In this case, logtail chooses one to collect the log, and the others will be discarded. Click the error message to view details, and then handle the error according to the config path and other information.
SPLIT_FIND_ALARM The split_char and split_string plug-ins cannot find the fields specified by SourceKey in the config from the log. Click the error message to view details, and then check whether SourceKey settings are incorrect or an invalid log exists.
SPLIT_LOG_ALARM The processor_split_char and processor_split_string plug-in are incorrect. The number of parsed fields varies with the key specified by SplitKeys. Click the error message to view details, and then check whether SourceKey settings are incorrect or an invalid log exists.
STAT_FILE_ALARM An error occurs when the plug-in collects files through the LogFileReader object. Click the error message to view details, and then handle the error according to the file path and other information.
SERVICE_SYSLOG_INIT_ALARM The service_syslog plug-in initialization fails. Click the error message to view details, and check whether Address in the config is correct.
SERVICE_SYSLOG_STREAM_ALARM An error occurs when the service_syslog Plug-In collects data through TCP. Click the error message to view details, which may contain the following error types. Handle the error accordingly.
  • Accepterror...: an error occurs during Accept execution. In this case, the plug-in waits for a while and tries again.
  • setKeepAliveerror...: The Keep Alive plug-in fails to be set. In this case, the plug-in skips the error and runs properly.
  • connectioni/otimeout...: reading data through TCP expires. In this case, the plug-in resets the expiration duration and reads data properly.
  • scanerror...: an error occurs when the plug-in reads data through TCP. In this case, the plug-in waits for a while and tries again.
SERVICE_SYSLOG_PACKET_ALARM An error occurs when the service_syslog Plug-In collects data through UDP. Click the error message to view details, which may contain the following error types. Handle the error accordingly.
  • connectioni/otimeout...: Reading timeout over UDP, the plug-in will reset the timeout and continue reading.
  • read from error...: an error occurs when the plug-in reads data through UDP. In this case, the plug-in waits for a while and tries again.

Application scope

  • Log Service