Logtail plug-in that is used to parse NGINX logs - Simple Log Service

You can use a Logtail plug-in to parse NGINX logs into structured data based on log_format. In this case, a log is parsed into multiple key-value pairs.

Introduction to NGINX logs

NGINX servers generate NGINX access logs based on log_format and access_log. The following code shows the default configuration of log_format and access_log:

log_format main  '$remote_addr - $remote_user [$time_local] "$request" '
                 '$request_time $request_length '
                 '$status $body_bytes_sent "$http_referer" '
                 '"$http_user_agent"';
access_log /var/logs/nginx/access.log main

The following table describes the log fields.

Field	Description
remote_addr	The IP address of the client.
remote_user	The username that is used by the client to send the request.
time_local	The system time of the server. The value must be enclosed in brackets [].
request	The URI and HTTP protocol of the request.
request_time	The time that is required to process the request. Unit: seconds.
request_length	The length of the request. The request line, request headers, and request body are all counted.
status	The status of the request.
body_bytes_sent	The number of bytes in a response that is sent to the client. The response header is not counted.
http_referer	The URL of a source web page.
http_user_agent	The browser information of the client.

Entry point

If you want to use a Logtail plug-in to process logs, you can add a Logtail plug-in configuration when you create or modify a Logtail configuration. For more information, see Overview.

Configuration description

Parameter	Description
NGINX Log Configuration	The log configuration section that you specify in the NGINX configuration file. The log configuration section starts with log_format. Example: `log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$request_time $request_length ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent"';` For more information, see Introduction to NGINX logs.
Original Field	The original field that stores the log content before parsing. Default value: content.
Extracted Field	The key that is automatically extracted based on the value of the NGINX Log Configuration field.
Retain Original Field if Parsing Fails	Specifies whether to retain the original field in the new log that is obtained after the raw log fails to be parsed.
Retain Original Field if Parsing Succeeds	Specifies whether to retain the original field in the new log that is obtained after parsing.
New Name of Original Field	The new name of the original field that you want to retain. If you select Retain Original Field if Parsing Fails or Retain Original Field if Parsing Succeeds, you can rename the original field that stores the original log content.