All Products
Search
Document Center

:Parsing in NGINX mode

Last Updated:Dec 20, 2023

You can use a Logtail plug-in to parse NGINX logs into structured data based on log_format. In this case, a log is parsed into multiple key-value pairs.

Introduction to NGINX logs

NGINX servers generate NGINX access logs based on log_format and access_log. The following code shows the default configuration of log_format and access_log:

log_format main  '$remote_addr - $remote_user [$time_local] "$request" '
                 '$request_time $request_length '
                 '$status $body_bytes_sent "$http_referer" '
                 '"$http_user_agent"';
access_log /var/logs/nginx/access.log main

The following table describes the log fields.

Field

Description

remote_addr

The IP address of the client.

remote_user

The username that is used by the client to send the request.

time_local

The system time of the server. The value must be enclosed in brackets [].

request

The URI and HTTP protocol of the request.

request_time

The time that is required to process the request. Unit: seconds.

request_length

The length of the request. The request line, request headers, and request body are all counted.

status

The status of the request.

body_bytes_sent

The number of bytes in a response that is sent to the client. The response header is not counted.

http_referer

The URL of a source web page.

http_user_agent

The browser information of the client.

Entry point

If you want to use a Logtail plug-in to process logs, you can add a Logtail plug-in configuration when you create or modify a Logtail configuration. For more information, see Overview.

Configuration description

Parameter

Description

NGINX Log Configuration

The log configuration section that is specified in the NGINX configuration file. The section starts with log_format. Example:

log_format main  '$remote_addr - $remote_user [$time_local] "$request" '
                 '$request_time $request_length '
                 '$status $body_bytes_sent "$http_referer" '
                 '"$http_user_agent"';

For more information, see Introduction to NGINX logs.

Original Field

The original field that stores log content before parsing. Default value: content.

Extracted Field

The key that is automatically extracted based on NGINX Log Configuration.

Retain Original Field if Parsing Fails

After you select Retain Original Field if Parsing Fails, the original field is retained if parsing fails.

Retain Original Field if Parsing Succeeds

After you select Retain Original Field if Parsing Succeeds, the original field is retained if parsing is successful.

New Name of Original Field

If you select Retain Original Field if Parsing Fails or Retain Original Field if Parsing Succeeds, you can rename the original field that stores log content.