All Products
Search

This Product

    Elasticsearch:Use an Elastic Agent to collect NGINX log data

    Document Center

    Elasticsearch:Use an Elastic Agent to collect NGINX log data

    Last Updated:Mar 26, 2026

    Use Fleet Server in Kibana to centrally manage Elastic Agents running on multiple Elastic Compute Service (ECS) instances. The agents collect NGINX access and error logs and send them to Elasticsearch, where you can visualize and query the data in Kibana.

    This guide shows how to configure a shared agent policy for two ECS instances that both run NGINX.

    Limitations

    This feature requires an Elasticsearch cluster that uses the Basic Management (V2) architecture. Clusters using the cloud-native Next-Gen Management (V3) architecture do not support creating agent policies in Kibana Fleet. A replacement solution will be available later. To check your cluster's architecture, go to Basic Information > Control Architecture Type.

    Key concepts

    Term Description
    Fleet A centralized management solution in Elasticsearch that manages Elastic Agents across your environment.
    Fleet Agent A lightweight data collection agent that runs on multiple operating systems and gathers data from source systems.
    Fleet Server A service that transfers data collected by Elastic Agents to Elasticsearch.

    Prerequisites

    Before you begin, ensure that you have:

    • An Alibaba Cloud Elasticsearch cluster (version 7.10 or later, excluding 7.16). This guide uses Elasticsearch 8.5.

    • Two ECS instances in the same virtual private cloud (VPC) as the Elasticsearch cluster, each with an elastic IP address (EIP) associated. The ECS instances act as the source servers from which Elastic Agents collect NGINX logs.

    Create an agent policy and add integrations

    Step 1: Create an agent policy

    1. Log on to the Kibana console. For more information, see Log on to the Kibana console.

    2. Go to Management > Fleet.

    3. On the Fleet page, click the Agent policies tab, then click Create agent policy.

    4. In the Create agent policy panel, configure the following settings:

      1. In the Name field, enter nginx-log.

      2. Clear Collect System Logs and Metrics.

      3. Click Advanced options. Under Agent monitoring, clear both Collect agent logs and Collect agent metrics. image.png > Note: This setup collects only NGINX logs. Disable system logs, metrics, and agent-specific data to avoid unnecessary data collection.

    5. Click Create agent policy.

    Step 2: Add a Fleet Server integration

    1. On the Agent policies tab, click the nginx-log policy.

    2. On the Integrations tab, click Add integration.

    3. Search for Fleet Server and select it.

    4. Install the Fleet Server assets:

      1. On the Fleet Server page, click the Settings tab.

      2. Click Install Fleet Server assets, then confirm by clicking Install Fleet Server. > Note: After installation, the integration version appears on the Settings tab.

    5. In the upper-right corner, click Add Fleet Server.

    6. On the Add Fleet Server integration page, enter a name for the integration. The nginx-log agent policy is selected by default.

    7. Click Save and continue, then click Add Elastic Agent later.

    Step 3: Add an NGINX integration

    1. On the nginx-log agent policy page, click the Integrations tab, then click Add integration.

    2. Search for NGINX and select it.

    3. Install the NGINX assets:

      1. On the NGINX page, click the Settings tab.

      2. Click Install NGINX assets, then confirm by clicking Install NGINX. > Note: After installation, the integration version appears on the Settings tab.

    4. In the upper-right corner, click Add NGINX.

    5. On the Add NGINX integration page, configure the integration:

      1. In the Configure integration section, keep the default name nginx-1 in the Integration name field.

      2. Next to Collect logs from NGINX instances, click Change defaults to set a custom log path.

      3. In the Where to add this integration section, click the Existing hosts tab, then select nginx-log from the agent policy list.

    6. Click Save and continue, then click Add Elastic Agent later.

    Add Elastic Agent

    Step 1: Configure a Fleet Server host

    1. Go to Management > Fleet.

    2. On the Fleet page, click the Settings tab.

    3. Under Fleet server hosts, click Edit hosts.

    4. In the Fleet Server hosts panel, enter the private endpoint of your ECS instance in the format https://<Private IP address>:<Port>, for example https://172.16.*.***:8220.

      This example uses the primary private IP address of an ECS instance. For details, see Fleet Server hosts.

    5. Under Outputs, click the edit icon in the Actions column.

    6. In the Edit output panel, enter the private endpoint of your Elasticsearch cluster in the format http://<Private endpoint>:<Port>, for example http://es-cn-uqm3auln80001****.elasticsearch.aliyuncs.com:9200.

    7. Click Save and apply settings, then confirm by clicking Save and deploy.

    Step 2: Add an Elastic Agent to each ECS instance

    1. On the Fleet page, click the Agent policies tab.

    2. Find the nginx-log policy, click the actions icon in the Actions column, and select Add Agent.

    3. In the Select a policy for Fleet Server section, keep the default nginx-log agent policy.

    4. In the Choose a deployment mode for security section, keep the default Quick start.

    5. Under Add your Fleet Server host, click Add host.

    6. Under Generate a service token, click Generate service token.

    7. Under Install Fleet Server to a centralized host, copy the generated code block and run it on the first ECS instance. When Successfully appears in the output, Elastic Agent is installed and running on the instance.

      image.png

    8. Repeat the previous step to add Elastic Agent to the second ECS instance.

    View NGINX logs

    After Elastic Agents start collecting data, view the logs using any of the following methods.

    View logs on Dashboards

    1. Go to Management > Fleet.

    2. On the Fleet page, click the Data streams tab. NGINX logs appear in the dataset list.

    3. In the Actions column for the NGINX dataset, click the actions icon and select View Dashboards.

    4. Select a dashboard, for example [Metrics Nginx] Overview.

      image.png

    Explore logs in Discover

    1. Go to Analytics > Discover.

    2. In the upper-right corner, select a data view, for example metrics-*.

      image.png

    Query logs in Dev Tools

    1. Go to Management > Dev Tools.

    2. Run the following queries to retrieve NGINX log data:

      GET logs-nginx.access-*/_search
GET logs-nginx.error-*/_search