All Products
Search
Document Center

Elasticsearch:Use Metricbeat to collect system data and NGINX service data

Last Updated:Oct 24, 2023

This topic describes how to use Alibaba Cloud Metricbeat to collect system data and NGINX service data and then generate visual charts. The system data includes CPU utilization, memory usage, disk I/O, and network I/O.

Prerequisites

Use Metricbeat to collect system data

  1. Log on to the Alibaba Cloud Elasticsearch console.
  2. In the Create Shipper section of the page that appears, click Metricbeat.

  3. Install and configure the shipper.

    For more information, see Collect the logs of an ECS instance and Prepare a YML configuration file for a shipper. The following figure shows the configurations that are used in this topic.Metricbeat

    Note
    • If you select Enable Kibana Monitoring, the system enables Metricbeat service monitoring in the Kibana console.

    • If you select Enable Kibana Dashboard, the system generates charts in the Kibana console. You do not need to configure the YML file. Alibaba Cloud Kibana is configured in a VPC. You must enable the Kibana private network access feature on the Kibana configuration page. For more information, see Configure a public or private IP address whitelist for Kibana.

    • The system module is enabled by default. You do not need to specify Shipper YML Configuration.

  4. Click Next.

  5. Select the target ECS instance.

    Select the target ECS instance
    Important

    If you are installing a shipper for the first time, click Authorize Now. Then, follow the instructions to authorize Elasticsearch to access ECS instances.

  6. Enable the shipper and check whether the shipper installation succeeds.

    1. Click Enable.

      Then, the Enable Shipper message appears.

    2. Click Back to Beats Shippers. In the Manage Shippers section of the Beats Data Shippers page, view the installed shipper.

    3. After the state of the shipper changes to Enabled 1/1, click View Instances in the Actions column.

    4. In the View Instances pane, check whether the shipper installation on the ECS instance succeeds. If the value of Installed Shippers is Heartbeat Normal, the shipper installation succeeds.

      Shipper installation succeeded
  7. View the collected data.

    1. Log on to the Kibana console of your Elasticsearch cluster.

      For more information, see Log on to the Kibana console.

    2. In the left-side navigation pane, click Dashboard.

    3. In the Dashboards section, click [Metricbeat System] Overview. On the page that appears, click the target Metricbeat system and view the dashboard for the system.

      Dashboard

Use Metricbeat to collect NGINX service data

Prerequisites: The stub_status module is enabled for the NGINX service. The ngx_http_stub_status_module module measures the number of requests received and that of requests processed by the NGINX service. Therefore, you must enable stub_status in the nginx.conf file.

location /status {
           stub_status on;
           access_log off;
        }
Important

The value of server_status_path in the metricbeat.yml file must be the same as that of status in the nginx.conf file.

  1. Log on to the Alibaba Cloud Elasticsearch console.
  2. In the Create Shipper section of the page that appears, click Metricbeat.

  3. Install and configure the shipper.

    For more information, see Collect the logs of an ECS instance and Prepare a YML configuration file for a shipper. The following figure shows the configurations that are used in this topic.Configure the shipper for the NGINX service

    Add the following script in the metricbeat.yml file:

    metricbeat.modules:
    - module: nginx
      metricsets: ["stubstatus"]
      enabled: true
      period: 10s
      # Nginx hosts
      hosts: ["http://121.41.**.**"]
      # Path to server status. Default server-status
      server_status_path: "status"
    Note
    • If you select Enable Kibana Monitoring, the system enables Metricbeat service monitoring in the Kibana console.

    • If you select Enable Kibana Dashboard, the system generates charts in the Kibana console. You do not need to configure the YML file. Alibaba Cloud Kibana is configured in a VPC. You must enable the Kibana private network access feature on the Kibana configuration page. For more information, see Configure a public or private IP address whitelist for Kibana.

  4. Click Next.

  5. Select the target ECS instance.

    Select the target ECS instance
    Important

    If you are installing a shipper for the first time, click Authorize Now. Then, follow the instructions to authorize Elasticsearch to access ECS instances.

  6. Enable the shipper and check whether the shipper installation succeeds.

    For more information, see Use Metricbeat to collect system data.

  7. View the collected data.

    1. In the address bar of your browser, enter <IP address of the NGINX server>/status to access the monitoring page of the NGINX server.

      Enter the monitoring page
    2. Log on to the Kibana console of your Elasticsearch cluster.

      For more information, see Log on to the Kibana console.

    3. In the left-side navigation pane, click Dashboard.

    4. In the Dashboards section, click [Metricbeat Nginx] Overview. On the page that appears, view the dashboard for the NGINX service.

      Dashboard for the NGINX service