All Products
Search
Document Center

Elasticsearch:Use Metricbeat to collect system data and NGINX service data

Last Updated:May 13, 2024

This topic describes how to use Alibaba Cloud Metricbeat to collect system data and NGINX service data and then generate visual charts. The system data includes CPU utilization, memory usage, disk I/O, and network I/O.

Prerequisites

The following operations are performed:

Use Metricbeat to collect system data

  1. Log on to the Alibaba Cloud Elasticsearch console.
  2. In the left-side navigation pane, click Beats Data Shippers.

  3. In the Create Shipper section of the page that appears, click Metricbeat.

  4. Install and configure a shipper.

    For more information, see Collect the logs of an ECS instance and Prepare a YML configuration file for a shipper. The following figure shows the configurations that are used in this topic.Metricbeat

    Note
    • If you select Enable Kibana Monitoring, the system enables Metricbeat service monitoring in the Kibana console.

    • If you select Enable Kibana Dashboard, the system generates charts in the Kibana console. You do not need to configure the YML file. Alibaba Cloud Kibana is deployed in a VPC. You must enable the Private Network Access feature for Kibana on the Kibana Configuration page. For more information, see Configure a public or private IP address whitelist for Kibana.

    • The system module is enabled by default. You do not need to specify Shipper YML Configuration.

  5. Click Next.

  6. Select the ECS instance on which you want to install the shipper.

    选择采集器安装的实例

    Note

    If you are installing a shipper for the first time, click Authorize Now. Then, follow the instructions to authorize Elasticsearch to access ECS instances.

  7. Enable the shipper and check whether the shipper is installed.

    1. Click Enable.

      Then, the Enable Shipper message appears.

    2. Click Back to Beats Shippers. In the Manage Shippers section of the Beats Data Shippers page, view the installed shipper.

    3. After the state of the shipper changes to Enabled 1/1, click View Instances in the Actions column.

    4. In the View Instances pane, check whether the shipper is installed on the ECS instance. If the value of Installed Shippers is Heartbeat Normal, the shipper is installed.

      采集器安装成功

  8. View the result.

    1. Log on to the Kibana console of the Elasticsearch cluster.

      For more information, see Log on to the Kibana console.

    2. In the left-side navigation pane, click Dashboard.

    3. In the Dashboards section, click [Metricbeat System] Overview. On the page that appears, click the desired Metricbeat system and view the dashboard for the system.

      仪表板

Use Metricbeat to collect NGINX service data

Prerequisites: The stub_status module is enabled for the NGINX service. The ngx_http_stub_status_module module measures the number of requests received by the NGINX service and the number of requests processed by the NGINX service. Therefore, you must enable stub_status in the nginx.conf file.

location /status {
           stub_status on;
           access_log off;
        }
Important

The value of server_status_path in the metricbeat.yml file must be the same as that of status in the nginx.conf file.

  1. Log on to the Alibaba Cloud Elasticsearch console.
  2. In the left-side navigation pane, click Beats Data Shippers.

  3. In the Create Shipper section of the page that appears, click Metricbeat.

  4. Install and configure a shipper.

    For more information, see Collect the logs of an ECS instance and Prepare a YML configuration file for a shipper.

    Add the following script in the metricbeat.yml file:

    image

    metricbeat.modules:
    - module: nginx
      metricsets: ["stubstatus"]
      enabled: true
      period: 10s
      # Nginx hosts
      hosts: ["http://121.41.**.**"]
      # Path to server status. Default server-status
      server_status_path: "status"
    Note
    • If you select Enable Kibana Monitoring, the system enables Metricbeat service monitoring in the Kibana console.

    • If you select Enable Kibana Dashboard, the system generates charts in the Kibana console. You do not need to configure the YML file. Alibaba Cloud Kibana is deployed in a VPC. You must enable the Private Network Access feature for Kibana on the Kibana Configuration page. For more information, see Configure a public or private IP address whitelist for Kibana.

  5. Click Next.

  6. Select the ECS instance on which you want to install the shipper.

    选择采集器安装的实例

    Note

    If you are installing a shipper for the first time, click Authorize Now. Then, follow the instructions to authorize Elasticsearch to access ECS instances.

  7. Enable the shipper and check whether the shipper is installed.

    For more information, see Use Metricbeat to collect system data.

  8. View the result.

    1. In the address bar of your browser, enter <IP address of the NGINX server>/status to access the monitoring page of the NGINX server.

      访问监控页面

    2. Log on to the Kibana console of the Elasticsearch cluster.

      For more information, see Log on to the Kibana console.

    3. In the left-side navigation pane, click Dashboard.

    4. In the Dashboards section, click [Metricbeat Nginx] Overview. On the page that appears, view the dashboard for the NGINX service.

      nginx监控仪表板