This topic shows you how to use Beats together with Alibaba Cloud Elasticseach to create a management dashboard in Kibana. In this example, Metricbeat is used to collect data from a computer that runs macOS.

Terms

Beats is a platform for single-purpose data shippers. They send data from thousands of machines and systems to Logstash or Elasticsearch.

Metricbeat is a lightweight shipper that collects metrics from your systems and services. System metrics include CPU and memory metrics. Service metrics include Redis and NGINX metrics.

Overview

The overall procedure is as follows:
  1. Complete preparations.
  2. Configure an Alibaba Cloud Elasticsearch cluster.
  3. Configure Metricbeat.
  4. View the dashboard in Kibana.
Note You can also use Metricbeat to collect metrics from a computer running Linux or Windows and then send the metrics to your Alibaba Cloud Elasticsearch cluster.

Precautions

The following example includes the following IP addresses in the whitelist of the Elasticsearch cluster: 0.0.0.0/1,128.0.0.0/1. This configuration allows almost all public IP addresses to access the Elasticsearch cluster. We recommend that you evaluate the risks before you make the same configuration.

Preparations

Configure an Alibaba Cloud Elasticsearch cluster

  1. Log on to the Alibaba Cloud Elasticsearch console. In the top navigation bar, select the region where your Elasticsearch cluster resides. In the left-side navigation pane, click Elasticsearch Clusters. Then, find your Elasticsearch cluster and click its ID. In the left-side navigation pane of the page that appears, click Security.
  2. Turn on the Public Network Access switch. After this switch turns green, click Update next to Public Network Whitelist and add the public IP address of your computer running macOS to the whitelist.
    Notice If you are using a public network, add the IP address of the jump server that controls outbound traffic of the public network to the whitelist. If you cannot obtain this IP address, we recommend that you add 0.0.0.0/1,128.0.0.0/1 to the whitelist. This configuration allows almost all public IP addresses to access the Elasticsearch cluster. We recommend that you evaluate the risks before you make the same configuration.
  3. Go back to the Basic Information page and record the public endpoint of your Elasticsearch cluster.Obtain the public endpoint of an Elasticsearch cluster
  4. In the left-side navigation pane, click Cluster Configuration. On the page that appears, click Modify Configuration on the right side of YML Configuration. In the YML Configuration pane that appears, set Auto Indexing to Enable.Enable auto indexing
    Notice This configuration only takes effect after your Elasticsearch cluster is restarted. To prevent impacts on your business, exercise caution when you change the setting of Auto Indexing.
  5. Select This operation will restart the cluster. Continue? and click OK.

    During the restart, you can view the progress in the Tasks message. After your Elasticsearch cluster is restarted, the configuration of the cluster takes effect.

Configure Metricbeat

  1. Decompress the Metricbeat installation package you have downloaded during preparations, and go to the Metricbeat folder.Go to the Metricbeat folder
  2. Open the metricbeat.yml file and edit the Elasticsearch output section in it. You must uncomment the involved content.Edit the metricbeat.yml file
    Parameter Description
    hosts The internal or public endpoint of your Alibaba Cloud Elasticsearch cluster. This section uses the public endpoint as an example.
    protocol Set this parameter to http.
    username The default value is elastic.
    password The password that you set when you purchase your Alibaba Cloud Elasticsearch cluster.
  3. Run the following command to start Metricbeat:
    ./metricbeat -e -c metricbeat.yml
    After Metricbeat is started, it starts to ship data to your Alibaba Cloud Elasticsearch cluster.Metricbeat started successfully

View the dashboard in Kibana

Log on to the Kibana console of your Alibaba Cloud Elasticsearch cluster. In the left-side navigation pane, click Dashboard.
Note If you have not created index patterns in the Kibana console, the Dashboard page does not display relevant information. In this situation, you can click Index Patterns on the Management page of the Kibana console and create an index pattern based on instructions. Then go back to the Dashboard page.
  • List of metricsList of metrics
  • CPU metricsCPU metrics
    Note You can configure metrics to be refreshed at 5-second intervals. The system generates reports for the metrics based on the intervals. You can also connect to WebHook to configure alerts.

Reference: Use Beats to quickly build a visualized O&M system