All Products
Search
Document Center

Elasticsearch:Use Uptime to monitor Alibaba Cloud Elasticsearch clusters in real time

Last Updated:Oct 30, 2023

Heartbeat shippers can detect the statuses of network endpoints based on the HTTP or HTTPS, TCP, and ICMP services on a regular basis. They can also send the collected detection data to the Uptime application in Kibana. Then, the Uptime application monitors the availability and response time of applications and services in real time and reports errors before your business is affected. This topic describes how to use Uptime to monitor an Alibaba Cloud Elasticsearch cluster in real time.

Background information

Uptime must be used with the following services:

  • Heartbeat

  • Elasticsearch

  • Kibana

Note

You can also use the Alerting and Actions feature of Kibana V7.7 to configure monitoring and alerting. For more information, see Alerting and Actions.

Deployment architecture

  • Deployment of a single Heartbeat shipper

    A single Heartbeat shipper is deployed at a single position to monitor a single service. The Heartbeat shipper sends monitoring data to Elasticsearch. You can view the heartbeat data of the service on the Uptime page of Kibana and determine the service status based on the heartbeat data.单实例部署

  • Deployment of multiple Heartbeat shippers

    Two Heartbeat shippers are deployed at different positions to monitor the same service. The two Heartbeat shippers send monitoring data to Elasticsearch. You can view the heartbeat data of the service on the Uptime page of Kibana and determine the service status based on the heartbeat data. If the Heartbeat shipper at one position becomes faulty, the Heartbeat shipper at the other position can help locate the fault.多实例部署

For more information about the deployment architecture, see Deployment Architecture.

Make preparations

  1. Create an Alibaba Cloud Elasticsearch cluster and enable the Auto Indexing feature for the cluster.

  2. Create an Elastic Compute Service (ECS) instance, which is used to deploy a Heartbeat shipper. The ECS instance must reside in the same virtual private cloud (VPC) as the Elasticsearch cluster.

    For more information, see Create an instance by using the wizard.

    Important

    Beats supports only the following operating systems: Alibaba Cloud Linux, Red Hat Enterprise Linux (RHEL), and CentOS. Therefore, you must select one of the preceding operating systems when you create the ECS instance.

  3. Install Cloud Assistant and Docker on the ECS instance.

Create a Heartbeat shipper

  1. Log on to the Alibaba Cloud Elasticsearch console.
  2. In the left-side navigation pane, click Beats Data Shippers.

  3. In the Create Shipper section of the Beats Data Shippers page, click Heartbeat.

  4. Install and configure a shipper.

    For more information, see Collect the logs of an ECS instance and Prepare a YML configuration file for a shipper.配置Heartbeat

    Table 1. Configurations for heartbeat.monitors

    Parameter

    Description

    type

    In this topic, http is used.

    Note

    A Heartbeat shipper can monitor HTTP or HTTPS, TCP, and ICMP services. If you use an HTTP or HTTPS monitor, response code, request bodies, and request headers can be monitored. If you use a TCP monitor, port numbers and strings can be monitored.

    urls

    The URLs that you want to check. You can specify multiple HTTP services. In this topic, an Alibaba Cloud Elasticsearch cluster is checked. This parameter is set to the internal endpoint of the Elasticsearch cluster.

    schedule

    The interval at which the check is performed. The value @every 10s indicates that the check is performed every 10 seconds.

  5. Click Next.

  6. In the Install Shipper step, select the ECS instance on which you want to install the shipper.

    选择采集器安装的实例
  7. Enable the shipper and check whether the shipper is installed.

    For more information, see Collect the logs of an ECS instance.

    If the state of the Heartbeat shipper is Enabled, and the installation state of the Heartbeat shipper is Heartbeat Normal, the Heartbeat shipper is installed.采集器安装成功

View the monitoring information on the Uptime page

  1. Log on to the Kibana console.

    The Kibana console is the one that corresponds to the Elasticsearch cluster that you specified for Output when you create the Heartbeat shipper. For more information, see Log on to the Kibana console.

  2. In the left-side navigation pane, click Uptime. On the Uptime page, you can view the monitoring information.

    查看Uptime监控信息
    • Color red: indicates that the Elasticsearch cluster is in an abnormal state. Check the communication status of the Heartbeat shipper or the status of the Elasticsearch cluster.

    • Color blue: indicates that the Elasticsearch cluster is in a normal state.