Elasticsearch:Installation failures of Beats shippers

Procedure

1. Check whether the Elastic Compute Service (ECS) instance on which Beats is installed runs Aliyun Linux, Red Hat Enterprise Linux (RHEL), or CentOS.
2. Check whether the ECS instance on which Beats is installed resides in the same virtual private cloud (VPC) as your Alibaba Cloud Elasticsearch or Logstash cluster.
3. Check whether Cloud Assistant and Docker are installed on the ECS instance on which Beats is installed.
   To perform the check, connect to the ECS instance and run the following commands:

   • Check the status of Cloud Assistant

     systemctl status aliyun.service

     If Cloud Assistant is in a normal state, the result shown in the following figure is returned.

     For more information about how to install Cloud Assistant, see Install the Cloud Assistant client.

   • Check the status of Docker

     systemctl status docker

     If Docker is in a normal state, the result shown in the following figure is returned.

     For more information about how to install Docker, see Deploy and use Docker on Alibaba Cloud Linux 2 instances.

4. Check whether the following parameters are configured in the YML configuration file of the shipper:

   - type: log
     # Change to true to enable this input configuration.
        enabled: true              
     # Paths that should be crawled and fetched. Glob based paths.
     paths:
       - /var/log/*.log

   Parameter	Description
   enabled	Set this parameter to true when you use Beats. Default value: false.
   paths	You can use a wildcard to specify this parameter, such as *.log. This facilitates the query of the log file.

   Important

   • The value of paths is different from Filebeat Log File Path that you specified on the configuration page. For Filebeat, collected data can be stored in the file indicated by paths only after the path specified by paths is mapped by Docker. However, Docker maps the path specified by Filebeat Log File Path. Therefore, we recommend that you set paths and Filebeat Log File Path to the same value.
   • If you already specify Output on the configuration page, you cannot specify it again in the YML configuration file. Otherwise, the system prompts an installation error.
   • Exercise caution when you modify the parameters that are commented out by using number signs (#) in the YML configuration file of the shipper, such as X-Pack-related parameters. Otherwise, the shipper fails to be installed.
5. Connect to the ECS instance, check whether a Beats instance is created under the /opt/aliyunbeats/ path, and check whether the conf, data, and logs folders exist.
   

   You can also query Beats logs in the logs folder to locate problems.
6. Check the status of the Docker container in which the Beats service is provided. Then, locate problems based on logs.
   1. Check the status of the Docker container.

      docker ps -a | grep filebeat

      
   2. If the Docker container is in the exited state, check the logs that the container generates.

      docker logs -f Container ID