Overview

Through Beats, you can collect log files, network data, server metrics and other data in ECS instances and send them to Alibaba Cloud Elasticsearch or Logstash for monitoring, analysis and other operations. When you install the collector and encounter installation failure or abnormal heartbeat, you can refer to the method in this article for troubleshooting.

Possible causes and solutions

Follow these steps to troubleshoot the location problem and fix it:

  1. Check whether the operating system of the ECS instance on which Beats is installed is Alibaba Cloud Linux 2, RedHat, or CentOS. If the operating system of the ECS instance is not Alibaba Cloud Linux 2, RedHat, or CentOS, see Change the operating system.
  2. Check whether the ECS instance on which Beats is installed is in the same VPC Virtual Private Cloud as the Elasticsearch or Logstash instance. If it is not in the same Virtual Private Cloud, see Change the VPC of an ECS instance.
  3. Perform the following steps to check whether Cloud Assistant and Docker are installed in the ECS instance where the Beats service is installed:
    1. Log on to the Linux instance through SSH or VNC. For more information, see Overview of connection methods.
    2. Run the following command to view the Cloud Assistant service status: 
      systemctl status aliyun.service
      The system display is similar to the following, confirming that the Cloud Assistant service is running.
      p142955.png
    3. If the Cloud Assistant is not installed, see Installing Cloud Assistant Client for installation.
    4. Run the following command to view the status of Docker: 
      systemctl status docker
      The system display is similar to the following, confirm that Docker is in the running state.
      p142962.png
    5. If Docker is not installed, see Deploy and use Docker.
  4. Check the configuration of the .yml file and confirm that the following information is configured. If the .yml file configuration information is inconsistent with the following content, please modify it to be consistent. 
    - type: log
  # Change to true to enable this input configuration.
     enabled: true              
  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - /var/log/*.log
    Description:
    • Enabled: The default is false, which must be set to true when using.
    • Paths: To specify the path of the log file, fuzzy matching, such as *.log, can be used.
    • The .yml file's paths parameter is different from the Filebeat file directory filled in the console configuration page. The Filebeat file directory is the directory mapped by Docker. The files specified by the paths can only be collected if they are mapped to the collection directory. Therefore, we recommend that you set paths and Filebeat Log File Path to the same value.
    • After you specify the collector Output in the console configuration page, you cannot re-specify Output under the .yml file configuration, otherwise an installation error will be prompted.
    • For the parameters that have been commented out by default using pound numbers (#) in the collector .yml file, be carefully modified, such as parameter settings related to X-Pack, otherwise the installation will fail.
  5. After you perform the preceding steps to locate and fix the problem, run the following command to enter the specified directory: 
    cd opt/aliyunbeats/
  6. Execute the following command to confirm that the corresponding Beats has been generated 
    ls
  7. Enter the /filebeat directory and execute the following command to confirm that Beats has conf, data and logs directories. 
    ls
    The following figure shows the sample command output.
    {6C1B230E-8898-4074-89D3-F85792FAE868}_20200821161948.jpg
  8. If you encounter problems later, you can also enter the logs directory and execute the following command to view the Beats log to facilitate the location of the problem. 
    tail -n 2 filebeat
    The system display is similar to the following.
    p142998.png
  9. Alternatively, you can view the running status of the container where Beats is located and analyze log location problems:
    1. Run the following command to view the filebeat container running status: 
      docker ps -a | grep filebeat
      The filebeat container is in the running state.
      p143002.png
    2. If the container is in the Exited state, run the following command to view the output logs of the container, locate and fix the problem. 
      docker logs -f [$Docker_ID]
      Description: [$ Docker_ID] is the ID of the filebeat container.

  10. The above troubleshooting is normal (the same VPC, Cloud Assistant, Docker, configuration, etc. are normal), but the Alibaba Cloud Beat collector is always in effect, then perform the following troubleshooting:

    1. Log on to the RAM console and check whether the following roles and policies are deleted.

      • AliyunServiceRoleForElasticsearchCollector(AliyunServiceRolePolicyForElasticsearchCollector)

      • AliyunElasticsearchAccessingOOSRole(AliyunElasticsearchAccessingOOSRolePolicy)

      • AliyunOOSAccessingECS4ESRole(AliyunOOSAccessingECS4ESRolePolicy)

    2. If the AliyunElasticsearchAccessingOOSRole, AliyunOOSAccessingECS 4ESRole and Policy do not exist, click Cloud Resource Access Authorization for quick authorization.

    3. AliyunServiceRoleFor Elasticsearch Collector service association role authorization, see Elasticsearch service association roles for configuration.

    4. If the problem still exists after the investigation, you can submit a ticket and contact Alibaba Cloud technical support for troubleshooting.

References

Application scope

  • Elasticsearch