You can use Beats to collect the data of an Elastic Compute Service (ECS) instance. The data includes logs, network data, and metrics. Beats then sends the collected data to Alibaba Cloud Elasticsearch or Logstash for further processing, such as monitoring and analysis. This topic describes how to use a Filebeat shipper to collect the logs of an ECS instance.
Prerequisites
- An Alibaba Cloud Elasticsearch or Logstash cluster is created. In this example, a
V6.7.0 cluster is used.
For more information, see Create an Alibaba Cloud Elasticsearch cluster and Create an Alibaba Cloud Logstash cluster.
- The Auto Indexing feature is enabled for the Elasticsearch cluster.
For security purposes, Alibaba Cloud Elasticsearch disables the Auto Indexing feature by default. However, Beats depends on this feature when it collects the logs of ECS instances. If you want to select Elasticsearch for Output, you must enable the Auto Indexing feature for the Elasticsearch cluster. For more information, see Configure the YML file.
- An ECS instance is created in the same virtual private cloud (VPC) as the Elasticsearch
or Logstash cluster.
When you create the ECS instance, select one of the following types of operating systems: Alibaba Cloud Linux, Red Hat Enterprise Linux (RHEL), and Community Enterprise Operating System (CentOS). Beats supports only the three types of operating systems. For more information, see Create an instance by using the wizard.
Notice The default installation directory of Beats is /opt/aliyunbeats/. After you install Beats, the conf, logs, and data directories are generated on the ECS instance. The conf directory contains the configuration file, the logs directory contains the Beats log file, and the data directory contains the Beats data file. We recommend that you do not delete or modify these files. Otherwise, errors may occur, or data may be incorrect. If an error occurs, you can view the Beats logs in the logs directory to locate the error. - Cloud Assistant and Docker are installed on the ECS instance.
For more information, see Install the Cloud Assistant client and Deploy and use Docker on Alibaba Cloud Linux 2 instances.
Procedure
FAQ
How do I use a Filebeat shipper to collect logs from multiple directories?
paths
parameter in Shipper YML Configuration. 
References
API operation for creating a shipper: CreateCollector