ApsaraMQ for Kafka:Filebeat

Filebeat

Filebeat is a lightweight shipper that forwards and centralizes log data. It monitors specified log files or locations, collects log events, and forwards them to Elasticsearch or Logstash for indexing. Filebeat works as follows:

1. Filebeat starts one or more inputs. The inputs look for log data in specified locations.

2. Filebeat starts a harvester for each log it finds. The harvester reads the log and sends the log data to libbeat.

3. libbeat aggregates the data and then sends the aggregated data to the configured output.

Benefits

Connecting ApsaraMQ for Kafka to Filebeat provides the following benefits:

• Asynchronous processing: Prevents traffic bursts.

• Application decoupling: Isolates downstream failures from the upstream service.

• Reduced overhead: Lowers the resource consumption of Filebeat.

Connection methods

You can connect ApsaraMQ for Kafka to Filebeat by using the following methods:

• Connect as an input in a VPC environment

• Connect as an output in a VPC environment

• Connect as an input over the Internet

• Connect as an output over the Internet