You can modify and enable the YML configuration of a shipper to complete specific data collection tasks. This topic discusses specific parameters in the YML configuration files and describes how to modify the YML configuration.
Preparations
The Auto Indexing feature is enabled for your Alibaba Cloud Elasticsearch cluster.
For security purposes, Alibaba Cloud Elasticsearch disables Auto Indexing by default. However, Beats depends on this feature. If you select Elasticsearch for Output when you create a shipper, you must enable the Auto Indexing feature. For more information, see Enable auto indexing.
metricbeat.modules:
- module: system
metricsets: ["diskio","network"]
diskio.include_devices: []
period: 1sFilebeat configuration
filebeat.inputs in filebeat.yml to determine how to search for or handle input data sources. The following figure
shows an example of simple input configuration.
filebeat.inputs:
- type: log
paths:
- /opt/test/logs/t1.log
- /opt/test/logs/t2/*
fields:
alilogtype: usercenter_serverlog- If you specify Output when you install a shipper, you do not need to specify it again in Shipper YML Configuration. If you specify it again, the system prompts a shipper installation error. For more information, see Install a shipper.
- An input data source starts with a hyphen (
-). You can use multiple hyphens to specify multiple input data sources.
| Parameter | Description |
|---|---|
type |
The input type. Examples of valid values: stdin, redis, tcp, and syslog. Default value: log.
|
paths |
The paths of logs you want to monitor. You can specify either a file or a directory to map to Docker. |
enabled |
Specifies whether the configuration takes effect. The value true indicates that the configuration takes effect. The value false indicates that the configuration does not take effect.
|
fields |
The optional fields. Below this parameter, you can indent with two spaces to add fields.
For example, enter alilogtype: usercenter_serverlog to add this field in each output log to identify the type of the log source. If logs
are shipped to Logstash, they can be classified and processed based on this field.
|
For more information, visit Log input in the open-source Filebeat documentation.
Metricbeat configuration
metricbeat.modules in metricbeat.yml to configure a module.
metricbeat.modules:
- module: system
metricsets: ["diskio","network"]
hosts: ["http://XX.XX.XX.XX/"]
period: 10s
fields:
dc: west
tags: ["tag"]| Parameter | Description |
|---|---|
module |
The name of the module you want to run. For more information about supported modules, visit Modules. |
metricsets |
The list of metricsets you want to execute. For more information about metricsets, visit Modules. |
enabled |
Specifies whether the configuration takes effect. The value true indicates that the configuration takes effect. The value false indicates that the configuration does not take effect.
|
period |
Specifies how often the metricsets are executed. If the system is not reachable, Metricbeat returns an error for each period. |
hosts |
Optional. The list of hosts from which you want to fetch information. |
fields |
Optional. The fields that are sent with the metricset event. |
tags |
Optional. The list of tags that are sent with the metricset event. |
For more information, visit Configuration in the open-source Metricbeat documentation.
Heartbeat configuration
Heartbeat can be installed on a remote server in a lightweight manner to periodically check the status of your services and determine whether they are available. Heartbeat is unlike Metricbeat. Heartbeat checks whether your services are reachable but Metricbeat checks whether your services are running.
heartbeat.monitors in heartbeat.yml to specify the services you want to monitor.
heartbeat.monitors:
- type: http
name: ecs_monitor
enabled: true
urls: ["http://localhost:9200"]
schedule: '@every 5s'
fields:
dc: west| Parameter | Description |
|---|---|
type |
The monitor type. Valid values: icmp, tcp, and http.
|
name |
The monitor name. This value appears in the exported fields of the monitor field, and is considered as the job name. The type field is considered as the job type.
|
enabled |
Specifies whether the configuration takes effect. The value true indicates that the configuration takes effect. The value false indicates that the configuration does not take effect.
|
urls |
Optional. The list of servers you want to ping. |
schedule |
The task schedule. If you set the value to @every 5s, the system runs the task every 5 seconds from the time Heartbeat is started. If
you set the value to */5 * * * * * *, the system runs the task every 5 seconds.
|
fields |
The optional fields that you can add to the output as additional information. |
For more information, visit Configuration in the open-source Heartbeat documentation.
Auditbeat configuration
service auditd status command to query the service status.
You can specify auditbeat.modules in auditbeat.yml to configure the Auditbeat shipper. auditbeat.yml consists of two parts: module and output. If you want to enable the auditd and file_integrity modules, add the following script to auditbeat.yml:
auditbeat.modules:
- module: auditd
audit_rules: |
-w /etc/passwd -p wa -k identity
-a always,exit -F arch=b32 -S open,creat,truncate,ftruncate,openat,open_by_handle_at -F exit=-EPERM -k access
- module: file_integrity
paths:
- /bin
- /usr/bin
- /sbin
- /usr/sbin
- /etcFor more information about auditbeat.yml configuration, visit Configuration in the open-source Auditbeat documentation. For more information about module configuration, visit Modules.