You can modify and enable the YML configuration of a shipper to complete specific data collection tasks. This topic discusses specific parameters in the YML configuration files and describes how to modify the YML configuration.
Prerequisites
An Alibaba Cloud Elasticsearch cluster is created, and the Auto Indexing feature is enabled for the cluster. For more information about how to create an Elasticsearch cluster, see Create an Alibaba Cloud Elasticsearch cluster.
For security purposes, Alibaba Cloud Elasticsearch disables the Auto Indexing feature by default. However, Beats depends on this feature. If you select Elasticsearch for Output when you install a shipper, you must enable the Auto Indexing feature. For more information, see Access and configure an Elasticsearch cluster.
metricbeat.modules:
- module: system
metricsets: ["diskio","network"]
diskio.include_devices: []
period: 1sFilebeat configuration
filebeat.inputs in
filebeat.yml to determine how to search for or handle input data sources. The following figure shows an example of a simple input configuration.
filebeat.inputs:
- type: log
enabled: true
paths:
- /opt/test/logs/t1.log
- /opt/test/logs/t2/*
fields:
alilogtype: usercenter_serverlog- If you specify Output when you install a shipper, you do not need to specify it again in Shipper YML Configuration. Otherwise, the system prompts a shipper installation error.
- Each input data source starts with a hyphen (
-). You can use multiple hyphens to specify multiple input data sources.
| Parameter | Description |
|---|---|
type |
The input type. Examples of valid values: stdin, redis, tcp, and syslog. Default value: log. |
paths |
The paths of the logs you want to monitor. You can specify a file or a directory to map to Docker. |
enabled |
Specifies whether the configuration takes effect. The value true indicates that the configuration takes effect. The value false indicates that the configuration does not take effect. |
fields |
Optional. Below this parameter, you can indent with two spaces to add fields. For example, enter alilogtype: usercenter_serverlog to add this field to each output log to identify the type of the log source. If logs are shipped to Logstash, they can be classified and processed based on this field. |
For more information, see Log input in the open source Filebeat documentation.
Metricbeat configuration
metricbeat.modules in
metricbeat.yml to configure a
module.
metricbeat.modules:
- module: system
metricsets: ["diskio","network"]
enabled: true
hosts: ["http://XX.XX.XX.XX/"]
period: 10s
fields:
dc: west
tags: ["tag"]| Parameter | Description |
|---|---|
module |
The name of the module you want to run. For more information about supported modules, see Modules. |
metricsets |
Specifies the metricsets you want to execute. For more information about metricsets, see Modules. |
enabled |
Specifies whether the configuration takes effect. The value true indicates that the configuration takes effect. The value false indicates that the configuration does not take effect. |
period |
Specifies how often the metricsets are executed. If the system is inaccessible, Metricbeat returns an error for each period. |
hosts |
Optional. This parameter specifies the hosts from which you want to obtain information. |
fields |
Optional. This parameter specifies the fields that are sent with the metricset event. |
tags |
Optional. This parameter specifies the tags that are sent with the metricset event. |
For more information, see open source Metricbeat documentation.
Heartbeat configuration
Heartbeat can be installed on a remote server in a lightweight manner. You can use Heartbeat to periodically check the status of your services and determine whether they are available. Unlike Metricbeat, Heartbeat checks whether your services are available but Metricbeat checks whether your services are running.
heartbeat.monitors in
heartbeat.yml to specify the services you want to monitor.
heartbeat.monitors:
- type: http
name: ecs_monitor
enabled: true
urls: ["http://localhost:9200"]
schedule: '@every 5s'
fields:
dc: west| Parameter | Description |
|---|---|
type |
The monitor type. Valid values: icmp, tcp, and http. |
name |
The monitor name. This value appears in Exported fields of the monitor field and is used as the job name. The type field is used as the job type. |
enabled |
Specifies whether the configuration takes effect. The value true indicates that the configuration takes effect. The value false indicates that the configuration does not take effect. |
urls |
Optional. This parameter specifies the servers to which you want to connect. |
schedule |
The task schedule. If you set the value to @every 5s, the system runs the task every five seconds from the time Heartbeat is started. If you set the value to */5 * * * * * *, the system runs the task every five seconds. |
fields |
Optional. You can add the fields to output as additional information. |
For more information, see open source Heartbeat documentation.
Auditbeat configuration
service auditd status command to query its status.
You can specify auditbeat.modules in auditbeat.yml to configure the Auditbeat shipper. auditbeat.yml consists of two parts: module and output. If you want to enable a module, you must add specific parameters to auditbeat.yml. The following configurations use the auditd and file_integrity modules as examples:
auditbeat.modules:
- module: auditd
audit_rules: |
-w /etc/passwd -p wa -k identity
-a always,exit -F arch=b32 -S open,create,truncate,ftruncate,openat,open_by_handle_at -F exit=-EPERM -k access
- module: file_integrity
paths:
- /bin
- /usr/bin
- /sbin
- /usr/sbin
- /etcFor more information about auditbeat.yml configuration, see Step 2: Configure Auditbeat in the open source Auditbeat documentation. For more information about module configuration, see Modules.