monitoring logs - Elasticsearch - Alibaba Cloud Documentation Center

By default, the X-Pack monitoring client collects cluster metrics every 10 seconds and stores them in daily rolling indices with the .monitoring-* prefix. Over time, these monitoring indices can consume significant disk space and affect your instance's performance. You can control this disk space usage by adjusting the data retention policy and collection scope of the monitoring indices.

Procedure

Before you begin, log in to the Kibana console. For instructions, see Connect to a cluster by using Kibana.

The following steps use an Alibaba Cloud Elasticsearch 6.7.0 cluster as an example. UI labels may vary slightly in other versions.

Check the disk space used by monitoring indices.
1. In the Kibana console, click the icon in the upper-left corner to expand the navigation pane on the left, and then choose Management > Stack Monitoring.
2. In the Elasticsearch section, click Indices.
3. On the Indices tab, turn on System indices to view the disk space used by monitoring indices, such as .monitoring-es-* and .monitoring-kibana-*. The .monitoring-es-* indices typically use the most space because they store information such as cluster state, cluster statistics, node statistics, and index statistics.
Configure the monitoring indices. By default, system monitoring indices retain data for the last seven days. The size of these indices depends on the number of indices (including system indices) and nodes in your instance. You can control disk space usage with one or both of the following methods.
1. In the navigation pane on the left, click Dev Tools.
2. Set the retention period for monitoring indices. In the Console, run the following command to set the retention period to one day. You can adjust this value as needed. The minimum retention period is one day.
```
PUT _cluster/settings
{"persistent": {"xpack.monitoring.history.duration":"1d"}}
```
3. Specify which indices to monitor. Run the following command to reduce the disk space used by .monitoring-es-* indices. The following example excludes system indices from monitoring.
```
PUT _cluster/settings
{"persistent": {"xpack.monitoring.collection.indices": "*,-.*"}}
```
  Monitoring data for excluded indices does not appear on the Monitoring page in Kibana. However, you can still view the indices and their status by running the GET _cat/indices command.