This topic describes how to configure monitoring indexes for your Alibaba Cloud Elasticsearch cluster. After the configuration, you can view monitoring log data. This configuration prevents your storage space from being exhausted by the log data.

Prerequisites

An Elasticsearch cluster is created. If no cluster is created, first perform the operations described in Create an Elasticsearch cluster. This topic describes how to create an Elasticsearch V6.7 cluster of the Standard Edition.

Background information

By default, the X-Pack monitoring component collects the monitoring data every 10 seconds and saves the data to the indexes that have prefix .monitoring-* in your Elasticsearch cluster.

The .monitoring-es-6-* and .monitoring-kibana-6-* indexes are used to store the monitoring data. The Elasticsearch cluster rolls over to a new index each day. The name of a .monitoring-es-6- index ends with the date when the monitoring data is saved.

A .monitoring-es-6-* index stores information about the cluster status, cluster statistics, node statistics, and index statistics, which consumes a large amount of disk space.

Procedure

  1. Log on to the Kibana console of your Elasticsearch cluster.
    For more information, see Log on to the Kibana console.
  2. In the left-side navigation pane, click Monitoring.
  3. In the Elasticsearch section, click Indices.
    Indices
  4. On the Indices tab, turn on the System indices switch to query the storage space that is consumed by monitoring indexes.
    View monitoring index information
  5. In the left-side navigation pane, click Dev Tools.
  6. On the Console tab, create a monitoring index.
    By default, the system retains the indexes that are created in the last seven days. Indexes that store the monitoring data, such as .monitoring-es-6-*, consume the storage space of your Elasticsearch cluster. The size of each index depends on the numbers of nodes and indexes that include system indexes in the cluster. You can use one or both of the following methods to prevent the indexes that store the monitoring data from consuming a large amount of disk space for your cluster.
    • Configure the index retention period 
      PUT _cluster/settings
{"persistent": {"xpack.monitoring.history.duration":"1d"}}

      You can specify the index retention period as required. The minimum retention period is one day.

    • Specify the indexes to be collected
      Call the API operation to specify the inclusion or exclusion list for index collection. This allows you to reduce the size of the .monitoring-es-6-* indexes. The following example shows how to create an exclusion list for collecting system indexes. 
      PUT _cluster/settings
{"persistent": {"xpack.monitoring.collection.indices": "*,-. *"}}
      Note The excluded indexes are not collected or displayed on the Monitoring page in the Kibana console. However, these indexes are listed in the index list retrieved by calling the GET _cat/indices operation. The status of open or close for the indexes is also displayed.