Use commands to check cluster health status - Elasticsearch

You can run commands in the Kibana Dev Tools console to check cluster health, node load, shard allocation, and index information for routine monitoring and troubleshooting.

Prerequisites

You must have an Alibaba Cloud Elasticsearch instance. For more information, see Create an Alibaba Cloud Elasticsearch cluster.

Procedure

Log on to the Kibana console of your Elasticsearch cluster.
For detailed instructions, see Log on to the Kibana console. All commands in this topic can be run in the Dev Tools console.
In the left-side navigation pane, click Dev Tools.

In the Console, run the following command to check the cluster's health status.

GET /_cluster/health

A successful request returns the following response:

{
  "cluster_name" : "es-cn-45xxxxxxxxxxxxk1q",
  "status" : "green",
  "timed_out" : false,
  "number_of_nodes" : 2,
  "number_of_data_nodes" : 2,
  "active_primary_shards" : 18,
  "active_shards" : 36,
  "relocating_shards" : 0,
  "initializing_shards" : 0,
  "unassigned_shards" : 0,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 100.0
}

The status field indicates the cluster health status. Valid values are green, yellow, and red.

Status	State	Description
red	Not all primary shards are available.	One or more primary shards are unassigned.
yellow	All primary shards are available, but not all replica shards are available.	One or more replica shards are unassigned.
green	All primary and replica shards are available.	All primary and replica shards are assigned.

A yellow cluster status can significantly slow down operations such as password changes and instance upgrades. Before performing these operations, restore your cluster to the green status. A common reason for a yellow status is that the number of replicas for an index is greater than the number of available nodes minus one. To fix this, follow these steps:

Find the indices with a yellow status:

curl -u <username>:<password> http://<host>:9200/_cat/indices

Adjust the number of replicas for the affected index, setting the value to one less than the number of available nodes.
```
curl -XPUT -u <username>:<password> http://<host>:9200/<problematic_index_name>/_settings -H 'Content-Type: application/json' -d '{"index":{"number_of_replicas":(<number_of_nodes - 1>)}'
```
Note
After restarting or scaling an instance, adjust the number of replicas for your indices to match the new node count.

Cluster commands

Command	Description
GET /_cat/health?v	Returns the cluster health status (green, yellow, or red). For status descriptions, see the table above.
GET /_cluster/health?pretty=true	Returns the health status of the cluster. The `pretty=true` parameter formats the JSON output for readability. You can also add other query parameters, such as: `level=indices`: Displays the status of each index. `level=shards`: Displays information about shards.
GET /_cluster/stats	Returns cluster statistics, including CPU and JVM information.
GET /_cluster/state	Returns detailed information about the cluster state, including nodes and shards.
GET /_cluster/pending_tasks	Returns the pending tasks in the cluster.
GET /_cluster/settings	Returns the cluster-wide settings.

Node commands

Command	Description
GET /_cat/master?v	Returns information about the master node.
GET /_cat/nodes?v	Returns the current status of each node, including CPU usage, heap memory usage, and load.
GET /_cat/nodeattrs?v	Returns the custom attributes of each node.
GET /_nodes/stats?pretty=true	Returns node statistics.
GET /_nodes/process	Returns process information for nodes.
GET /_nodes/hot_threads	Returns the tasks that are executed by hot threads consuming high amounts of CPU. This command is commonly used to identify the cause of high CPU utilization in a cluster. By analyzing the thread stacks, such as Lucene Merge Thread, and CPU utilization in the response, you can identify the specific tasks or operations that cause high load.
GET /_nodes/<nodeip>/jvm,process,os	Returns JVM, process, and operating system (OS) information for a specified node.
GET /_cat/plugins?v	Returns the plugins installed on each node.
GET /_cat/thread_pool?v	Returns thread pool statistics for each node, including thread pool type, number of active threads, and queue size.

Shard commands

Command	Description
GET /_cat/shards?v	Returns detailed information about each shard, such as its index name, shard ID, type (primary or replica), status, document count, and disk usage. The output includes a failure reason for any failed shards. To view shard information for a specific index, run `GET _cat/shards/<index>?v`.
GET /_cat/allocation?v	Returns the shard allocation for each node.
GET /_cat/recovery?v	Returns the recovery progress for each shard.

Segment commands

GET /_cat/segments?v

Returns segment information for each index, such as segment name, the shard it belongs to, memory or disk usage, and whether it has been flushed to disk. You can also view segment information for a specific index by running GET _cat/segments/<index>?v.

Index commands

Command	Description
GET /_cat/indices?v	Returns detailed information for all indices, including health status, state, shard and replica counts, and document count. To view information for a specific index, run `GET _cat/indices/<index>?v`.
GET /_cat/aliases?v	Returns information about all index aliases, including the indices they point to and their routing configurations.

Mapping commands

Command	Description
GET /_mapping	Returns the mappings for all indices.
GET /<index>/_mapping	Returns the mapping for a specific index.

Document commands

Command	Description
GET /_cat/count?v	Returns the total document count in the cluster. To view the count for a specific index, run `GET _cat/count/<index>?v`.
GET /<index>/_doc/<id>	Retrieves the data from a specific document.

Snapshot commands

Command	Description
GET _snapshot/_all	Returns all snapshots.
GET _snapshot/<snapshot_name>/_status	Returns the progress of a specified snapshot.

FAQ

How do I view all search tasks that are being executed in Elasticsearch?

In the Dev Tools console of Kibana, run the following command to query all running search tasks on the current node and their details:

GET /_tasks?actions=indices:data/read/search*&detailed=true&nodes=_local

This command is suitable for scenarios such as troubleshooting search latency and checking the execution status of search tasks.

How do I troubleshoot the 503 Service Unavailable error returned for an Elasticsearch search request?

The 503 error is usually caused by high cluster load or high disk usage. Perform the following steps to troubleshoot the error:

Run GET /_cluster/health to check the cluster health status and determine whether the cluster is in the red or yellow state or whether unassigned shards exist.
Run GET /_cat/nodes?v to check the CPU usage, heap memory usage, and disk usage of the nodes.
If the cluster status is abnormal, resolve the shard allocation issue or resource bottleneck first, and then retry the search request.