This topic provides answers to some frequently asked questions about Alibaba Cloud Elasticsearch clusters.

When I purchase an Elasticsearch cluster, I performed an incorrect configuration. How do I modify the configuration after the cluster is created?

If you find that the configurations of your cluster do not meet your expectations after the cluster is created, you can refer to the methods provided in the following table to modify the configurations.
Warning If you need to cancel the subscription of or release your cluster, first back up your data. For more information about how to back up data, see Create manual snapshots and restore data from manual snapshots. After the cluster subscription cancellation or cluster release, your data stored on the cluster is deleted and cannot be restored.
Configuration item Solution
Billing method

If you purchased a pay-as-you-go cluster, you can switch the billing method of the cluster to subscription. For more information, see Change the billing method of a cluster from pay-as-you-go to subscription.

Version This configuration item can be modified if one of the following conditions is met:
  • The version of the cluster that you purchased is V5.5.3, and you want to upgrade the version to V5.6.16.
  • The version of the cluster that you purchased is V5.6.16, and you want to upgrade the version to V6.3.2.
  • The version of the cluster that you purchased is V6.3.2, and you want to upgrade the version to V6.7.0.

For more information about how to upgrade the version of a cluster, see Upgrade the version of a cluster. For other types of version upgrades, we recommend that you cancel the subscription of or release your cluster and purchase another cluster of the desired version.

Region You cannot modify this configuration item. We recommend that you cancel the subscription of or release your cluster and purchase another cluster based on your business requirements.
Zone You can migrate nodes to the desired zone. For more information, see Migrate nodes in a zone.
Note Before you migrate nodes, make sure that your cluster is in the Active state.
Number of zones You cannot modify this configuration item. We recommend that you cancel the subscription of or release your cluster and purchase another cluster based on your business requirements.
Specifications You can modify this configuration item. For more information, see Upgrade the configuration of a cluster.
Storage type You can modify this configuration item. For more information, see Upgrade the configuration of a cluster.
Cloud disk encryption You cannot modify this configuration item. We recommend that you cancel the subscription of or release your cluster and purchase another cluster based on your business requirements.
Storage space per node You can modify this configuration item. For more information, see Upgrade the configuration of a cluster.
Number of data nodes You can modify this configuration item. For more information, see Upgrade the configuration of a cluster.
Network type, VPC, and vSwitch You cannot modify these configuration items. We recommend that you cancel the subscription of or release your cluster and purchase another cluster based on your business requirements.
Note The network type of an Elasticsearch cluster can only be VPC.
Username The default username is elastic. You cannot modify this configuration item. You can create a user in the Kibana console and grant the required permissions to the user. For more information, see Use the RBAC mechanism provided by Elasticsearch X-Pack to implement access control.
Password You can modify this configuration item. For more information, see Reset the access password for an Elasticsearch cluster.

For the configuration items that are not provided in the preceding table, check whether you can modify the items on the configuration upgrade page. For more information, see Upgrade the configuration of a cluster and Downgrade the configuration of a cluster.

What are the mappings between versions on the Elasticsearch buy page and specific Elasticsearch versions?

Version on the buy page Specific version
7.10 7.10.0
7.7 7.7.1
6.8 6.8.6
6.7 6.7.0
6.3 6.3.2
5.6 5.6.16
5.5 5.5.3

If you have a self-managed Elasticsearch cluster, we recommend that you select a version that is nearest to the version of the cluster when you purchase an Alibaba Cloud Elasticsearch cluster. For example, you can select a version whose minor version is nearest to that of your self-managed Elasticsearch cluster. If you do not have a self-managed Elasticsearch cluster, we recommend that you select the latest version.

When I purchase an Elasticsearch cluster, no VPCs are available. What do I do?

Check whether the RAM user that you use is granted the permissions to obtain the list of virtual private clouds (VPCs). For more information, see View the basic information about a RAM user. If the RAM user that you use is not granted the permissions, grant the permissions to the RAM user. For more information, see Create a custom policy.

When I purchase an Elasticsearch cluster, I select a VPC, but no vSwitches are available. What do I do?

You must check whether vSwitches are created in the selected region. If no vSwitches are created in the selected region, you must create one. For more information, see Create an IPv4 VPC.

After I cancel the subscription of or release an Elasticsearch cluster, I purchase another cluster. Does the endpoint of the new cluster remain the same as that of the original cluster?

No, the endpoint of the new cluster is different from that of the original cluster. After you purchase the new cluster, we recommend that you modify the client code and cancel the subscription of or release the original cluster to avoid service interruptions.

How do I release an Elasticsearch cluster?

On the Elasticsearch Clusters page, find the cluster that you want to release and choose More > Release in the Actions column. For more information, see Release a cluster.

When does the system release an Elasticsearch cluster after the cluster is suspended?

The system releases the cluster one day after the cluster is suspended. After the cluster is released, all the data stored in the cluster is permanently deleted and cannot be recovered. For more information, see Overdue payments and cluster suspension or release.

Can I purchase an Elasticsearch cluster that has only one node?

No, you cannot purchase an Elasticsearch cluster that has only one node. An Elasticsearch cluster must have a minimum of two data nodes. For more information, see Parameters on the buy page.

When I purchase an Elasticsearch cluster, resources of a specific category are sold out. What do I do?

Take one of the following measures:
  • Select another region.
  • Select another zone.
  • Select another category.

If the resources that you want to purchase are still unavailable after you take all of the preceding measures, try again later. Resources are dynamic. If resources are insufficient, Alibaba Cloud replenishes the resources at the earliest opportunity.

Why do I need to upgrade the data nodes with the specifications of 1 vCPU and 2 GiB of memory in Elasticsearch clusters?

Data nodes with the specifications of 1 vCPU and 2 GiB of memory may affect the performance of Elasticsearch clusters. Alibaba Cloud Elasticsearch no longer provides data nodes with such specifications since May 2021. Existing data nodes with such specifications can still be used. Data nodes with the specifications of 1 vCPU and 2 GiB of memory are designed only for testing purposes. Clusters that contain such data nodes are not suitable for production environments. The service level agreement (SLA) does not apply to these clusters. Therefore, we recommend that you upgrade your data nodes with the specifications of 1 vCPU and 2 GiB of memory. For more information, see Upgrade the configuration of a cluster.

Why is an Elasticsearch cluster in the Initializing state for a long period of time after it is purchased?

A period of time is required before an Elasticsearch cluster can provide services after it is purchased. The period of time varies based on the specifications, data structure, and data volume of the cluster. In most cases, a few hours are required.

Do I need to purchase a Kibana node after I purchase an Elasticsearch cluster?

No, you do not need to purchase a Kibana node after you purchase an Elasticsearch cluster. When you purchase an Elasticsearch cluster, Alibaba Cloud Elasticsearch offers you a Kibana node with the specifications of 1 vCPU and 2 GiB of memory free of charge by default. If you want to use a Kibana node with higher specifications, you must select the desired specifications for the Kibana node when you purchase an Elasticsearch cluster. For more information, see Create an Alibaba Cloud Elasticsearch cluster.

What do I do if I cannot find the Elasticsearch cluster that I created?

Check whether the region that you selected in the top navigation bar of the Elasticsearch console is the region where you cluster resides. If the region that you selected is correct but you cannot find your cluster, we recommend that you clear the browser cache or use an on-premises network.

Can I upgrade or downgrade the version of an Elasticsearch cluster?

Upgrades are supported, whereas downgrades are not supported. You can upgrade the versions of clusters only from V5.5.3 to V5.6.16, from V5.6.16 to V6.3.2, or from V6.3.2 to V6.7.0. For more information, see Upgrade the version of a cluster.

If you want to perform upgrades between other versions or perform downgrades, purchase an Elasticsearch cluster of the desired version. Then, migrate data from the original cluster to the new cluster and cancel the subscription of or release the original cluster.

Can I log on to an Elasticsearch cluster over SSH and modify the configuration of the cluster?

For security purposes, you are not allowed to log on to your Elasticsearch cluster over SSH. If you want to modify the configuration of your cluster, use the cluster configuration feature of Elasticsearch. For more information, see Overview.

Is Logstash V6.7 compatible with Elasticsearch V6.3?

Yes, Logstash V6.7 is compatible with Elasticsearch V6.3. For more information, see Compatibility matrixes.

Can Elasticsearch be used as a data source of Quick BI?

No, Elasticsearch cannot be used as a data source of Quick BI. You can use Kibana to analyze data and present analysis results.

Does Elasticsearch support scoring plug-ins?

Yes, Elasticsearch supports scoring plug-ins. When you create an index, Elasticsearch allows you to create a tokenizer. This way, when you search for data, Elasticsearch uses a scoring plug-in to sort search results by score. For more information, see Step 5: Search for data.

Does Elasticsearch support LDAP?

Yes, Elasticsearch supports LDAP. If you want to use LDAP to authenticate the requests that are sent to your Elasticsearch cluster, you must deploy an on-premises Elasticsearch cluster of the same version and use this cluster to conduct an authentication test. If LDAP runs as expected, configure your cluster to support LDAP based on the configurations of the on-premises Elasticsearch cluster. For more information, see Use X-Pack to configure LDAP authentication.

Does Alibaba Cloud provide Elasticsearch SDK for Java?

Yes, Alibaba Cloud provides Elasticsearch SDK for Java. Different Elasticsearch versions use different SDKs. For more information, see Java API.

How do I view the kernel version of an Elasticsearch cluster?

By default, Elasticsearch clusters use the kernel of the latest version. For more information about kernel versions, see AliES release notes. If your cluster does not use the kernel of the latest version, the A new kernel patch is available message appears on the Basic Information page of your cluster. You can click the message to view the current kernel version of your cluster. View the kernel version of a cluster

When am I allowed to forcefully restart an Elasticsearch cluster? What is the impact of a force restart?

If your Elasticsearch cluster is in an abnormal state (indicated by the color red or yellow) and you want to restart the cluster, you can forcefully restart the cluster. During the force restart, your Elasticsearch service may be unstable, data may be lost, or data read and write operations may fail. Proceed with caution.

How do I check whether the vulnerability in Apache Log4j 2 is fixed for my Elasticsearch cluster?

The vulnerability is fixed after your Elasticsearch cluster is restarted. For more information, see Vulnerability announcement | RCE vulnerability in Apache Log4j 2.

How long is required to restart an Elasticsearch cluster or node?

When you restart an Elasticsearch cluster or node, the system displays the required time. The time is estimated based on the specifications, data structure, and data volume of the cluster or node. In most cases, a few hours are required to restart a cluster. For more information, see Restart a cluster or node.

Does the system restart an Elasticsearch cluster after I enable or disable the Public Network Access feature for the cluster?

No, the system does not restart an Elasticsearch cluster after you enable or disable the Public Network Access feature for the cluster. Only the status of the Public Network Access feature changes. This does not affect your cluster.

Does the system restart an Elasticsearch cluster after I reset the access password for the cluster?

No, the system does not restart an Elasticsearch cluster after you reset the access password for the cluster. After you reset the access password for a cluster, the system only reloads the data of the cluster but does not restart the cluster. For more information, see Reset the access password for an Elasticsearch cluster.

Is the restart of an Elasticsearch cluster affected if the indexes in the cluster have no replica shards?

Yes, if the indexes in an Elasticsearch cluster have no replica shards, the restart of the cluster is affected. Services may be interrupted during the restart of the cluster. In most cases, if the load of a cluster is not high and the indexes in the cluster have replica shards, the cluster can still provide services during a restart. However, access timeouts may occur during a restart in some cases. For example, if a number of nodes in the cluster are forced to restart at the same time, the cluster is heavily loaded and is not accessible, the indexes in the cluster do not have replica shards, or large amounts of data are written or queried during a restart or forced restart, access timeouts may occur. In these cases, we recommend that you design a retry mechanism on your client first and restart the cluster during off-peak hours.

How do I restart a specific type of node, such as Kibana node, or a single node?

  • Restart a specific type of node

    On the Basic Information page of your Elasticsearch cluster, click Restart in the upper-right corner. In the Restart dialog box, select Node Role for Object and select the type of node that you want to restart in the Node field. For more information, see Restart a cluster or node.

  • Restart a single node
    You can use one of the following methods to restart a single node:
    • On the Basic Information page of your Elasticsearch cluster, click Restart in the upper-right corner. In the Restart dialog box, select Node for Object and select the node that you want to restart in the Node field. For more information, see Restart a cluster or node.
    • In the Node Visualization section of the Basic Information page of your Elasticsearch cluster, move the pointer over the node that you want to restart. In the popover that appears, click Restart. For more information, see View the cluster status and node information.

What do I do if the restart process of an Elasticsearch cluster or a node is stuck?

View the status and load information of the cluster on the Cluster Monitoring page of the cluster. If the cluster is in an abnormal state or is heavily loaded, the restart process of the cluster or a node is stuck. We recommend that you troubleshoot the issue based on the monitoring data of the cluster or by upgrading the configuration of the cluster. For more information, see View cluster monitoring data or Upgrade the configuration of a cluster.

The CPU utilization of and loads on some nodes in an Elasticsearch cluster are normal, whereas other nodes are in the idle state. What do I do?

This issue is caused by unbalanced loads on the cluster. Unbalanced loads may be caused by several reasons, which include inappropriate shard settings, uneven segment sizes, unseparated hot and cold data, and persistent connections that are used for Service Load Balancer (SLB) instances and multi-zone architecture. Resolve the issue based on the actual scenario. For more information, see Unbalanced loads on a cluster.
Notice Before you resolve the issue, check the specifications of your cluster. If the specifications of your cluster are 1 vCPU and 2 GiB of memory, upgrade the specifications to 2 vCPUs and 4 GiB of memory or higher. The specifications of 1 vCPU and 2 GiB of memory are used only for tests. For more information about how to upgrade the specifications, see Upgrade the configuration of a cluster.

What do I do if an Elasticsearch cluster enters an abnormal state indicated by the color yellow?

  • Cause

    If the number of replica shards that you specify for an index is greater than the number of nodes minus 1, the cluster enters an abnormal state indicated by the color yellow.

  • Solution
    Run the GET _cat/indices?v command to query the distribution of shards for indexes and identify the index that is in an abnormal state indicated by the color yellow. Then, change the number of replica shards for the index to 0. After the cluster recovers to a normal state, change the number of replica shards for the index from 0 to the original setting.
    Warning After the number of replica shards is changed to 0, if nodes stop providing services due to errors, data stored on the nodes may be lost. Proceed with caution when you change the number of replica shards. After the cluster recovers to a normal state, change the number of replica shards from 0 to the original setting at your earliest opportunity. The recovery of the cluster requires approximately 1 minute.
    PUT test/_settings
    {
      "index" : {
        "number_of_replicas":"0"
      }
    }                                

What do I do if an Elasticsearch cluster enters an abnormal state indicated by the color red due to heavy loads?

If an error occurs on the node on which primary shards are distributed, the cluster enters an abnormal state indicated by the color red. You can run the GET /_cat/indices?v command to query the distribution of shards for indexes and identify the index that is in an abnormal state indicated by the color red. Then, troubleshoot the issue based on the causes and solutions described in the following table.

Cause Solution
The resources of the cluster are insufficient due to unbalanced loads on nodes. Change the total number of primary and replica shards to an integral multiple of the number of data nodes in the cluster to balance loads on nodes. For more information, see What do I do if shards are not evenly distributed on nodes in an Elasticsearch cluster?.
The cluster stores invalid indexes. Clear invalid indexes at a regular basis, such as monitoring indexes whose names start with .monitoring. For more information about how to configure monitoring indexes, see Configure monitoring indexes.
Shards are not allocated to nodes. Run the GET /_cluster/allocation/explain?pretty command to query the reason why shards are not allocated to nodes and troubleshoot the issue based on the actual situation. After the issue is resolved, run the POST /_cluster/reroute?retry_failed=true command to reallocate shards to nodes.
The cache occupies a large number of resources. Run the POST /<Index name>/_cache/clear?fielddata=true command to clear the cache.
A cluster update operation such as configuration upgrade is being performed on the cluster. Pause the update operation and select Forced Update on the Upgrade/Downgrade page to forcefully update the cluster. For more information, see Upgrade the configuration of a cluster.
The resources of the cluster are insufficient because the cluster uses low specifications such as 1 vCPU and 2 GiB of memory or 2 vCPUs and 4 GiB of memory. Upgrade the configuration of the cluster. For more information, see Upgrade the configuration of a cluster.

Monitoring data or an alert shows that the CPU utilization of my Elasticsearch cluster is excessively high. What do I do?

Troubleshoot the issue based on the causes and solutions described in the following table.
Cause Solution
The write throughput of the cluster is excessively high, or the cluster stores invalid indexes. Reduce the concurrency for write operations, or delete invalid indexes such as monitoring indexes whose names start with .monitoring to reduce resource usage. You can specify a retention duration for such indexes. For more information, see Configure monitoring indexes.
The cache for indexes occupies a large number of resources. Run the POST /Index name/_cache/clear?fielddata=true command to clear the cache.
The cluster uses low specifications. Upgrade the configuration of the cluster. For more information, see Upgrade the configuration of a cluster.
Loads on nodes in the cluster are unbalanced. Change the total number of primary and replica shards to an integral multiple of the number of data nodes in the cluster and make sure that shards are evenly distributed on nodes to balance loads on the nodes. For more information, see What do I do if shards are not evenly distributed on nodes in an Elasticsearch cluster?.

What do I do if the disk usage of my Elasticsearch cluster is excessively high?

Run the DELETE /Index name command to delete invalid indexes. After the disk usage is lower than 75%, forcefully upgrade the configuration of disks in the Elasticsearch console. For more information, see Upgrade the configuration of a cluster. If the disk usage of a node is excessively high, you must optimize the configuration of shards. For more information, see What do I do if shards are not evenly distributed on nodes in an Elasticsearch cluster?.
Note To avoid the impact of high disk usage on Alibaba Cloud Elasticsearch, we recommend that you enable disk usage monitoring and alerting. You must view the alerting text message in time and take appropriate measures in advance. For more information, see View cluster monitoring data. When the disk usage of a node exceeds different thresholds, impacts on the cluster are different. The following descriptions provide the related information in detail:
  • 85%: If the disk usage of a node exceeds 85%, the system no longer allocates new shards to the node.
  • 90%: If the disk usage of a node exceeds 90%, the system migrates the shards on the node to other data nodes with low disk usage.
  • 95%: If the disk usage of a node exceeds 95%, the system forcefully adds the read_only_allow_delete attribute to all indexes in the cluster. As a result, data cannot be written to the indexes. You can only read data from the indexes or delete the indexes.

Monitoring data or an alert shows that the memory usage of my Elasticsearch cluster is excessively high. What do I do?

Troubleshoot the issue based on the causes and solutions described in the following table.
Cause Solution
The cache for the cluster occupies a large amount of memory. If the cache for the cluster occupies a large amount of memory for a short period of time, run the POST /Index name/_cache/clear?fielddata=true command to clear the cache. If the cache for the cluster occupies a large amount of memory for a long period of time, upgrade the configuration of the cluster. For more information, see Upgrade the configuration of a cluster. The memory usage of the cluster may periodically increase but no alert is generated, which may be caused by business fluctuations or memory reclaim of the cluster. This is a normal phenomenon.
The read or write throughput of the cluster is high. Stop the read or write operation, install a throttling plug-in, and then enable the throttling feature of the plug-in. For more information, see Use the aliyun-qos plug-in.
Invalid indexes occupy a large amount of memory. Delete invalid indexes such as monitoring indexes whose names start with .monitoring to release resources. You can specify a retention duration for such indexes. For more information, see Configure monitoring indexes.
Shards are not evenly distributed on nodes, and loads on nodes are unbalanced. Change the total number of primary and replica shards to an integral multiple of the number of data nodes in the cluster and make sure that shards are evenly distributed on nodes to balance loads on the nodes. For more information, see What do I do if shards are not evenly distributed on nodes in an Elasticsearch cluster?.

What do I do if shards are not evenly distributed on nodes in an Elasticsearch cluster?

Appropriately plan shards and reallocate shards for nodes. Make sure that the total number of primary and replica shards is an integral multiple of the number of data nodes in the cluster. This ensures that data is evenly distributed on each data node and prevents heavy loads on a node due to uneven shard distribution. The following descriptions provide examples on how to allocate primary and replica shards for nodes:
  • If the cluster has three data nodes, you can configure three primary shards and one replica shard for each primary shard. The total number of primary and replica shards that you can configure is six.
  • If the cluster has eight data nodes, you can configure four primary shards and one replica shard for each primary shard. The total number of primary and replica shards that you can configure is eight. Alternatively, you can configure eight primary shards and one replica shard for each primary shard. In this case, the total number of primary and replica shards that you can configure is 16.
Note
  • After you adjust the number of shards for nodes, you must make sure that the data stored on the nodes can be evenly distributed on each node. To achieve this, we recommend that you reindex the data during off-peak hours. The availability and query performance of an Elasticsearch cluster increase with the number of replica shards. However, more memory space of the cluster will be occupied.
  • Both the number of shards and the size of each shard contribute to the stability and performance of an Elasticsearch cluster. You must appropriately plan shards for all indexes in an Elasticsearch cluster. This prevents numerous shards from affecting cluster performance when it is difficult to define business scenarios. For more information about how to plan shards for indexes, see Shard evaluation.
  • Uneven shard distribution leads to unbalanced cluster loads. You can use one of the following methods to check whether shards are evenly allocated for nodes in an Elasticsearch cluster:
    • View the monitoring data of the cluster. If shards are not evenly distributed on nodes in the cluster, the CPU utilization, memory usage, or disk usage of a node is high or the node is heavily loaded.
    • Run the GET _cat/shards?v command to query the shard information of the indexes in the cluster. If nodes with heavy loads have a large number of shards, shards are not evenly distributed on all nodes.

My Elasticsearch cluster is heavily loaded, and the cluster logs contain the following error message: java.lang.StackOverflowError for the entire cluster. What do I do?

The error message indicates that a stack overflow error occurs because the amount of data written to the stack by using Lucene exceeds the upper limit. This issue is related to regular expression-based queries and fuzzy match. This issue is fixed in Elasticsearch V6.0 and later. We recommend that you upgrade the configuration of the cluster at your earliest opportunity or optimize the query statement that you use. For more information, see java.lang.StackOverflowError for the entire cluster.

What do I do if no result is returned for a query in an Elasticsearch cluster or a long period of time is required before results can be returned?

If no result is returned for a query in an Elasticsearch cluster or a long period of time is required before results can be returned, the query is a slow query. You can view slow query logs or view the monitoring data of the cluster to identify the causes of the issue. The following table describes common causes and related solutions.
Cause Solution
Loads on nodes are unbalanced due to uneven shard distribution. Change the total number of primary and replica shards to an integral multiple of the number of data nodes in the cluster to balance loads on nodes. For more information, see What do I do if shards are not evenly distributed on nodes in an Elasticsearch cluster?.
The slow query isolation feature is not used. Use the slow query isolation feature to reduce the impact of a slow query on the performance of the cluster. For more information, see Use the slow query isolation feature.
The resources of the cluster are insufficient. If queries that consume a large number of resources are performed on the cluster, we recommend that you optimize the statements used for the queries or upgrade the configuration of the cluster. The queries can be aggregate queries, term queries, script queries, or fuzzy match. For more information about how to upgrade the configuration of an Elasticsearch cluster, see Upgrade the configuration of a cluster.
Note The query performance of an Elasticsearch cluster is related to the health status of the cluster. If the memory usage of an Elasticsearch cluster is lower than 80% and loads on nodes in the cluster are balanced, the cluster can provide high query performance.

What do I do if the following error message is reported when data is being written to my Elasticsearch cluster: Data too large... which is larger than the limit of?

  • Cause

    The error message indicates that the write throughput of the cluster is excessively high and circuit breaking is triggered for the cluster due to insufficient resources of the cluster.

  • Solution
    Notice If the operations in the following solutions cannot be performed, you must stop all query and write operations and forcefully restart the cluster. After the cluster recovers to a normal state, use one of the following solutions to resolve the issue.
    • Reduce the concurrency for the write operations and delete invalid indexes such as monitoring indexes whose names start with .monitoring to release resources. We recommend that you specify a retention duration for such indexes. For more information, see Configure monitoring indexes.
    • Run the POST /Index name/_cache/clear?fielddata=true command to clear the cache for indexes.
    • Upgrade the configuration of the cluster.

Can I delete multiple indexes in an Elasticsearch cluster at a time?

Yes, you can delete multiple indexes in an Elasticsearch cluster at a time. To perform this operation, you must set the Index Deletion parameter to Allow Wildcards in the YML configuration file of the cluster and restart the cluster. After the cluster is restarted, the indexes are deleted at a time by using wildcards. For more information, see Configure the YML file.
Warning Deleted indexes cannot be recovered. Exercise caution when you configure the Index Deletion parameter.

When I create an index in my Elasticsearch cluster, the error message index uuid conflicted is reported, which indicates that the universally unique identifier (UUID) of the index is conflicted, and data cannot be written to documents in the index. What do I do?

This is a known issue. To troubleshoot the issue, upgrade the kernel version of the cluster to V1.5.0 or V1.6.0. For more information, see Upgrade the version of a cluster.

How do I plan resources before I use Elasticsearch, such as cluster specifications, the number of shards, and the size of each shard?

You can evaluate the total amount of the resources that you need to purchase based on your business requirements. For more information about how to evaluate the amount of the required resources, see Evaluate specifications and storage capacity.

How do I view the configuration of an Elasticsearch cluster?

You can view the configuration of the Elasticsearch cluster on the Basic Information page of the cluster. For more information, see View the basic information of a cluster.

When you use Transport Client to access an Elasticsearch cluster, set the cluster.name parameter to the ID of your cluster. For more information, see Transport Client (5.x).

Are services affected when I modify the configuration of an Elasticsearch cluster?

The system restarts the cluster after you modify the configuration of the cluster. The system uses the rolling restart method to restart a cluster. Before the restart, make sure that the cluster is in the Active state (indicated by the color green), each index has at least one replica shard for each primary shard, and resource usage is not high. For example, the value of NodeCPUUtilization(%) is about 80%, that of NodeHeapMemoryUtilization is about 50%, and that of NodeLoad_1m is less than the number of vCPUs of the current node. If all the conditions are met, the cluster can still provide services during the restart. You can view the resource usage on the Cluster Monitoring page. However, we recommend that you modify the configuration of your cluster during off-peak hours.

Does the system reallocate shards for nodes in an Elasticsearch cluster after the number of nodes in the cluster is changed?

No, the system does not reallocate shards for nodes in an Elasticsearch cluster after the number of nodes in the cluster is changed. This may lead to unbalanced loads on the nodes. For more information about the analysis for and solutions to unbalanced loads on nodes, see Unbalanced loads on a cluster.

Can I change the cloud disk type of an Elasticsearch cluster?

Yes, you can change the cloud disk type of an Elasticsearch cluster. The following types of cloud disks are supported: ultra disks, standard SSDs, and enhanced SSDs (ESSDs). These types of cloud disks are listed in ascending order of their storage performance.

Can I convert other types of nodes in an Elasticsearch cluster to warm nodes?

No, you cannot convert other types of nodes in an Elasticsearch cluster to warm nodes. The conversion can cause your cluster to be unstable. For more information, see "Hot-Warm" Architecture in Elasticsearch 5.x.

Can I downgrade the configuration of an Elasticsearch cluster? If yes, what do I do?

Yes, you can downgrade the configuration of an Elasticsearch cluster. For more information about how to perform this operation, see Scale in a cluster or Downgrade the configuration of a cluster.

How do I modify the configuration of an Elasticsearch cluster to ensure that services run as expected when a temporary business surge occurs?

We recommend that you add nodes to the cluster when the temporary business surge occurs and remove the nodes after the business surge. For more information, see Upgrade the configuration of a cluster and Scale in a cluster. For the changes to take effect, the system restarts the cluster. Before the restart, take note of the following items:
  • The cluster is in the Active state (indicated by the color green).
  • Each index of the cluster has at least one replica shard for each primary shard, and the resource usage of the cluster is not high. For example, the value of NodeCPUUtilization(%) is about 80%, that of NodeHeapMemoryUtilization is about 50%, and that of NodeLoad_1m is less than the number of vCPUs of the current node. You can view the resource usage on the Cluster Monitoring page of the cluster.

When I upgrade the configuration of an Elasticsearch cluster, the system displays the following error message: UpgradeVersionMustFromConsole. What do I do?

The error message is returned because the version change does not meet requirements. You can upgrade the versions of clusters only from V5.5.3 to V5.6.16, from V5.6.16 to V6.3.2, or from V6.3.2 to V6.7.0.

How long is required to upgrade the version of an Elasticsearch cluster?

The required time is determined by the data volume, data structure, and specifications of your cluster. The version upgrade requires about 1 hour.

Are services affected when I upgrade the version of an Elasticsearch cluster?

When you upgrade the version of an Elasticsearch cluster, you can still read data from or write data to the cluster but cannot make other changes. We recommend that you perform a version upgrade during off-peak hours. For more information about the precautions and procedure for a version upgrade, see Upgrade the version of a cluster.

What do I do if an error is reported when I upgrade the configuration of an Elasticsearch cluster or when a configuration upgrade for an Elasticsearch cluster times out?

In most cases, this issue occurs because the cluster is in an abnormal state. In this case, we recommend that you pause the query and write operations, and troubleshoot the issue by following the instructions described in What do I do if an Elasticsearch cluster enters an abnormal state indicated by the color red due to heavy loads?. After the cluster recovers to a normal state, upgrade the configuration of the cluster again. You can also ignore the health status of the cluster and perform a forced update when you upgrade the configuration of the cluster. However, the forced update may affect the services provided by the cluster. Proceed with caution.

If this issue occurs due to other causes, troubleshoot the issue based on the error message that is reported.

Can I use the YML configuration file of an Elasticsearch cluster to configure the http.max_content_length and discovery.zen.ping_timeout parameters?

You are not allowed to configure the two parameters. If you want to add these parameters to the configuration file, contact Alibaba Cloud Elasticsearch technical engineers. Before you add the parameters, make sure that the parameter settings are correct and you accept the impact caused by parameter modifications. If the parameter settings are incorrect, the system fails to perform a rolling restart for the cluster.
Note In most cases, you do not need to change the settings of the following parameters: discovery.zen.ping_timeout, discovery.zen.fd.ping_timeout, discovery.zen.fd.ping_interval, and discovery.zen.fd.ping_retries.

Can I switch the VPC of an Elasticsearch cluster?

No, you cannot switch the VPC of an Elasticsearch cluster. You can purchase an Elasticsearch cluster in the desired VPC and migrate data from the original cluster to the new cluster. Then, cancel the subscription of or release the original cluster.

Will the existing data in an Elasticsearch cluster be lost if I change the cloud disk type of the cluster?

No, the existing data in an Elasticsearch cluster will not be lost if you change the cloud disk type of the cluster. However, new data that is continuously written to the cluster may be lost. We recommend that you change the cloud disk type during off-peak hours or after you stop the data write operations. For more information about how to change the cloud disk type of an Elasticsearch cluster, see Upgrade the configuration of a cluster.

When I upgrade the configuration of an Elasticsearch cluster, the system displays a prompt message indicating that the cluster is in an unhealthy state, but the cluster is in a normal state indicated by the color green. What do I do?

Some indexes in the cluster may be in the close state. You can run the POST /<index_name>/_open command to temporarily open the indexes. For more information, see Upgrade the configuration of a cluster.

Can I upgrade the vCPU configurations of an Elasticsearch cluster without migrating data?

No, you cannot upgrade the vCPU configurations of an Elasticsearch cluster without migrating data. If you upgrade or downgrade the vCPU configurations of an Elasticsearch cluster, the system performs a blue-green update for the cluster. After the blue-green update, the IP addresses of nodes in the cluster are changed, and data is migrated from the original nodes to the new nodes.

When I remove data nodes from my Elasticsearch cluster, the following error message is reported: "This operation may cause a shard allocation error or insufficient storage, CPU, or memory resources." What do I do?

Possible cause Solution
The resources of the cluster are insufficient.

After data nodes are removed, the cluster does not have sufficient resources to store system data or handle workloads. The resources include disks, memory, and vCPUs.

Run the GET _cat/indices?v command to check whether the resource usage of your cluster, such as disk usage, is greater than the related threshold. Make sure that the cluster has sufficient resources to store data and process requests. If these requirements are not met, upgrade the configuration of the cluster. For more information, see Upgrade the configuration of a cluster.
Errors occur on shard allocation.

Elasticsearch is based on Lucene principles. This indicates that Elasticsearch does not migrate two or more replica shards of an index on a data node to the same data node. In this case, after data nodes are removed, the number of replica shards in a cluster may be greater than or equal to the number of data nodes. This results in shard allocation errors.

Run the GET _cat/indices?v command to check whether the number of replica shards in the cluster is less than the number of data nodes after specific data nodes are removed. If this requirement is not met, change the number of replica shards. For more information, see Index Templates. The following code provides an example on how to change the number of replica shards to 2 in the index template:
PUT _template/template_1
{
  "template": "*",
  "settings": {
    "number_of_replicas": 2
  }
}  

When I remove data nodes from my Elasticsearch cluster, the error message "The cluster is in an abnormal state or has ongoing tasks." is reported. What do I do?

Enable the Intelligent Maintenance feature for the cluster to diagnose the cluster and troubleshoot the issue based on the diagnostic results and suggestions. For more information, see Perform a diagnostic on an Elasticsearch cluster.

When I remove data nodes from my Elasticsearch cluster, the error message "The number of nodes that you reserve must be more than two and more than half the current number of nodes." is reported. What do I do?

To ensure cluster reliability and stability, at least two data nodes must be reserved after data node removal, and the number of data nodes selected for removal or data migration must be no more than half the current number of data nodes. If the requirements are not met, adjust the data nodes to remove, or upgrade the configuration of the cluster. For more information about how to upgrade the configuration of an Elasticsearch cluster, see Upgrade the configuration of a cluster.

When I remove data nodes from my Elasticsearch cluster, the error message "The operation is not supported." is reported. What do I do?

Run the GET _cluster/settings command to query the configurations of the cluster and check whether the cluster contains the configuration "cluster.routing.allocation.enable" : "none". This configuration does not allow data distribution. If the cluster contains the configuration, you can temporarily change the configuration to "cluster.routing.allocation.enable" : "all". If the configuration affects your other operations, you can change the configuration to the original setting after data node removal.

What do I do if data nodes fail to be removed or data fails to be migrated due to the auto_expand_replicas index setting?

  • Cause

    You may use the access control feature provided by the X-Pack plug-in. In earlier Elasticsearch versions, this feature applies the "index.auto_expand_replicas" : "0-all" setting to the .security index by default. This causes errors when you migrate data or remove data nodes.

  • Solution
    1. Run the following command to query index settings:
      GET .security/_settings
      The following result is returned:
      {
        ".security-6" : {
          "settings" : {
            "index" : {
              "number_of_shards" : "1",
              "auto_expand_replicas" : "0-all",
              "provided_name" : ".security-6",
              "format" : "6",
              "creation_date" : "1555142250367",
              "priority" : "1000",
              "number_of_replicas" : "9",
              "uuid" : "9t2hotc7S5OpPuKEIJ****",
              "version" : {
                "created" : "6070099"
              }
            }
          }
        }
      }
    2. Use one of the following methods to modify the auto_expand_replicas index setting:
      • Method 1
        PUT .security/_settings
        {
          "index" : {
            "auto_expand_replicas" : "0-1"
          }
        }
      • Method 2
        PUT .security/_settings
        {
          "index" : {
            "auto_expand_replicas" : "false",
            "number_of_replicas" : "1"
          }
        }
        Notice The number_of_replicas parameter specifies the number of replica shards for each index. You can configure this parameter based on your business requirements. Make sure that the value of this parameter is greater than or equal to 1 but no more than the number of available data nodes.

How do I clear the cache for an Elasticsearch cluster?

Log on to the Kibana console of the cluster and run one of the following commands to clear the cache:
  • Clear the cache of a specific index
    POST /<Index name>/_cache/clear?fielddata=true
  • Clear all cache
    POST /_cache/clear

How do I migrate nodes in an Elasticsearch cluster from one zone to another?

Perform the steps described in Migrate nodes in a zone to migrate nodes from one zone to another.

Can I update only the disk configuration of an Elasticsearch cluster during a configuration upgrade of the cluster?

Yes, you can update only the disk configuration of an Elasticsearch cluster during a configuration upgrade of the cluster. For more information, see Upgrade the configuration of a cluster.
Notice When you update the disk configuration of an Elasticsearch cluster, the system performs a rolling restart for the cluster. We recommend that you update the disk configuration of an Elasticsearch cluster during off-peak hours.

Can I change the JVM parameter settings of an Elasticsearch cluster?

Alibaba Cloud Elasticsearch clusters use JVM parameter settings that are recommended by open source Elasticsearch. The settings cannot be changed. By default, JVM heap memory is half of cluster memory.

How do I update dictionary content when I use the IK analysis plug-in?

You can use the standard update or rolling update feature of the IK analysis plug-in to update dictionary content. For more information, see Use the analysis-ik plug-in.

When I use the IK analysis plug-in, the error message the "ik startOffset" is reported. What do I do?

The error message is returned because of an Elasticsearch V6.7 bug. You must restart your cluster. For more information, see Restart a cluster or node.

The IK dictionary files on my on-premises machine are lost. Can I retrieve them on the cluster management page?

No, you cannot retrieve them on the cluster management page. You can only delete or update dictionary files on the cluster management page. We recommend that you download the official main and stopword dictionary files. Then, change the tokens in the files to those in your system dictionary file and upload the files to your cluster.

How do I apply updated IK dictionaries to existing data?

You must perform a reindex operation. If indexes are configured with IK tokens, the updated dictionaries apply only to new data in these indexes. If you want to apply the updated dictionaries to all the data in these indexes, you must perform a reindex operation. For more information, see Configure a remote reindex whitelist.

Is a specific threshold provided for full GC?

Full GC is used to clean the entire heap memory. Whether full GC is correctly performed needs to be analyzed based on the service latency, heap memory size before full GC, and heap memory size after full GC. The CMS collector starts to collect garbage when the memory usage is 75%. This is because some space is reserved for burst traffic.

Can I remove plug-ins that are not used?

You can remove only some plug-ins. On the Plug-ins page of your Elasticsearch cluster, you can view plug-ins that can be removed on the Built-in Plug-ins tab. If Remove is displayed in the Actions column of a plug-in, the plug-in can be removed. For more information about how to remove a plug-in, see Install and remove a built-in plug-in.

Are the dictionaries provided by the IK analysis plug-in of Alibaba Cloud Elasticsearch the same as the dictionaries provided by the IK analysis plug-in of open source Elasticsearch?

Yes, the dictionaries provided by the IK analysis plug-in of Alibaba Cloud Elasticsearch are the same as the dictionaries provided by the IK analysis plug-in of open source Elasticsearch. For more information, see IK Analysis for Elasticsearch.

Can a custom plug-in access an external network, such as reading dictionary files on GitHub?

No, custom plug-ins cannot access external networks. If you want your Elasticsearch cluster to access external files, upload the files to OSS and connect your Elasticsearch cluster to OSS.

Does a custom plug-in support the rolling update method?

No, custom plug-ins do not support the rolling update method. If you want a custom plug-in to support this method, configure the plug-in based on the rolling update method of the IK analysis plug-in. For more information, see IK Analysis for Elasticsearch.

How do I configure the analysis-aliws plug-in? What is the format of the dictionary file for this plug-in?

For more information about how to configure the plug-in, see Use the analysis-aliws plug-in.

The dictionary file must meet the following requirements:
  • Name: aliws_ext_dict.txt.
  • Encoding format: UTF-8.
  • Content: Each row contains one word and ends with \n (line feed in UNIX or Linux). No whitespace characters are used before or after this word. If the dictionary file is generated in Windows, you must use the dos2unix tool to convert the file before you upload it.

What are the differences among Elasticsearch synonyms, IK tokens, and AliNLP tokens?

Token type Usage Description Supported file type Tokenizer and analyzer
Synonym You can upload a synonym dictionary file on the Cluster Configuration page of your cluster to enable the cluster to use it. After you write several synonyms to the file, the system displays all the synonyms when you query one of them. The synonym dictionary file must be a TXT file encoded in UTF-8. Custom tokenizer and analyzer
IK token The IK tokens are used based on the analysis-ik plug-in. The system splits a paragraph based on the main.dic file. If you send a query request that contains one or more words split from the paragraph, the system returns the entire paragraph in the query result. The analysis-ik plug-in also provides a stopword file named stop.dic. The query result does not include the stopwords in the stop.dic file. You can view the dictionary file from the official documentation. The main and stopword dictionary files must be DIC files encoded in UTF-8. Tokenizer:
  • ik_smart
  • ik_max_word
AliNLP token The AliNLP tokens are used based on the analysis-aliws plug-in. The analysis-aliws plug-in works in a similar way as the analysis-ik plug-in, but the analysis-aliws plug-in does not provide a separate stopword dictionary file. Stopwords are integrated into the main dictionary file aliws_ext_dict.txt. The file is invisible to you. In addition, you are not allowed to customize stopwords. The dictionary file name must be aliws_ext_dict.txt. The file must be encoded in UTF-8.
  • Analyzer: aliws, which does not return function words, function phrases, or symbols
  • Tokenizer: aliws_tokenizer

How do I install the analysis-ik plug-in?

analysis-ik is an IK analysis plug-in provided by Alibaba Cloud Elasticsearch. This plug-in is a built-in plug-in and cannot be removed. You can use the standard or rolling update method to update the built-in IK main dictionary and stopword list of the analysis-ik plug-in. Then, you can use the updated dictionary and stopword list when you configure mappings for an index. For more information about how to use the analysis-ik plug-in, see Use the analysis-ik plug-in.

If I use the rolling update method to update dictionaries that are dynamically loaded from OSS and the dictionaries stored in OSS are updated, will the dictionaries on all nodes in my Elasticsearch cluster be automatically updated?

No, the dictionaries on all nodes in your Elasticsearch cluster will not be automatically updated. Alibaba Cloud Elasticsearch does not support the automatic update of dictionaries on nodes in an Elasticsearch cluster after a rolling update of the dictionaries stored in Object Storage Service (OSS). After the dictionaries stored in OSS are changed, you must manually upload the dictionary file for the updated dictionary file to take effect. For indexes that are configured with IK tokens, synonyms, or AliNLP tokens, new dictionaries take effect only for data that is inserted after a standard or rolling update. If you also want the new dictionaries to take effect for existing data, you must reindex the existing data.

Can I specify a retention period for the .security indexes of an Elasticsearch cluster?

No, you cannot specify a retention period for the .security indexes of an Elasticsearch cluster. Elasticsearch does not automatically delete expired indexes. You must manually delete the expired .security indexes. For more information, see Step 6: (Optional) Delete the index.

How do I view the logs that are generated over the last seven days?

You can call the ListSearchLog API operation to obtain all the logs that you require. For more information, see ListSearchLog.

I cannot view the search and update logs of an Elasticsearch cluster. What do I do?

You can configure slow logs and reduce the timestamp precision of log entries. For more information, see References.

How do I configure and view the slow logs of an Elasticsearch cluster?

By default, Elasticsearch logs only read and write operations that require 5 seconds to 10 seconds to complete as slow logs. You can log on to the Kibana console of the cluster and run the related command to reduce the timestamp precision of log entries. This helps capture more logs. For more information, see References.
Note You are not allowed to change the format of slow logs.

How do I obtain the slow logs of an Elasticsearch cluster on a regular basis?

You can call the ListSearchLog API operation to obtain the slow logs of your cluster on a regular basis. For more information, see ListSearchLog.

Can I restore data from the snapshots of an Elasticsearch cluster to an Elasticsearch cluster of a different version?

For automatic snapshots, you can restore data from the snapshots only to the original cluster. For more information, see Create automatic snapshots and restore data from automatic snapshots.

For manual snapshots, you can restore data from the snapshots to a cluster other than the original cluster. We recommend that you use a destination cluster whose version is the same as the original cluster. If the versions are different, compatibility issues may occur. For more information, see Create manual snapshots and restore data from manual snapshots.

When I back up data for an Elasticsearch cluster, the system displays a message indicating that the cluster is unhealthy. What do I do?

When an Elasticsearch cluster is unhealthy, you cannot use the Auto Snapshot feature and configure shared OSS repositories for the cluster. You can purchase an OSS bucket that resides in the same region as your Elasticsearch cluster. Then, create an OSS repository and manually create snapshots. For more information, see Create manual snapshots and restore data from manual snapshots.

I enable the Auto Snapshot feature but do not configure shared OSS repositories for an Elasticsearch cluster. Are snapshots created?

Elasticsearch provides an OSS bucket for your cluster by default. You can log on to the Kibana console of your cluster and run the GET _snapshot/aliyun_auto_snapshot/_all command to obtain automatic snapshots. For more information about how to log on to the Kibana console, see Log on to the Kibana console.

When I restore data from snapshots, the destination Elasticsearch cluster displays a message indicating that shards are abnormal. After I run the POST /_cluster/reroute?retry_failed=true command to reroute the shards, the issue persists. What do I do?

The following figure shows the issue.

Data restoration issue
Delete the problematic index and call the _restore API to restore it. You must add the max_restore_bytes_per_sec parameter to the request. This parameter is used to limit the restoration rate. The default value of this parameter is 40mb. This value indicates that the index is restored at a speed of 40 MB per second.
POST /_snapshot/aliyun_snapshot_from_instanceId/es-cn-instanceId_datetime/_restore
{
    "indices": "myIndex",
    "settings": {
    "max_restore_bytes_per_sec" : "150mb" 
    }
}
Note You can also add the following parameters:
  • compress: specifies whether to enable data compression. Default value: true.
  • max_snapshot_bytes_per_sec: specifies the rate at which snapshots are created for each node. Default value: 40mb.

Can I export data from an Elasticsearch cluster to my on-premises machine?

You can use the data backup feature provided by Elasticsearch to export data. For more information, see Data backup overview. You can create snapshots, store them in OSS, and then download objects from OSS. For more information, see Download objects.

How do I use the email notification feature of X-Pack Watcher?

You can configure specific actions for X-Pack Watcher. For more information, see Watcher settings in Elasticsearch.
Notice X-Pack Watcher of Elasticsearch cannot directly access the Internet. You must use the internal endpoint of an Elasticsearch cluster to access the Internet. Therefore, you must create an ECS instance that can access both the Internet and the Elasticsearch cluster. Then, use the ECS instance as a proxy to perform actions. For more information, see Configure a DingTalk chatbot to receive alert notifications from X-Pack Watcher.

What do I do if the system reports an alert indicating that memory cannot be allocated to the garbage collector?

Possible causes include heavy loads, high query QPS, or large amounts of data to write. Troubleshoot the issue based on the following instructions:
  • Heavy loads: For more information, see High disk usage and read-only indexes.
  • High query QPS or large amounts of data to write: We recommend that you install the aliyun-qos plug-in on your Elasticsearch cluster to implement read/write throttling. For more information, see Use the aliyun-qos plug-in.
    Note For image retrieval, we recommend that you install the aliyun-knn plug-in on your cluster and plan your cluster and indexes. For more information, see Use the aliyun-knn plug-in.

How do I use a client to access an Alibaba Cloud Elasticsearch cluster? What is the difference between access to an Alibaba Cloud Elasticsearch cluster and access to an open source Elasticsearch cluster?

Access an Alibaba Cloud Elasticsearch cluster by using its internal or public endpoint. Access an open source Elasticsearch cluster by using its address. For more information, see Use a client to access an Alibaba Cloud Elasticsearch cluster.

When I use a client to access an Elasticsearch cluster, can I disable the basic access authentication feature?

No, you cannot disable the basic authentication feature. The basic authentication feature is a Kibana authentication mechanism provided by the built-in Elasticsearch plug-in X-Pack. Therefore, you cannot disable the feature.

I purchase an ECS instance that resides in the same VPC as but a different zone from an Elasticsearch cluster. Can I use the ECS instance to access the Elasticsearch cluster over an internal network?

Yes, you can use the ECS instance to access the Elasticsearch cluster over an internal network. You can use an ECS instance to access an Elasticsearch cluster over an internal network if they reside in the same VPC.

How do I access an Elasticsearch cluster over the Internet?

You can access the cluster over the Internet by using its public endpoint and configuring a public IP address whitelist. For more information, see Configure a public or private IP address whitelist for an Elasticsearch cluster. When you access the cluster, you must configure parameters, such as the domain name, username, and password. For more information, see Use a client to access an Alibaba Cloud Elasticsearch cluster.