This topic provides answers to some commonly asked questions about Alibaba Cloud Elasticsearch clusters.

When I purchase an Elasticsearch cluster, I selected an incorrect zone. How do I change the zone?

After the cluster is created and in the Active state, migrate nodes from the incorrect zone to your desired zone. For more information, see Migrate nodes in a zone.

What are the mappings between versions on the Elasticsearch buy page and specific Elasticsearch versions?

Version on the buy page Specific version
7.7 7.7.1
7.4 7.4.0
6.8 6.8.6
6.7 6.7.0
6.3 6.3.2
5.6 5.6.16
5.5 5.5.3

After I cancel the subscription of or release an Elasticsearch cluster, I purchase another cluster. Does the endpoint of the new cluster remain the same as that of the original cluster?

No, after you purchase the new cluster, we recommend that you modify the client code and cancel the subscription of or release the original cluster to avoid service interruptions.

How do I access an Elasticsearch cluster from the classic network?

You can use the ClassicLink feature to access an Elasticsearch cluster from the classic network. For more information, see Access to an Alibaba Cloud Elasticsearch cluster from the classic network.

How do I release an Elasticsearch cluster?

On the Elasticsearch Clusters page, find the cluster that you want to release. Then, in the Actions column, choose More > Release. For more information, see Release a cluster.

When is an Elasticsearch cluster released after it is suspended?

The cluster is released 24 hours after it is suspended. After it is released, all data in the cluster is permanently deleted and cannot be recovered. For more information, see Overdue payments and cluster release.

Can I purchase an Elasticsearch cluster that has only one node?

No, an Elasticsearch cluster must have a minimum of two data nodes. For more information, see Parameters on the buy page.

When I purchase an Elasticsearch cluster, resources of a specific category are sold out. What do I do?

Take one of the following measures:
  • Select another region.
  • Select another zone.
  • Select another category.

If the resources that you want to purchase are still unavailable after you take all of the preceding measures, try again later. Resources are dynamic. If resources are insufficient, Alibaba Cloud replenishes them as soon as possible.

Can I upgrade or downgrade the version of an Elasticsearch cluster?

Upgrades are supported, whereas downgrades are not supported. You can upgrade Elasticsearch clusters only from V6.3.2 to V6.7.0. For more information, see Upgrade the version of a cluster.

If you want to perform upgrades between other versions or downgrades, purchase an Elasticsearch cluster of the desired version. Then, migrate data from the original cluster to the new cluster and cancel the subscription of or release the original cluster.

Can I log on to an Elasticsearch cluster over SSH and modify the configuration of the cluster?

No, for security purposes, you are not allowed to log on to your Elasticsearch cluster over SSH. If you want to modify the configuration of your cluster, use the cluster configuration feature of Elasticsearch. For more information, see Overview.

Is Logstash V6.7 compatible with Elasticsearch V6.3?

Yes, for more information, see Compatibility matrixes.

Can Elasticsearch be used as a data source of Quick BI?

No, you can use Kibana to analyze and present analysis results.

Does Elasticsearch support scoring plug-ins?

Yes, when you create an index, Elasticsearch allows you to create a tokenizer. When you search for data, Elasticsearch uses a scoring plug-in to sort search results by score. For more information, see Search for data.

Does Elasticsearch support LDAP?

No, if you want to use Lightweight Directory Access Protocol (LDAP) to authenticate requests sent to your Elasticsearch cluster, you must deploy an on-premises Elasticsearch cluster of the same version. Then, use the on-premises Elasticsearch cluster to conduct an authentication test. If LDAP runs as expected, send related configurations to Alibaba Cloud Elasticsearch technical engineers. Then, the engineers can configure your cluster to support LDAP based on the configurations. For more information, see Integrate X-Pack with LDAP.

Does Alibaba Cloud provide Elasticsearch SDK for Java?

Yes, different Elasticsearch versions use different SDKs. For more information, see Java API.

How do I view the kernel version of an Elasticsearch cluster?

By default, Elasticsearch clusters use the kernel of the latest version. For more information about kernel versions, see AliES release notes. If your cluster does not use the kernel of the latest version, the A new kernel patch is available message appears on the Basic Information page of your cluster. You can click the message to view the current kernel version of your cluster.View the kernel version of a cluster

How long is required to restart an Elasticsearch cluster or node?

When you restart an Elasticsearch cluster or node, the system displays the required time. The time is estimated based on your cluster specifications, data structure, and data volume. Typically, it requires a few hours to restart a cluster. For more information, see Restart a cluster or node.

Does the system restart an Elasticsearch cluster after I enable or disable the Public Network Access feature for the cluster?

No, only the status of the Public Network Access feature changes, which does not affect your cluster.

The CPU utilization and loads of some nodes in an Elasticsearch cluster are normal, whereas other nodes are in the idle state. What do I do?

This issue is caused by unbalanced loads on the cluster. Unbalanced loads may be caused by several reasons. These reasons include inappropriate shard settings, uneven segment sizes, unseparated hot and cold data, and persistent connections that are used for Service Load Balancer (SLB) instances and multi-zone architecture. Resolve the issue based on the actual situation. For more information, see Unbalanced loads on a cluster.
Notice Before you resolve the issue, check the specifications of your cluster. If the specifications of your cluster are 1 vCPU and 2 GiB of memory, upgrade the specifications to 2 vCPUs and 4 GiB of memory or higher. The specifications of 1 vCPU and 2 GiB of memory are used only for tests. For more information about how to upgrade the specifications, see Upgrade the configuration of a cluster.

How do I plan resources before I use Elasticsearch, such as cluster specifications, the number of shards, and the size of each shard?

Evaluate the specifications and storage capacity of your Elasticsearch cluster. For more information, see Evaluate specifications and storage capacity. You can purchase an Elasticsearch cluster or upgrade the configuration of the cluster based on the evaluation results.

How do I view the configuration of an Elasticsearch cluster?

You can view the configuration of your Elasticsearch cluster on the Basic Information page of the cluster. For more information, see View the basic information of a cluster.

When you use Transport Client to access an Elasticsearch cluster, set the cluster.name parameter to the ID of your cluster. For more information, see Transport Client (5.x).

Are services affected when I modify the configuration of an Elasticsearch cluster?

The system restarts the cluster after you modify its configuration. The system uses the rolling restart method to restart a cluster. Before the restart, make sure that the cluster is in the Active state (indicated by the color green), each index has at least one replica shard for each primary shard, and resource usage is not high. For example, the value of NodeCPUUtilization(%) is about 80%, that of NodeHeapMemoryUtilization is about 50%, and that of NodeLoad_1m is less than the number of vCPUs of the current node. If all the conditions are met, the cluster can still provide services during the restart. You can view the resource usage on the Cluster Monitoring page. However, we recommend that you modify the configuration of your cluster during off-peak hours.

Can I change the cloud disk type of an Elasticsearch cluster?

No, if you want to change the cloud disk type of your cluster, purchase another cluster based on your requirements and migrate data from the original cluster to the new cluster. Then, cancel the subscription of or release the original cluster. For more information about how to migrate data, see Configure a shared OSS repository.

Can I convert other types of nodes in an Elasticsearch cluster to warm nodes?

No, the conversion can cause your cluster to be unstable. For more information, see "Hot-Warm" Architecture in Elasticsearch 5.x.

Can I downgrade the specifications of an Elasticsearch cluster? If yes, how do I do?

No, you can scale in your cluster. For more information, see Scale in an Elasticsearch cluster.

In the event of a temporary business surge, how do I modify the configuration of an Elasticsearch cluster to ensure that services run as expected?

We recommend that you add nodes to the cluster when the temporary business surge occurs and remove the nodes after the business surge. For more information, see Upgrade the configuration of a cluster and Scale in an Elasticsearch cluster. For the changes to take effect, the system restarts the cluster. Before the restart, take note of the following items:
  • The cluster is in the Active state (indicated by the color green).
  • Each index of the cluster has at least one replica shard for each primary shard, and the resource usage of the cluster is not high. For example, the value of NodeCPUUtilization(%) is about 80%, that of NodeHeapMemoryUtilization is about 50%, and that of NodeLoad_1m is less than the number of vCPUs of the current node. You can view the resource usage on the Cluster Monitoring page of the cluster.

When I upgrade the configuration of an Elasticsearch cluster, the system displays the "UpgradeVersionMustFromConsole" error message. What do I do?

The error message returned because the version change does not meet requirements. You can upgrade clusters only from V6.3.2 to V6.7.0.

How long is required to upgrade the version of an Elasticsearch cluster?

The required time is determined by the data volume, data structure, and cluster specifications. The version upgrade requires about one hour.

Are services affected when I upgrade the version of an Elasticsearch cluster?

When you upgrade the version of an Elasticsearch cluster, you can still read data from or write data to the cluster but cannot make other changes. We recommend that you perform a version upgrade during off-peak hours. For more information about the precautions and procedure for a version upgrade, see Upgrade the version of a cluster.

Can I use the YML configuration file of an Elasticsearch cluster to change the settings of the http.max_content_length and discovery.zen.ping_timeout parameters?

You are not allowed to configure the two parameters. If you want to add these parameters to the configuration file, contact Alibaba Cloud Elasticsearch technical engineers. Before you add the parameters, make sure that the parameter settings are correct and you accept the impact caused by parameter modifications. If the parameter settings are incorrect, the system fails to perform a rolling restart for the cluster.
Note In most cases, you do not need to change the settings of the discovery.zen.ping_timeout, discovery.zen.fd.ping_timeout, discovery.zen.fd.ping_interval, and discovery.zen.fd.ping_retries parameters.

Can I switch the VPC of an Elasticsearch cluster?

No, you can purchase an Elasticsearch cluster in the desired virtual private cloud (VPC) and migrate data from the original cluster to the new cluster. Then, cancel the subscription of or release the original cluster.

Can I change the JVM parameter settings of an Elasticsearch cluster?

Alibaba Cloud Elasticsearch clusters use JVM parameter settings that are recommended by open source Elasticsearch. We recommend that you do not change the settings. By default, JVM heap memory is half of cluster memory.

How do I update dictionary content when I use the IK analysis plug-in?

You can use the standard update or rolling update feature of the IK analysis plug-in to update dictionary content. For more information, see Use the analysis-ik plug-in.

When I use the IK analysis plug-in, the system displays the "ik startOffset" error message. What do I do?

The error message returned because of an Elasticsearch V6.7 bug. You must restart your cluster. For more information, see Restart a cluster or node. We will fix the bug as soon as possible.

The IK dictionary files on my on-premises machine are lost. Can I retrieve them on the cluster management page?

No, you can only delete or update dictionary files on the cluster management page. We recommend that you download the official main and stopword dictionary files. Then, change the tokens in the files to those in your system dictionary file and upload the files to your cluster.

After I update IK dictionaries, how do I apply the new dictionaries to existing data?

You must perform a reindex operation. If indexes are configured with IK tokens, the new dictionaries apply only to new data in these indexes. If you want to apply the new dictionaries to all the data in these indexes, you must perform a reindex operation. For more information, see Configure the YML file.

Is there a specific threshold for full GC?

Full garbage collection (GC) is used to clean the entire heap memory. Whether full GC is correctly performed needs to be analyzed based on the service latency, heap memory size before full GC, and heap memory size after full GC. The CMS collector starts to collect garbage when the memory usage is 75%. This is because some space is reserved for burst traffic.

Can I remove plug-ins that are not used?

You can remove only some plug-ins. On the Built-in Plug-ins tab of the Plug-ins page of your Elasticsearch cluster, you can view plug-ins that can be removed. If the system displays Remove in the Actions column of a plug-in, the plug-in can be removed. For more information about how to remove a plug-in, see Install and remove a built-in plug-in.

Are the dictionaries provided by the IK analysis plug-in of Alibaba Cloud Elasticsearch the same as those provided by the IK analysis plug-in of open source Elasticsearch?

Yes, for more information, see IK Analysis for Elasticsearch.

Can a custom plug-in access an external network, such as reading dictionary files on GitHub?

No, if you want your Elasticsearch cluster to access external files, upload the files to Alibaba Cloud Object Storage Service (OSS) and connect your Elasticsearch cluster to OSS.

Does a custom plug-in support the rolling update method?

No, if you want a custom plug-in to support this method, configure the plug-in based on the rolling update method of the IK analysis plug-in. For more information, see IK Analysis for Elasticsearch.

How do I configure the analysis-aliws plug-in? What is the format of the dictionary file for this plug-in?

For more information about how to configure the plug-in, see Use the analysis-aliws plug-in.

The dictionary file must meet the following requirements:
  • Name: aliws_ext_dict.txt.
  • Encoding format: UTF-8.
  • Content: Each row contains one word and ends with \n (line break in UNIX or Linux). No whitespace characters are used before or after this word. If the dictionary file is generated in Windows, you must use the dos2unix tool to convert the file and upload the file to your cluster.

What are the differences among Elasticsearch synonyms, IK tokens, and AliNLP tokens?

Token type Usage Description Supported file type Tokenizer and analyzer
Synonym You can upload a synonym dictionary file on the Cluster Configuration page of your cluster to enable the cluster to use it. After you write several synonyms in the file, the system displays all the synonyms when you query one of them. The synonym dictionary file must be a TXT file encoded in UTF-8. Custom tokenizer and analyzer
IK token The IK tokens are used based on the analysis-ik plug-in. The system splits a paragraph based on the main.dic file. If you send a query request that contains one or more split words, the system returns the entire paragraph in the query result. The analysis-ik plug-in also provides a stopword file named stop.dic. The query result does not include the stopwords in the stop.dic file You can view the dictionary file from the official documentation. The main and stopword dictionary files must be DIC files encoded in UTF-8. Tokenizer:
  • ik_smart
  • ik_max_word
AliNLP token The AliNLP tokens are used based on the analysis-aliws plug-in. The analysis-aliws plug-in works in a similar way as the analysis-ik plug-in, but the analysis-aliws plug-in does not provide a separate stopword dictionary file. Stopwords are integrated into the main dictionary file aliws_ext_dict.txt. The file is invisible to you. In addition, you are not allowed to customize stopwords. The dictionary file name must be aliws_ext_dict.txt. The file must be encoded in UTF-8.
  • Analyzer: aliws, which does not return function words, function phrases, or symbols
  • Tokenizer: aliws_tokenizer

Can I specify a retention period for the .security indexes of an Elasticsearch cluster?

No, Elasticsearch does not automatically delete expired indexes. You must manually delete the expired .security indexes. For more information, see Delete an index.

I can view Elasticsearch cluster logs that are generated only over the last seven days. How do I view more logs?

You can call the ListSearchLog operation to obtain all logs that you require. For more information, see ListSearchLog.

I cannot view the search and update logs of an Elasticsearch cluster. What do I do?

You can configure slow logs and reduce the timestamp precision of log entries. For more information, see Configure slow logs.

How do I configure and view the slow logs of an Elasticsearch cluster?

By default, Elasticsearch logs only read and write operations that require 5 to 10 seconds to complete as slow logs. You can log on to the Kibana console of the cluster and run the related command to reduce the timestamp precision of log entries. This helps capture more logs. For more information, see Configure slow logs.
Note You are not allowed to change the format of slow logs.

How do I obtain the slow logs of an Elasticsearch cluster on a regular basis?

You can call the ListSearchLog operation on a regular basis to obtain the slow logs of your cluster. For more information, see ListSearchLog.

Can I restore data from the snapshots of an Elasticsearch cluster to an Elasticsearch cluster of a different version?

For automatic snapshots, you can restore data from the snapshots only to the original cluster. For more information, see Create automatic snapshots and restore data from automatic snapshots.

For manual snapshots, you can restore data from the snapshots to a cluster other than the original cluster. We recommend that the versions of the destination cluster and original cluster must be the same. If the versions are different, compatibility issues may occur. For more information, see Commands to create snapshots and restore data.

When I back up data for an Elasticsearch cluster, the system displays a message indicating that the cluster is unhealthy. What do I do?

When an Elasticsearch cluster is unhealthy, you cannot use the Auto Snapshot feature and specify shared OSS repositories. You can purchase an OSS bucket that resides in the same region as your Elasticsearch cluster. Then, create an OSS repository and manually create snapshots. For more information, see Commands to create snapshots and restore data.

I enable the Auto Snapshot feature but do not specify shared OSS repositories for an Elasticsearch cluster. Are snapshots created?

Elasticsearch provides an OSS bucket for your cluster by default. You can log on to the Kibana console of your cluster and run the GET _snapshot/aliyun_auto_snapshot/_all command to obtain automatic snapshots. For more information about how to log on to the Kibana console, see Log on to the Kibana console.

When I restore data from snapshots, the destination Elasticsearch cluster displays a message. This message indicates that shards are abnormal. After I run the POST /_cluster/reroute?retry_failed=true command to reroute the shards, the issue persists. What do I do?

Data restoration issue
Delete the problematic index and call the _restore operation to restore it. You must add the max_restore_bytes_per_sec parameter to the request. This parameter is used to limit the restoration rate. The default value of this parameter is 40mb. This value indicates that the index is restored at a speed of 40 MB per second.
POST /_snapshot/aliyun_snapshot_from_instanceId/es-cn-instanceId_datetime/_restore
{
    "indices": "myIndex",
    "settings": {
    "max_restore_bytes_per_sec" : "150mb" 
    }
}
Note You can also add the following parameters:
  • compress: specifies whether to enable data compression. Default value: true.
  • max_snapshot_bytes_per_sec: specifies the snapshot creation rate of each node. Default value: 40mb.

Can I export data from an Elasticsearch cluster to my on-premises machine?

Yes, Elasticsearch provides the snapshot feature. For more information, see View the snapshot feature. You can create and store snapshots in OSS and download objects from OSS. For more information, see Download objects.

How do I use the email notification feature of X-Pack Watcher?

You can configure specific actions for X-Pack Watcher. For more information, see Watcher settings in Elasticsearch.
Notice X-Pack Watcher of Elasticsearch cannot directly access the Internet. You must use the internal endpoint of an Elasticsearch cluster to access the Internet. Therefore, you must create an ECS instance that can access both the Internet and the Elasticsearch cluster. Then, use the ECS instance as a proxy to perform actions. For more information, see Configure X-Pack Watcher.

What do I do if the system reports an alert indicating that memory cannot be allocated to the garbage collector?

Possible causes include heavy loads, high query QPS, or large amounts of data to write. Troubleshoot the issue based on the following instructions:

How do I use a client to access an Alibaba Cloud Elasticsearch cluster? What is the difference between access to an Alibaba Cloud Elasticsearch cluster and access to an open source Elasticsearch cluster?

Access an Alibaba Cloud Elasticsearch cluster by using its internal or public endpoint. Access an open source Elasticsearch cluster by using its address. For more information, see Use a client to access an Alibaba Cloud Elasticsearch cluster.

When I use a client to access an Elasticsearch cluster, can I disable the basic authentication feature?

No, the basic authentication feature is a Kibana authentication mechanism provided by the built-in Elasticsearch plug-in X-Pack. Therefore, you cannot disable the feature.

I purchase an ECS instance that resides in the same VPC as but different zone from an Elasticsearch cluster. Can I use the ECS instance to access the Elasticsearch cluster from an internal network?

Yes, you can use an ECS instance to access an Elasticsearch cluster from an internal network if they reside in the same VPC.

How do I access an Elasticsearch cluster from the Internet?

You can access the cluster from the Internet by using its public endpoint and configuring a public IP address whitelist. For more information, see Configure a whitelist to access an Elasticsearch cluster over the Internet or a VPC. When you access the cluster, you must configure parameters, such as the domain name, username, and password. For more information, see Use a client to access an Alibaba Cloud Elasticsearch cluster.