Connect Elasticsearch clusters

Precautions

Due to the adjustment made to the Alibaba Cloud Elasticsearch network architecture, clusters created after October 2020 do not support some features. These features include X-Pack Watcher, LDAP authentication, cross-cluster reindexing, cross-cluster searches, and cluster interconnection. The features will be available soon.

Prerequisites

Connect Elasticsearch clusters

1. Log on to the Alibaba Cloud Elasticsearch console.
2. In the left-side navigation pane, click Elasticsearch Clusters.
3. In the top navigation bar, select a resource group and a region. On the Clusters page, click the ID of the desired cluster.
4. In the left-side navigation pane of the Basic Information page, click Security.
5. On the page that appears, click Edit next to Cluster Interconnection.
6. In the Edit Configuration panel, click Add Cluster.
7. In the Add Cluster dialog box, select the IDs of the remote Elasticsearch clusters that you want to connect.
   

   Notice

   • The RAM users of an Alibaba Cloud account can query all Elasticsearch clusters that belong to the account only after the ListInstance permission is granted to the RAM users. For more information, see Types of resources that can be authorized.
   • After you connect the current Elasticsearch cluster to a remote cluster, you can view the ID of the current cluster in the Edit Configuration panel of the remote cluster. This indicates that the communication between the two clusters is bidirectional.
8. Click OK.
   After you add a cluster, you can find the cluster in the Connected Clusters section of the Edit Configuration panel.

   Note If you no longer require the added cluster, click Remove in the Actions column to remove the cluster.

After you complete the configuration, you must also perform the operations described in Configure the cross-cluster search feature. This way, you can search for data in the remote Elasticsearch cluster from the current Elasticsearch cluster.

Configure the cross-cluster search feature

1. Log on to the Kibana console of the remote Elasticsearch cluster.
   For more information, see Log on to the Kibana console.
2. Create an index, add a document to the index, and insert data into the document in the remote Elasticsearch cluster.
   • Create an index

     PUT /twitter
     {
         "settings" : {
             "index" : {
                 "number_of_shards" : 3, 
                 "number_of_replicas" : 2 
             }
         }
     }

   • Create a document and insert data into the document

     POST twitter/_doc/
     {
         "user" : "kimchy",
         "post_date" : "2009-11-15T14:12:12",
         "message" : "trying out Elasticsearch"
     }

   Note The index and document are used to test the cross-cluster search feature.
3. Log on to the Kibana console of the current Elasticsearch cluster.
4. Use one of the following methods to configure the cross-cluster search feature in the current Elasticsearch cluster.
   In the following methods, an Elasticsearch V6.7.0 cluster is used. The methods that are used to configure the cross-cluster search feature in other Elasticsearch versions are similar. For more information, see Cross-cluster search in Elasticsearch V7.X, Cross-cluster search in Elasticsearch V6.3, and Cross-cluster search in Elasticsearch V5.5.

   • Method 1: Use the internal endpoint of the remote Elasticsearch cluster

     PUT _cluster/settings
     {
       "persistent": {
         "cluster": {
           "remote": {
             "cluster_one": {
               "seeds": [
                 "es-cn-o4xxxxxxxxxxxx4f1.elasticsearch.aliyuncs.com:9300"
               ]
             }
           }
         }
       }
     }

   • Method 2: Use the IP addresses of the nodes in the remote Elasticsearch cluster

     PUT _cluster/settings
     {
       "persistent": {
         "cluster": {
           "remote": {
             "cluster_one": {
               "seeds": [
                 "10.8.xx.xx:9300",
                 "10.8.xx.xx:9300",
                 "10.8.xx.xx:9300"
               ]
             }
           }
         }
       }
     }

   Notice

   • If the current Elasticsearch cluster is a single-zone cluster, you can use Method 1 or Method 2. If the current Elasticsearch cluster is a multi-zone cluster, you can use only Method 2. Both methods can be used to connect multiple remote Elasticsearch clusters to the current Elasticsearch cluster.
   • You can query the index data in the remote Elasticsearch cluster from the current Elasticsearch cluster. This applies only if you have configured the domain name or node IP addresses of the remote Elasticsearch cluster in the current Elasticsearch cluster for cross-cluster searches. You cannot run similar commands in the remote Elasticsearch cluster to search for data in the current Elasticsearch cluster. To search for data in the current Elasticsearch cluster from the remote Elasticsearch cluster, you must add the domain name or node IP addresses of the current cluster to the remote cluster.
5. Run the following command to check whether the cross-cluster search feature is correctly configured:

   POST /cluster_one:twitter/doc/_search
   {
     "query": {
       "match_all": {}
     }
   }

   If the cross-cluster search feature is correctly configured, the following result is returned:

   {
     "took" : 78,
     "timed_out" : false,
     "_shards" : {
       "total" : 3,
       "successful" : 3,
       "skipped" : 0,
       "failed" : 0
     },
     "_clusters" : {
       "total" : 1,
       "successful" : 1,
       "skipped" : 0
     },
     "hits" : {
       "total" : 1,
       "max_score" : 1.0,
       "hits" : [
         {
           "_index" : "cluster_one:twitter",
           "_type" : "doc",
           "_id" : "qudxxxxxxxxxx_7ie6J",
           "_score" : 1.0,
           "_source" : {
             "user" : "kimchy",
             "post_date" : "2009-11-15T14:12:12",
             "message" : "trying out Elasticsearch"
           }
         }
       ]
     }
   }