Elasticsearch:Enable cross-cluster search

Step 1: Configure network interconnection

1. Log on to the Alibaba Cloud Elasticsearch console.

2. In the left navigation menu, choose Elasticsearch Clusters.

3. Navigate to the target cluster.

   1. In the top navigation bar, select the resource group to which the cluster belongs and the region where the cluster resides.

   2. On the Elasticsearch Clusters page, find the cluster and click its ID.

4. In the navigation pane on the left, choose Configuration and Management > Security.

5. In the Network Settings section, next to Cluster Interconnection, click Modify.

6. On the Modify Configuration panel, click Add Cluster.

7. In the Add Cluster dialog box, select the remote cluster to connect.

   Important

   • A Resource Access Management (RAM) user requires ListInstance permission to retrieve all instances in the corresponding Alibaba Cloud account. See Grant permissions to an Elasticsearch instance.

   • This network connection is bi-directional. If you configure a connection from cluster A to cluster B, cluster B will also show a connection to cluster A on its Cluster Interconnection page.

8. Click OK.

   After adding an instance, you can view the list of connected instances on the Modify Configuration page.

   Note

   You can remove an interconnected instance by clicking Remove on this page if the connection is no longer needed.

Step 2: Configure cross-cluster search

After network connectivity is established, you must configure cross-cluster search within Elasticsearch itself to enable data querying.

1. (Optional) Prepare test data.

   If your remote Elasticsearch instance already contains data for testing, skip this step.

   1. Log on to the Kibana console of the remote Elasticsearch cluster and go to the Kibana homepage.

      For instructions, see Log on to the Kibana console.

      Note

      Examples here use Elasticsearch V6.7.0. Operations may vary slightly for other versions.

   2. In the left navigation pane, click Dev Tools.

   3. In the Console, run the following commands to create an index and insert a document:

      • Create an index

        PUT /twitter
        {
            "settings" : {
                "index" : {
                    "number_of_shards" : 3, 
                    "number_of_replicas" : 2 
                }
            }
        }

      • Create a document and insert data

        POST twitter/_doc/
        {
            "user" : "kimchy",
            "post_date" : "2009-11-15T14:12:12",
            "message" : "trying out Elasticsearch"
        }

2. Configure cross-cluster search in your primary cluster.

   1. Log on to the Kibana console of your primary Elasticsearch cluster.

   2. In Dev Tools, execute the _cluster/settings command to define the remote cluster. Replace cluster_one with a descriptive name for your remote cluster, and 10.8.xx.xx:9300 with the actual IP addresses of the remote cluster's nodes (can be found n the Node Visualization section on the Basic Information page of the cluster).

      PUT _cluster/settings
      {
        "persistent": {
          "cluster": {
            "remote": {
              "cluster_one": {
                "seeds": [
                  "10.8.xx.xx:9300",
                  "10.8.xx.xx:9300",
                  "10.8.xx.xx:9300"
                ]
              }
            }
          }
        }
      }

      Note

      • Configuring cross-cluster search using domain names is not supported. You must use node IP addresses.

      • This configuration enables searching from the primary cluster to the remote cluster. To allow the remote cluster to search back to the primary, you must perform a similar _cluster/settingsconfiguration on the remote instance, pointing to the primary cluster's nodes.

      • For more information on version-specific configurations, refer to the official Elastic documentation: see Cross-cluster search for Elasticsearch 7.x, Cross-cluster search for Elasticsearch 6.3, and Cross-cluster search for Elasticsearch 5.5.

   3. Verify cross-cluster search:

      POST /cluster_one:twitter/_doc/_search
      {
        "query": {
          "match_all": {}
        }
      }

      Expected successful response:

      {
        "took" : 78,
        "timed_out" : false,
        "_shards" : {
          "total" : 3,
          "successful" : 3,
          "skipped" : 0,
          "failed" : 0
        },
        "_clusters" : {
          "total" : 1,
          "successful" : 1,
          "skipped" : 0
        },
        "hits" : {
          "total" : 1,
          "max_score" : 1.0,
          "hits" : [
            {
              "_index" : "cluster_one:twitter",
              "_type" : "_doc",
              "_id" : "qudxxxxxxxxxx_7ie6J",
              "_score" : 1.0,
              "_source" : {
                "user" : "kimchy",
                "post_date" : "2009-11-15T14:12:12",
                "message" : "trying out Elasticsearch"
              }
            }
          ]
        }
      }