All Products
Search
Document Center

Elasticsearch:Use a client to access an Alibaba Cloud Elasticsearch cluster

Last Updated:Feb 13, 2023

This topic describes how to use a PHP, Python, Java, or Go client to access an Alibaba Cloud Elasticsearch cluster. This topic also provides sample code and usage notes of each type of client.

Preparations

  • Install a JDK. The JDK version must be 1.8 or later.

    For more information, see Install a JDK.

  • Create an Alibaba Cloud Elasticsearch cluster.

    For more information, see Create an Alibaba Cloud Elasticsearch cluster.

    Note

    To ensure compatibility, we recommend that you create an Alibaba Cloud Elasticsearch cluster of the same version as the client. For more information about the compatibility between Elasticsearch and clients, see Compatibility.

  • Enable the Auto Indexing feature for the Elasticsearch cluster.

    For more information, see Configure the YML file.

  • Configure a whitelist for the cluster to ensure normal communication among networks.

    • If the server that runs code is located in an Internet environment, you can access the cluster by using its public endpoint. Before you access the cluster, you must enable the Public Network Access feature for the cluster and add the public IP address of the server to a public IP address whitelist of the cluster. For more information, see Configure a public or private IP address whitelist for an Elasticsearch cluster.

      Important
      • If you are using a public network, add the IP address of the jump server that controls outbound traffic of the public network to the whitelist.

      • You can also add 0.0.0.0/0 to the whitelist to allow requests from all IPv4 addresses. If you make this configuration, all public IP addresses can be used to access the cluster. This poses security risks. We recommend that you evaluate the risks before you make this configuration.

      • If no whitelist is configured or the whitelist is incorrectly configured, the system reports the "Timeout connecting" error message to indicate a connection timeout error.

      • If you want to access the Kibana node in your cluster from a client, you must configure a whitelist for the Kibana service. For more information, see Configure a public or private IP address whitelist for Kibana.

    • If the server that runs code is located in the same virtual private cloud (VPC) as the cluster, you can access the cluster by using its internal endpoint. Before you access the cluster, make sure that the private IP address of the server is added to a private IP address whitelist of the cluster. By default, 0.0.0.0/0 is added to the whitelist.

Precautions

  • The default connection pool provided by the PHP client of open source Elasticsearch is not suitable for accessing services on the cloud. Alibaba Cloud Elasticsearch handles requests sent from clients based on load balancing. Therefore, a PHP client must use SimpleConnectionPool as the connection pool. If SimpleConnectionPool is not used as the connection pool, a connection error occurs when your Alibaba Cloud Elasticsearch cluster is restarted. A PHP client must be configured to reconnect to an Elasticsearch cluster after the client is disconnected. If the PHP client cannot reconnect to the cluster, an error may occur (for example, the "No enabled connection" error message is reported) during a restart of the cluster even if SimpleConnectionPool is used as the connection pool.

  • Java clients include Transport Client, Low Level REST Client, and High Level REST Client. For more information about the sample code for each type of client, see Overview. In this example, High Level REST Client 6.7 is used.

  • Java Transport Client communicates with an Elasticsearch cluster over TCP. If Java Transport Client communicates with an Elasticsearch cluster of a version that does not match the version of Java Transport Client, incompatibility issues may occur. In later versions of open source Elasticsearch, Transport Client is deprecated. If you use Transport Client 5.5 to access an Elasticsearch V5.5 cluster or use Transport Client 5.6 to access an Elasticsearch V5.6 cluster, the system displays the "NoNodeAvailableException" error message. To ensure version compatibility, we recommend that you use Transport Client 5.3.3 or Java Low Level REST Client to access an Elasticsearch cluster.

  • Before you use a Go client to access an Elasticsearch cluster, you must install a compilation environment for Go. For more information, see The Go Programming Language. In this example, a Go 1.19.1 client is used.

Sample code

This section provides sample code for each type of client that you can use to access an Alibaba Cloud Elasticsearch cluster.

// In this example, a Go 1.19.1 client is used. 
package main

import (
  "log"
  "github.com/elastic/go-elasticsearch/v7"
)

func main() {
  cfg := elasticsearch.Config {
    Addresses: []string{
      "<YourEsHost>",
    },
    Username: "<UserName>",
    Password: "<YourPassword>",
  }

  es, err := elasticsearch.NewClient(cfg)
  if err != nil {
    log.Fatalf("Error creating the client: %s", err)
  }

  res, err := es.Info()
  if err != nil {
    log.Fatalf("Error getting response: %s", err)
  }

  defer res.Body.Close()
  log.Println(res)
}
// In this example, High Level REST Client 6.7 is used. 
import org.apache.http.HttpHost;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;

import org.elasticsearch.action.delete.DeleteRequest;
import org.elasticsearch.action.delete.DeleteResponse;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.action.index.IndexResponse;
import org.elasticsearch.client.*;

import java.io.IOException;
import java.util.HashMap;
import java.util.Map;

public class RestClientTest67 {

    private static final RequestOptions COMMON_OPTIONS;

    static {
        RequestOptions.Builder builder = RequestOptions.DEFAULT.toBuilder();

        // The default cache size is 100 MiB. Change it to 30 MiB. 
        builder.setHttpAsyncResponseConsumerFactory(
                new HttpAsyncResponseConsumerFactory
                        .HeapBufferedResponseConsumerFactory(30 * 1024 * 1024));
        COMMON_OPTIONS = builder.build();
    }

    public static void main(String[] args) {
        // Use basic access authentication for the Elasticsearch cluster. 
        final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
       // Use the username and password that are specified when you create the Elasticsearch cluster. You can also use the username and password to log on to the Kibana console. 
        credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials("<UserName>", "<YourPassword>"));

        // Create a Java REST client by using the builder and configure HttpClientConfigCallback for the HTTP client. 
       // Specify the public endpoint of the Elasticsearch cluster. You can obtain the endpoint from the Basic Information page of the cluster. 
        RestClientBuilder builder = RestClient.builder(new HttpHost("<YourEsHost>", 9200, "http"))
                .setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
                    @Override
                    public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
                        return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
                    }
                });

        // Use the REST low-level client builder to create a RestHighLevelClient instance. 
        RestHighLevelClient highClient = new RestHighLevelClient(builder);

        try {
            // Create a request. 
            Map<String, Object> jsonMap = new HashMap<>();
           jsonMap.put("<YourEsField1>", "<YourEsFieldValue1>");
           jsonMap.put("<YourEsField2>", "<YourEsFieldValue2>");
           IndexRequest indexRequest = new IndexRequest("<YourEsIndex>", "<YourEsType>", "<YourEsId>").source(jsonMap);

            // Run the following code in parallel and use the custom configuration of RequestOptions: 
            IndexResponse indexResponse = highClient.index(indexRequest, COMMON_OPTIONS);

            long version = indexResponse.getVersion();

            System.out.println("Index document successfully! " + version);

            highClient.close();

        } catch (IOException ioException) {
            // Handle exceptions. 
        }
    }
}
<?php
require 'vendor/autoload.php';
use Elasticsearch\ClientBuilder;

$client = ClientBuilder::create()->setHosts([
  [
    'host'   => '<YourEsHost>',
    'port'   => '9200',
    'scheme' => 'http',
    'user'   => '<UserName>',
    'pass'   => '<YourPassword>'
  ]
])->setConnectionPool('\Elasticsearch\ConnectionPool\SimpleConnectionPool', [])
  ->setRetries(10)->build();

$indexParams = [
  'index'  => '<YourEsIndex>',
  'type'   => '<YourEsType>',
  'id'     => '<YourEsId>',
  'body'   => ['<YourEsField>' => '<YourEsFieldValue>'],
  'client' => [
    'timeout'         => 10,
    'connect_timeout' => 10
  ]
];
$indexResponse = $client->index($indexParams);
print_r($indexResponse);

$searchParams = [
  'index'  => '<YourEsIndex>',
  'type'   => '<YourEsType>',
  'body'   => [
    'query' => [
      'match' => [
        '<YourEsField>' => '<YourEsFieldValue>'
      ]
    ]
  ],
  'client' => [
    'timeout'         => 10,
    'connect_timeout' => 10
  ]
];
$searchResponse = $client->search($searchParams);
print_r($searchResponse);
?>
from elasticsearch import Elasticsearch, RequestsHttpConnection
import certifi
es = Elasticsearch(
    ['<YourEsHost>'],
    http_auth=('<UserName>', '<YourPassword>'),
    port=9200,
    use_ssl=False
)
res = es.index(index="<YourEsIndex>", doc_type="<YourEsType>", id=<YourEsId>, body={"<YourEsField1>": "<YourEsFieldValue1>", "<YourEsField2>": "<YourEsFieldValue2>"})
res = es.get(index="<YourEsIndex>", doc_type="<YourEsType>", id=<YourEsId>)
print(res['_source'])

When you use the preceding code, you must replace the parameters listed in the following table with your actual values.

Parameter

Description

<YourEsHost>

The internal or public endpoint of your Alibaba Cloud Elasticsearch cluster. You can obtain the endpoint from the Basic Information page of your cluster. For more information, see View the basic information of a cluster.

<UserName>

The username that is used to access your Alibaba Cloud Elasticsearch cluster. The default username is elastic.

<YourPassword>

The password that is used to access your Alibaba Cloud Elasticsearch cluster. The password is specified when you create the cluster. If you forget the password, you can reset it. For more information about the procedure and precautions for resetting the password, see Reset the access password for an Elasticsearch cluster.

<YourEsIndex>

The name of the index in your Alibaba Cloud Elasticsearch cluster.

<YourEsType>

The type of documents in the index.

Important

In versions earlier than Elasticsearch V7.0, you can customize a type for your documents. In Elasticsearch V7.0 and later, the type of documents can only be _doc.

<YourEsId>

The ID of a document in the index.

<YourEsField>

The name of a field in the document.

<YourEsFieldValue>

The value of the field.

References