All Products
Search
Document Center

Elasticsearch:Use a client to access an Alibaba Cloud Elasticsearch cluster

Last Updated:Feb 27, 2024

This topic describes how to use a PHP, Python, Java, or Go client to access an Alibaba Cloud Elasticsearch cluster. This topic also provides sample code and usage notes for each type of client.

Prerequisites

  • Create an Alibaba Cloud Elasticsearch cluster. For more information, see Create an Alibaba Cloud Elasticsearch cluster.

  • Install an Elasticsearch client that uses the required programming language.

    To prevent incompatibility issues, we recommend that you install a client whose version is the same as the version of the created Elasticsearch cluster. For more information about the version compatibility between an Elasticsearch cluster and clients, see Compatibility.

    • For information about an Elasticsearch Go client, see Elasticsearch Go Client.

      Note

      Before you use a Go client to access an Elasticsearch cluster, you must install a compilation environment for Go. For more information, see The Go Programming Language. In this example, Go 1.19.1 is used.

    • For information about an Elasticsearch Java client, see Elasticsearch Java API Client.

      Note
      • Java clients include Transport Client, Low Level REST Client, High Level REST Client, and Java API Client. For more information about the sample code for each type of client, see Overview. In this example, High Level REST Client 6.7 is used.

      • Java Transport Client communicates with an Elasticsearch cluster over TCP. If Java Transport Client communicates with an Elasticsearch cluster of a version that does not match the version of Java Transport Client, incompatibility issues may occur. In later versions of open source Elasticsearch, Transport Client is deprecated. If you use Transport Client 5.5 to access an Elasticsearch V5.5 cluster or use Transport Client 5.6 to access an Elasticsearch V5.6 cluster, the system reports the NoNodeAvailableException error message. To ensure version compatibility, we recommend that you use Transport Client 5.3.3 or Java Low Level REST Client to access an Elasticsearch cluster.

    • For information about an Elasticsearch PHP client, see Elasticsearch PHP Client.

      Note

      The default connection pool provided by the PHP client of open source Elasticsearch is not suitable for accessing services in the cloud. Alibaba Cloud Elasticsearch handles requests sent from clients based on load balancing. Therefore, a PHP client must use SimpleConnectionPool as the connection pool. If SimpleConnectionPool is not used as the connection pool, a connection error occurs when the Alibaba Cloud Elasticsearch cluster is restarted. In addition, a PHP client must be configured to reconnect to an Elasticsearch cluster after the client is disconnected. This is because a connection error may also occur during the restart of the Alibaba Cloud Elasticsearch cluster even if the PHP client uses SimpleConnectionPool as the connection pool. For example, the No enabled connection error message may be reported.

    • For information about an Elasticsearch Python client, see Elasticsearch Python Client.

    • For information about other Elasticsearch clients, see Elasticsearch Clients.

  • Enable the Auto Indexing feature for the Elasticsearch cluster. For more information, see Configure the YML file.

  • Configure the IP address whitelist of the Elasticsearch cluster to ensure network connectivity.

    • If the server that runs code is located in the same virtual private cloud (VPC) as the Elasticsearch cluster, you can access the cluster by using the internal endpoint of the cluster. Before you access the Elasticsearch cluster, make sure that the private IP address of the server is added to the private IP address whitelist of the cluster. By default, 0.0.0.0/0 is added to the IP address whitelist.

    • If the server that runs code is located on the Internet, you can access the Elasticsearch cluster by using the public endpoint of the cluster. You must enable the Public Network Access feature for the Elasticsearch cluster and add the public IP address of the server to the public IP address whitelist of the Elasticsearch cluster. For more information, see Configure a public or private IP address whitelist for an Elasticsearch cluster.

      Important
      • If you use a public network such as Wi-Fi or a broadband network, add the IP address of the jump server that controls outbound traffic of the public network to the IP address whitelist of the Elasticsearch cluster.

      • You can add 0.0.0.0/0 to the IP address whitelist of the Elasticsearch cluster to allow access from all IPv4 addresses to the cluster. If you make this configuration, all public IPv4 addresses can be used to access the Elasticsearch cluster. This poses security risks. We recommend that you evaluate the risks before you make this configuration.

      • If no IP address whitelist is configured or the IP address whitelist is incorrectly configured, the system reports the Timeout connecting error message to indicate a connection timeout error.

      • If you want to access the Kibana node in your cluster from a client, you must configure an IP address whitelist for the Kibana service. For more information, see Configure a public or private IP address whitelist for Kibana.

Sample code

This section provides sample code for each type of client that you can use to access an Alibaba Cloud Elasticsearch cluster.

// In this example, a Go 1.19.1 client is used. 
package main

import (
  "log"
  "github.com/elastic/go-elasticsearch/v7"
)

func main() {
  cfg := elasticsearch.Config {
    Addresses: []string{
      "<YourEsHost>",
    },
    Username: "<UserName>",
    Password: "<YourPassword>",
  }

  es, err := elasticsearch.NewClient(cfg)
  if err != nil {
    log.Fatalf("Error creating the client: %s", err)
  }

  res, err := es.Info()
  if err != nil {
    log.Fatalf("Error getting response: %s", err)
  }

  defer res.Body.Close()
  log.Println(res)
}
// In this example, High Level REST Client 6.7 is used. 
import org.apache.http.HttpHost;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;

import org.elasticsearch.action.delete.DeleteRequest;
import org.elasticsearch.action.delete.DeleteResponse;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.action.index.IndexResponse;
import org.elasticsearch.client.*;

import java.io.IOException;
import java.util.HashMap;
import java.util.Map;

public class RestClientTest67 {

    private static final RequestOptions COMMON_OPTIONS;

    static {
        RequestOptions.Builder builder = RequestOptions.DEFAULT.toBuilder();

        // The default cache size is 100 MiB. Change it to 30 MiB. 
        builder.setHttpAsyncResponseConsumerFactory(
                new HttpAsyncResponseConsumerFactory
                        .HeapBufferedResponseConsumerFactory(30 * 1024 * 1024));
        COMMON_OPTIONS = builder.build();
    }

    public static void main(String[] args) {
        // Use basic authentication for the Elasticsearch cluster. 
        final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
       // Use the username and password that are specified when you create the Elasticsearch cluster. You can also use the username and password to log on to the Kibana console of the Elasticsearch cluster. 
        credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials("<UserName>", "<YourPassword>"));

        // Create a Java REST client by using the builder and configure HttpClientConfigCallback for the HTTP client. 
       // Specify the public endpoint of the Elasticsearch cluster. You can find the Elasticsearch cluster on the Elasticsearch Clusters page in the Elasticsearch console, click the cluster ID, and then obtain the endpoint from the Basic Information page of the Elasticsearch cluster. 
        RestClientBuilder builder = RestClient.builder(new HttpHost("<YourEsHost>", 9200, "http"))
                .setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
                    @Override
                    public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
                        return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
                    }
                });

        // Use the REST low-level client builder to create a RestHighLevelClient instance. 
        RestHighLevelClient highClient = new RestHighLevelClient(builder);

        try {
            // Create a request. 
            Map<String, Object> jsonMap = new HashMap<>();
           jsonMap.put("<YourEsField1>", "<YourEsFieldValue1>");
           jsonMap.put("<YourEsField2>", "<YourEsFieldValue2>");
           IndexRequest indexRequest = new IndexRequest("<YourEsIndex>", "<YourEsType>", "<YourEsId>").source(jsonMap);

            // Implement synchronous execution and use the custom configuration (COMMON_OPTIONS) of RequestOptions: 
            IndexResponse indexResponse = highClient.index(indexRequest, COMMON_OPTIONS);

            long version = indexResponse.getVersion();

            System.out.println("Index document successfully! " + version);

            highClient.close();

        } catch (IOException ioException) {
            // Handle exceptions. 
        }
    }
}
<?php
require 'vendor/autoload.php';
use Elasticsearch\ClientBuilder;

$client = ClientBuilder::create()->setHosts([
  [
    'host'   => '<YourEsHost>',
    'port'   => '9200',
    'scheme' => 'http',
    'user'   => '<UserName>',
    'pass'   => '<YourPassword>'
  ]
])->setConnectionPool('\Elasticsearch\ConnectionPool\SimpleConnectionPool', [])
  ->setRetries(10)->build();

$indexParams = [
  'index'  => '<YourEsIndex>',
  'type'   => '<YourEsType>',
  'id'     => '<YourEsId>',
  'body'   => ['<YourEsField>' => '<YourEsFieldValue>'],
  'client' => [
    'timeout'         => 10,
    'connect_timeout' => 10
  ]
];
$indexResponse = $client->index($indexParams);
print_r($indexResponse);

$searchParams = [
  'index'  => '<YourEsIndex>',
  'type'   => '<YourEsType>',
  'body'   => [
    'query' => [
      'match' => [
        '<YourEsField>' => '<YourEsFieldValue>'
      ]
    ]
  ],
  'client' => [
    'timeout'         => 10,
    'connect_timeout' => 10
  ]
];
$searchResponse = $client->search($searchParams);
print_r($searchResponse);
?>
from elasticsearch import Elasticsearch, RequestsHttpConnection
import certifi
es = Elasticsearch(
    ['<YourEsHost>'],
    http_auth=('<UserName>', '<YourPassword>'),
    port=9200,
    use_ssl=False
)
res = es.index(index="<YourEsIndex>", doc_type="<YourEsType>", id=<YourEsId>, body={"<YourEsField1>": "<YourEsFieldValue1>", "<YourEsField2>": "<YourEsFieldValue2>"})
res = es.get(index="<YourEsIndex>", doc_type="<YourEsType>", id=<YourEsId>)
print(res['_source'])

The preceding code shows only how a client is used to access the Elasticsearch cluster for which HTTP is enabled. If HTTPS is enabled for the Elasticsearch cluster, you must change the value of use_ssl to True and add verify_certs=True, as shown in the following code.

es = Elasticsearch(
['<YourEsHost>'],
http_auth=('<UserName>', '<YourPassword>'),
port=9200,
use_ssl=True,
verify_certs=True
)

When you use the preceding code, you must replace the parameters listed in the following table with the actual values.

Parameter

Description

<YourEsHost>

The public or internal endpoint of the Elasticsearch cluster. You can go to the Basic Information page of the Elasticsearch cluster in the Elasticsearch console to obtain the public or internal endpoint.

<UserName>

The username of the Elasticsearch cluster. The username is elastic.

<YourPassword>

The password of the Elasticsearch cluster.

If you forget the password, you can go to the Security page to reset the password. For more information, see Reset the access password for an Elasticsearch cluster.

<YourEsIndex>

The name of the index in the Elasticsearch cluster.

<YourEsType>

The type of a document.

Important

If you use an Elasticsearch cluster earlier than V7.0, you can specify a document type based on your business requirements. For Elasticsearch clusters V7.0 or later, the document type is fixed as _doc.

<YourEsId>

The ID of a document.

<YourEsField>

The name of a field.

<YourEsFieldValue>

The value of a field that is specified by <YourEsField>.