Elasticsearch:Java API Client (8.x)

Background information

Elasticsearch whose version is earlier than 7.17 uses the Java REST Client. From Elasticsearch 7.17, the Java REST Client is deprecated, and the Java API Client is recommended.

Introduction to the Java API Client

The Java API Client is a Java client library used to establish communication with the Elasticsearch server. The Java API Client can help developers establish communication between an Elasticsearch client and the Elasticsearch server and easily develop and maintain code.

The Java API Client consists of the following components:

1. ElasticsearchClient class: The ElasticsearchClient class is the core class of the Java API Client. The ElasticsearchClient class provides methods that can be used to establish communication with the Elasticsearch server. The ElasticsearchClient class encapsulates the Transport protocol and provides methods that are used for making synchronous and asynchronous calls and streaming and function calls.

2. JSON ObjectMapper: It is a library used for serialization and deserialization. JSON ObjectMapper is seamlessly integrated with the Jackson library and can serialize Java objects into JSON objects.

3. Common capabilities: The Java API Client provides common capabilities, such as the connection pool, retry, and JSON object-related serialization. The Java API Client enhances code readability and maintainability and facilitates code development operations of developers.

Make preparations

• Install a Java Development Kit (JDK). The JDK version must be 1.8 or later. For more information, see Install a JDK.

• Create an Alibaba Cloud Elasticsearch cluster whose version is later than or equal to the version of the Java API Client. In this example, an Alibaba Cloud Elasticsearch V8.X cluster is created. For more information, see Create an Alibaba Cloud Elasticsearch cluster.

  Note

  To ensure that you can use the features of the Java API Client, we recommend that you use the Java API Client whose version is the same as the version of your Elasticsearch cluster.

• Enable the Auto Indexing feature for the Elasticsearch cluster. For more information, see Configure the YML file. If the Auto Indexing feature is not enabled, the error shown in the following figure is reported.

• Configure an IP address whitelist for the Elasticsearch cluster to ensure normal communication among networks.

  • If the server that runs Java code is located in an Internet environment, you can access the cluster by using its public endpoint. Before you access the cluster, you must enable the Public Network Access feature for the cluster and add the public IP address of the server to a public IP address whitelist of the cluster. For more information, see Configure a public or private IP address whitelist for an Elasticsearch cluster.

    Important

    • If your client is in a home network or in a LAN of an office, you must add the IP address of the Internet egress to the whitelist rather than the private IP address of the client.

    • You can also add 0.0.0.0/0 to the whitelist to allow requests from all IPv4 addresses. If you make this configuration, all public IP addresses can be used to access the cluster. This poses security risks. We recommend that you evaluate the risks before you make this configuration.

    • If no IP address whitelist is configured or the IP address whitelist is incorrectly configured, the system reports the "Timeout connecting" error message to indicate a connection timeout error.

    • If you want to access the Kibana node in your cluster from a client, you must configure an IP address whitelist for Kibana. For more information, see Configure a public or private IP address whitelist for Kibana.

  • If the server that runs Java code is located in the same virtual private cloud (VPC) as the Elasticsearch cluster, you can access the cluster by using its internal endpoint. Before you access the Elasticsearch cluster, make sure that the private IP address of the server is added to a private IP address whitelist of the cluster. By default, 0.0.0.0/0 is added to the whitelist.

Configurations for establishing communication between an Elasticsearch client and the Elasticsearch server

Elasticsearch initializes the following types of clients:

• Low-level client

      // Create the low-level client
      RestClient restClient = RestClient.builder(
          new HttpHost("localhost", 9200,"http")).build();

• Transport client that uses the JacksonJsonpMapper class to parse data

      // Create the transport with a Jackson mapper
      ElasticsearchTransport transport = new RestClientTransport(
          restClient, new JacksonJsonpMapper());

• Blocked Java client

      // And create the API client
      ElasticsearchClient elasticsearchClient = new ElasticsearchClient(transport);
      System.out.println("elasticsearchClient = " + elasticsearchClient);

POM dependencies

When you use the following Project Object Model (POM) dependencies, you must replace 8.x with the version of the Java API Client that you use.

<dependency>
    <groupId>co.elastic.clients</groupId>
    <artifactId>elasticsearch-java</artifactId>
    <version>8.x</version>
</dependency>
<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
    <version>2.12.3</version>
</dependency>
<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-core</artifactId>
    <version>2.20.0</version>
</dependency>
<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-api</artifactId>
    <version>2.20.0</version>
</dependency>

Example

In the following sample code, replace the parameters enclosed in {} with your actual values.

import co.elastic.clients.elasticsearch.ElasticsearchClient;
import co.elastic.clients.elasticsearch.indices.*;
import co.elastic.clients.json.jackson.JacksonJsonpMapper;
import co.elastic.clients.transport.ElasticsearchTransport;
import co.elastic.clients.transport.rest_client.RestClientTransport;
import org.apache.http.HttpHost;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;
import org.elasticsearch.client.*;


import java.io.IOException;

public class RestClientTest {
    public static void main(String[] args) {

        // Basic authentication is required for Alibaba Cloud Elasticsearch clusters.
        final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();

        // The username and password are the ones you set when you create the Alibaba Cloud Elasticsearch instance. They are also used to log on to the Kibana console.
        credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials("{Username}", "{Password}"));
        // Create a REST client using the builder and configure the HttpClientConfigCallback of the HTTP client.
        // Click the ID of the created Elasticsearch instance. On the Basic Information page, obtain the public endpoint. This is the ES cluster endpoint.
        RestClient restClient = RestClient.builder(new HttpHost("{ES cluster endpoint}", 9200, "http"))
                .setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
                    @Override
                    public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
                        return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
                    }
                }).build();

        // Create the transport using a Jackson mapper.
        ElasticsearchTransport transport = new RestClientTransport(restClient, new JacksonJsonpMapper());

        // Create the API client.
        ElasticsearchClient elasticsearchClient = new ElasticsearchClient(transport);
        //System.out.println("elasticsearchClient = " + elasticsearchClient);


        // Create and delete an index, and view information about all indexes.
        // Create an alias named "foo" and set it as the write index. This means all write operations, such as indexing, updating, and deleting documents, are automatically routed to the index that corresponds to this alias.
        try {
            // Create an index.
            CreateIndexResponse indexRequest = elasticsearchClient.indices().create(createIndexBuilder -> createIndexBuilder
                    .index("{index_name}")
                    .aliases("{foo}", aliasBuilder -> aliasBuilder
                            .isWriteIndex(true)
                    )
                );
            // Check whether the operation in the "indexRequest" request is acknowledged by the Elasticsearch cluster.
            boolean acknowledged = indexRequest.acknowledged();
            System.out.println("Index document successfully! " + acknowledged);

            // Delete the index.
            DeleteIndexResponse deleteResponse = elasticsearchClient.indices().delete(createIndexBuilder -> createIndexBuilder
                    .index("{index_name}")
            );
            System.out.println("Delete document successfully! \n" + deleteResponse.toString());

            // View information about all indexes (health, status, index, uuid, pri, and rep).
            IndicesResponse indicesResponse = elasticsearchClient.cat().indices();
            indicesResponse.valueBody().forEach(info -> System.out.println(info.health() + "\t"+  info.status() + "\t" + info.index() + "\t" + info.uuid() +"\t" + info.pri() + "\t" + info.rep()));


            transport.close();
            restClient.close();
        } catch (IOException ioException) {
            // Handle exceptions.
        }
    }
}

In a high-concurrency scenario, you must add the configurations that are related to the number of client connections. The following code provides an example:

httpClientBuilder.setMaxConnTotal(500); 
httpClientBuilder.setMaxConnPerRoute(300);

Sample code:

String host = "localhost";
int port = 9200;
String username = "elastic";
String password = "passwd";
final int max_conn_total = 500;
final int max_conn_per_route = 300;

    RestClient restClient = RestClient.builder(new HttpHost("localhost", 9200, "http"))
                .setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
                    @Override
                    public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
                        httpClientBuilder.setMaxConnTotal(max_conn_total);
                        httpClientBuilder.setMaxConnPerRoute(max_conn_per_route);
                        return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
                    }
                }).build();

For more information about the features of the Java API Client, see the official Java API Client documentation.