Elasticsearch:Use the Elasticsearch Java API Client (8.x)

Prerequisites

Before you begin, make sure you have:

• JDK 1.8 or later installed. See Install a JDK.

• An Alibaba Cloud Elasticsearch cluster at version 8.x. See Create an Alibaba Cloud Elasticsearch cluster.

  Use a Java API Client version that matches your cluster version. For example, for a cluster at 8.17.0, set the client version to 8.17.0.

• Auto Indexing enabled on the cluster. See Configure the YML file. Without Auto Indexing, document indexing requests return an error.

• Network access configured:

  • Internet access: Enable Public Network Access on the cluster, then add your server's public IP address to the cluster's public IP address whitelist. If your server is behind a NAT (home network or office LAN), add the Internet egress IP, not the private IP of the machine. See Configure a public or private IP address whitelist for an Elasticsearch cluster.

  • VPC access: Use the cluster's internal endpoint. By default, 0.0.0.0/0 is already in the private IP address whitelist.

Add Maven dependencies

Add the following dependencies to your pom.xml. Replace 8.x with your actual client version (for example, 8.17.0 for a cluster at version 8.17.0).

<dependency>
    <groupId>co.elastic.clients</groupId>
    <artifactId>elasticsearch-java</artifactId>
    <version>8.x</version>
</dependency>
<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
    <version>2.12.3</version>
</dependency>
<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-core</artifactId>
    <version>2.20.0</version>
</dependency>
<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-api</artifactId>
    <version>2.20.0</version>
</dependency>

Initialize the client

The Java API Client is built from three layers:

1. A low-level RestClient that handles HTTP connections.

2. A RestClientTransport that adds JSON serialization using JacksonJsonpMapper.

3. An ElasticsearchClient that provides the typed API.

RestClient restClient = RestClient.builder(
    new HttpHost("localhost", 9200, "http")).build();                      // <1>

ElasticsearchTransport transport = new RestClientTransport(
    restClient, new JacksonJsonpMapper());                                  // <2>

ElasticsearchClient elasticsearchClient = new ElasticsearchClient(transport); // <3>

1. Create the low-level HTTP client.

2. Wrap it in a transport layer with Jackson serialization.

3. Create the typed API client.

Alibaba Cloud Elasticsearch clusters require basic authentication. Pass credentials through HttpClientConfigCallback when building the RestClient:

final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(AuthScope.ANY,
    new UsernamePasswordCredentials("<username>", "<password>")); // <1> Don't hardcode credentials in production.

RestClient restClient = RestClient.builder(
    new HttpHost("<ES cluster endpoint>", 9200, "http"))          // <2>
    .setHttpClientConfigCallback(httpClientBuilder ->
        httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider))
    .build();

1. The username and password are the credentials you set when creating the cluster. They are also used to log in to Kibana. Use environment variables or a secrets manager instead of hardcoding credentials.

2. Get the public or internal endpoint from the Basic Information page of your cluster in the Alibaba Cloud console.

Complete example

The following example connects to an Alibaba Cloud Elasticsearch cluster with basic authentication, creates an index with a write alias, verifies the result, deletes the index, and lists all indexes.

Replace <username>, <password>, <ES cluster endpoint>, <index_name>, and <foo> with your actual values.

import co.elastic.clients.elasticsearch.ElasticsearchClient;
import co.elastic.clients.elasticsearch.indices.*;
import co.elastic.clients.json.jackson.JacksonJsonpMapper;
import co.elastic.clients.transport.ElasticsearchTransport;
import co.elastic.clients.transport.rest_client.RestClientTransport;
import org.apache.http.HttpHost;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;
import org.elasticsearch.client.*;

import java.io.IOException;

public class RestClientTest {
    public static void main(String[] args) {

        final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
        // Don't hardcode credentials in production — use environment variables or a secrets manager.
        credentialsProvider.setCredentials(AuthScope.ANY,
            new UsernamePasswordCredentials("<username>", "<password>"));

        RestClient restClient = RestClient
            .builder(new HttpHost("<ES cluster endpoint>", 9200, "http"))
            .setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
                @Override
                public HttpAsyncClientBuilder customizeHttpClient(
                        HttpAsyncClientBuilder httpClientBuilder) {
                    return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
                }
            }).build();

        ElasticsearchTransport transport = new RestClientTransport(
            restClient, new JacksonJsonpMapper());

        ElasticsearchClient elasticsearchClient = new ElasticsearchClient(transport);

        try {
            // Create an index with a write alias.
            // Setting isWriteIndex(true) routes all write operations (index, update, delete)
            // to this index through the alias.
            CreateIndexResponse indexRequest = elasticsearchClient.indices().create(
                b -> b.index("<index_name>")
                      .aliases("<foo>", a -> a.isWriteIndex(true))
            );
            boolean acknowledged = indexRequest.acknowledged();
            System.out.println("Index created: " + acknowledged);

            // Delete the index.
            DeleteIndexResponse deleteResponse = elasticsearchClient.indices().delete(
                b -> b.index("<index_name>")
            );
            System.out.println("Index deleted: " + deleteResponse.toString());

            // List all indexes (health, status, index name, UUID, primary shards, replicas).
            IndicesResponse indicesResponse = elasticsearchClient.cat().indices();
            indicesResponse.valueBody().forEach(info ->
                System.out.println(
                    info.health() + "\t" + info.status() + "\t" + info.index() + "\t" +
                    info.uuid() + "\t" + info.pri() + "\t" + info.rep()
                )
            );

            transport.close();
            restClient.close();
        } catch (IOException e) {
            // Handle connection and API errors.
        }
    }
}

Tune for high concurrency

For high-concurrency workloads, increase the maximum number of HTTP connections on the RestClient. The defaults are too low for production traffic.

httpClientBuilder.setMaxConnTotal(500);      // total connections across all routes
httpClientBuilder.setMaxConnPerRoute(300);   // connections to a single ES endpoint

Full example with connection pool settings:

final int maxConnTotal = 500;
final int maxConnPerRoute = 300;

RestClient restClient = RestClient
    .builder(new HttpHost("<ES cluster endpoint>", 9200, "http"))
    .setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
        @Override
        public HttpAsyncClientBuilder customizeHttpClient(
                HttpAsyncClientBuilder httpClientBuilder) {
            httpClientBuilder.setMaxConnTotal(maxConnTotal);
            httpClientBuilder.setMaxConnPerRoute(maxConnPerRoute);
            return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
        }
    }).build();

Troubleshooting

What's next

• Explore the full Java API Client feature set in the official Java API Client documentation.

• Learn how to manage index mappings and perform search queries using the typed builder API.