Connect LindormSearch via Java Low Level REST Client with Auth - Lindorm

The Java Low Level REST Client sends raw HTTP requests without encoding or decoding the request or response body, giving you direct control over request handling. Use it when you need to connect to LindormSearch (compatible with Elasticsearch 7.10 and earlier) and want to manage serialization yourself.

Prerequisites

Before you begin, ensure that you have:

JDK 1.8 or later installed
LindormSearch activated for your Lindorm instance. See Activation guide
Your client IP address added to the Lindorm instance whitelist. See Configure a whitelist

Install the client

Add the following dependencies to the dependencies section of your pom.xml or build file.

Maven:

<dependency>
  <groupId>org.elasticsearch.client</groupId>
  <artifactId>elasticsearch-rest-client</artifactId>
  <version>7.10.0</version>
</dependency>
<dependency>
  <groupId>org.apache.logging.log4j</groupId>
  <artifactId>log4j-core</artifactId>
  <version>2.8.2</version>
</dependency>
<dependency>
  <groupId>org.apache.logging.log4j</groupId>
  <artifactId>log4j-api</artifactId>
  <version>2.7</version>
</dependency>

Gradle:

dependencies {
  implementation 'org.elasticsearch.client:elasticsearch-rest-client:7.10.0'
  implementation 'org.apache.logging.log4j:log4j-core:2.8.2'
  implementation 'org.apache.logging.log4j:log4j-api:2.7'
}

Configure the connection

Build a RestClient with the LindormSearch endpoint and credentials:

String search_url = "ld-t4n5668xk31ui****-proxy-search-public.lindorm.rds.aliyuncs.com";
int search_port = 30070;

// For demo purposes only. Do not hardcode credentials in production code.
String username = "user";
String password = "test";

final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(username, password));

RestClientBuilder restClientBuilder = RestClient.builder(new HttpHost(search_url, search_port));
restClientBuilder.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
  @Override
  public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
    return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
  }
});

Parameters:

Parameter	Description
`search_url`	The LindormSearch endpoint for Elasticsearch. To get the endpoint, see View endpoints. Use the Virtual Private Cloud (VPC) endpoint for applications running on an ECS instance, or the Internet endpoint for local applications connecting over the public network.
`search_port`	The port for the LindormSearch endpoint. Fixed value: `30070`.
`username`	The username for the search engine. To get the default credentials, go to the Lindorm console, click Database Connections in the left navigation pane, and then click the Search Engine tab.
`password`	The password for the search engine. Found in the same location as `username`.

Important

Use a VPC connection for applications deployed on an ECS instance — it provides lower latency and better security than a public network connection. To connect over the public network, first enable the public endpoint: in the Lindorm console, click Database Connections in the left navigation pane, click the Search Engine tab, and then click Enable Public Endpoint in the upper-right corner.

Create an index

try (RestClient restClient = restClientBuilder.build()) {
  String indexName = "lindorm_index";

  Request indexRequest = new Request("PUT", "/" + indexName);
  indexRequest.setJsonEntity("{" +
    "  \"settings\":{" +
    "    \"index.number_of_shards\": 1" +
    "  }," +
    "  \"mappings\":{" +
    "    \"properties\":{" +
    "      \"name\":{" +
    "        \"type\":\"text\"" +
    "      }" +
    "    }" +
    "  }" +
    "}");
  Response response = restClient.performRequest(indexRequest);
  String responseBody = EntityUtils.toString(response.getEntity());
  System.out.println("responseBody = " + responseBody);
}

A successful response looks like this:

{"acknowledged":true,"shards_acknowledged":true,"index":"lindorm_index"}

Write documents in bulk

Use the Bulk API to write multiple documents in a single request. The following example writes 5,000 documents to lindorm_index:

Random random = new Random();
Request bulkRequest = new Request("POST", "/_bulk");
StringBuilder bulkJsonBuilder = new StringBuilder();
for (int i = 0; i < 5000; i++) {
  // Replace field names and values with those from your application.
  bulkJsonBuilder.append("{\"index\":{\"_index\":\"").append(indexName).append("\",\"_id\":\"").append(i).append("\"}}").append("\n");
  String value = random.nextInt() + "";
  bulkJsonBuilder.append("{\"field1\":\"").append(value).append("\",\"field2\":\"").append(value).append("\"}").append("\n");
}
bulkRequest.setJsonEntity(bulkJsonBuilder.toString());
response = restClient.performRequest(bulkRequest);

After the bulk write, send a refresh request to make the documents visible for search:

response = restClient.performRequest(new Request("POST", "/" + indexName + "/_refresh"));
responseBody = EntityUtils.toString(response.getEntity());
System.out.println("responseBody = " + responseBody);

A successful refresh response looks like this:

{"_shards":{"total":1,"successful":1,"failed":0}}

Query documents

Query all documents in the index (returns up to 10 results by default):

response = restClient.performRequest(new Request("GET", "/" + indexName + "/_search"));
responseBody = EntityUtils.toString(response.getEntity());
System.out.println("responseBody = " + responseBody);

Query a document by ID:

response = restClient.performRequest(new Request("GET", "/" + indexName + "/_doc/0"));
responseBody = EntityUtils.toString(response.getEntity());
System.out.println("responseBody = " + responseBody);

A successful query-by-ID response looks like this:

{"_index":"lindorm_index","_id":"0","_version":1,"_seq_no":0,"_primary_term":1,"found":true,"_source":{"field1":"2127821774","field2":"2127821774"}}

Delete an index

response = restClient.performRequest(new Request("DELETE", "/" + indexName));
responseBody = EntityUtils.toString(response.getEntity());
System.out.println("responseBody = " + responseBody);

A successful delete response looks like this:

{"acknowledged":true}

Complete example

The following is the complete sample program combining all the operations above:

import org.apache.http.HttpHost;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;
import org.apache.http.util.EntityUtils;
import org.elasticsearch.client.Request;
import org.elasticsearch.client.Response;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestClientBuilder;

import java.util.Random;

public class RestLClientTest {
  public static void main(String[] args) {
    String search_url = "ld-t4n5668xk31ui****-proxy-search-public.lindorm.rds.aliyuncs.com";
    int search_port = 30070;

    // For demo purposes only. Do not hardcode credentials in production code.
    String username = "user";
    String password = "test";

    final CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
    credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(username, password));
    RestClientBuilder restClientBuilder = RestClient.builder(new HttpHost(search_url, search_port));
    restClientBuilder.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
      @Override
      public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
        return httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider);
      }
    });

    try (RestClient restClient = restClientBuilder.build()) {
      String indexName = "lindorm_index";

      // Create a search index.
      Request indexRequest = new Request("PUT", "/" + indexName);
      indexRequest.setJsonEntity("{" +
        "  \"settings\":{" +
        "    \"index.number_of_shards\": 1" +
        "  }," +
        "  \"mappings\":{" +
        "    \"properties\":{" +
        "      \"name\":{" +
        "        \"type\":\"text\"" +
        "      }" +
        "    }" +
        "  }" +
        "}");
      Response response = restClient.performRequest(indexRequest);
      String responseBody = EntityUtils.toString(response.getEntity());
      System.out.println("responseBody = " + responseBody);

      // Write 5,000 documents in a batch.
      Random random = new Random();
      Request bulkRequest = new Request("POST", "/_bulk");
      StringBuilder bulkJsonBuilder = new StringBuilder();
      for (int i = 0; i < 5000; i++) {
        // Replace field names and values with those from your application.
        bulkJsonBuilder.append("{\"index\":{\"_index\":\"").append(indexName).append("\",\"_id\":\"").append(i).append("\"}}").append("\n");
        String value = random.nextInt() + "";
        bulkJsonBuilder.append("{\"field1\":\"").append(value).append("\",\"field2\":\"").append(value).append("\"}").append("\n");
      }
      bulkRequest.setJsonEntity(bulkJsonBuilder.toString());
      response = restClient.performRequest(bulkRequest);

      // Refresh the index to make the written documents visible.
      response = restClient.performRequest(new Request("POST", "/" + indexName + "/_refresh"));
      responseBody = EntityUtils.toString(response.getEntity());
      System.out.println("responseBody = " + responseBody);

      // Query all documents in the index (returns up to 10 by default).
      response = restClient.performRequest(new Request("GET", "/" + indexName + "/_search"));
      responseBody = EntityUtils.toString(response.getEntity());
      System.out.println("responseBody = " + responseBody);

      // Query the document with ID 0.
      response = restClient.performRequest(new Request("GET", "/" + indexName + "/_doc/0"));
      responseBody = EntityUtils.toString(response.getEntity());
      System.out.println("responseBody = " + responseBody);

      // Delete the index.
      response = restClient.performRequest(new Request("DELETE", "/" + indexName));
      responseBody = EntityUtils.toString(response.getEntity());
      System.out.println("responseBody = " + responseBody);

    } catch (Exception e) {
      System.out.println("msg: " + e.getMessage());
    }
  }
}