使用用戶端串連PolarSearch
PolarSearch完全相容OpenSearch的官方用戶端,您可以使用相應的用戶端與PolarSearch進行互動。這使得您能夠通過Java或Python等常用程式設計語言高效地管理索引、操作文檔(包括增、刪、改、查)以及執行複雜搜尋,從而將搜尋功能無縫整合到您的應用程式中。
準備工作
已建立含有PolarSearch節點的叢集並設定了節點的管理員帳號。
擷取串連地址:在叢集的数据库节点地區,將滑鼠懸浮在搜索节点,根據您的業務環境,擷取PolarSearch節點的私網或公網地址。
串連叢集
OpenSearch Java client
OpenSearch Java用戶端允許您通過Java方法和資料結構與OpenSearch叢集互動,而不是使用HTTP方法和原始JSON。比如,您可以使用對象向叢集提交請求,通過用戶端內建的方法來建立索引、向文檔寫入資料,或完成其他動作。有關該用戶端完整的API文檔和更多樣本,請參閱javadoc。
1. 選擇傳輸層並添加依賴
OpenSearch Java Client需要搭配一個傳輸層架構來處理HTTP請求。
PolarSearch 1.x版本:Java Client僅能使用RestClient Transport。
PolarSearch 3.x版本:Java Client則可以使用Apache HttpClient 5 Transport或者RestClient Transport。
PolarSearch 1.x版本
Maven樣本:在pom.xml檔案中添加以下依賴:
<!-- OpenSearch Java Client核心庫 -->
<dependency>
<groupId>org.opensearch.client</groupId>
<artifactId>opensearch-java</artifactId>
<version>1.0.0</version>
</dependency>
<!-- RestClient Transport 傳輸層 -->
<dependency>
<groupId>org.opensearch.client</groupId>
<artifactId>opensearch-rest-client</artifactId>
<version>1.3.20</version>
</dependency>PolarSearch 3.x版本
Apache HttpClient5 Transport Maven樣本:在pom.xml檔案中添加以下依賴:
<!-- OpenSearch Java Client核心庫 -->
<dependency>
<groupId>org.opensearch.client</groupId>
<artifactId>opensearch-java</artifactId>
<version>3.3.0</version>
</dependency>
<!-- Apache HttpClient5 傳輸層 -->
<dependency>
<groupId>org.apache.httpcomponents.client5</groupId>
<artifactId>httpclient5</artifactId>
<version>5.4.3</version>
</dependency>2. 設定資料類
建立一個測試資料類,以供後續測試PolarSearch功能使用。
static class IndexData {
private String title;
private String text;
public IndexData() {}
public IndexData(String title, String text) {
this.title = title;
this.text = text;
}
public String getTitle() {
return title;
}
public void setTitle(String title) {
this.title = title;
}
public String getText() {
return text;
}
public void setText(String text) {
this.text = text;
}
@Override
public String toString() {
return String.format("IndexData{title='%s', text='%s'}", title, text);
}
}3. 初始化用戶端
根據您選擇的傳輸層,使用PolarSearch的串連資訊初始化用戶端。在實際使用中,您可以根據以下代碼配置將PolarSearch的相關資訊設定為環境變數,或直接將其賦值為參數的預設值,以便於測試使用。
PolarSearch 1.x版本
以下樣本展示了如何使用RestClient Transport初始化一個用戶端,當前範例程式碼禁用了SSL。
public static OpenSearchTransport createTransport() throws Exception {
var env = System.getenv();
var hostname = env.getOrDefault("HOST", "<polarsearch_host>");
var port = Integer.parseInt(env.getOrDefault("PORT", "<polarsearch_port>"));
var scheme = env.getOrDefault("SCHEME", "http");
var user = env.getOrDefault("USERNAME", "<polarsearch_username>");
var pass = env.getOrDefault("PASSWORD", "<polarsearch_password>");
final HttpHost host = new HttpHost(hostname, port, scheme);
final BasicCredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(AuthScope.ANY,
new UsernamePasswordCredentials(user, pass));
final SSLContext sslContext = SSLContextBuilder.create()
.loadTrustMaterial(null, (chains, authType) -> true)
.build();
RestClientBuilder builder = RestClient.builder(host)
.setHttpClientConfigCallback(httpClientBuilder ->
httpClientBuilder
.setDefaultCredentialsProvider(credentialsProvider)
.setSSLContext(sslContext));
final RestClient restClient = builder.build();
return new RestClientTransport(restClient, new JacksonJsonpMapper());
}
transport = createTransport();
var client = new OpenSearchClient(transport);PolarSearch 3.x版本
以下樣本展示了如何使用Apache HttpClient5 Transport初始化一個用戶端,當前範例程式碼禁用了SSL。
public static OpenSearchTransport createTransport() throws NoSuchAlgorithmException, KeyStoreException, KeyManagementException {
var env = System.getenv();
var hostname = env.getOrDefault("HOST", "<polarsearch_host>");
var port = Integer.parseInt(env.getOrDefault("PORT", "<polarsearch_port>"));
var scheme = env.getOrDefault("SCHEME", "http");
var user = env.getOrDefault("USERNAME", "<polarsearch_username>");
var pass = env.getOrDefault("PASSWORD", "<polarsearch_password>");
final HttpHost host = new HttpHost(scheme, hostname, port);
final BasicCredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(new AuthScope(host), new UsernamePasswordCredentials(user, pass.toCharArray()));
final SSLContext sslContext = SSLContextBuilder
.create()
.loadTrustMaterial(null, (chains, authType) -> true)
.build();
final ApacheHttpClient5TransportBuilder builder = ApacheHttpClient5TransportBuilder.builder(host);
builder.setHttpClientConfigCallback(httpClientBuilder -> {
final TlsStrategy tlsStrategy = ClientTlsStrategyBuilder.create()
.setSslContext(sslContext)
.setTlsDetailsFactory(new Factory<SSLEngine, TlsDetails>() {
@Override
public TlsDetails create(final SSLEngine sslEngine) {
return new TlsDetails(sslEngine.getSession(), sslEngine.getApplicationProtocol());
}
})
.build();
final PoolingAsyncClientConnectionManager connectionManager = PoolingAsyncClientConnectionManagerBuilder
.create()
.setTlsStrategy(tlsStrategy)
.build();
return httpClientBuilder
.setDefaultCredentialsProvider(credentialsProvider)
.setConnectionManager(connectionManager);
});
final OpenSearchTransport transport = builder.build();
return transport;
}
transport = createTransport();
var client = new OpenSearchClient(transport);4. 執行基本操作
以下程式碼片段展示了如何執行常見的索引和文檔操作。
建立索引
// 1. Creating an index
final var index = "my-index";
if (!client.indices().exists(r -> r.index(index)).value()) {
CreateIndexRequest createIndexRequest = new CreateIndexRequest.Builder().index(index)
.build();
client.indices().create(createIndexRequest);
}寫入文檔
// 2. Indexing data
IndexData indexData = new IndexData("first_name", "Bruce");
IndexRequest<IndexData> indexRequest = new IndexRequest.Builder<IndexData>().index(index).id("1").document(indexData).build();
client.index(indexRequest);搜尋文檔
// 3. Searching for documents
SearchResponse<IndexData> searchResponse = client.search(s -> s.index(index), IndexData.class);
for (int i = 0; i< searchResponse.hits().hits().size(); i++) {
LOGGER.info(searchResponse.hits().hits().get(i).source());
} 刪除文檔
// 4. Deleting a document
client.delete(b -> b.index(index).id("1"));刪除索引
// 5. Deleting an index
DeleteIndexRequest deleteIndexRequest = new DeleteIndexRequest.Builder().index(index).build();
DeleteIndexResponse deleteIndexResponse = client.indices().delete(deleteIndexRequest);OpenSearch Java high-level REST client
建議您使用OpenSearch Java client,因為Java high-level REST client已在OpenSearch中被廢棄,未來版本將不再支援該用戶端。
1. 添加依賴
在pom.xml檔案中添加以下依賴:
PolarSearch 1.x版本
<!-- OpenSearch Java rest high level client核心庫 -->
<dependency>
<groupId>org.opensearch.client</groupId>
<artifactId>opensearch-rest-high-level-client</artifactId>
<version>1.3.20</version>
</dependency>PolarSearch 3.x版本
<!-- OpenSearch Java rest high level client核心庫 -->
<dependency>
<groupId>org.opensearch.client</groupId>
<artifactId>opensearch-rest-high-level-client</artifactId>
<version>3.3.2</version>
</dependency>2. 初始化用戶端
在實際使用中,您可以根據以下代碼配置將PolarSearch的相關資訊設定為環境變數,或直接將其賦值為參數的預設值,以便於測試使用。
PolarSearch 1.x版本
以下樣本展示了如何初始化一個1.x版本的用戶端,當前範例程式碼禁用了SSL。
public static RestHighLevelClient createClient() throws Exception {
var env = System.getenv();
var hostname = env.getOrDefault("HOST", "<polarsearch_host>");
var port = Integer.parseInt(env.getOrDefault("PORT", "<polarsearch_port>"));
var scheme = env.getOrDefault("SCHEME", "http");
var user = env.getOrDefault("USERNAME", "<polarsearch_username>");
var pass = env.getOrDefault("PASSWORD", "<polarsearch_password>");
final HttpHost host = new HttpHost(hostname, port, scheme);
final BasicCredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(AuthScope.ANY,
new UsernamePasswordCredentials(user, pass));
final SSLContext sslContext = SSLContextBuilder.create()
.loadTrustMaterial(null, (chains, authType) -> true)
.build();
RestClientBuilder builder = RestClient.builder(host)
.setHttpClientConfigCallback(httpClientBuilder ->
httpClientBuilder
.setDefaultCredentialsProvider(credentialsProvider)
.setSSLContext(sslContext));
return new RestHighLevelClient(builder);
}
RestHighLevelClient client = createClient();PolarSearch 3.x版本
以下樣本展示了如何初始化一個3.x版本的用戶端,當前範例程式碼禁用了SSL。
public static RestHighLevelClient createClient() throws Exception {
var env = System.getenv();
var hostname = env.getOrDefault("HOST", "<polarsearch_host>");
var port = Integer.parseInt(env.getOrDefault("PORT", "<polarsearch_port>"));
var scheme = env.getOrDefault("SCHEME", "http");
var user = env.getOrDefault("USERNAME", "<polarsearch_username>");
var pass = env.getOrDefault("PASSWORD", "<polarsearch_password>");
final HttpHost host = new HttpHost(scheme, hostname, port);
final BasicCredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(new AuthScope(host),
new UsernamePasswordCredentials(user, pass.toCharArray()));
final SSLContext sslContext = SSLContextBuilder.create()
.loadTrustMaterial(null, (chains, authType) -> true)
.build();
final TlsStrategy tlsStrategy = ClientTlsStrategyBuilder.create()
.setSslContext(sslContext)
.build();
final PoolingAsyncClientConnectionManager connectionManager = PoolingAsyncClientConnectionManagerBuilder
.create()
.setTlsStrategy(tlsStrategy)
.build();
RestClientBuilder builder = RestClient.builder(host)
.setHttpClientConfigCallback(httpClientBuilder ->
httpClientBuilder
.setDefaultCredentialsProvider(credentialsProvider)
.setConnectionManager(connectionManager));
return new RestHighLevelClient(builder);
}
RestHighLevelClient client = createClient();3. 執行基本操作
以下程式碼片段展示了如何執行常見的索引和文檔操作。
建立索引
// 1. Creating an index
final var index = "my-index";
boolean exists = client.indices().exists(new GetIndexRequest(index), RequestOptions.DEFAULT);
if (!exists) {
CreateIndexRequest createIndexRequest = new CreateIndexRequest(index);
client.indices().create(createIndexRequest, RequestOptions.DEFAULT);
}寫入文檔
// 2. Indexing data
IndexData indexData = new IndexData("first_name", "Bruce");
Map<String, Object> document = new HashMap<>();
document.put("title", indexData.getTitle());
document.put("text", indexData.getText());
IndexRequest indexRequest = new IndexRequest(index).id("1").source(document);
client.index(indexRequest, RequestOptions.DEFAULT);搜尋文檔
// 3. Searching for documents
SearchRequest searchRequest = new SearchRequest(index);
searchRequest.source(new SearchSourceBuilder().query(QueryBuilders.matchAllQuery()));
SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
for (SearchHit hit : searchResponse.getHits().getHits()) {
Map<String, Object> sourceMap = hit.getSourceAsMap();
IndexData data = new IndexData((String) sourceMap.get("title"), (String) sourceMap.get("text"));
LOGGER.info(data);
}刪除文檔
// 4. Deleting a document
client.delete(new DeleteRequest(index, "1"), RequestOptions.DEFAULT);刪除索引
// 5. Deleting an index
DeleteIndexRequest deleteIndexRequest = new DeleteIndexRequest(index);
AcknowledgedResponse deleteIndexResponse = client.indices().delete(deleteIndexRequest, RequestOptions.DEFAULT);Elasticsearch Java High Level REST Client
PolarSearch 1.x版本100%相容Elasticsearch Java High Level REST Client的7.0.0 ~ 7.13.4的所有版本,若使用的用戶端版本為上述相容版本,則可以直接修改串連地址,而無需改造任何業務代碼。
1. 添加依賴
在pom.xml檔案中添加以下依賴:
<!-- Elasticsearch Java rest high level client核心庫 -->
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>7.13.4</version>
</dependency>2. 初始化用戶端
在實際使用中,您可以根據以下代碼配置將PolarSearch的相關資訊設定為環境變數,或直接將其賦值為參數的預設值,以便於測試使用。
以下樣本展示了如何初始化一個1.x版本的用戶端,當前範例程式碼禁用了SSL。
public static RestHighLevelClient createClient() throws Exception {
var env = System.getenv();
var hostname = env.getOrDefault("HOST", "<polarsearch_host>");
var port = Integer.parseInt(env.getOrDefault("PORT", "<polarsearch_port>"));
var scheme = env.getOrDefault("SCHEME", "http");
var user = env.getOrDefault("USERNAME", "<polarsearch_username>");
var pass = env.getOrDefault("PASSWORD", "<polarsearch_password>");
final HttpHost host = new HttpHost(hostname, port, scheme);
final BasicCredentialsProvider credentialsProvider = new BasicCredentialsProvider();
credentialsProvider.setCredentials(AuthScope.ANY,
new UsernamePasswordCredentials(user, pass));
final SSLContext sslContext = SSLContextBuilder.create()
.loadTrustMaterial(null, (chains, authType) -> true)
.build();
RestClientBuilder builder = RestClient.builder(host)
.setHttpClientConfigCallback(httpClientBuilder ->
httpClientBuilder
.setDefaultCredentialsProvider(credentialsProvider)
.setSSLContext(sslContext));
return new RestHighLevelClient(builder);
}
RestHighLevelClient client = createClient();3. 執行基本操作
以下程式碼片段展示了如何執行常見的索引和文檔操作。
建立索引
// 1. Creating an index
final var index = "my-index";
boolean exists = client.indices().exists(new GetIndexRequest(index), RequestOptions.DEFAULT);
if (!exists) {
CreateIndexRequest createIndexRequest = new CreateIndexRequest(index);
client.indices().create(createIndexRequest, RequestOptions.DEFAULT);
}寫入文檔
// 2. Indexing data
IndexData indexData1 = new IndexData("first_name", "Bruce");
Map<String, Object> document1 = new HashMap<>();
document1.put("title", indexData1.getTitle());
document1.put("text", indexData1.getText());
client.index(new IndexRequest(index).id("1").source(document1), RequestOptions.DEFAULT);搜尋文檔
// 3. Searching for documents
SearchRequest searchRequest = new SearchRequest(index);
searchRequest.source(new SearchSourceBuilder().query(QueryBuilders.matchAllQuery()));
SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
for (SearchHit hit : searchResponse.getHits().getHits()) {
Map<String, Object> sourceMap = hit.getSourceAsMap();
IndexData data = new IndexData((String) sourceMap.get("title"), (String) sourceMap.get("text"));
LOGGER.info(data);
}刪除文檔
// 4. Deleting a document
client.delete(new DeleteRequest(index, "1"), RequestOptions.DEFAULT);刪除索引
// 5. Deleting an index
DeleteIndexRequest deleteIndexRequest = new DeleteIndexRequest(index);
AcknowledgedResponse deleteIndexResponse = client.indices().delete(deleteIndexRequest, RequestOptions.DEFAULT);Low-level Python client
建議您使用Low-level Python client,因為High-level Python client在OpenSearch 2.1.0版本後已被廢棄。
1.配置環境
根據實際業務需求,進入指定的專案目錄。此處以
/home/PolarSearchTestPython為例。mkdir /home/PolarSearchTestPython cd /home/PolarSearchTestPython在
/home/PolarSearchTestPython目錄中,建立虛擬環境(venv)隔離專案依賴,避免全域汙染。python3 -m venv myenv啟用虛擬環境
source myenv/bin/activate安裝所需的Python依賴庫。
pip3 install opensearch-py
2. 串連PolarSearch
在您的Python代碼中匯入OpenSearch類,並使用PolarSearch的串連資訊建立用戶端執行個體。在實際使用中,您可以根據以下代碼配置將PolarSearch的相關資訊設定為環境變數,或直接將其賦值為參數的預設值,以便於測試使用。
from opensearchpy import OpenSearch
host = os.getenv("HOST", default="<polarsearch_host>")
port = int(os.getenv("PORT", <polarsearch_port>))
auth = (os.getenv("USERNAME", "<polarsearch_username>"), os.getenv("PASSWORD", "<polarsearch_password>"))
client = OpenSearch(
hosts=[{"host": host, "port": port}],
http_auth=auth,
use_ssl=False,
verify_certs=False,
ssl_show_warn=False,
)3. 操作樣本
以下程式碼片段展示了如何執行常見的索引和文檔操作。
建立索引
使用client.indices.create()方法建立一個新索引。
index_name = 'python-test-index'
index_body = {
'settings': {
'index': {
'number_of_shards': 4
}
}
}
response = client.indices.create(index=index_name, body=index_body)寫入文檔
使用client.index()方法向指定索引中添加一個文檔。
document = {
'title': 'Moneyball',
'director': 'Bennett Miller',
'year': '2011'
}
response = client.index(
index = 'python-test-index',
body = document,
id = '1',
refresh = True
)搜尋文檔
使用client.search()方法根據查詢條件搜尋文檔。
q = 'miller'
query = {
'size': 5,
'query': {
'multi_match': {
'query': q,
'fields': ['title^2', 'director']
}
}
}
response = client.search(
body = query,
index = 'python-test-index'
)刪除文檔
使用client.delete()方法刪除指定ID的文檔。
response = client.delete(
index = 'python-test-index',
id = '1'
)刪除索引
使用client.indices.delete()方法刪除整個索引。
response = client.indices.delete(
index = 'python-test-index'
)完整範例程式碼
下面是一個完整的樣本,示範了從建立索引到最終刪除索引的全過程。
OpenSearch Java client
範例程式碼
PolarSearch 1.x版本
PolarSearch 3.x版本
運行方式
mvn clean compile exec:java -Dexec.mainClass=samples.OpenSearchClientExampleOpenSearch Java high-level REST client
範例程式碼
PolarSearch 1.x版本
PolarSearch 3.x版本
運行方式
mvn clean compile exec:java -Dexec.mainClass=samples.OpenSearchClientExampleElasticsearch Java High Level REST Client
樣本程式
Low-level Python client
依賴配置
pip3 install opensearch-py樣本程式
import os
from opensearchpy import OpenSearch
host = os.getenv("HOST", default="<polarsearch_host>")
port = int(os.getenv("PORT", <polarsearch_port>))
auth = (os.getenv("USERNAME", "<polarsearch_username>"), os.getenv("PASSWORD", "<polarsearch_password>"))
client = OpenSearch(
hosts=[{"host": host, "port": port}],
http_auth=auth,
use_ssl=False,
verify_certs=False,
ssl_show_warn=False,
)
# Create an index with non-default settings.
index_name = 'python-test-index'
index_body = {
'settings': {
'index': {
'number_of_shards': 4
}
}
}
response = client.indices.create(index=index_name, body=index_body)
print('\nCreating index:')
print(response)
# Add a document to the index.
document = {
'title': 'Moneyball',
'director': 'Bennett Miller',
'year': '2011'
}
id = '1'
response = client.index(
index = index_name,
body = document,
id = id,
refresh = True
)
print('\nAdding document:')
print(response)
# Perform bulk operations
movies = '{ "index" : { "_index" : "my-dsl-index", "_id" : "2" } } \n { "title" : "Interstellar", "director" : "Christopher Nolan", "year" : "2014"} \n { "create" : { "_index" : "my-dsl-index", "_id" : "3" } } \n { "title" : "Star Trek Beyond", "director" : "Justin Lin", "year" : "2015"} \n { "update" : {"_id" : "3", "_index" : "my-dsl-index" } } \n { "doc" : {"year" : "2016"} }'
client.bulk(body=movies)
# Search for the document.
q = 'miller'
query = {
'size': 5,
'query': {
'multi_match': {
'query': q,
'fields': ['title^2', 'director']
}
}
}
response = client.search(
body = query,
index = index_name
)
print('\nSearch results:')
print(response)
# Delete the document.
response = client.delete(
index = index_name,
id = id
)
print('\nDeleting document:')
print(response)
# Delete the index.
response = client.indices.delete(
index = index_name
)
print('\nDeleting index:')
print(response)