PolarSearch offers two major versions, 1.x and 3.x, built on different OpenSearch kernels. This topic compares the two versions based on kernel features, client compatibility, and common scenarios to help you select the version that fits your business needs.
Selection overview
Choosing a PolarSearch version is a trade-off between compatibility and performance. When adding a search node, you must decide between version 1.x and 3.x based on your long-term business plans and your tolerance for code refactoring costs. Use the following guide to make the best choice:
Decision factor | Choose PolarSearch 1.x (Compatibility-first) | Choose PolarSearch 3.x (Performance-first) |
Core business need | Your business primarily uses traditional full-text search and has no short-term plans for vector search. | Your business involves vector search scenarios, such as AI search or retrieval-augmented generation (RAG), or you want to prepare for these capabilities in the future. |
Client code refactoring cost | You require strict compatibility with existing code. Your application currently uses Elasticsearch Java high-level REST Client version 7.13.4 or earlier, and you want a smooth migration with zero-code changes. | You are willing to refactor code. You are prepared to switch from the Elasticsearch client to the OpenSearch client for better long-term benefits. |
Performance and feature expectations | Meeting the current performance baseline for traditional search is sufficient. | You want better query and write performance, stronger security controls, and access to the latest features from the OpenSearch community. |
Version comparison
The following table compares the core specifications of PolarSearch 1.x and 3.x:
Feature | PolarSearch 1.x | PolarSearch 3.x |
Lucene version | 8.10.1 | 10.3.1 |
Elasticsearch client compatibility | 100% compatible with Elasticsearch Java high-level REST Client versions 7.0.0 to 7.13.4 | Not compatible |
AWS OpenSearch kernel compatibility | 100% compatible with the OpenSearch 1.x kernel. | 100% compatible with OpenSearch 2.x and 3.x kernels. |
AWS OpenSearch client compatibility | Client version compatibility is identical to AWS OpenSearch. For more information, see the compatibility matrix for the client of your specific language. | |
Vector search (k-NN) | Basic support with low performance | Full support with high performance |
Query performance | Lucene 8.x Benchmark | Based on Lucene 10.x for better indexing and query performance |
Security capabilities | Basic authentication | Enhanced authentication and access control |
Scenarios | Migration from Elasticsearch 7.x | New projects, AI scenarios |
Client compatibility
The two PolarSearch versions support different client libraries. You must select the client that corresponds to your PolarSearch version.
PolarSearch 1.x clients
Client | Recommended version | Description |
7.10.2 to 7.13.4 |
| |
Elasticsearch Java Low Level REST Client | Any version | Stable and flexible, but you must handle the serialization of HTTP requests and responses. |
1.3.20 | Natively supported. Recommended for new integration projects with PolarSearch 1.x. |
PolarSearch 3.x clients
Client | Recommended version | Description |
3.3.0 | Natively supported. Supports all features of version 3.x, including vector search (k-NN). | |
Elasticsearch Java Low Level REST Client | Any version | Stable and flexible, but you must handle the serialization of HTTP requests and responses. |
Scenarios
Migrating from Elasticsearch
If your application currently uses the Elasticsearch Java high-level REST Client (versions 7.0.0 to 7.13.4), your choice of PolarSearch version depends on whether you plan to refactor your application code:
No code refactoring: Choose PolarSearch 1.x. This version is API-compatible with Elasticsearch 7.x, which allows for a smooth migration from Elasticsearch without code refactoring. You can directly reuse your existing query DSL, index mappings, and client configurations.
Code refactoring required: PolarSearch 3.x is not compatible with the Elasticsearch client. During migration, you must change the client library in your application and perform major code refactoring.
AI search and RAG
If your application requires vector search, semantic similarity matching, or retrieval-augmented generation (RAG), choose PolarSearch 3.x. The built-in k-NN plugin supports storing high-dimensional embedding vectors and performing approximate nearest neighbor (ANN) queries. Typical scenarios include the following:
RAG knowledge bases for large language model (LLM) applications.
Semantic search for documents, images, or multimodal data.
Recommendation systems based on embedding similarity.
Hybrid queries that combine full-text search and vector search.
New projects
If your project has no legacy dependencies on Elasticsearch, choose PolarSearch 3.x. This version runs on the latest OpenSearch kernel, offers better query performance, provides native vector search capabilities, and has a longer support period. Use the native OpenSearch client from the start to take full advantage of all version 3.x features.
Traditional full-text search
Both PolarSearch 1.x and 3.x support full-text search capabilities, including inverted indexes, tokenization, and relevance scoring. If your business primarily uses traditional full-text search:
Choose 1.x if you need compatibility with existing Elasticsearch Java high-level REST Client code.
Choose 3.x if you want better indexing and query performance or high-performance vector search (k-NN).
Migration recommendations
PolarSearch provides clear paths to help you smoothly migrate or upgrade your existing services to different PolarSearch versions. The following sections describe the procedures for two typical scenarios:
Migrating from Elasticsearch to PolarSearch 1.x
This path is designed for Elasticsearch instances where you want to minimize code changes during migration. It takes full advantage of the compatibility of PolarSearch 1.x with legacy clients.
Create a new cluster
Create a new PolarSearch 1.x search node in your PolarDB cluster.Confirm client version compatibility
Check and ensure that the Elasticsearch Java high-level REST Client version used in your application is within the 7.0.0 to 7.13.4 range. If the version is 7.14.0 or later, you must downgrade it.Migrate data
Use Snapshot/Restore or Reindex to perform a full migration of data from your existing Elasticsearch cluster to the new PolarSearch 1.x search node.Test core APIs
Point your application's connection to the new PolarSearch 1.x search node and test all REST API calls for compatibility, such as index creation, document reads and writes, search, and aggregation.Verify business features and performance
Fully test your application to ensure that all search-dependent features, such as product search and log queries, work as expected and return correct results.Gradually switch traffic
Use load balancing or application layer configurations to gradually switch online traffic from the legacy Elasticsearch cluster to the new PolarSearch 1.x search node.Decommission the legacy instance
After confirming that the new cluster has been running stably for a period, you can safely decommission the legacy Elasticsearch instance to complete the migration.
Upgrading from PolarSearch 1.x to 3.x
This path is for version 1.x users who want to use the new features of version 3.x, such as high-performance vector search. Note that this procedure involves client replacement and code refactoring.
Create a new cluster
Create a new PolarSearch 3.x search node in your PolarDB cluster.Migrate data
Use Reindex to perform a full migration of data from the PolarSearch 1.x search node to the new PolarSearch 3.x search node.Switch clients and refactor code
Replace the client library in your application from the Elasticsearch Java high-level REST Client or an older OpenSearch Java Client to a native client compatible with OpenSearch 3.x, such as OpenSearch Java Client 3.3.0. This step typically involves major code refactoring.Verify business features
After refactoring the code, fully test your application to ensure all features work correctly on the new client and the 3.x kernel.Verify performance
Focus on the performance of search, indexing, and aggregation to ensure it meets or exceeds the performance of version 1.x.Gradually switch traffic
Use load balancing or application layer configurations to gradually switch online traffic from the 1.x search node to the new PolarSearch 3.x search node.Decommission the legacy search node
After confirming that the new cluster has been running stably for a period, you can safely decommission the legacy PolarSearch 1.x search node to complete the upgrade.
FAQ
Can I use an Elasticsearch client with PolarSearch 3.x?
No. PolarSearch 3.x is built on OpenSearch 3.x and is not compatible with Elasticsearch client libraries. Elasticsearch Java high-level REST Client versions 7.14.0 and later, and Java API Client 8.x, both include an Elastic license check that prevents them from connecting to PolarSearch. To connect to PolarSearch 3.x, you must use a native OpenSearch client, such as OpenSearch Java Client 3.x or later.
Why can't Elasticsearch Java high-level REST Client 7.14.0 and later connect to PolarSearch?
Starting with version 7.14.0, the Elasticsearch Java high-level REST Client includes a security check that fails to correctly identify the PolarSearch server-side. This causes the client to throw an exception during startup.
For PolarSearch 1.x, downgrade the client to a version between 7.0.0 and 7.13.4.
For PolarSearch 3.x, switch to a native OpenSearch client.
Does PolarSearch 1.x support vector search?
PolarSearch 1.x provides basic support for vector search, but it is not recommended. The kernel version of PolarSearch 1.x is older. Although it offers basic vector search features, its performance is low and not suitable for production environments that require high performance. If you need high-performance vector search, you must choose PolarSearch 3.x.