Alibaba Cloud Elasticsearch version comparison, features by version - Elasticsearch

Alibaba Cloud Elasticsearch (ES) provides two edition types: Self-developed Enhanced Edition and Standard Edition. Self-developed Enhanced Editions are deeply optimized on open-source Elasticsearch and deliver higher performance and AI search capabilities, whereas the Standard Edition is 100% compatible with open-source features and includes a free license for the official Platinum-level advanced features. Compare the editions and the features of each version to choose the ES version that fits your workload.

Self-developed enhanced editions

Alibaba Cloud ES provides two self-developed enhanced instance types: Vector Enhanced Edition and Kernel-enhanced Edition. We recommend version 8.17 (Vector Enhanced Edition) or version 7.10 (Kernel-enhanced Edition). The following table compares the two instance types.

Item	Vector Enhanced Edition	Kernel-enhanced Edition
Supported versions	8.17 and 8.15	7.16, 7.10, and 6.7
Main features	100% compatible with open-source Elasticsearch. Includes a free license for all Platinum-level advanced features. Supports the FalconSeek cloud-native kernel, which is based on the Alibaba self-developed Havenask engine. A C++ columnar memory model and a fully asynchronous framework significantly improve query performance for complex aggregations, high-cardinality terms, and vector retrieval. Supports the searchable snapshot storage-compute separation architecture. All data is stored in Object Storage Service (OSS), which greatly reduces storage costs for massive volumes of cold data. Supports the cluster health event center, which detects cluster resource exceptions and risks automatically through scheduled inspections and monitoring alerts, and then triggers alerts and automatic recovery in time.	100% compatible with open-source Elasticsearch. Includes a free license for all Platinum-level advanced features. Uses the deeply optimized AliES kernel. The kernel optimizes costs and improves instance performance and stability in multiple scenarios.
Scenarios	All ES scenarios, such as information retrieval, search, log analysis, and vector retrieval.	All ES scenarios. Especially suitable for enterprise-grade scenarios that require high read and write performance, and for log search and analytics scenarios with heavy writes and light reads.
Target users	You know ES and can tune performance for your own scenarios. You have clear resource planning.	You require higher optimization of cluster write and query performance. You want to reduce the configuration and O&M costs of ES in the cloud. You want to improve cluster stability when workloads fluctuate. You want to reduce the storage costs of massive volumes of data.
Billable items	You are charged based on the node specifications, storage space, and number of nodes of the ES cluster.	You are charged based on the node specifications, storage space, and number of nodes of the ES cluster. Basic enhancements: provided as plug-ins for free. You can install them on demand. Advanced enhancements: enabled on demand. After you enable them, you are charged for the additional write traffic and storage space. Currently, only Kernel-enhanced Edition 7.10 supports advanced enhancements, and the feature is available only in the China (Hong Kong) region. Support for other regions is coming soon.

Open-source features by version

The open-source features in this section apply to all Alibaba Cloud ES instance types, including the Standard Edition and both Self-developed Enhanced Editions. All Alibaba Cloud ES versions are 100% compatible with open-source Elasticsearch features and include a free license for the official Platinum-level advanced features (formerly the X-Pack commercial plug-in). Open-source features vary by Alibaba Cloud ES version as follows:

Version 9.3

New open-source features:

Agent Builder is generally available (GA). In Kibana, you can interact with Elasticsearch data through conversations and build AI chat applications. Built-in agents provide out-of-the-box capabilities.
The Elastic Inference Service (EIS) is extended to support Jina AI models, which further enriches the model ecosystem of the inference service.
DiskBBQ searches quantized vectors directly from disk without loading complete vectors into memory. Latency can be less than 20 ms with 100 MB of memory. NVIDIA GPU acceleration for vectorized processing is also supported.
The ACORN filtered vector search algorithm integrates filtering logic into HNSW graph traversal. Filtered search speed increases by 5 times without affecting accuracy.
Built on Lucene 10, which further improves the index compression ratio and the efficiency of inverted index retrieval.
LOOKUP JOIN is generally available (GA). You can run cross-index join operations directly in the ES|QL query pipeline. Multi-field matching, expression operations, and execution across remote clusters are also supported.
ES|QL query performance for time series data is significantly optimized, and latency is reduced by up to 5 times. New time series aggregation commands include RATE, *_OVER_TIME, TBUCKET, and TS.
Inline Stats supports multi-stage logic. You can use the stats command to aggregate data in the middle of the pipeline while retaining row-level details.
Garbage collection (GC) is optimized for memory jitter in high-concurrency write scenarios, which improves system stability.
Alibaba Cloud Elasticsearch releases its latest version based on 9.3. For AI capabilities, Agent Builder lets you build AI chat applications in Kibana, and the EIS inference service adds support for Jina AI models. For vector search, DiskBBQ and the ACORN algorithm greatly reduce memory usage and improve filtered search performance, and GPU acceleration is supported. For query and analysis, ES|QL adds LOOKUP JOIN for cross-index association, Inline Stats for multi-stage aggregation, and time series query optimization that reduces latency by up to 5 times. The underlying engine is upgraded to Lucene 10, and GC stability is improved in high-concurrency scenarios.

For more information about the changes, see What’s new in 9.x.

Version 8.17

New open-source features:

dense_vector introduces the Better Binary Quantization (BBQ) quantization type. Vector indexes achieve 32x memory compression, which greatly reduces memory capacity.
The Inference API is generally available (GA). For more information, see Inference APIs.
Reciprocal rank fusion (RRF) is generally available (GA). For more information, see Reciprocal rank fusion.
The logsdb index mode is generally available (GA). Log indexing uses about three times less storage space. For more information, see Logs data stream.
The built-in Elastic Rerank model is introduced. For more information, see Elastic Rerank.
The best_compression codec uses zstd, which reduces storage by about 12% and improves write throughput by 14%.
Several ES|QL features are optimized, including support for full-text search. For more information, see ES|QL.
Alibaba Cloud Elasticsearch releases the latest enhanced edition based on 8.17. A built-in model service lets you build AI search applications flexibly, any external AI model service can be called, and Better Binary Quantization (BBQ) saves more than 10 times the memory cost.

For more information about the changes, see What’s new in 8.17 and What’s new in 8.16.

Version 8.15

New open-source features:

Vector index fields are optimized. For more information, see dense-vector.
The int8_hnsw type replaces hnsw as the default type, and int8 quantization is enabled by default.
int4 quantization is supported, which saves 8 times the memory capacity.
The bit vector type is added.
SIMD instructions accelerate the merge performance of int8 quantized indexes on the aarch64 architecture. Merge performance improves by about 3 times.
A rerank phase is supported, and text_similarity_reranker can use a rerank model. For more information, see text-similarity-reranker-retriever.
The retriever query syntax is added to support multi-channel recall. For more information, see retriever.
The semantic_text field type is added for easier semantic search. For more information, see semantic-text.
Sparse queries use the sparse_vector syntax instead of text_expansion. For more information, see query-dsl-sparse-vector-query.
The query rules API is generally available (GA). For more information, see query-rules-apis.
Index Sorting supports nested fields. For more information, see index-modules-index-sorting.
The efficient logsdb index for log scenarios is added. For more information, see logs-data-stream.
Lucene is upgraded to 9.11, which improves memory efficiency and query performance. For more information, see apache-lucenetm-9110-available.
For more information about the changes, see What’s new in 8.15 and What’s new in 8.14.

Version 8.13

New open-source features:

Vector dimensions are increased to 4,096. For more information, see 4096 dimension dense vector.
Vector indexes support Scalar Quantization, which reduces the memory of vector indexes by nearly 75%. For more information, see scalar-quantization-in-lucene.
Sparse vectors are supported, and the sparse_vector type is added. For more information, see Sparse vector.
Query concurrency on a single shard is supported. For more information, see Query parallelization.
Vector fields support the nested type. You can split a document into passages and create vector indexes for the passages. For more information, see Multiple results from the same doc with nested vectors.
Learning To Rank is added, which supports reranking of results in the rescore phase. For more information, see Learning To Rank.
A new inference API is supported, which integrates external model services. For more information, see inference APIs.
SIMD improves vector query performance. For more information, see Accelerating vector search with SIMD instructions.
For more information about the changes, see What’s new in 8.13.

Version 8.9

New open-source features:

Hybrid ranking of text and vector recall results is supported. For more information, see Reciprocal rank fusion (RRF).
The maximum vector dimensions are increased to 2,048. For more information, see Increase max number of vector dims to 2048.
Brute-force retrieval performance is improved. For more information, see Improve brute force vector search speed.
KNN queries support querying multiple fields at the same time. For more information, see Allow more than one KNN search clause.
The ELSER model is built in. For more information, see ELSER inference integration.
Distributed NLP model scheduling and management is stably supported. For more information, see Make native inference generally available.
Write performance with primary keys is improved. For more information, see Primary key optimization.
Query performance for constant keyword fields is improved. For more information, see Skip shards when querying constant keyword fields.
Time series data stream (TSDS) and Downsample are stably supported. For more information, see TSDS and Downsample.
Source text memory is optimized, and the use of ThreadLocal is removed. For more information, see Remove uses of deprecated LeafReader.
For more information about the changes, see What’s new in 8.9.

Version 8.5

New open-source features:

Vector similarity search is implemented based on the HNSW algorithm. For more information, see k-nearest neighbor (kNN) search.
The time series feature (TSDS) is added. For more information, see Time series data stream (TSDS).
Geo grid queries are added. For more information, see Geo grid query.
Security configuration is simplified. For more information, see Start the Elastic Stack with security enabled automatically.
The Lucene compression algorithm is improved to reduce index size.
Range query performance is enhanced.
The lookup runtime field type is supported. For more information, see lookup-runtime-fields.
Random sampler aggregate queries are implemented. For more information, see Random sampler aggregation.
Heap memory consumption on master and data nodes is reduced.
_type is removed. However, 8.x remains compatible with 7.x requests. For more information about compatibility, see rest-api-compatibility.
Index protection is provided. By default, the elastic user can read only built-in ES indexes.
For more information about the changes, see Breaking changes in 8.5.

Version 7.16

New open-source features:

SQL queries are supported in cross-cluster search.
The ingest pipeline supports enrich policies of the range type.
The cache is optimized to improve query performance.
You can add indexes to and remove indexes from a data stream.
The cluster UUID and name are added to the audit log.
For more information about the changes, see Breaking changes in 7.16.

Version 7.10

New open-source features:

The compression capability for stored fields is improved to reduce storage costs.
ES security is enhanced by Event Query Language (EQL).
The default value of search.max_buckets is increased from 10,000 to 65,535.
Case-insensitive queries are supported. To run a case-insensitive query, set the optional case_insensitive parameter to true.
For more information about the changes, see Breaking changes in 7.10.

Version 7.7

New open-source features:

When you create an index, the default number of shards changes from five to one.
Mapping types are removed. You do not need to specify a type when you define index mappings and templates. For more information, see Removal of mapping types.
A search request returns a maximum of 10,000 documents by default. If the total number exceeds this value, ES returns only 10,000 documents. For more information, see track_total_hits 10000 default.
A single data node can contain a maximum of 1,000 shards by default. You can configure this limit by using the cluster.max_shards_per_node parameter. For more information, see cluster shard limit.
The total number of scroll objects is limited to 500 by default. You can configure this limit by using the search.max_open_scroll_context parameter. For more information, see Scroll Search Context.
The parent circuit breaker uses the actual available memory as the baseline (indices.breaker.total.use_real_memory). The default value is 95% of the JVM heap memory, which maximizes memory usage and avoids out-of-memory errors. For more information, see Circuit Breaker.
Support for the _all field is deprecated, which improves search performance.
Intervals Queries are added. You can retrieve documents based on the order and proximity of multiple strings in the text.
After you enable audit logging, audit events are persisted to the <clustername>_audit.json file on the host file system. Output to an index is not supported. For more information, see Enabling audit logging.
For more information about the changes in the 7.x series, see Breaking changes in 7.0.

Version 6.x (6.7 and 6.8)

New open-source features:

An index can have only one type. We recommend that you use _doc.
Starting from 6.6.0, index lifecycle management (ILM) is added to reduce index maintenance costs.
The Rolling up historical data feature is added to summarize historical data.
Starting from 6.3, X-Pack SQL is supported. SQL statements are converted to DSL statements, which reduces the learning cost of DSL.
Aggregate functions are enriched. The Composite, Parent, and Weighted Avg functions are supported.
For more information about the changes, see Breaking changes in 6.0.

Version 5.x (5.6)

New open-source features:

An index can have multiple types, and custom types are supported.
The string field type is removed and replaced by text or keyword.
The value of the index field changes from not_analyzed or no to true or false.
float is used instead of double to reduce storage costs.
The Java High Level REST Client replaces the legacy Java transport client.
For more information about the changes, see Breaking changes in 5.0.

References

You can view the instance type and ES version on the Basic Information page of the instance. For more information, see View basic cluster information.
To purchase an Alibaba Cloud ES instance, see Create an Alibaba Cloud Elasticsearch cluster.
To evaluate the ES specifications and capacity that you need, see Evaluate specifications and storage capacity.