Boost Elasticsearch Query Performance with FalconSeek - Elasticsearch - Alibaba Cloud - Elasticsearch

ES can face performance bottlenecks when handling complex aggregation queries, high-cardinality term queries, and vector search. The FalconSeek cloud-native kernel, built on Alibaba's proprietary Havenask engine, uses C++ columnar memory and a fully asynchronous framework to significantly improve query performance while remaining fully compatible with the ES ecosystem. This document explains how to enable FalconSeek to accelerate queries, covering instance creation, index and field-level configuration, and query strategy management. It also details optimization practices for performance-sensitive scenarios like vector search.

Limitations

FalconSeek is in a limited, invitation-only free preview.
Version limitation: This feature requires Elasticsearch 8.17.0. Upgrading or downgrading the version is not supported.
Instance limitation: You can only enable FalconSeek for newly purchased Vector Enhanced Edition instances. Existing instances are not supported.

Create an instance and enable FalconSeek

You can only enable the FalconSeek feature when creating a new instance.

On the instance purchase page, enable FalconSeek.
- Cluster Type: You must select Vector Enhanced Edition.
- Cluster Type: You must select 8.17.0 (Recommended for vector scenarios).
- Advanced Enhanced Features: Select FalconSeek Cloud-Native Kernel.
Configure other parameters, such as VPC and Cluster Specification, based on your business needs, and then complete the payment.
After the instance is created, run the following command in the Kibana console to confirm that FalconSeek has been enabled at the cluster level.
```
GET _cluster/settings?include_defaults&filter_path=defaults.havenask.engine.enabled
```
If the enabled field in the response is true, the feature has been successfully enabled.
```
{
  "defaults": {
    "havenask": {
      "engine": {
        "enabled": "true"
      }
    }
  }
}
```

Use FalconSeek in an index

After you enable FalconSeek for an instance, it is not enabled on any index by default. You must explicitly enable the engine for each index you want to accelerate in the index settings.

Enable FalconSeek for a new index

When you create an index, add "index.havenask.engine.enabled": true to settings to enable FalconSeek.

PUT falcon_seek_test
{
  "settings": {
    "index.havenask.engine.enabled": true    // Enable the FalconSeek feature
  },
  "mappings": {
    "properties": {
      "foo": {
        "type": "keyword"
      }
    }
  }
}

Ingest data.

POST falcon_seek_test/_bulk
{"index":{}}
{"foo":"hello"}
{"index":{}}
{"foo":"world"}
{"index":{}}
{"foo":"cpp"}
{"index":{}}
{"foo":"java"}

Query the data. By default, the query uses FalconSeek.

GET falcon_seek_test/_search
{
  "query": {
    "term":{
      "foo":{
        "value": "hello"
      }
    }
  },
  "sort": [
    {
      "_doc": {
        "order": "desc"
      }
    }
  ]
}

Enable FalconSeek for an existing index

The following command creates an index but does not enable acceleration.

PUT my_existing_index
{
  "settings": {
    "number_of_shards": 5,
    "number_of_replicas": 1
  },
  "mappings": {
    "properties": {
      "foo": {
        "type": "keyword"
      }
    }
  }
}

Use the _settings API to update the index configuration to dynamically enable FalconSeek.
```
PUT my_existing_index/_settings
{
  "index.havenask.engine.enabled": true
}
```
For the change to take effect, you must close the index by running POST /my_existing_index/_close and then reopen it by running POST /my_existing_index/_open.

Manage query execution strategies

FalconSeek provides flexible query execution strategies that allow you to control whether a request is processed by FalconSeek or falls back to the Elasticsearch native engine. You can configure this at the index level with index.havenask.engine.search.type or at the per-query level with havenask_search_type. The query-level parameter takes precedence.

Strategy	Description	Recommended use case
`native` (default)	Queries are first processed by FalconSeek. If an unsupported syntax or feature is encountered, the request automatically falls back to the Elasticsearch native engine.	General production workloads that require both performance and compatibility.
`es`	Forces queries to be executed by the Elasticsearch native engine.	For performance comparisons or to bypass unknown issues in FalconSeek.
`native-direct`	Forces the FalconSeek engine to execute queries. If an unsupported syntax or feature is encountered, the system returns an error instead of falling back.	Debugging.

Set the index-level query strategy

The following example sets all queries on my_index to use the Elasticsearch native engine by default.

PUT my_index/_settings
{
  "index.havenask.engine.search.type": "es"
}

Override the strategy per query

Assume my_index uses the native strategy by default. To force a specific query to use the Elasticsearch native engine, add the havenask_search_type parameter to the request.

Method 1: As a URL parameter

GET my_index/_search?havenask_search_type=es
{
  "query": {
    "match_all": {}
  }
}

Method 2: As a request body parameter

GET my_index/_search
{
  "query": {
    "match_all": {}
  },
  "havenask_search_type": "es"
}

Use FalconSeek to accelerate vector search

FalconSeek adds a C++-based vector index on top of the existing Elasticsearch index structure. This proprietary vector index, developed by Alibaba, powers core services across Alibaba Group, including search, recommendations, and reverse image search on Taobao and Tmall. Use the high-performance vector index in the FalconSeek cloud-native kernel to build efficient AI applications like reverse image search and semantic search.

Create a vector index. In the mappings, define a field of type dense_vector named image_vector to store image or text vectors. Ensure that FalconSeek is enabled for this index.

PUT vector_index
{
  "settings": {
    "index.havenask.engine.enabled": true
  },
  "mappings": {
    "properties": {
      "image_vector": {
        "type": "dense_vector",
        "dims": 128 ,
        "index_options": {
          "type": "havenask_native"
        }
      },
      "title": {
        "type": "keyword"
      }
    }
  }
}

Ingest vector data. Use the _bulk or _index API to write documents. The vector field value must be an array of floating-point numbers.

POST vector_index/_bulk
{"index":{"_id":"1"}}
{"image_vector":[0.12, -0.05, 0.08, 0.24, -0.17, 0.31, 0.02, -0.19, 0.11, 0.28,
    -0.03, 0.15, 0.22, -0.11, 0.09, 0.33, -0.07, 0.14, 0.26, -0.21,
    0.18, 0.29, -0.13, 0.06, 0.35, -0.08, 0.16, 0.23, -0.15, 0.12, 
    0.27, -0.22, 0.19, 0.32, -0.14, 0.07, 0.25, -0.18, 0.13, 0.30,
    -0.09, 0.17, 0.24, -0.16, 0.10, 0.34, -0.10, 0.20, 0.31, -0.23,
    0.15, 0.28, -0.12, 0.11, 0.26, -0.19, 0.14, 0.29, -0.17, 0.08,
    0.22, -0.20, 0.16, 0.27, -0.15, 0.09, 0.25, -0.21, 0.18, 0.30,
    -0.13, 0.07, 0.24, -0.22, 0.19, 0.32, -0.16, 0.10, 0.26, -0.18,
    0.12, 0.28, -0.14, 0.06, 0.23, -0.19, 0.15, 0.29, -0.11, 0.05,
    0.21, -0.17, 0.13, 0.27, -0.10, 0.04, 0.20, -0.15, 0.11, 0.25,
    -0.09, 0.03, 0.19, -0.13, 0.10, 0.24, -0.08, 0.02, 0.18, -0.12,
    0.09, 0.23, -0.07, 0.01, 0.17, -0.11, 0.08, 0.22, 0.05, -0.06,
    0.09, 0.23, -0.07, 0.01, 0.17, -0.11, 0.08, 0.22],"title":"Image One"}
{"index":{"_id":"2"}}
{"image_vector":[0.12, -0.05, 0.08, 0.24, -0.17, 0.31, 0.02, -0.19, 0.11, 0.28,
    -0.03, 0.15, 0.22, -0.11, 0.09, 0.33, -0.07, 0.14, 0.26, -0.21,
    0.18, 0.29, -0.13, 0.06, 0.35, -0.08, 0.16, 0.23, -0.15, 0.12, 
    0.27, -0.22, 0.19, 0.32, -0.14, 0.07, 0.25, -0.18, 0.13, 0.30,
    -0.09, 0.17, 0.24, -0.16, 0.10, 0.34, -0.10, 0.20, 0.31, -0.23,
    0.15, 0.28, -0.12, 0.11, 0.26, -0.19, 0.14, 0.29, -0.17, 0.08,
    0.22, -0.20, 0.16, 0.27, -0.15, 0.09, 0.25, -0.21, 0.18, 0.30,
    -0.13, 0.07, 0.24, -0.22, 0.19, 0.32, -0.16, 0.10, 0.26, -0.18,
    0.12, 0.28, -0.14, 0.06, 0.23, -0.19, 0.15, 0.29, -0.11, 0.05,
    0.21, -0.17, 0.13, 0.27, -0.10, 0.04, 0.20, -0.15, 0.11, 0.25,
    -0.09, 0.03, 0.19, -0.13, 0.10, 0.24, -0.08, 0.02, 0.18, -0.12,
    0.09, 0.23, -0.07, 0.01, 0.17, -0.11, 0.08, 0.22, 0.05, -0.06,
    0.09, 0.23, -0.07, 0.01, 0.17, -0.11, 0.08, 0.22],"title":"Image Two"}

Perform a k-NN vector search. Use the knn query clause to find the top K results that are most similar to a given vector.

 GET vector_index/_search
{
  "knn": {
    "field": "image_vector",
    "query_vector": [0.12, -0.05, 0.08, 0.24, -0.17, 0.31, 0.02, -0.19, 0.11, 0.28,
    -0.03, 0.15, 0.22, -0.11, 0.09, 0.33, -0.07, 0.14, 0.26, -0.21,
    0.18, 0.29, -0.13, 0.06, 0.35, -0.08, 0.16, 0.23, -0.15, 0.12, 
    0.27, -0.22, 0.19, 0.32, -0.14, 0.07, 0.25, -0.18, 0.13, 0.30,
    -0.09, 0.17, 0.24, -0.16, 0.10, 0.34, -0.10, 0.20, 0.31, -0.23,
    0.15, 0.28, -0.12, 0.11, 0.26, -0.19, 0.14, 0.29, -0.17, 0.08,
    0.22, -0.20, 0.16, 0.27, -0.15, 0.09, 0.25, -0.21, 0.18, 0.30,
    -0.13, 0.07, 0.24, -0.22, 0.19, 0.32, -0.16, 0.10, 0.26, -0.18,
    0.12, 0.28, -0.14, 0.06, 0.23, -0.19, 0.15, 0.29, -0.11, 0.05,
    0.21, -0.17, 0.13, 0.27, -0.10, 0.04, 0.20, -0.15, 0.11, 0.25,
    -0.09, 0.03, 0.19, -0.13, 0.10, 0.24, -0.08, 0.02, 0.18, -0.12,
    0.09, 0.23, -0.07, 0.01, 0.17, -0.11, 0.08, 0.22, 0.05, -0.06,
    0.09, 0.23, -0.07, 0.01, 0.17, -0.11, -0.08,-0.22],
    "k": 10,
    "num_candidates": 100
  },
  "fields": ["title"]
}

For more information about using the FalconSeek vector index, see the FalconSeek Vector Index User Guide.

How it works

The FalconSeek cloud-native kernel replaces parts of the underlying Lucene components in Elasticsearch with a highly optimized C++ engine. When you send a query, it first reaches the FalconSeek plugin layer. This layer determines whether the FalconSeek engine can process the query:

Supported queries: The request is forwarded to the high-performance FalconSeek engine for execution.
Unsupported queries: Depending on your configured strategy, the query automatically falls back to the Elasticsearch native engine, or the system returns an error.

This mechanism ensures high performance while maintaining compatibility and stability.

FAQ

Q: Can I enable FalconSeek on an existing Elasticsearch instance?

A: No. FalconSeek can only be enabled for newly purchased instances. For existing instances, we recommend creating a new, compliant instance and migrating your data using a tool like Data Transmission Service (DTS) or Logstash.

Q: What happens if I run a query that FalconSeek does not support?

A: It depends on your query execution strategy. In the default native mode, the query automatically falls back to the Elasticsearch native engine. You will still get the correct result, but without the performance acceleration from FalconSeek. In native-direct mode, the system will return an error.