DiskBBQ is an index type for vector fields. Standard indexes and Collections share the same bbq_disk field configuration. When a vector field of this type is searched within a Slice scope in a Collection, Sliced DiskBBQ is used, and no additional set of Mapping parameters is required. The following sections describe the DiskBBQ field configuration and KNN query parameters and explain how to read vector values on demand. To get started, work through the complete example for a standard index. The following table provides a quick reference to the request endpoints.
Operation | Standard index | Collection |
Add a vector field |
|
|
Query nearest neighbors |
|
|
The dimensions of document vectors and query vectors must match dims. The examples in the following reference sections target a Collection; if the target is a standard index, use the actual index name. The _slice parameter applies to Collection queries only: use _slice to specify the Slice to search, or let the scope be selected automatically based on the vector cluster recall mode. The complete example at the end of this topic uses a standard index.
For information about the benefits, scenarios, and trade-offs of the two-tier capability, see DiskBBQ two-tier capability.
Field configuration and query parameters
Configure a mapping
Getting Started already configures DiskBBQ when you create a Collection. For other Collections that do not yet have a vector field, use the Mapping API to add one:
PUT /{collection}/_mapping
{
"properties": {
"embedding": {
"type": "dense_vector",
"dims": 4,
"index": true,
"similarity": "cosine",
"index_options": {
"type": "bbq_disk"
}
}
}
}When you explicitly specify "type": "bbq_disk", the server supplies the following default index_options values. These default values are independent of the vector dimensions:
{
"index_options": {
"type": "bbq_disk",
"cluster_size": 384,
"flat_index_threshold": -1,
"default_visit_percentage": 0.0,
"rescore_vector": { "oversample": 3.0 },
"bits": 1
}
}default_visit_percentage is the Mapping-level default value. It is used when the query-level visit_percentage is not set. The query-level parameter takes precedence. rescore_vector.oversample can be overridden in a query. cluster_size, flat_index_threshold, and bits are maintained by the server and do not need to be adjusted.
Keep the vector dimensions and similarity consistent with the model used to generate the vectors. To switch the index type of an existing vector field, create the target Collection, configure a new Mapping, and then use Reindex to migrate the data and run the evaluation again.
Run a KNN query
In a query, use knn.field to reference the vector field configured in the Mapping. The following request uses visit_percentage to adjust the access scope of DiskBBQ within the specified Slice:
POST /knowledge-chunks/_search?_slice=tenant-a
{
"knn": {
"field": "embedding",
"query_vector": [0.80, 0.12, 0.05, 0.03],
"k": 10,
"visit_percentage": 10.0
},
"_source": ["document_id", "title", "content"]
}Parameter | Description |
| The number of nearest neighbors selected globally. The actual number of returned results is also limited by the top-level |
| The percentage of vectors visited on each shard. Valid values: 0 to 100. A larger value typically improves recall but also increases computing and read overhead. A value explicitly set in the query overrides the Mapping default. When the final effective value is greater than |
| When the final effective |
| Adds structured filtering to reduce irrelevant candidates. |
| Uses the original vectors to rescore the candidates returned by quantized retrieval. It can override the default |
By default, the effective value of visit_percentage is 0, so num_candidates determines the candidate visit scope. To control the visit scope by percentage, set default_visit_percentage in the Mapping or visit_percentage in the query. When you use vector cluster recall, you must also configure the scope of candidate Slices. For more information, see Vector cluster recall reference.
Read vector values on demand
For regular searches, keep the default setting that does not return vectors, and use only _source to select the fields your application needs, such as the title and content in the preceding example. KNN retrieval and rescoring are performed on the server, so vectors do not need to be returned to your application.
Explicitly retrieve the required fields only when you need to verify or export vector values. For example, read the embedding field of a document:
GET /knowledge-chunks/_doc/chunk-1001?_slice=tenant-a&_source_exclude_vectors=falseUse _source_exclude_vectors=false to explicitly retrieve vectors. This parameter returns both the business fields and the vectors in _source. Returning vectors increases data read, reconstruction, and network transmission overhead, so use this parameter only when you need to verify or export vectors.
The returned values depend on the storage precision of the vector field. The example uses float (float32) by default. Floating-point rounding can occur both when the written decimal values are converted to float32 and during the normalization and restoration performed for cosine similarity. If the field uses element_type: bfloat16, the storage precision is lower and the error is more noticeable than with float32. When DiskBBQ retrieves vectors, it uses the retained vector values rather than deriving the original values from the quantized encoding. If you need the original model output for bit-level verification or precision evaluation, store the original vectors separately.
Complete example for a standard index
Your account must have the permissions required to create indexes and to write and read data. After you create and connect to an Agent Engine Edition instance, run the following requests in sequence in Kibana Dev Tools. This example uses the standard index vector-demo and manually constructed four-dimensional vectors only to verify the configuration and query workflow.
Step 1: Create a vector index
PUT /vector-demo
{
"settings": {
"number_of_shards": 1,
"number_of_replicas": 2
},
"mappings": {
"properties": {
"title": { "type": "text" },
"embedding": {
"type": "dense_vector",
"dims": 4,
"similarity": "cosine",
"index": true,
"index_options": { "type": "bbq_disk" }
}
}
}
}Use an index name that does not exist yet. In production scenarios, keep the vector dimensions and similarity consistent with the model output. The shard and replica values in this example are for demonstration only. For more information, see Node and replica settings.
Step 2: Write data and verify query visibility
POST /vector-demo/_bulk
{"index":{"_id":"1"}}
{"title":"Knowledge base search","embedding":[0.80,0.12,0.05,0.03]}
{"index":{"_id":"2"}}
{"title":"Product search","embedding":[0.10,0.75,0.10,0.05]}For regular writes to a standard index, do not pass refresh, and do not call _refresh in your application code. Verify that errors=false is returned in the Bulk response, and then wait about 5 seconds for the automatic background refresh to make the data searchable. If the results do not contain the new documents yet, wait for another automatic refresh cycle and try again. For information about the applicable conditions for verifying writes by ID and for synchronous waiting, see Refresh and query visibility.
Step 3: Query similar vectors
POST /vector-demo/_search
{
"knn": {
"field": "embedding",
"query_vector": [0.80, 0.12, 0.05, 0.03],
"k": 2,
"visit_percentage": 10.0
},
"_source": ["title"]
}Verify that timed_out=false and _shards.failed=0, and make sure that hits.hits contains the two documents titled "Knowledge base search" and "Product search". This example verifies only the request workflow. Evaluate recall and performance by using your actual vectors, query sets, and concurrent loads.