When the size of a vector dataset exceeds the available memory, in-memory indexes such as HNSW and IVF_FLAT are limited by memory capacity and cannot meet both performance and cost requirements. DiskANN offloads index data to disk and can still deliver high search accuracy and query performance when the data size exceeds the memory capacity. It is suitable for vector retrieval scenarios with billions of vectors or more.
Limits
Item | Description |
How to enable the index | Clusters whose Compute Node CU Type is Storage-optimized enable DiskANN by default. For other instances, you can explicitly specify |
Version requirements for deployment modes | Disk mode requires an Alibaba Cloud Milvus cluster of version 2.6.18 or later. Version 2.6.3 supports only performance mode. |
How it works
DiskANN combines the Vamana graph and RaBitQ quantization to achieve efficient vector search on disk.
Vamana graph
The Vamana graph is the core structure of the disk-based strategy of DiskANN. Unlike the multi-layer structure of HNSW, Vamana uses a single-layer sparse graph. It is built with two rounds of pruning, which maintains graph connectivity while introducing more long-range edges, reducing the number of hops required for the search to converge.
The build process is as follows:
Initial random connections: Each vector is a node in the graph. Nodes are initially connected at random to form a dense network, typically with about 500 edges per node, to ensure connectivity.
Two rounds of pruning optimization: Redundant edges are pruned. Low-quality connections are removed based on the distances between nodes, and high-quality edges are preferentially retained. The maximum number of edges per node is limited by
MaxDegree. Long-range edges are introduced to connect data points that are far apart in the vector space, forming shortcuts that speed up navigation on the graph. The breadth of the neighbor search during graph construction is determined bySearchListSize.
Open source DiskANN stores the neighbor list of each node and its full-precision vector in the same disk sector. During a search, a single disk read obtains both the neighbor relationships and the original vector, achieving implicit reranking. However, the entire search process involves a large amount of serial disk I/O. Alibaba Cloud Milvus reorganizes the Vamana graph index in memory so that the search process generates no disk I/O, and original vectors are read from disk only in the final rerank phase.
RaBitQ quantization
RaBitQ (Random Bit Quantization) normalizes vectors and maps them to the vertices of a hypercube. Each dimension requires only 1 bit, which compresses storage overhead and accelerates approximate distance calculation between vectors.
In high-dimensional spaces, the angles between random vectors are highly concentrated, and the error of quantization to hypercube vertices converges at a rate of O(1/√d). Therefore, the higher the dimensionality, the smaller the quantization error. In a 768-dimensional space, the error of 1-bit quantization is already very small. On top of standard 1-bit RaBitQ, Alibaba Cloud Milvus uses a 4-bit extended mode that encodes residual information with 4 bits per dimension to balance compression ratio and accuracy.
Enable the DiskANN index
Step 1: Create a capacity-oriented cluster
When you create an Alibaba Cloud Milvus instance, set the CU Type of Compute Node to Storage-optimized. After the cluster is created, no additional configuration is required, and the DiskANN RaBitQ index type is used by default. If the index type is set to AUTOINDEX when you create a collection, the system automatically matches DiskANN as the underlying index implementation.
For more information about how to create an instance, see Quick start: Create a Milvus instance.
When AUTOINDEX is used, the index type returned by describe_index is still AUTOINDEX, and the automatically matched underlying index is not displayed. To confirm the index type that actually takes effect, query segment-level information. If index_name is DISKANN, DiskANN has been matched:
from pymilvus import connections, utility
connections.connect(uri="http://<endpoint>:19530", token="<user>:<password>")
for seg in utility.get_query_segment_info("<collection_name>"):
print(seg.segmentID, seg.num_rows, seg.index_name)Step 2: Select a deployment mode
The same DiskANN RaBitQ index supports two deployment modes. You can select the loading and search method based on your business requirements:
Deployment mode | Loading behavior | Scenarios |
Performance mode (default) | Both the Vamana graph and the RaBitQ quantization codes are loaded into memory | Online retrieval scenarios that require high QPS and low latency |
Disk mode | The Vamana graph is stored on disk, and only the RaBitQ codes are loaded into memory | Cost-sensitive scenarios that can tolerate a certain decrease in QPS and an increase in latency |
If you do not have an explicit cost reduction requirement, we recommend that you keep the default performance mode. Switch to disk mode only when memory cost is the main constraint and your business can tolerate degraded query performance. Disk mode requires a cluster version of 2.6.18 or later.
Step 3: Plan memory and instance types
Use the following formulas to calculate the theoretical minimum memory. The actual memory usage is slightly higher than the theoretical value, so we recommend that you reserve resources based on the theoretical value × 1.5.
Performance mode:
Memory size = number of entities × (dimensions / 2 + 228) bytesDisk mode:
Memory size = number of entities × (dimensions / 2) bytesTake 100 million 768-dimensional vectors as an example:
Deployment mode | Theoretical memory | Recommended memory (× 1.5) | Recommended instance type |
Performance mode | 100 million × (384 + 228) bytes ≈ 57 GB | About 85 GB | Two 16C64G compute nodes |
Disk mode | 100 million × 384 bytes ≈ 36 GB | About 54 GB | One 16C64G compute node |
Step 4: Switch the deployment mode (optional)
Performance mode is used by default. Switching the deployment mode does not require rebuilding the index. The switch is completed only in the load phase. The process is: release the collection, modify the collection properties, and load the collection again.
During the switch, the collection is released and loaded again. The query capability of this collection is briefly unavailable during this period, so we recommend that you perform the switch during off-peak hours.
Use the following Python script to complete the switch:
from pymilvus import MilvusClient
CLUSTER_ENDPOINT = "http://xxx-internal.milvus.aliyuncs.com:19530"
TOKEN = "<user>:<password>"
COLLECTION_NAME = "<collection_name>"
def set_diskann_mode(mode: str):
client = MilvusClient(uri=CLUSTER_ENDPOINT, token=TOKEN, timeout=7200)
# 1. release collection
client.release_collection(COLLECTION_NAME)
# 2. Modify the deployment mode
client.alter_collection_properties(
COLLECTION_NAME,
properties={"diskann.rabitq_mode": mode},
)
# 3. Load again
client.load_collection(COLLECTION_NAME, timeout=7200)
client.close()
if __name__ == '__main__':
set_diskann_mode("disk") # Switch to disk mode
# set_diskann_mode("perf") # Switch back to performance modeParameters:
Parameter | Description |
| The access endpoint of the Milvus instance. You can use either the internal or public endpoint. Both must include port 19530. |
| The access credential, in the |
| The name of the destination collection. |
| Valid values for the deployment mode: |
After the switch is complete, you can check the current mode by viewing diskann.rabitq_mode in the properties returned by describe_collection.
Parameter configuration
By tuning DiskANN parameters, you can balance speed, accuracy, and memory overhead for a specific dataset and search workload.
The current engine does not reject parameters that exceed the recommended value ranges described below. Entering values that are out of range or that do not satisfy the recommended relationships does not return an error, but may cause abnormal recall or query performance. Make sure that the values are reasonable before you configure them.
Index building parameters
The following parameters affect how the index is built. Adjusting them affects the index size, build time, and search quality.
Parameter | Description | Value | Tuning recommendation |
| Controls the maximum number of connections (edges) for each data point in the Vamana graph. | Integer. Recommended range: [1, 512]. Default value: | The larger the value, the denser the graph and the higher the recall, but memory usage and build time also increase. For most scenarios, we recommend a value in the range of [10, 100]. |
| The size of the candidate pool used to search for the nearest neighbors of each node during index building. For each node added to the graph, a list of the | Integer. We recommend a value that is not less than | The larger the value, the more likely the true nearest neighbors are found for each node, and the better the graph quality and recall. However, the index build time increases significantly. If the value is less than |
Search parameters
The following parameters affect search behavior. Adjusting them affects search speed, latency, and resource consumption.
Parameter | Description | Value | Tuning recommendation |
| The size of the candidate pool maintained when traversing the graph during a search. | Integer. Default value: | The larger the value, the higher the probability of finding the true nearest neighbors (higher recall), but search latency also increases. We recommend that you set it equal to or slightly greater than the number of results to retrieve, |
| Specifies whether to use original vectors for fine ranking during a search. | Boolean. Default value: | In scenarios that require high QPS, you can disable fine ranking. QPS improves significantly but recall decreases. The improvement is especially significant in high |
| The early termination threshold for the search. The search stops when there is no improvement for N consecutive steps. | Integer. Recommended range: (0, 200]. Default value: | The larger the value, the more thorough the search and the higher the recall. The smaller the value, the earlier the search stops, which improves QPS but decreases recall. |
The preceding search parameters take effect only when the data volume is large enough. For a collection with a small amount of data, exact retrieval is performed directly without going through the ANN index. In this case, adjusting search_list, use_refine, or early_termination_threshold has no observable impact on recall or the returned Distance. Parameter tuning should be validated on a dataset close to the production scale.
If a collection has too many segments, query performance is noticeably affected. We recommend that you periodically trigger compaction from the management page or through the client API to reduce the number of segments.