Manage vector indexes - Vector Retrieval Service for Milvus

Vector indexes use advanced index schemas and algorithms, such as Inverted File (IVF) and Hierarchical Navigable Small World (HNSW), to compress the vector space and locate the most similar data points in large datasets. By creating indexes on vector fields, you can significantly improve recall rates and response speeds for applications such as image recognition, voice retrieval, and recommendation systems.

Background information

Milvus supports multiple index types for efficient similarity retrieval and provides three distance metrics: Cosine Similarity (COSINE), Euclidean Distance (L2), and Inner Product (IP). You can create indexes on frequently queried vector and scalar fields to optimize retrieval performance.

Prerequisites

The PyMilvus library is installed on your local client and is updated to the latest version.
To install or update the PyMilvus library, run the following command.
```
pip install --upgrade pymilvus
```
A Milvus instance has been created. For more information, see Create a Milvus instance.

Preparations

Before managing indexes, create a collection. There are two approaches:

Option 1: Create a collection with indexes

To create and load an index along with the collection:

Declare the dimension of the vector field (dimension)
Provide index-related parameters (including metric_type and index configuration)

Option 2: Create a collection without an index

The following code snippet uses this approach. The created collection does not contain an index and is not automatically loaded into memory:

from pymilvus import MilvusClient, DataType

client = MilvusClient(
    uri="http://c-xxxx.milvus.aliyuncs.com:19530",  # The public endpoint of the Milvus instance.
    token="<yourUsername>:<yourPassword>",  # The username and password for the Milvus instance.
    db_name="default"  # The name of the database to connect to. This example uses the default database.
)

schema = MilvusClient.create_schema(
    auto_id=False,
    enable_dynamic_field=True,
)

schema.add_field(field_name="id", datatype=DataType.INT64, is_primary=True)
schema.add_field(field_name="vec", datatype=DataType.FLOAT_VECTOR, dim=5)

client.create_collection(
    collection_name="<yourCollectionName>",
    schema=schema,
)

Create an index

To create an index, call the create_index function and pass in the defined index_params.

index_params = MilvusClient.prepare_index_params()
# Define the index parameters.
index_params.add_index(
    field_name="vec",          # Specify the vector field name, such as "vec".
    metric_type="L2",          # Set the metric type, such as L2.
    index_type="IVF_PQ",       # Set the index type, such as IVF_PQ.
    index_name="vector_index"  # Set the index name as needed.
)

# Create the index.
client.create_index(
    collection_name="<yourCollectionName>",
    index_params=index_params
)

View index

res = client.describe_index(
    collection_name="<yourCollectionName>",
    index_name="<yourIndexName>"
)

print(res)

Delete an index

client.drop_index(
    collection_name="<yourCollectionName>",
    index_name="<yourIndexName>"
)