Vector search is a key technology for efficient similarity search. This topic describes how to quickly implement vector search with detailed examples.
Prerequisites
The PyMilvus library is installed on your local client and is updated to the latest version.
To install or update the PyMilvus library, run the following command.
pip install --upgrade pymilvusA Milvus instance has been created. For more information, see Create a Milvus instance.
Notes
Alibaba Cloud AISearch for Milvus (Milvus) supports connections over an internal network and the Internet. Before you connect to a Milvus instance, make sure that your client has the required network access permissions. For more information, see Network access and security settings.
Procedure
Step 1: Connect to a Milvus instance
You can use the following code to connect to a Milvus instance.
from pymilvus import MilvusClient
# Create a Milvus client.
client = MilvusClient(
uri="http://c-xxxx.milvus.aliyuncs.com:19530", # The public endpoint of the Milvus instance.
token="<yourUsername>:<yourPassword>", # The username and password to log on to the Milvus instance.
db_name="default" # The name of the database to connect to. This example uses the default database.
)
Step 2: Create a collection
You can use the following code to create a collection. For more custom parameter options, see Manage collections.
client.create_collection(
collection_name="demo", # The name of the collection.
dimension=5 # The vector dimensions.
)This code sets the collection name and vector dimensions. It also automatically applies the following configurations:
Uses the default names for the primary key field "id" and the vector field "vector".
The "metric_type" property uses the COSINE metric type by default.
Sets the primary key field "id" as an integer. Its value does not automatically increment.
Adds an extra "$meta" field to store data for fields not defined in the schema as key-value pairs.
Step 3: Insert data
After you create a collection, the system automatically loads the collection and its index into memory. You can use the following code to insert test data into the collection.
Insert a small amount of data
This code inserts 10 predefined entities. Each entity has a fixed vector and color tag.
data=[{'id': 0, 'vector': [-0.493313706583155, -0.172001225836391, 0.16825615330139554, -0.0198911518739604, -0.9756816265213708], 'color': 'green_5760'}, {'id': 1, 'vector': [0.6695699219225086, 0.49952523907354496, -0.49870548178008534, 0.8824655547230731, -0.7182693622931615], 'color': 'blue_2330'}, {'id': 2, 'vector': [-0.6057771959702387, 0.9141473782193543, 0.32053983678483466, -0.32126010092015655, 0.725222856037071], 'color': 'grey_9673'}, {'id': 3, 'vector': [0.14082089434165868, 0.9924029949938447, 0.7943279666144052, -0.7898608705081103, -0.9941425813199956], 'color': 'white_2829'}, {'id': 4, 'vector': [-0.46180540826224026, 0.33216876051895783, 0.5786699695956004, 0.8891120357625131, 0.04872530176990697], 'color': 'pink_9061'}, {'id': 5, 'vector': [-0.6097452740606673, 0.35648319550551144, -0.5699789153006387, 0.15085357921088316, -0.8817226997144627], 'color': 'pink_8525'}, {'id': 6, 'vector': [0.7843522543512762, -0.7663837586858071, -0.8681839054724569, 0.6880645348647785, -0.5151293183261791], 'color': 'green_5016'}, {'id': 7, 'vector': [-0.9967116931989293, 0.5741923070732655, -0.019126124261334976, -0.34163875885482753, -0.8189843931354175], 'color': 'brown_7434'}, {'id': 8, 'vector': [0.7347243385915765, -0.7358853080124825, -0.23737428377511716, 0.06980552357261627, -0.30613964550461437], 'color': 'blue_5059'}, {'id': 9, 'vector': [-0.21187155428455862, -0.3288541717216129, -0.32564136453418824, -0.14054963599686743, 0.5491320339870627], 'color': 'yellow_9887'}]
res = client.insert(
collection_name="demo",
data=data
)
Insert more data
This code uses a list comprehension to dynamically generate many entities. The vectors and color tags of these entities are randomly generated within a specified range.
import random
colors = ["green", "blue", "yellow", "red", "black", "white", "purple", "pink", "orange", "brown", "grey"]
data = [ {
"id": i,
"vector": [ random.uniform(-1, 1) for _ in range(5) ],
"color": f"{random.choice(colors)}_{str(random.randint(1000, 9999))}"
} for i in range(1000) ]
res = client.insert(
collection_name="demo",
data=data[1:]
)
print(res)Step 4: Vector search
Data insertion is an asynchronous process. After you insert data, the system does not immediately update the search index. To query the latest data, you must wait a few seconds for the index to update before you search.
Single vector search
To perform a similarity search, you can provide a list that contains a single query vector.
query_vectors = [
[-0.8832567462711804, -0.2999882617491647, 0.9921295273224382, -0.272575369985379, -0.688914679645338]
]
res = client.search(
collection_name="demo", # The collection to query
data=query_vectors, # The query vectors.
limit=3, # The number of entities to return.
)
print(res)
Batch vector search
For a batch search with multiple query vectors, you can use a list of vectors as the input parameter. The following code provides an example.
query_vectors = [
[0.06586461994037252, 0.7693023529849932, 0.8199991781350795, -0.6988017611187176, 0.408383847889378],
[0.8988257992203861, 0.021911711196309414, 0.19086900086430836, 0.63590610476426, -0.6713237387993141]
]
res = client.search(
collection_name="demo",
data=query_vectors,
limit=3,
)
print(res)Step 5: Filtered search
You can use the fields defined in the schema to set filter conditions. This narrows the search scope and improves search efficiency.
Filter by a numeric field
The following example shows how to filter based on the value range of the id field.
query_vectors = [
[-0.30932351869632435, -0.7132856078639205, 0.6006201320181415, 0.40140510356426784, -0.21223937444001328]
]
res = client.search(
collection_name="demo",
data=query_vectors,
filter="3 < id < 5", # The filter condition for a numeric field range.
limit=3
)
print(res)
Filter by a metadata field ($meta)
The following example shows how to search for records where the color property value starts with "green". It also specifies that the color field is included in the output.
query_vectors = [
[0.9636568288732006, -0.5900490884830603, 0.2504591754023724, 0.7120903924474389, 0.7620604497390009]
]
res = client.search(
collection_name="demo",
data=query_vectors,
filter='$meta["color"] like "green%"', # The filter condition to match a metadata field property value.
limit=3,
output_fields=["color"] # The fields to include in the returned results.
)
print(res)