All Products
Search
Document Center

Vector Retrieval Service for Milvus:Alibaba Cloud Milvus AI Center: one-stop text and multimodal vectorization

Last Updated:Oct 19, 2026

Alibaba Cloud Milvus AI Center is a managed model service capability built into Milvus. After you enable it, you can pass in raw text or multimodal data (images and videos) directly when writing and retrieving data, and the system automatically generates the vectors. You do not need to deploy a separate Embedding inference service.

Feature overview

image

Feature

Description

One-stop management in the console

View and configure model services and obtain call examples in the console, without switching between multiple platforms.

Managed model capabilities

The platform provides managed model inference services, so you do not need to build your own inference service or maintain the underlying infrastructure.

Direct write and retrieval of raw data

Pass in raw text or multimodal content directly when inserting, updating, and querying data, and the system completes the vectorization automatically.

Flexible switching between multiple models

Multiple vectorization models are available, so you can flexibly balance effectiveness, latency, and cost based on your business scenario.

Call volume and token statistics

Token usage and the number of calls are counted by two dimensions: cluster and model.

Call monitoring

You can view operational metrics such as the request Success Rate and the Average RT (average response time).

In addition to vectorization, the AI Center provides unified model capabilities such as reranking, text generation, multimodal understanding, and video editing, which can be used directly in scenarios such as semantic retrieval, RAG knowledge bases, enterprise intelligent Q&A, content recommendation, and multimodal search. This topic uses vectorization as an example to describe the complete integration process for text retrieval and multimodal retrieval.

Procedure

View the available vectorization models

Log on to the Milvus console. In the left-side navigation pane, choose AI Center. On the Model Services tab, view the Vectorization group, where you can obtain the input type and output dimensions of each model as well as the API example.

The following vectorization models are currently available:

Model

Supported input

Description

qwen3.7-text-embedding

Text

The recommended text vector model. The default number of dimensions is 1024.

text-embedding-v4

Text

A multilingual text vector model that supports custom vector dimensions.

text-embedding-v3

Text

A general-purpose text vector model.

text-embedding-v2

Text

A general-purpose text vector model. It will be discontinued soon and is not recommended for new business.

qwen3-vl-embedding

Text, images, and videos

The recommended multimodal vector model. The default number of dimensions is 2560. You can specify other dimensions by using dim.

tongyi-embedding-vision-plus

Text, images, and videos

A multimodal vector model.

Note

The AI Center provides services independently by region. After you switch to another region, you must view and configure the services again in the new region.

Prepare a Milvus instance

The vectorization capability requires a Milvus 2.6 instance. After you create a 2.6 instance, you can declare a Function in a collection to call a model directly. You do not need to separately bind the model to the instance.

To access the instance over the Internet, go to the Security Configuration tab on the instance details page, enable Internet Access, and configure an Internet access whitelist.

View usage and monitoring data

Go to AI Center and click the Monitoring tab. You can view call information from two perspectives: By Milvus Cluster or By Model:

  • Token usage / number of calls: lists the token usage and call count details by cluster and by model.

  • Total tokens and total text input tokens: the aggregated token consumption within the selected scope.

  • Success Rate: the request success rate.

  • Average RT: the average response time.

You can select a cluster or a model, set the sampling interval (interval), and filter by quick ranges such as 1 hour, 3 hours, 6 hours, 12 hours, 1 day, and 7 days, or by a custom time window.

Case 1: text-to-text semantic retrieval

Scenario description

Write a batch of raw text into Milvus without generating vectors in advance. During a query, enter a natural language question directly, and Milvus automatically vectorizes the query and returns the most semantically relevant results. This applies to scenarios such as knowledge base Q&A, document search, and FAQ retrieval.

Procedure

  1. Create a collection and define the raw text field document and the vector field dense.

  2. Bind the qwen3.7-text-embedding model by using Function.

  3. Insert test text data and call flush to ensure the data is retrievable.

  4. Initiate retrieval with a natural language question to return the most relevant text snippets.

Note

After inserting data, you must call flush. Otherwise, a retrieval performed immediately afterwards may return empty results.

Sample code

from pymilvus import MilvusClient, DataType, Function, FunctionType

client = MilvusClient(
    uri="http://c-xxxx.milvus.aliyuncs.com:19530",
    token='root:xxx',
)

# ========== Create the collection ==========
collection_name = 'demo1'
schema = client.create_schema()

schema.add_field("id", DataType.INT64, is_primary=True, auto_id=False)
schema.add_field("document", DataType.VARCHAR, max_length=9000)
schema.add_field("dense", DataType.FLOAT_VECTOR, dim=1024)
text_embedding_function = Function(
    name="dashscope_api_test123",
    function_type=FunctionType.TEXTEMBEDDING,
    input_field_names=["document"],
    output_field_names=["dense"],
    params={
        "provider": "aliyun_milvus",
        "model_name": "qwen3.7-text-embedding"
    }
)

schema.add_function(text_embedding_function)
index_params = client.prepare_index_params()

index_params.add_index(
    field_name="dense",
    index_type="AUTOINDEX",
    metric_type="COSINE"
)
client.drop_collection(collection_name)
client.create_collection(
    collection_name=collection_name,
    schema=schema,
    index_params=index_params
)

insert_data = []
record_id = 1

mock_texts = [
    "A vector database converts text into high-dimensional vectors to enable semantic retrieval and understand the real intent of a query.",
    "Deep learning models can learn feature representations from large volumes of image data for tasks such as image classification and object detection.",
    "Retrieval-augmented generation recalls relevant documents to provide external knowledge to large models, reducing hallucinations and improving answer reliability.",
    "In customer service scenarios, semantic search can quickly locate historical tickets, improving first-response efficiency and the issue resolution rate.",
    "Similarity search often uses cosine distance to measure how close two vector directions are, which suits text semantic matching tasks.",
    "Data cleansing is an important step before vectorization; removing noise and unifying formats can significantly improve recall quality.",
    "A higher embedding dimension is not always better; you need to balance effectiveness, latency, and storage cost.",
    "Splitting long documents into small segments with a chunking strategy can improve the retrieval hit rate and reduce context redundancy.",
    "Adding a source field to each piece of knowledge improves the explainability of results and makes it easier to display citations in the frontend.",
    "Multilingual retrieval can achieve cross-language matching through a unified vector space, improving the experience of internationalized systems.",
    "Index parameters such as ef and M affect the recall rate and query performance of HNSW and must be tuned based on the business.",
    "Running offline evaluations before release can quantify the retrieval quality differences between different models and parameter combinations.",
    "Metadata filtering in a vector store can be combined with semantic recall to achieve more precise scoped search.",
    "For high-frequency questions, you can cache retrieval results for a short time to reduce backend computing pressure and response time.",
    "The semantic retrieval pipeline should log queries so that zero-result queries can be analyzed and the knowledge base continuously optimized.",
    "When knowledge content is updated frequently, an incremental indexing strategy can reduce the resource consumption of full rebuilds.",
    "Adding a reranking model to a Q&A system can improve the relevance and readability of the top results.",
    "Establishing access control policies for sensitive data is a basic security requirement for enterprise-grade vector retrieval systems.",
    "Setting topK properly balances coverage and noise, avoiding returning too many low-relevance results.",
    "Feeding user feedback back into the training and evaluation process continuously improves the overall retrieval and Q&A effectiveness.",
]

for text in mock_texts:
    insert_data.append({
        "id": record_id,
        "document": text,
    })
    record_id += 1

BATCH_SIZE = 30
print(f"Preparing to insert {len(insert_data)} records, batch_size={BATCH_SIZE}...")
for batch_start in range(0, len(insert_data), BATCH_SIZE):
    batch = insert_data[batch_start:batch_start + BATCH_SIZE]
    client.insert(collection_name, batch)
    print(f"  Inserted {min(batch_start + BATCH_SIZE, len(insert_data))}/{len(insert_data)} records")
client.flush(collection_name)
print("Data insertion completed!\n")

# ========== Test 1: text-to-text (pure text semantic retrieval through the dense field) ==========
print("=" * 60)
print("Test 1: Text-to-text - query content related to vector databases")
print("=" * 60)
results = client.search(
    collection_name=collection_name,
    data=['What is semantic retrieval? How does a vector database work?'],
    anns_field='dense',
    limit=3,
    output_fields=['document'],
)
for hits in results:
    for hit in hits:
        print(f"  id={hit['id']}, distance={hit['distance']:.4f}, document={hit['entity']['document'][:50]}")

Running result

Preparing to insert 20 records, batch_size=30...
  Inserted 20/20 records
Data insertion completed!

============================================================
Test 1: Text-to-text - query content related to vector databases
============================================================
  id=1, distance=0.8668, document=A vector database converts text into high-dimensio
  id=13, distance=0.5989, document=Metadata filtering in a vector store can be combin
  id=15, distance=0.5414, document=The semantic retrieval pipeline should log queries

The retrieval results are sorted by semantic relevance, and the text closest to the query intent ranks first. The actual similarity scores vary with the model version and the data content.

Case 2: multimodal retrieval

Scenario description

In retail, e-commerce, and content platform scenarios, the retrieval objects include images and videos in addition to text. The multimodal vector models of the AI Center support incorporating text and visual content into a unified vectorization and retrieval process, enabling cross-modal retrieval capabilities such as text-to-image/video and image-to-image/video search.

This case defines two vector fields in the same collection: dense, which is generated by the text model and used for text semantic retrieval, and dense_mm, which is generated by the multimodal model and used for cross-modal retrieval.

Procedure

  1. Upload the images or videos to object storage such as OSS and generate URLs that can be accessed by the model service.

  2. Create a collection and define the text field document, the multimedia address field url, the text vector field dense, and the multimodal vector field dense_mm.

  3. Bind qwen3.7-text-embedding and qwen3-vl-embedding to the text field and the multimedia field respectively.

  4. Insert the test materials and their corresponding text descriptions.

  5. Perform retrieval verification such as text-to-text, text-to-image/video, and image-to-image/video search.

Important

When you bind a multimodal model to a multimedia field, you must set is_multimodal to true in params. Otherwise, an error is reported when you create the collection because the model cannot parse the input. The dim value must be consistent with the number of dimensions declared for the vector field.

You can also write plain text directly into a multimedia field. The multimodal model generates a text vector in the same vector space for it, which supports mixed image-and-text retrieval.

Sample code

from pymilvus import MilvusClient, DataType, Function, FunctionType

client = MilvusClient(
    uri="http://c-xxxx.milvus.aliyuncs.com:19530",
    token='root:xxx',
)

# Replace with your own image or video address, which must be accessible to the model service
IMAGE_URL = "https://example.com/your-image.jpg"
VIDEO_URL = "https://example.com/your-video.mp4"

# ========== Create the collection ==========
collection_name = 'demo11'
schema = client.create_schema()

schema.add_field("id", DataType.INT64, is_primary=True, auto_id=False)
schema.add_field("document", DataType.VARCHAR, max_length=9000)
schema.add_field("url", DataType.VARCHAR, max_length=9000)
schema.add_field("dense", DataType.FLOAT_VECTOR, dim=1024)
schema.add_field("dense_mm", DataType.FLOAT_VECTOR, dim=1024)

text_embedding_function = Function(
    name="dashscope_api_test123",
    function_type=FunctionType.TEXTEMBEDDING,
    input_field_names=["document"],
    output_field_names=["dense"],
    params={
        "provider": "aliyun_milvus",
        "model_name": "qwen3.7-text-embedding"
    }
)
mm_embedding_function = Function(
    name="dashscope_api_mm123",
    function_type=FunctionType.TEXTEMBEDDING,
    input_field_names=["url"],
    output_field_names=["dense_mm"],
    params={
        "provider": "aliyun_milvus",
        "model_name": "qwen3-vl-embedding",
        "dim": "1024",
        "is_multimodal": "true"
    }
)

schema.add_function(text_embedding_function)
schema.add_function(mm_embedding_function)
index_params = client.prepare_index_params()

index_params.add_index(
    field_name="dense",
    index_type="AUTOINDEX",
    metric_type="COSINE"
)
index_params.add_index(
    field_name="dense_mm",
    index_type="AUTOINDEX",
    metric_type="COSINE"
)
client.drop_collection(collection_name)
client.create_collection(
    collection_name=collection_name,
    schema=schema,
    index_params=index_params
)

# ========== Insert data: pass a URL for multimedia materials and the text itself for plain text ==========
insert_data = [
    {"id": 1, "document": "An apparel product image showing the details of the top worn by a person.", "url": IMAGE_URL},
    {"id": 2, "document": "A short apparel showcase video presenting how the garment looks in motion.", "url": VIDEO_URL},
    {"id": 3,
     "document": "A vector database converts text into high-dimensional vectors to enable semantic retrieval and understand the real intent of a query.",
     "url": "A vector database converts text into high-dimensional vectors to enable semantic retrieval and understand the real intent of a query."},
    {"id": 4,
     "document": "Deep learning models can learn feature representations from large volumes of image data for tasks such as image classification and object detection.",
     "url": "Deep learning models can learn feature representations from large volumes of image data for tasks such as image classification and object detection."},
    {"id": 5,
     "document": "Apparel e-commerce platforms often need to retrieve product cover images by style, color, and material.",
     "url": "Apparel e-commerce platforms often need to retrieve product cover images by style, color, and material."},
]

print(f"Preparing to insert {len(insert_data)} records...")
client.insert(collection_name, insert_data)
client.flush(collection_name)
print("Data insertion completed!\n")

# ========== Test 1: text-to-text (pure text semantic retrieval through the dense field) ==========
print("=" * 60)
print("Test 1: Text-to-text - query content related to apparel products")
print("=" * 60)
results = client.search(
    collection_name=collection_name,
    data=['Appearance display of an apparel product'],
    anns_field='dense',
    limit=3,
    output_fields=['document', 'url'],
)
for hits in results:
    for hit in hits:
        print(f"  id={hit['id']}, distance={hit['distance']:.4f}, document={hit['entity']['document'][:40]}")

# ========== Test 2: text-to-image / video (match multimedia content with a text query and filter out URL records) ==========
print("\n" + "=" * 60)
print("Test 2: Text-to-image / video - search apparel-related materials with a text description")
print("=" * 60)
results = client.search(
    collection_name=collection_name,
    data=['A display of a garment being worn'],
    anns_field='dense_mm',
    limit=3,
    filter='url like "http%"',
    output_fields=['document', 'url'],
)
for hits in results:
    for hit in hits:
        is_media = "media" if str(hit['entity']['url']).startswith('http') else "text"
        print(f"  id={hit['id']}, distance={hit['distance']:.4f}, [{is_media}] document={hit['entity']['document'][:40]}")

# ========== Test 3: image-to-image / video (query multimedia content with an image address) ==========
print("\n" + "=" * 60)
print("Test 3: Image-to-image / video - search similar materials with an image address")
print("=" * 60)
results = client.search(
    collection_name=collection_name,
    data=[IMAGE_URL],
    anns_field='dense_mm',
    limit=3,
    output_fields=['document', 'url'],
)
for hits in results:
    for hit in hits:
        is_media = "media" if str(hit['entity']['url']).startswith('http') else "text"
        print(f"  id={hit['id']}, distance={hit['distance']:.4f}, [{is_media}] document={hit['entity']['document'][:40]}")

Running result

Preparing to insert 5 records...
Data insertion completed!

============================================================
Test 1: Text-to-text - query content related to apparel products
============================================================
  id=1, distance=0.6412, document=An apparel product image showing the det
  id=5, distance=0.5901, document=Apparel e-commerce platforms often need
  id=2, distance=0.5689, document=A short apparel showcase video presentin

============================================================
Test 2: Text-to-image / video - search apparel-related materials with a text description
============================================================
  id=2, distance=0.1861, [media] document=A short apparel showcase video presentin
  id=1, distance=0.1153, [media] document=An apparel product image showing the det

============================================================
Test 3: Image-to-image / video - search similar materials with an image address
============================================================
  id=1, distance=1.0000, [media] document=An apparel product image showing the det
  id=2, distance=0.0732, [media] document=A short apparel showcase video presentin
  id=3, distance=0.0294, [text] document=A vector database converts text into hig

In Test 3, the similarity between the query image and the same image in the collection is 1.0000, which indicates that the image-to-image search path works. The similarity magnitude of cross-modal retrieval (querying multimedia with text) is usually lower than that of same-modal retrieval, which is normal. On the business side, you should judge the effectiveness by the relative ranking within the same retrieval scenario rather than by comparing absolute scores across scenarios.