Alibaba Cloud Milvus AI Center is a managed model service capability built into Milvus. After you enable it, you can pass in raw text or multimodal data (images and videos) directly when writing and retrieving data, and the system automatically generates the vectors. You do not need to deploy a separate Embedding inference service.
Feature overview

Feature | Description |
One-stop management in the console | View and configure model services and obtain call examples in the console, without switching between multiple platforms. |
Managed model capabilities | The platform provides managed model inference services, so you do not need to build your own inference service or maintain the underlying infrastructure. |
Direct write and retrieval of raw data | Pass in raw text or multimodal content directly when inserting, updating, and querying data, and the system completes the vectorization automatically. |
Flexible switching between multiple models | Multiple vectorization models are available, so you can flexibly balance effectiveness, latency, and cost based on your business scenario. |
Call volume and token statistics | Token usage and the number of calls are counted by two dimensions: cluster and model. |
Call monitoring | You can view operational metrics such as the request Success Rate and the Average RT (average response time). |
In addition to vectorization, the AI Center provides unified model capabilities such as reranking, text generation, multimodal understanding, and video editing, which can be used directly in scenarios such as semantic retrieval, RAG knowledge bases, enterprise intelligent Q&A, content recommendation, and multimodal search. This topic uses vectorization as an example to describe the complete integration process for text retrieval and multimodal retrieval.
Procedure
View the available vectorization models
Log on to the Milvus console. In the left-side navigation pane, choose AI Center. On the Model Services tab, view the Vectorization group, where you can obtain the input type and output dimensions of each model as well as the API example.
The following vectorization models are currently available:
Model | Supported input | Description |
| Text | The recommended text vector model. The default number of dimensions is 1024. |
| Text | A multilingual text vector model that supports custom vector dimensions. |
| Text | A general-purpose text vector model. |
| Text | A general-purpose text vector model. It will be discontinued soon and is not recommended for new business. |
| Text, images, and videos | The recommended multimodal vector model. The default number of dimensions is 2560. You can specify other dimensions by using |
| Text, images, and videos | A multimodal vector model. |
The AI Center provides services independently by region. After you switch to another region, you must view and configure the services again in the new region.
Prepare a Milvus instance
The vectorization capability requires a Milvus 2.6 instance. After you create a 2.6 instance, you can declare a Function in a collection to call a model directly. You do not need to separately bind the model to the instance.
To access the instance over the Internet, go to the Security Configuration tab on the instance details page, enable Internet Access, and configure an Internet access whitelist.
View usage and monitoring data
Go to AI Center and click the Monitoring tab. You can view call information from two perspectives: By Milvus Cluster or By Model:
Token usage / number of calls: lists the token usage and call count details by cluster and by model.
Total tokens and total text input tokens: the aggregated token consumption within the selected scope.
Success Rate: the request success rate.
Average RT: the average response time.
You can select a cluster or a model, set the sampling interval (interval), and filter by quick ranges such as 1 hour, 3 hours, 6 hours, 12 hours, 1 day, and 7 days, or by a custom time window.
Case 1: text-to-text semantic retrieval
Scenario description
Write a batch of raw text into Milvus without generating vectors in advance. During a query, enter a natural language question directly, and Milvus automatically vectorizes the query and returns the most semantically relevant results. This applies to scenarios such as knowledge base Q&A, document search, and FAQ retrieval.
Procedure
Create a collection and define the raw text field
documentand the vector fielddense.Bind the
qwen3.7-text-embeddingmodel by usingFunction.Insert test text data and call
flushto ensure the data is retrievable.Initiate retrieval with a natural language question to return the most relevant text snippets.
After inserting data, you must call flush. Otherwise, a retrieval performed immediately afterwards may return empty results.
Sample code
from pymilvus import MilvusClient, DataType, Function, FunctionType
client = MilvusClient(
uri="http://c-xxxx.milvus.aliyuncs.com:19530",
token='root:xxx',
)
# ========== Create the collection ==========
collection_name = 'demo1'
schema = client.create_schema()
schema.add_field("id", DataType.INT64, is_primary=True, auto_id=False)
schema.add_field("document", DataType.VARCHAR, max_length=9000)
schema.add_field("dense", DataType.FLOAT_VECTOR, dim=1024)
text_embedding_function = Function(
name="dashscope_api_test123",
function_type=FunctionType.TEXTEMBEDDING,
input_field_names=["document"],
output_field_names=["dense"],
params={
"provider": "aliyun_milvus",
"model_name": "qwen3.7-text-embedding"
}
)
schema.add_function(text_embedding_function)
index_params = client.prepare_index_params()
index_params.add_index(
field_name="dense",
index_type="AUTOINDEX",
metric_type="COSINE"
)
client.drop_collection(collection_name)
client.create_collection(
collection_name=collection_name,
schema=schema,
index_params=index_params
)
insert_data = []
record_id = 1
mock_texts = [
"A vector database converts text into high-dimensional vectors to enable semantic retrieval and understand the real intent of a query.",
"Deep learning models can learn feature representations from large volumes of image data for tasks such as image classification and object detection.",
"Retrieval-augmented generation recalls relevant documents to provide external knowledge to large models, reducing hallucinations and improving answer reliability.",
"In customer service scenarios, semantic search can quickly locate historical tickets, improving first-response efficiency and the issue resolution rate.",
"Similarity search often uses cosine distance to measure how close two vector directions are, which suits text semantic matching tasks.",
"Data cleansing is an important step before vectorization; removing noise and unifying formats can significantly improve recall quality.",
"A higher embedding dimension is not always better; you need to balance effectiveness, latency, and storage cost.",
"Splitting long documents into small segments with a chunking strategy can improve the retrieval hit rate and reduce context redundancy.",
"Adding a source field to each piece of knowledge improves the explainability of results and makes it easier to display citations in the frontend.",
"Multilingual retrieval can achieve cross-language matching through a unified vector space, improving the experience of internationalized systems.",
"Index parameters such as ef and M affect the recall rate and query performance of HNSW and must be tuned based on the business.",
"Running offline evaluations before release can quantify the retrieval quality differences between different models and parameter combinations.",
"Metadata filtering in a vector store can be combined with semantic recall to achieve more precise scoped search.",
"For high-frequency questions, you can cache retrieval results for a short time to reduce backend computing pressure and response time.",
"The semantic retrieval pipeline should log queries so that zero-result queries can be analyzed and the knowledge base continuously optimized.",
"When knowledge content is updated frequently, an incremental indexing strategy can reduce the resource consumption of full rebuilds.",
"Adding a reranking model to a Q&A system can improve the relevance and readability of the top results.",
"Establishing access control policies for sensitive data is a basic security requirement for enterprise-grade vector retrieval systems.",
"Setting topK properly balances coverage and noise, avoiding returning too many low-relevance results.",
"Feeding user feedback back into the training and evaluation process continuously improves the overall retrieval and Q&A effectiveness.",
]
for text in mock_texts:
insert_data.append({
"id": record_id,
"document": text,
})
record_id += 1
BATCH_SIZE = 30
print(f"Preparing to insert {len(insert_data)} records, batch_size={BATCH_SIZE}...")
for batch_start in range(0, len(insert_data), BATCH_SIZE):
batch = insert_data[batch_start:batch_start + BATCH_SIZE]
client.insert(collection_name, batch)
print(f" Inserted {min(batch_start + BATCH_SIZE, len(insert_data))}/{len(insert_data)} records")
client.flush(collection_name)
print("Data insertion completed!\n")
# ========== Test 1: text-to-text (pure text semantic retrieval through the dense field) ==========
print("=" * 60)
print("Test 1: Text-to-text - query content related to vector databases")
print("=" * 60)
results = client.search(
collection_name=collection_name,
data=['What is semantic retrieval? How does a vector database work?'],
anns_field='dense',
limit=3,
output_fields=['document'],
)
for hits in results:
for hit in hits:
print(f" id={hit['id']}, distance={hit['distance']:.4f}, document={hit['entity']['document'][:50]}")Running result
Preparing to insert 20 records, batch_size=30...
Inserted 20/20 records
Data insertion completed!
============================================================
Test 1: Text-to-text - query content related to vector databases
============================================================
id=1, distance=0.8668, document=A vector database converts text into high-dimensio
id=13, distance=0.5989, document=Metadata filtering in a vector store can be combin
id=15, distance=0.5414, document=The semantic retrieval pipeline should log queriesThe retrieval results are sorted by semantic relevance, and the text closest to the query intent ranks first. The actual similarity scores vary with the model version and the data content.
Case 2: multimodal retrieval
Scenario description
In retail, e-commerce, and content platform scenarios, the retrieval objects include images and videos in addition to text. The multimodal vector models of the AI Center support incorporating text and visual content into a unified vectorization and retrieval process, enabling cross-modal retrieval capabilities such as text-to-image/video and image-to-image/video search.
This case defines two vector fields in the same collection: dense, which is generated by the text model and used for text semantic retrieval, and dense_mm, which is generated by the multimodal model and used for cross-modal retrieval.
Procedure
Upload the images or videos to object storage such as OSS and generate URLs that can be accessed by the model service.
Create a collection and define the text field
document, the multimedia address fieldurl, the text vector fielddense, and the multimodal vector fielddense_mm.Bind
qwen3.7-text-embeddingandqwen3-vl-embeddingto the text field and the multimedia field respectively.Insert the test materials and their corresponding text descriptions.
Perform retrieval verification such as text-to-text, text-to-image/video, and image-to-image/video search.
When you bind a multimodal model to a multimedia field, you must set is_multimodal to true in params. Otherwise, an error is reported when you create the collection because the model cannot parse the input. The dim value must be consistent with the number of dimensions declared for the vector field.
You can also write plain text directly into a multimedia field. The multimodal model generates a text vector in the same vector space for it, which supports mixed image-and-text retrieval.
Sample code
from pymilvus import MilvusClient, DataType, Function, FunctionType
client = MilvusClient(
uri="http://c-xxxx.milvus.aliyuncs.com:19530",
token='root:xxx',
)
# Replace with your own image or video address, which must be accessible to the model service
IMAGE_URL = "https://example.com/your-image.jpg"
VIDEO_URL = "https://example.com/your-video.mp4"
# ========== Create the collection ==========
collection_name = 'demo11'
schema = client.create_schema()
schema.add_field("id", DataType.INT64, is_primary=True, auto_id=False)
schema.add_field("document", DataType.VARCHAR, max_length=9000)
schema.add_field("url", DataType.VARCHAR, max_length=9000)
schema.add_field("dense", DataType.FLOAT_VECTOR, dim=1024)
schema.add_field("dense_mm", DataType.FLOAT_VECTOR, dim=1024)
text_embedding_function = Function(
name="dashscope_api_test123",
function_type=FunctionType.TEXTEMBEDDING,
input_field_names=["document"],
output_field_names=["dense"],
params={
"provider": "aliyun_milvus",
"model_name": "qwen3.7-text-embedding"
}
)
mm_embedding_function = Function(
name="dashscope_api_mm123",
function_type=FunctionType.TEXTEMBEDDING,
input_field_names=["url"],
output_field_names=["dense_mm"],
params={
"provider": "aliyun_milvus",
"model_name": "qwen3-vl-embedding",
"dim": "1024",
"is_multimodal": "true"
}
)
schema.add_function(text_embedding_function)
schema.add_function(mm_embedding_function)
index_params = client.prepare_index_params()
index_params.add_index(
field_name="dense",
index_type="AUTOINDEX",
metric_type="COSINE"
)
index_params.add_index(
field_name="dense_mm",
index_type="AUTOINDEX",
metric_type="COSINE"
)
client.drop_collection(collection_name)
client.create_collection(
collection_name=collection_name,
schema=schema,
index_params=index_params
)
# ========== Insert data: pass a URL for multimedia materials and the text itself for plain text ==========
insert_data = [
{"id": 1, "document": "An apparel product image showing the details of the top worn by a person.", "url": IMAGE_URL},
{"id": 2, "document": "A short apparel showcase video presenting how the garment looks in motion.", "url": VIDEO_URL},
{"id": 3,
"document": "A vector database converts text into high-dimensional vectors to enable semantic retrieval and understand the real intent of a query.",
"url": "A vector database converts text into high-dimensional vectors to enable semantic retrieval and understand the real intent of a query."},
{"id": 4,
"document": "Deep learning models can learn feature representations from large volumes of image data for tasks such as image classification and object detection.",
"url": "Deep learning models can learn feature representations from large volumes of image data for tasks such as image classification and object detection."},
{"id": 5,
"document": "Apparel e-commerce platforms often need to retrieve product cover images by style, color, and material.",
"url": "Apparel e-commerce platforms often need to retrieve product cover images by style, color, and material."},
]
print(f"Preparing to insert {len(insert_data)} records...")
client.insert(collection_name, insert_data)
client.flush(collection_name)
print("Data insertion completed!\n")
# ========== Test 1: text-to-text (pure text semantic retrieval through the dense field) ==========
print("=" * 60)
print("Test 1: Text-to-text - query content related to apparel products")
print("=" * 60)
results = client.search(
collection_name=collection_name,
data=['Appearance display of an apparel product'],
anns_field='dense',
limit=3,
output_fields=['document', 'url'],
)
for hits in results:
for hit in hits:
print(f" id={hit['id']}, distance={hit['distance']:.4f}, document={hit['entity']['document'][:40]}")
# ========== Test 2: text-to-image / video (match multimedia content with a text query and filter out URL records) ==========
print("\n" + "=" * 60)
print("Test 2: Text-to-image / video - search apparel-related materials with a text description")
print("=" * 60)
results = client.search(
collection_name=collection_name,
data=['A display of a garment being worn'],
anns_field='dense_mm',
limit=3,
filter='url like "http%"',
output_fields=['document', 'url'],
)
for hits in results:
for hit in hits:
is_media = "media" if str(hit['entity']['url']).startswith('http') else "text"
print(f" id={hit['id']}, distance={hit['distance']:.4f}, [{is_media}] document={hit['entity']['document'][:40]}")
# ========== Test 3: image-to-image / video (query multimedia content with an image address) ==========
print("\n" + "=" * 60)
print("Test 3: Image-to-image / video - search similar materials with an image address")
print("=" * 60)
results = client.search(
collection_name=collection_name,
data=[IMAGE_URL],
anns_field='dense_mm',
limit=3,
output_fields=['document', 'url'],
)
for hits in results:
for hit in hits:
is_media = "media" if str(hit['entity']['url']).startswith('http') else "text"
print(f" id={hit['id']}, distance={hit['distance']:.4f}, [{is_media}] document={hit['entity']['document'][:40]}")Running result
Preparing to insert 5 records...
Data insertion completed!
============================================================
Test 1: Text-to-text - query content related to apparel products
============================================================
id=1, distance=0.6412, document=An apparel product image showing the det
id=5, distance=0.5901, document=Apparel e-commerce platforms often need
id=2, distance=0.5689, document=A short apparel showcase video presentin
============================================================
Test 2: Text-to-image / video - search apparel-related materials with a text description
============================================================
id=2, distance=0.1861, [media] document=A short apparel showcase video presentin
id=1, distance=0.1153, [media] document=An apparel product image showing the det
============================================================
Test 3: Image-to-image / video - search similar materials with an image address
============================================================
id=1, distance=1.0000, [media] document=An apparel product image showing the det
id=2, distance=0.0732, [media] document=A short apparel showcase video presentin
id=3, distance=0.0294, [text] document=A vector database converts text into higIn Test 3, the similarity between the query image and the same image in the collection is 1.0000, which indicates that the image-to-image search path works. The similarity magnitude of cross-modal retrieval (querying multimedia with text) is usually lower than that of same-modal retrieval, which is normal. On the business side, you should judge the effectiveness by the relative ranking within the same retrieval scenario rather than by comparing absolute scores across scenarios.