Vector Retrieval Service for Milvus:Alibaba Cloud Milvus Embedding サービス：テキストおよびマルチモーダルデータのワンストップベクトル化

シナリオ

事前にベクターを生成せずに、生のテキストのバッチを Milvus にインジェストします。クエリ時には自然言語の質問を直接入力し、Milvus が自動的にクエリをベクトル化して、セマンティックに最も関連性の高い結果を返します。この機能は、ナレッジベース Q&A、ドキュメント検索、よくある質問の取得などのシナリオに適用できます。

操作手順

1. コレクションを作成し、生テキストフィールド document とベクトルフィールド dense を定義します。

2. text-embedding-v4 モデルを Function を使用してバインドします。

3. テスト用テキストデータを挿入します。

4. 自然言語の質問を使用して検索を実行し、最も関連性の高いテキストセグメントを取得します。

サンプルコード

import random
from pymilvus import MilvusClient, DataType, Function, FunctionType

client = MilvusClient(
    uri="http://c-xxxx.milvus.aliyuncs.com:19530",
    token='root:xxx',
)

# ========== Create a collection ==========
collection_name = 'demo1'
schema = client.create_schema()

schema.add_field("id", DataType.INT64, is_primary=True, auto_id=False)
schema.add_field("document", DataType.VARCHAR, max_length=9000)
schema.add_field("dense", DataType.FLOAT_VECTOR, dim=1024)
text_embedding_function = Function(
    name="dashscope_api_test123",
    function_type=FunctionType.TEXTEMBEDDING,
    input_field_names=["document"],
    output_field_names=["dense"],
    params={
        "provider": "aliyun_milvus",
        "model_name": "text-embedding-v4"
    }
)

schema.add_function(text_embedding_function)
index_params = client.prepare_index_params()

index_params.add_index(
    field_name="dense",
    index_type="AUTOINDEX",
    metric_type="COSINE"
)
client.drop_collection(collection_name)
client.create_collection(
    collection_name=collection_name,
    schema=schema,
    index_params=index_params
)


insert_data = []
record_id = 1

mock_texts = [
    "A vector database converts text into high-dimensional vectors for semantic search, understanding the true intent of queries.",
    "Deep learning models learn feature representations from large image datasets for tasks such as image classification and object detection.",
    "Retrieval-augmented generation recalls relevant documents to provide external knowledge for large language models, reducing hallucinations and improving answer reliability.",
    "In customer service scenarios, semantic search can quickly locate historical tickets, improving first-response efficiency and resolution rates.",
    "Similarity search commonly uses cosine distance to measure vector direction proximity, suitable for text semantic matching tasks.",
    "Data cleaning is a critical step before vectorization; denoising and format standardization significantly improve recall quality.",
    "Higher embedding dimensions are not always better; you need to balance effectiveness, latency, and storage cost.",
    "Chunking strategies that split long documents into small segments can improve search hit rates and reduce context redundancy.",
    "Adding a source field to each piece of knowledge aids result explainability, making it easy to display citation evidence on the frontend.",
    "Multilingual search leverages a unified vector space for cross-language matching, enhancing the experience of internationalized systems.",
    "Index parameters such as ef and M affect the recall rate and query performance of HNSW and need to be tuned based on your business requirements.",
    "Running offline evaluations before going live can quantify differences in search quality across different models and parameter combinations.",
    "Metadata filtering in a vector database can be combined with semantic recall for more precise scoped search.",
    "For high-frequency queries, caching search results briefly can reduce backend computation pressure and response time.",
    "A semantic search pipeline should log queries to facilitate analysis of zero-result queries and continuous optimization of the knowledge base.",
    "When knowledge content updates frequently, incremental indexing strategies can reduce the resource consumption of full rebuilds.",
    "Adding a reranking model in a Q&A system can improve the relevance and readability of the top results.",
    "Establishing access control policies for sensitive data is a fundamental security requirement for enterprise-grade vector search systems.",
    "Setting topK appropriately balances coverage and noise, avoiding the return of too many low-relevance results.",
    "Feeding user feedback back into the training and evaluation pipeline can continuously improve overall search and Q&A performance.",
]

for text in mock_texts:
    insert_data.append({
        "id": record_id,
        "document": text,
    })
    record_id += 1

BATCH_SIZE = 30
print(f"Preparing to insert {len(insert_data)} records, batch_size={BATCH_SIZE}...")
for batch_start in range(0, len(insert_data), BATCH_SIZE):
    batch = insert_data[batch_start:batch_start + BATCH_SIZE]
    client.insert(collection_name, batch)
    print(f"  Inserted {min(batch_start + BATCH_SIZE, len(insert_data))}/{len(insert_data)} records")
print("Data insertion complete!\n")

# ========== Test 1: Text-to-text search (semantic search via the dense field) ==========
print("=" * 60)
print("Test 1: Text-to-text search - query content related to vector databases")
print("=" * 60)
results = client.search(
    collection_name=collection_name,
    data=['What is semantic search? How does a vector database work?'],
    anns_field='dense',
    limit=3,
    output_fields=['document'],
)
for hits in results:
    for hit in hits:
        print(f"  id={hit['id']}, distance={hit['distance']:.4f}, document={hit['entity']['document'][:80]}")

出力

Preparing to insert 20 records, batch_size=30...
  Inserted 20/20 records
Data insertion complete!

============================================================
Test 1: Text-to-text search - query content related to vector databases
============================================================
  id=1, distance=0.8197, document=A vector database converts text into high-dimensional vectors for semantic search, understanding the true intent of queries.
  id=13, distance=0.6906, document=Metadata filtering in a vector database can be combined with semantic recall for more precise scoped search.
  id=18, distance=0.6563, document=Establishing access control policies for sensitive data is a fundamental security requirement for enterprise-grade vector search systems.

シナリオ

小売、E コマース、コンテンツプラットフォームなどのシナリオでは、検索対象がテキストだけでなく画像や動画も含まれます。Milvus Embedding サービスは、テキストおよび視覚コンテンツを統一されたベクトル化および検索パイプラインに組み込むことをサポートしており、Text-to-Image/Video や Image-to-Image/Video などのクロスモーダル検索機能を実現できます。

この例で使用する画像は、こちらからダウンロードできます。

操作手順

1. ローカルのテスト画像または動画を OSS にアップロードし、アクセス可能な署名付き URL を生成します。

2. コレクションを作成し、テキストフィールド document、マルチメディア URL フィールド url、テキストベクトルフィールド dense、マルチモーダルベクトルフィールド dense_mm を定義します。

3. text-embedding-v4 および qwen3-vl-embedding をそれぞれテキストフィールドおよびマルチメディアフィールドにバインドします。

4. テスト用のメディアアセットと対応するテキスト説明を挿入します。

5. テキストからテキスト、テキストから画像 / 動画、画像から画像 / 動画への検索検証を実行します。

サンプルコード

import glob
import os
import random
import oss2
from pymilvus import MilvusClient, DataType, Function, FunctionType

client = MilvusClient(
    uri="http://c-xxxx.milvus.aliyuncs.com:19530",
    token='root:xxx',
)

# ========== OSS configuration: upload multimedia resources and generate signed URLs ==========
SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
OSS_ACCESS_KEY_ID = os.environ['OSS_ACCESS_KEY_ID']
OSS_ACCESS_KEY_SECRET = os.environ['OSS_ACCESS_KEY_SECRET']
OSS_ENDPOINT = 'https://oss-cn-hangzhou.aliyuncs.com'
OSS_BUCKET_NAME = '002test'

auth = oss2.Auth(OSS_ACCESS_KEY_ID, OSS_ACCESS_KEY_SECRET)
bucket = oss2.Bucket(auth, OSS_ENDPOINT, OSS_BUCKET_NAME)


def upload_and_sign(relative_path, oss_key_prefix="milvus-embedding-test"):
    """Upload a local multimedia resource to OSS and return a signed URL (valid for 1 hour)"""
    full_path = os.path.join(SCRIPT_DIR, relative_path)
    oss_key = f"{oss_key_prefix}/{os.path.basename(relative_path)}"
    bucket.put_object_from_file(oss_key, full_path)
    signed_url = bucket.sign_url('GET', oss_key, 3600)
    print(f"  Upload successful: {oss_key} -> {signed_url[:80]}...")
    return signed_url


def upload_directory_with_patterns(
    directory_relative_path,
    patterns,
    oss_key_prefix="milvus-embedding-test",
    random_pick_count=None,
):
    """Upload multimedia files matching the given suffixes from a directory to OSS, return {filename: signed_url}"""
    full_dir = os.path.join(SCRIPT_DIR, directory_relative_path)
    matched_files = []
    for pattern in patterns:
        matched_files.extend(glob.glob(os.path.join(full_dir, pattern)))
    matched_files = sorted(set(matched_files))
    if random_pick_count is not None and len(matched_files) > random_pick_count:
        matched_files = random.sample(matched_files, random_pick_count)

    result = {}
    for file_path in matched_files:
        filename = os.path.basename(file_path)
        relative_path = os.path.join(directory_relative_path, filename)
        signed_url = upload_and_sign(relative_path, oss_key_prefix)
        result[filename] = signed_url
    return result


# Upload test assets from the banana and orange directories to OSS
print("Uploading test assets to OSS...")
banana_urls = upload_directory_with_patterns("qwen-vl/train/banana", ["*.JPEG"])
orange_urls = upload_directory_with_patterns("qwen-vl/train/orange", ["*.JPEG"])
print(f"Uploaded {len(banana_urls)} images from the banana directory, {len(orange_urls)} images from the orange directory")
print("Image asset upload complete!\n")

# Upload videos to OSS (same logic as image assets)
print("Uploading videos to OSS...")
selected_video_urls = upload_directory_with_patterns(
    "qwen-vl/short_video_10_of_42",
    ["*.mp4", "*.MP4", "*.mov", "*.MOV"],
    random_pick_count=5,
)
print(f"Uploaded and selected {len(selected_video_urls)} videos from the short_video_10_of_42 directory")
print("Video upload complete!\n")

banana_url_list = list(banana_urls.values())
orange_url_list = list(orange_urls.values())
selected_video_url_list = list(selected_video_urls.values())

# ========== Create a collection ==========
collection_name = 'demo11'
schema = client.create_schema()

schema.add_field("id", DataType.INT64, is_primary=True, auto_id=False)
schema.add_field("document", DataType.VARCHAR, max_length=9000)
schema.add_field("url", DataType.VARCHAR, max_length=9000)
schema.add_field("dense", DataType.FLOAT_VECTOR, dim=1024)
schema.add_field("dense_mm", DataType.FLOAT_VECTOR, dim=1024)
text_embedding_function = Function(
    name="dashscope_api_test123",
    function_type=FunctionType.TEXTEMBEDDING,
    input_field_names=["document"],
    output_field_names=["dense"],
    params={
        "provider": "aliyun_milvus",
        "model_name": "text-embedding-v4"
    }
)
mm_embedding_function = Function(
    name="dashscope_api_mm123",
    function_type=FunctionType.TEXTEMBEDDING,
    input_field_names=["url"],
    output_field_names=["dense_mm"],
    params={
        "provider": "aliyun_milvus",
        "model_name": "qwen3-vl-embedding",
        "dim": "1024"
    }
)

schema.add_function(text_embedding_function)
schema.add_function(mm_embedding_function)
index_params = client.prepare_index_params()

index_params.add_index(
    field_name="dense",
    index_type="AUTOINDEX",
    metric_type="COSINE"
)
index_params.add_index(
    field_name="dense_mm",
    index_type="AUTOINDEX",
    metric_type="COSINE"
)
client.drop_collection(collection_name)
client.create_collection(
    collection_name=collection_name,
    schema=schema,
    index_params=index_params
)

# ========== Insert Chinese test data, multimedia resources use OSS signed URLs ==========
banana_descriptions = [
    'A bunch of ripe yellow bananas hanging from a tree in a tropical orchard, with sunlight casting dappled shadows through the leaves.',
    'Neatly arranged bananas in the supermarket fruit section, with a price tag showing today\'s special, attracting many customers.',
    'A few bananas and a glass of milk on the kitchen table, a simple and healthy breakfast combination.',
    'Freshly picked green bananas neatly stacked in a bamboo basket, waiting to ripen naturally before going to market.',
    'A child holding a banana, happily walking and eating in the park, face beaming with satisfaction.',
    'A baker mashing overripe bananas to make a classic banana cake, the kitchen filled with a sweet aroma.',
    'Wild banana trees growing in clusters in the tropical rainforest, their large leaves gently swaying in the breeze.',
    'Sliced bananas on the breakfast table paired with oatmeal and yogurt, a nutritionally balanced healthy breakfast.',
    'A fruit vendor weighing a large bunch of fresh bananas for a customer, the scale showing exactly three jin.',
    'A few bananas displayed alongside apples and oranges in a fruit bowl, colorful and pleasing to the eye.',
]
orange_descriptions = [
    'Oranges hanging from branches in the orchard gleam golden in the sunlight, a joyful harvest season.',
    'A glass of freshly squeezed orange juice on the table, with cut oranges beside it, the flesh plump and juicy.',
    'Mom is peeling oranges in the kitchen, the air filled with a fresh citrus fragrance, as the children gather around waiting to eat.',
    'Navel oranges and blood oranges neatly lined up on supermarket shelves, each variety with its own distinctive characteristics.',
    'On a winter afternoon, a plate of sliced oranges on the coffee table is the whole family\'s favorite afternoon tea fruit.',
    'Grandma skillfully dries orange peels in the sun for tea, said to help regulate qi and strengthen the spleen.',
    'A fruit farmer carefully packs freshly picked oranges into cardboard boxes, ready to ship to customers across the country.',
    'The dessert shop window displays an exquisite orange mousse cake, its orange appearance very tempting.',
    'An orange cut in half reveals tender flesh, rich with juice, making one\'s mouth water.',
    'The fruit shop at the neighborhood entrance displays a sign announcing the arrival of Gannan navel oranges, attracting many residents to queue up.',
]
video_descriptions = [
    "A short fruit-themed video showing orchard picking and transportation.",
    "A short video of a fruit store display, with close-up shots of the fruit details.",
    "A short beverage-making video including slicing, pressing, and pouring into a glass.",
    "A short dessert-making video showing fruit decoration and the finished product.",
]

insert_data = []
record_id = 1
for idx, img_url in enumerate(banana_url_list):
    insert_data.append({
        'id': record_id,
        'document': banana_descriptions[idx],
        'url': img_url,
    })
    record_id += 1
for idx, img_url in enumerate(orange_url_list):
    insert_data.append({
        'id': record_id,
        'document': orange_descriptions[idx],
        'url': img_url,
    })
    record_id += 1
for idx, video_url in enumerate(selected_video_url_list):
    insert_data.append({
        "id": record_id,
        "document": video_descriptions[idx % len(video_descriptions)],
        "url": video_url,
    })
    record_id += 1
# Plain text data (the url field contains text, and dense_mm also generates a corresponding text vector)
insert_data.append({
    'id': record_id,
    'document': 'A vector database converts text into high-dimensional vectors for semantic search, understanding the true intent of queries.',
    'url': 'A vector database converts text into high-dimensional vectors for semantic search, understanding the true intent of queries.',
})
record_id += 1
insert_data.append({
    'id': record_id,
    'document': 'Deep learning models learn feature representations from large image datasets for tasks such as image classification and object detection.',
    'url': 'Deep learning models learn feature representations from large image datasets for tasks such as image classification and object detection.',
})

BATCH_SIZE = 20
print(f"Preparing to insert {len(insert_data)} records, batch_size={BATCH_SIZE}...")
for batch_start in range(0, len(insert_data), BATCH_SIZE):
    batch = insert_data[batch_start:batch_start + BATCH_SIZE]
    client.insert(collection_name, batch)
    print(f"  Inserted {min(batch_start + BATCH_SIZE, len(insert_data))}/{len(insert_data)} records")
print("Data insertion complete!\n")

# ========== Test 1: Text-to-text search (semantic search via the dense field) ==========
print("=" * 60)
print("Test 1: Text-to-text search - query content related to vector databases")
print("=" * 60)
results = client.search(
    collection_name=collection_name,
    data=['What is semantic search? How does a vector database work?'],
    anns_field='dense',
    limit=3,
    output_fields=['document', 'url'],
)
for hits in results:
    for hit in hits:
        print(f"  id={hit['id']}, distance={hit['distance']:.4f}, document={hit['entity']['document'][:80]}")

# ========== Test 2: Text-to-image/video search (query multimedia content via the dense_mm field using text) ==========
print("\n" + "=" * 60)
print("Test 2: Text-to-image/video search - search for banana-related images and video assets using a text description")
print("=" * 60)
results = client.search(
    collection_name=collection_name,
    data=['yellow bananas'],
    anns_field='dense_mm',
    limit=3,
    output_fields=['document', 'url'],
)
for hits in results:
    for hit in hits:
        has_image = "has image" if hit['entity'].get('url') else "no image"
        print(f"  id={hit['id']}, distance={hit['distance']:.4f}, [{has_image}] document={hit['entity']['document'][:80]}")

# ========== Test 3: Image-to-image/video search (query multimedia content via the dense_mm field using visual assets) ==========
print("\n" + "=" * 60)
print("Test 3: Image-to-image/video search - search for similar images and videos using a visual asset from the orange directory")
print("=" * 60)
results = client.search(
    collection_name=collection_name,
    data=[orange_url_list[0]],
    anns_field='dense_mm',
    limit=3,
    output_fields=['document', 'url'],
)
for hits in results:
    for hit in hits:
        has_media = "has media" if hit['entity'].get('url') else "no media"
        print(f"  id={hit['id']}, distance={hit['distance']:.4f}, [{has_media}] document={hit['entity']['document'][:80]}")

出力

Preparing to insert 27 records, batch_size=20...
  Inserted 20/27 records
  Inserted 27/27 records
Data insertion complete!

============================================================
Test 1: Text-to-text search - query content related to vector databases
============================================================
  id=26, distance=0.8197, document=A vector database converts text into high-dimensional vectors for semantic search, understanding the true intent of queries.
  id=27, distance=0.3091, document=Deep learning models learn feature representations from large image datasets for tasks such as image classification and object detection.
  id=22, distance=0.2337, document=A short video of a fruit store display, with close-up shots of the fruit details.

============================================================
Test 2: Text-to-image/video search - search for banana-related images and video assets using a text description
============================================================
  id=9, distance=0.4862, [has image] document=A fruit vendor weighing a large bunch of fresh bananas for a customer, the scale showing exactly three jin.
  id=1, distance=0.4834, [has image] document=A bunch of ripe yellow bananas hanging from a tree in a tropical orchard, with sunlight casting dappled shadows through the leaves.
  id=7, distance=0.4797, [has image] document=Wild banana trees growing in clusters in the tropical rainforest, their large leaves gently swaying in the breeze.

============================================================
Test 3: Image-to-image/video search - search for similar images and videos using a visual asset from the orange directory
============================================================
  id=11, distance=1.0000, [has media] document=Oranges hanging from branches in the orchard gleam golden in the sunlight, a joyful harvest season.
  id=12, distance=0.9898, [has media] document=A glass of freshly squeezed orange juice on the table, with cut oranges beside it, the flesh plump and juicy.
  id=13, distance=0.9858, [has media] document=Mom is peeling oranges in the kitchen, the air filled with a fresh citrus fragrance, as the children gather around waiting to eat.