Vector Retrieval Service for Milvus:Layanan Alibaba Cloud Milvus Embedding: Vektorisasi teks dan multimodal satu atap

Skenario

Ingesti batch teks mentah ke Milvus tanpa menghasilkan vektor terlebih dahulu. Saat kueri, masukkan pertanyaan dalam bahasa alami secara langsung, dan Milvus secara otomatis melakukan vektorisasi kueri serta mengembalikan hasil yang paling relevan secara semantik. Ini berlaku untuk skenario seperti Q&A basis pengetahuan, pencarian dokumen, dan pengambilan FAQ.

Prosedur

1. Buat koleksi dan definisikan bidang teks mentah document serta bidang vektor dense.

2. Bind model text-embedding-v4 menggunakan Function.

3. Masukkan data teks uji.

4. Lakukan pencarian menggunakan pertanyaan dalam bahasa alami dan ambil segmen teks yang paling relevan.

Kode contoh

import random
from pymilvus import MilvusClient, DataType, Function, FunctionType

client = MilvusClient(
    uri="http://c-xxxx.milvus.aliyuncs.com:19530",
    token='root:xxx',
)

# ========== Buat koleksi ==========
collection_name = 'demo1'
schema = client.create_schema()

schema.add_field("id", DataType.INT64, is_primary=True, auto_id=False)
schema.add_field("document", DataType.VARCHAR, max_length=9000)
schema.add_field("dense", DataType.FLOAT_VECTOR, dim=1024)
text_embedding_function = Function(
    name="dashscope_api_test123",
    function_type=FunctionType.TEXTEMBEDDING,
    input_field_names=["document"],
    output_field_names=["dense"],
    params={
        "provider": "aliyun_milvus",
        "model_name": "text-embedding-v4"
    }
)

schema.add_function(text_embedding_function)
index_params = client.prepare_index_params()

index_params.add_index(
    field_name="dense",
    index_type="AUTOINDEX",
    metric_type="COSINE"
)
client.drop_collection(collection_name)
client.create_collection(
    collection_name=collection_name,
    schema=schema,
    index_params=index_params
)


insert_data = []
record_id = 1

mock_texts = [
    "A vector database converts text into high-dimensional vectors for semantic search, understanding the true intent of queries.",
    "Deep learning models learn feature representations from large image datasets for tasks such as image classification and object detection.",
    "Retrieval-augmented generation recalls relevant documents to provide external knowledge for large language models, reducing hallucinations and improving answer reliability.",
    "In customer service scenarios, semantic search can quickly locate historical tickets, improving first-response efficiency and resolution rates.",
    "Similarity search commonly uses cosine distance to measure vector direction proximity, suitable for text semantic matching tasks.",
    "Data cleaning is a critical step before vectorization; denoising and format standardization significantly improve recall quality.",
    "Higher embedding dimensions are not always better; you need to balance effectiveness, latency, and storage cost.",
    "Chunking strategies that split long documents into small segments can improve search hit rates and reduce context redundancy.",
    "Adding a source field to each piece of knowledge aids result explainability, making it easy to display citation evidence on the frontend.",
    "Multilingual search leverages a unified vector space for cross-language matching, enhancing the experience of internationalized systems.",
    "Index parameters such as ef and M affect the recall rate and query performance of HNSW and need to be tuned based on your business requirements.",
    "Running offline evaluations before going live can quantify differences in search quality across different models and parameter combinations.",
    "Metadata filtering in a vector database can be combined with semantic recall for more precise scoped search.",
    "For high-frequency queries, caching search results briefly can reduce backend computation pressure and response time.",
    "A semantic search pipeline should log queries to facilitate analysis of zero-result queries and continuous optimization of the knowledge base.",
    "When knowledge content updates frequently, incremental indexing strategies can reduce the resource consumption of full rebuilds.",
    "Adding a reranking model in a Q&A system can improve the relevance and readability of the top results.",
    "Establishing access control policies for sensitive data is a fundamental security requirement for enterprise-grade vector search systems.",
    "Setting topK appropriately balances coverage and noise, avoiding the return of too many low-relevance results.",
    "Feeding user feedback back into the training and evaluation pipeline can continuously improve overall search and Q&A performance.",
]

for text in mock_texts:
    insert_data.append({
        "id": record_id,
        "document": text,
    })
    record_id += 1

BATCH_SIZE = 30
print(f"Preparing to insert {len(insert_data)} records, batch_size={BATCH_SIZE}...")
for batch_start in range(0, len(insert_data), BATCH_SIZE):
    batch = insert_data[batch_start:batch_start + BATCH_SIZE]
    client.insert(collection_name, batch)
    print(f"  Inserted {min(batch_start + BATCH_SIZE, len(insert_data))}/{len(insert_data)} records")
print("Data insertion complete!\n")

# ========== Uji 1: Pencarian teks-ke-teks (pencarian semantik melalui bidang dense) ==========
print("=" * 60)
print("Uji 1: Pencarian teks-ke-teks - kueri konten terkait database vektor")
print("=" * 60)
results = client.search(
    collection_name=collection_name,
    data=['What is semantic search? How does a vector database work?'],
    anns_field='dense',
    limit=3,
    output_fields=['document'],
)
for hits in results:
    for hit in hits:
        print(f"  id={hit['id']}, distance={hit['distance']:.4f}, document={hit['entity']['document'][:80]}")

Output

Preparing to insert 20 records, batch_size=30...
  Inserted 20/20 records
Data insertion complete!

============================================================
Uji 1: Pencarian teks-ke-teks - kueri konten terkait database vektor
============================================================
  id=1, distance=0.8197, document=A vector database converts text into high-dimensional vectors for semantic search, understanding the true intent of queries.
  id=13, distance=0.6906, document=Metadata filtering in a vector database can be combined with semantic recall for more precise scoped search.
  id=18, distance=0.6563, document=Establishing access control policies for sensitive data is a fundamental security requirement for enterprise-grade vector search systems.

Skenario

Dalam skenario ritel, e-commerce, dan platform konten, target pencarian tidak hanya mencakup teks tetapi juga gambar dan video. Layanan Embedding Milvus mendukung penggabungan konten teks dan visual ke dalam pipeline vektorisasi dan pencarian terpadu, memungkinkan kemampuan pencarian lintas modal seperti pencarian teks-ke-gambar/video dan gambar-ke-gambar/video.

Anda dapat mengunduh gambar yang digunakan dalam contoh ini dari sini.

Prosedur

1. Unggah gambar atau video uji lokal ke OSS dan hasilkan URL yang ditandatangani yang dapat diakses.

2. Buat koleksi dan definisikan bidang teks document, bidang URL multimedia url, bidang vektor teks dense, dan bidang vektor multimodal dense_mm.

3. Bind text-embedding-v4 dan qwen3-vl-embedding ke bidang teks dan bidang multimedia, masing-masing.

4. Masukkan aset media uji beserta deskripsi teksnya.

5. Lakukan verifikasi pencarian untuk pencarian teks-ke-teks, teks-ke-gambar/video, dan gambar-ke-gambar/video.

Kode contoh

import glob
import os
import random
import oss2
from pymilvus import MilvusClient, DataType, Function, FunctionType

client = MilvusClient(
    uri="http://c-xxxx.milvus.aliyuncs.com:19530",
    token='root:xxx',
)

# ========== Konfigurasi OSS: unggah sumber daya multimedia dan hasilkan URL yang ditandatangani ==========
SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
OSS_ACCESS_KEY_ID = os.environ['OSS_ACCESS_KEY_ID']
OSS_ACCESS_KEY_SECRET = os.environ['OSS_ACCESS_KEY_SECRET']
OSS_ENDPOINT = 'https://oss-cn-hangzhou.aliyuncs.com'
OSS_BUCKET_NAME = '002test'

auth = oss2.Auth(OSS_ACCESS_KEY_ID, OSS_ACCESS_KEY_SECRET)
bucket = oss2.Bucket(auth, OSS_ENDPOINT, OSS_BUCKET_NAME)


def upload_and_sign(relative_path, oss_key_prefix="milvus-embedding-test"):
    """Unggah sumber daya multimedia lokal ke OSS dan kembalikan URL yang ditandatangani (berlaku 1 jam)"""
    full_path = os.path.join(SCRIPT_DIR, relative_path)
    oss_key = f"{oss_key_prefix}/{os.path.basename(relative_path)}"
    bucket.put_object_from_file(oss_key, full_path)
    signed_url = bucket.sign_url('GET', oss_key, 3600)
    print(f"  Upload successful: {oss_key} -> {signed_url[:80]}...")
    return signed_url


def upload_directory_with_patterns(
    directory_relative_path,
    patterns,
    oss_key_prefix="milvus-embedding-test",
    random_pick_count=None,
):
    """Unggah file multimedia yang cocok dengan ekstensi tertentu dari direktori ke OSS, kembalikan {filename: signed_url}"""
    full_dir = os.path.join(SCRIPT_DIR, directory_relative_path)
    matched_files = []
    for pattern in patterns:
        matched_files.extend(glob.glob(os.path.join(full_dir, pattern)))
    matched_files = sorted(set(matched_files))
    if random_pick_count is not None and len(matched_files) > random_pick_count:
        matched_files = random.sample(matched_files, random_pick_count)

    result = {}
    for file_path in matched_files:
        filename = os.path.basename(file_path)
        relative_path = os.path.join(directory_relative_path, filename)
        signed_url = upload_and_sign(relative_path, oss_key_prefix)
        result[filename] = signed_url
    return result


# Unggah aset uji dari direktori banana dan orange ke OSS
print("Mengunggah aset uji ke OSS...")
banana_urls = upload_directory_with_patterns("qwen-vl/train/banana", ["*.JPEG"])
orange_urls = upload_directory_with_patterns("qwen-vl/train/orange", ["*.JPEG"])
print(f"Telah mengunggah {len(banana_urls)} gambar dari direktori banana, {len(orange_urls)} gambar dari direktori orange")
print("Pengunggahan aset gambar selesai!\n")

# Unggah video ke OSS (logika sama seperti aset gambar)
print("Mengunggah video ke OSS...")
selected_video_urls = upload_directory_with_patterns(
    "qwen-vl/short_video_10_of_42",
    ["*.mp4", "*.MP4", "*.mov", "*.MOV"],
    random_pick_count=5,
)
print(f"Telah mengunggah dan memilih {len(selected_video_urls)} video dari direktori short_video_10_of_42")
print("Pengunggahan video selesai!\n")

banana_url_list = list(banana_urls.values())
orange_url_list = list(orange_urls.values())
selected_video_url_list = list(selected_video_urls.values())

# ========== Buat koleksi ==========
collection_name = 'demo11'
schema = client.create_schema()

schema.add_field("id", DataType.INT64, is_primary=True, auto_id=False)
schema.add_field("document", DataType.VARCHAR, max_length=9000)
schema.add_field("url", DataType.VARCHAR, max_length=9000)
schema.add_field("dense", DataType.FLOAT_VECTOR, dim=1024)
schema.add_field("dense_mm", DataType.FLOAT_VECTOR, dim=1024)
text_embedding_function = Function(
    name="dashscope_api_test123",
    function_type=FunctionType.TEXTEMBEDDING,
    input_field_names=["document"],
    output_field_names=["dense"],
    params={
        "provider": "aliyun_milvus",
        "model_name": "text-embedding-v4"
    }
)
mm_embedding_function = Function(
    name="dashscope_api_mm123",
    function_type=FunctionType.TEXTEMBEDDING,
    input_field_names=["url"],
    output_field_names=["dense_mm"],
    params={
        "provider": "aliyun_milvus",
        "model_name": "qwen3-vl-embedding",
        "dim": "1024"
    }
)

schema.add_function(text_embedding_function)
schema.add_function(mm_embedding_function)
index_params = client.prepare_index_params()

index_params.add_index(
    field_name="dense",
    index_type="AUTOINDEX",
    metric_type="COSINE"
)
index_params.add_index(
    field_name="dense_mm",
    index_type="AUTOINDEX",
    metric_type="COSINE"
)
client.drop_collection(collection_name)
client.create_collection(
    collection_name=collection_name,
    schema=schema,
    index_params=index_params
)

# ========== Masukkan data uji dalam bahasa Mandarin, sumber daya multimedia menggunakan URL yang ditandatangani OSS ==========
banana_descriptions = [
    'A bunch of ripe yellow bananas hanging from a tree in a tropical orchard, with sunlight casting dappled shadows through the leaves.',
    'Neatly arranged bananas in the supermarket fruit section, with a price tag showing today\'s special, attracting many customers.',
    'A few bananas and a glass of milk on the kitchen table, a simple and healthy breakfast combination.',
    'Freshly picked green bananas neatly stacked in a bamboo basket, waiting to ripen naturally before going to market.',
    'A child holding a banana, happily walking and eating in the park, face beaming with satisfaction.',
    'A baker mashing overripe bananas to make a classic banana cake, the kitchen filled with a sweet aroma.',
    'Wild banana trees growing in clusters in the tropical rainforest, their large leaves gently swaying in the breeze.',
    'Sliced bananas on the breakfast table paired with oatmeal and yogurt, a nutritionally balanced healthy breakfast.',
    'A fruit vendor weighing a large bunch of fresh bananas for a customer, the scale showing exactly three jin.',
    'A few bananas displayed alongside apples and oranges in a fruit bowl, colorful and pleasing to the eye.',
]
orange_descriptions = [
    'Oranges hanging from branches in the orchard gleam golden in the sunlight, a joyful harvest season.',
    'A glass of freshly squeezed orange juice on the table, with cut oranges beside it, the flesh plump and juicy.',
    'Mom is peeling oranges in the kitchen, the air filled with a fresh citrus fragrance, as the children gather around waiting to eat.',
    'Navel oranges and blood oranges neatly lined up on supermarket shelves, each variety with its own distinctive characteristics.',
    'On a winter afternoon, a plate of sliced oranges on the coffee table is the whole family\'s favorite afternoon tea fruit.',
    'Grandma skillfully dries orange peels in the sun for tea, said to help regulate qi and strengthen the spleen.',
    'A fruit farmer carefully packs freshly picked oranges into cardboard boxes, ready to ship to customers across the country.',
    'The dessert shop window displays an exquisite orange mousse cake, its orange appearance very tempting.',
    'An orange cut in half reveals tender flesh, rich with juice, making one\'s mouth water.',
    'The fruit shop at the neighborhood entrance displays a sign announcing the arrival of Gannan navel oranges, attracting many residents to queue up.',
]
video_descriptions = [
    "A short fruit-themed video showing orchard picking and transportation.",
    "A short video of a fruit store display, with close-up shots of the fruit details.",
    "A short beverage-making video including slicing, pressing, and pouring into a glass.",
    "A short dessert-making video showing fruit decoration and the finished product.",
]

insert_data = []
record_id = 1
for idx, img_url in enumerate(banana_url_list):
    insert_data.append({
        'id': record_id,
        'document': banana_descriptions[idx],
        'url': img_url,
    })
    record_id += 1
for idx, img_url in enumerate(orange_url_list):
    insert_data.append({
        'id': record_id,
        'document': orange_descriptions[idx],
        'url': img_url,
    })
    record_id += 1
for idx, video_url in enumerate(selected_video_url_list):
    insert_data.append({
        "id": record_id,
        "document": video_descriptions[idx % len(video_descriptions)],
        "url": video_url,
    })
    record_id += 1
# Data teks biasa (bidang url berisi teks, dan dense_mm juga menghasilkan vektor teks yang sesuai)
insert_data.append({
    'id': record_id,
    'document': 'A vector database converts text into high-dimensional vectors for semantic search, understanding the true intent of queries.',
    'url': 'A vector database converts text into high-dimensional vectors for semantic search, understanding the true intent of queries.',
})
record_id += 1
insert_data.append({
    'id': record_id,
    'document': 'Deep learning models learn feature representations from large image datasets for tasks such as image classification and object detection.',
    'url': 'Deep learning models learn feature representations from large image datasets for tasks such as image classification and object detection.',
})

BATCH_SIZE = 20
print(f"Preparing to insert {len(insert_data)} records, batch_size={BATCH_SIZE}...")
for batch_start in range(0, len(insert_data), BATCH_SIZE):
    batch = insert_data[batch_start:batch_start + BATCH_SIZE]
    client.insert(collection_name, batch)
    print(f"  Inserted {min(batch_start + BATCH_SIZE, len(insert_data))}/{len(insert_data)} records")
print("Data insertion complete!\n")

# ========== Uji 1: Pencarian teks-ke-teks (pencarian semantik melalui bidang dense) ==========
print("=" * 60)
print("Uji 1: Pencarian teks-ke-teks - kueri konten terkait database vektor")
print("=" * 60)
results = client.search(
    collection_name=collection_name,
    data=['What is semantic search? How does a vector database work?'],
    anns_field='dense',
    limit=3,
    output_fields=['document', 'url'],
)
for hits in results:
    for hit in hits:
        print(f"  id={hit['id']}, distance={hit['distance']:.4f}, document={hit['entity']['document'][:80]}")

# ========== Uji 2: Pencarian teks-ke-gambar/video (kueri konten multimedia melalui bidang dense_mm menggunakan teks) ==========
print("\n" + "=" * 60)
print("Uji 2: Pencarian teks-ke-gambar/video - cari aset gambar dan video terkait pisang menggunakan deskripsi teks")
print("=" * 60)
results = client.search(
    collection_name=collection_name,
    data=['yellow bananas'],
    anns_field='dense_mm',
    limit=3,
    output_fields=['document', 'url'],
)
for hits in results:
    for hit in hits:
        has_image = "has image" if hit['entity'].get('url') else "no image"
        print(f"  id={hit['id']}, distance={hit['distance']:.4f}, [{has_image}] document={hit['entity']['document'][:80]}")

# ========== Uji 3: Pencarian gambar-ke-gambar/video (kueri konten multimedia melalui bidang dense_mm menggunakan aset visual) ==========
print("\n" + "=" * 60)
print("Uji 3: Pencarian gambar-ke-gambar/video - cari gambar dan video serupa menggunakan aset visual dari direktori jeruk")
print("=" * 60)
results = client.search(
    collection_name=collection_name,
    data=[orange_url_list[0]],
    anns_field='dense_mm',
    limit=3,
    output_fields=['document', 'url'],
)
for hits in results:
    for hit in hits:
        has_media = "has media" if hit['entity'].get('url') else "no media"
        print(f"  id={hit['id']}, distance={hit['distance']:.4f}, [{has_media}] document={hit['entity']['document'][:80]}")

Output

Preparing to insert 27 records, batch_size=20...
  Inserted 20/27 records
  Inserted 27/27 records
Data insertion complete!

============================================================
Uji 1: Pencarian teks-ke-teks - kueri konten terkait database vektor
============================================================
  id=26, distance=0.8197, document=A vector database converts text into high-dimensional vectors for semantic search, understanding the true intent of queries.
  id=27, distance=0.3091, document=Deep learning models learn feature representations from large image datasets for tasks such as image classification and object detection.
  id=22, distance=0.2337, document=A short video of a fruit store display, with close-up shots of the fruit details.

============================================================
Uji 2: Pencarian teks-ke-gambar/video - cari aset gambar dan video terkait pisang menggunakan deskripsi teks
============================================================
  id=9, distance=0.4862, [has image] document=A fruit vendor weighing a large bunch of fresh bananas for a customer, the scale showing exactly three jin.
  id=1, distance=0.4834, [has image] document=A bunch of ripe yellow bananas hanging from a tree in a tropical orchard, with sunlight casting dappled shadows through the leaves.
  id=7, distance=0.4797, [has image] document=Wild banana trees growing in clusters in the tropical rainforest, their large leaves gently swaying in the breeze.

============================================================
Uji 3: Pencarian gambar-ke-gambar/video - cari gambar dan video serupa menggunakan aset visual dari direktori jeruk
============================================================
  id=11, distance=1.0000, [has media] document=Oranges hanging from branches in the orchard gleam golden in the sunlight, a joyful harvest season.
  id=12, distance=0.9898, [has media] document=A glass of freshly squeezed orange juice on the table, with cut oranges beside it, the flesh plump and juicy.
  id=13, distance=0.9858, [has media] document=Mom is peeling oranges in the kitchen, the air filled with a fresh citrus fragrance, as the children gather around waiting to eat.