Génération multimodale - Vector Retrieval Service for Milvus

La fonction AI_MULTI_MODAL_GENERATE convertit du texte, des images, des vidéos ou de l'audio en sortie textuelle. Utilisez-la pour décrire des supports produits, résumer des vidéos, répondre à des questions sur des visuels ou transcrire la parole.

Format de la commande

La génération multimodale prend en charge deux modes d'utilisation :

API REST — Appelez directement l'endpoint pour générer du texte à partir de votre entrée. Privilégiez cette approche pour les tâches de génération ponctuelles ou par lots.
Fonction de collection — Définissez une fonction TEXTTRANSFORM dans le schéma de votre collection afin de générer automatiquement du texte lors de l'insertion des données. Cette méthode permet d'intégrer la génération directement dans votre pipeline d'ingestion de données.

Les exemples d'API REST nécessitent curl et jq. Les exemples Python requièrent pymilvus. Remplacez l'URL de l'endpoint et les identifiants dans chaque exemple par les valeurs correspondant à votre instance Milvus.

REST API

@id="cmdtabbmmg001"

REST API

@id="c0058k7q2t58"

POST /v2/vectordb/ai/multi_modal_generate
{
  "model_name": "<model_name>",
  "texts": ["<text_or_media_url>"],
  "params": {"media_type": "text | image | video | audio", "prompt": "<instruction>"}
}

@id="cmdrestsmmg01"

@id="cmdresthmmg01"

Python (Fonction de collection)

@id="cmdpysecmmg01"

Python (Fonction de collection)

@id="cmdpycodmmg01"

schema = MilvusClient.create_schema(auto_id=True, enable_dynamic_field=False)
schema.add_field("id", DataType.INT64, is_primary=True)
schema.add_field("content", DataType.VARCHAR, max_length=4096)
schema.add_field("generated", DataType.VARCHAR, max_length=1024)
schema.add_field("dummy_vector", DataType.FLOAT_VECTOR, dim=2)
schema.add_function(
    Function(
        name="generate_text",
        function_type=texttransform_function_type(),
        input_field_names=["content"],
        output_field_names=["generated"],
        params={
            "provider": "aliyun_milvus",
            "model_name": "<model_name>",
            "task": "ai_multi_modal_generate",
            "media_type": "text",
            "prompt": "Answer concisely: ${content}",
            "temperature": "0",
        },
    )
)

@id="cmdpyhmmg0001"

Paramètres

@id="p0067k7q2t67" Paramètre	@id="p0069k7q2t69" Description
@id="p0003k7q2t03" `model_name`	@id="p0005k7q2t05" Obligatoire. Pour le texte, les images et les vidéos, utilisez `qwen3.7-plus` (recommandé), `qwen3.6-plus`, `qwen3.6-flash`, `qwen3.5-flash` ou `kimi-k2.6`. Pour l'audio, utilisez `qwen3-asr-flash`.
@id="p0008k7q2t08" `texts`	@id="p0010k7q2t10" Obligatoire pour l'API REST. Pour le type de média `text`, transmettez le texte original. Pour les autres types de médias, transmettez une URL accessible par le modèle. Les images peuvent également être transmises en Base64. Les résultats respectent l'ordre des entrées.
@id="p0013k7q2t13" `media_type`	@id="p0015k7q2t15" Obligatoire. Valeurs valides : `text`, `image`, `video`, `audio`.
@id="p0018k7q2t18" `prompt`	@id="p0020k7q2t20" Obligatoire pour `image` et `video`. Facultatif pour `text`. Non pris en charge pour `audio`. Dans un schéma de fonction de collection, utilisez `${field_name}` pour référencer les valeurs des champs.
`temperature`	Facultatif. Contrôle le comportement de la génération de sortie. Transmis sous forme de nombre dans les appels d'API REST ou sous forme de chaîne dans les paramètres de la fonction de collection. Défini sur `0` dans tous les exemples.
@id="p0023k7q2t23" `language` / `enable_itn`	@id="p0025k7q2t25" Facultatif. Audio uniquement. `language` spécifie la langue de reconnaissance (par exemple, `zh`, `en`). `enable_itn` contrôle l'activation de la normalisation inverse du texte.
@id="p0028k7q2t28" `timeout_sec` / `max_concurrency`	@id="p0030k7q2t30" Facultatif. Contrôle le délai d'expiration par appel et la concurrence par lot.
@id="p0033k7q2t33" `output_mapping`	@id="p0035k7q2t35" Facultatif. Uniquement pour les scénarios de fonction de collection `text`, `image` et `video`. Mappe les chemins simples dans la sortie JSON du modèle vers plusieurs champs `VARCHAR`, `TEXT` ou `JSON`. Sans ce paramètre, un seul champ de sortie texte est autorisé.
@id="p0038k7q2t38" `provider` / `task`	@id="p0040k7q2t40" Obligatoire uniquement pour la fonction de collection. Valeurs fixes : `aliyun_milvus` et `ai_multi_modal_generate`.

Valeurs de retour

En cas de succès, la réponse contient les champs suivants :

**data.output.outputs** — Tableau de texte. Chaque élément correspond un à un au tableau d'entrée texts.
**data.usage** — Métriques d'utilisation pouvant inclure :
- input_tokens, output_tokens, total_tokens — Nombre de jetons.
- image_tokens, video_tokens, audio_tokens — Nombre de jetons spécifiques aux médias, le cas échéant.
- seconds — Durée du média, le cas échéant.

Exemple 1 : Générer une réponse textuelle concise (texte)

Cet exemple génère une réponse textuelle concise à partir d'une entrée textuelle en utilisant le type de média text.

REST API

@id="tab0076k7q2t76"

REST API

@id="c0079k7q2t79"

#!/usr/bin/env bash
set -euo pipefail

MILVUS_REST_BASE_URL="http://c-xxxx.milvus.aliyuncs.com:19530"
MILVUS_AUTH_TOKEN="<yourUsername>:<yourPassword>"

post_json() {
  local path="$1"
  local body="$2"
  curl -X POST \
    "$MILVUS_REST_BASE_URL$path" \
    -H "Authorization: Bearer $MILVUS_AUTH_TOKEN" \
    -H "Content-Type: application/json" \
    -d "$body"
}

BODY=$(cat <<JSON
{
  "model_name": "qwen3.7-plus",
  "texts": ["Explain in one sentence how Milvus is used in a RAG application."],
  "params": {"media_type": "text", "prompt": "Answer concisely.", "temperature": 0}
}
JSON
)
RESPONSE_BODY="$(post_json "/v2/vectordb/ai/multi_modal_generate" "$BODY")"
if command -v jq >/dev/null 2>&1; then
  echo "$RESPONSE_BODY" | jq .
  [ "$(echo "$RESPONSE_BODY" | jq -r '.code // -1')" = "0" ] || exit 1
else
  echo "$RESPONSE_BODY"
fi

@id="rt0077k7q2t77"

@id="h0078k7q2t78"

Python

@id="pt0080k7q2t80"

Python

@id="c0082k7q2t82"

from __future__ import annotations

from typing import Any

from pymilvus import DataType, Function, FunctionType, MilvusClient

MILVUS_URI = "http://c-xxxx.milvus.aliyuncs.com:19530"
MILVUS_TOKEN = "<yourUsername>:<yourPassword>"

DUMMY_VECTOR_DIM = 2
TEXTTRANSFORM_FUNCTION_TYPE = 9

def texttransform_function_type() -> Any:
    for type_name in ("TEXTTRANSFORM", "TEXT_TRANSFORM", "TextTransform"):
        function_type = getattr(FunctionType, type_name, None)
        if function_type is not None:
            return function_type
    # Alibaba Cloud Milvus provides TEXTTRANSFORM as a managed extension (function type value 9);
    # some pymilvus versions do not yet have this enum member built-in, and Function(...) validates via FunctionType(...).
    existing = getattr(FunctionType, "_value2member_map_", {}).get(TEXTTRANSFORM_FUNCTION_TYPE)
    if existing is not None:
        return existing
    extension = int.__new__(FunctionType, TEXTTRANSFORM_FUNCTION_TYPE)
    extension._name_ = "TEXTTRANSFORM"
    extension._value_ = TEXTTRANSFORM_FUNCTION_TYPE
    FunctionType._value2member_map_[TEXTTRANSFORM_FUNCTION_TYPE] = extension
    FunctionType._member_map_["TEXTTRANSFORM"] = extension
    return extension

def add_id(schema: Any) -> None:
    schema.add_field("id", DataType.INT64, is_primary=True)

def add_dummy_vector(schema: Any) -> None:
    schema.add_field("dummy_vector", DataType.FLOAT_VECTOR, dim=DUMMY_VECTOR_DIM)

def run_texttransform_example(*, client, collection_name, input_fields, output_field, function_name, function_params, rows) -> None:
    if client.has_collection(collection_name):
        client.drop_collection(collection_name)
    schema = MilvusClient.create_schema(auto_id=True, enable_dynamic_field=False)
    add_id(schema)
    for name, data_type, max_length in input_fields:
        field_params = {"max_length": max_length} if max_length is not None else {}
        schema.add_field(name, data_type, **field_params)
    output_name, output_data_type, output_max_length = output_field
    output_params = {"max_length": output_max_length} if output_max_length is not None else {}
    schema.add_field(output_name, output_data_type, **output_params)
    add_dummy_vector(schema)
    schema.add_function(
        Function(
            name=function_name,
            function_type=texttransform_function_type(),
            input_field_names=[name for name, _, _ in input_fields],
            output_field_names=[output_name],
            params=function_params,
        )
    )
    index_params = client.prepare_index_params()
    index_params.add_index(field_name="dummy_vector", index_type="AUTOINDEX", metric_type="COSINE")
    client.create_collection(collection_name=collection_name, schema=schema, index_params=index_params)
    client.insert(collection_name, rows)
    client.flush(collection_name)
    fields = [name for name, _, _ in input_fields] + [output_name]
    for row in client.query(collection_name, filter="", output_fields=fields, limit=len(rows)):
        print(row)

client = MilvusClient(uri=MILVUS_URI, token=MILVUS_TOKEN)

run_texttransform_example(
    client=client,
    collection_name="simple_ai_multi_modal_generate_text",
    input_fields=[("content", DataType.VARCHAR, 4096)],
    output_field=("generated", DataType.VARCHAR, 1024),
    function_name="generate_text",
    function_params={"provider": "aliyun_milvus", "model_name": "qwen3.7-plus", "task": "ai_multi_modal_generate", "media_type": "text", "prompt": "Answer concisely: ${content}", "temperature": "0"},
    rows=[{"content": "Explain how Milvus is used in a RAG application.", "dummy_vector": [0.1, 0.2]}],
)

@id="h0081k7q2t81"

Résultat attendu : Le champ generated contient une réponse concise décrivant l'utilisation de Milvus dans une application RAG. Dans la réponse de l'API REST, cette valeur se trouve à l'emplacement data.output.outputs[0].

Exemple 2 : Générer des descriptions pour la recherche à partir d'images de produits (image)

Cet exemple génère une description objective pour la recherche image-texte avant l'indexation des supports vestimentaires. L'exemple bash nécessite jq.

REST API

@id="tab0087k7q2t87"

REST API

@id="c0090k7q2t90"

#!/usr/bin/env bash
set -euo pipefail

MILVUS_REST_BASE_URL="http://c-xxxx.milvus.aliyuncs.com:19530"
MILVUS_AUTH_TOKEN="<yourUsername>:<yourPassword>"

post_json() {
  local path="$1"
  local body="$2"
  curl -X POST \
    "$MILVUS_REST_BASE_URL$path" \
    -H "Authorization: Bearer $MILVUS_AUTH_TOKEN" \
    -H "Content-Type: application/json" \
    -d "$body"
}

BODY=$(cat <<JSON
{
  "model_name": "qwen3.7-plus",
  "texts": ["https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20260415/hynnff/wan-video-edit-clothes.webp"],
  "params": {"media_type": "image", "prompt": "Describe the image in one concise sentence.", "temperature": 0}
}
JSON
)
RESPONSE_BODY="$(post_json "/v2/vectordb/ai/multi_modal_generate" "$BODY")"
if command -v jq >/dev/null 2>&1; then
  echo "$RESPONSE_BODY" | jq .
  [ "$(echo "$RESPONSE_BODY" | jq -r '.code // -1')" = "0" ] || exit 1
else
  echo "$RESPONSE_BODY"
fi

@id="rt0088k7q2t88"

@id="h0089k7q2t89"

Python

@id="pt0091k7q2t91"

Python

@id="c0093k7q2t93"

from __future__ import annotations

from typing import Any

from pymilvus import DataType, Function, FunctionType, MilvusClient

MILVUS_URI = "http://c-xxxx.milvus.aliyuncs.com:19530"
MILVUS_TOKEN = "<yourUsername>:<yourPassword>"

DUMMY_VECTOR_DIM = 2
TEXTTRANSFORM_FUNCTION_TYPE = 9

def texttransform_function_type() -> Any:
    for type_name in ("TEXTTRANSFORM", "TEXT_TRANSFORM", "TextTransform"):
        function_type = getattr(FunctionType, type_name, None)
        if function_type is not None:
            return function_type
    # Alibaba Cloud Milvus provides TEXTTRANSFORM as a managed extension (function type value 9);
    # some pymilvus versions do not yet have this enum member built-in, and Function(...) validates via FunctionType(...).
    existing = getattr(FunctionType, "_value2member_map_", {}).get(TEXTTRANSFORM_FUNCTION_TYPE)
    if existing is not None:
        return existing
    extension = int.__new__(FunctionType, TEXTTRANSFORM_FUNCTION_TYPE)
    extension._name_ = "TEXTTRANSFORM"
    extension._value_ = TEXTTRANSFORM_FUNCTION_TYPE
    FunctionType._value2member_map_[TEXTTRANSFORM_FUNCTION_TYPE] = extension
    FunctionType._member_map_["TEXTTRANSFORM"] = extension
    return extension

def add_id(schema: Any) -> None:
    schema.add_field("id", DataType.INT64, is_primary=True)

def add_dummy_vector(schema: Any) -> None:
    schema.add_field("dummy_vector", DataType.FLOAT_VECTOR, dim=DUMMY_VECTOR_DIM)

def run_texttransform_example(*, client, collection_name, input_fields, output_field, function_name, function_params, rows) -> None:
    if client.has_collection(collection_name):
        client.drop_collection(collection_name)
    schema = MilvusClient.create_schema(auto_id=True, enable_dynamic_field=False)
    add_id(schema)
    for name, data_type, max_length in input_fields:
        field_params = {"max_length": max_length} if max_length is not None else {}
        schema.add_field(name, data_type, **field_params)
    output_name, output_data_type, output_max_length = output_field
    output_params = {"max_length": output_max_length} if output_max_length is not None else {}
    schema.add_field(output_name, output_data_type, **output_params)
    add_dummy_vector(schema)
    schema.add_function(
        Function(
            name=function_name,
            function_type=texttransform_function_type(),
            input_field_names=[name for name, _, _ in input_fields],
            output_field_names=[output_name],
            params=function_params,
        )
    )
    index_params = client.prepare_index_params()
    index_params.add_index(field_name="dummy_vector", index_type="AUTOINDEX", metric_type="COSINE")
    client.create_collection(collection_name=collection_name, schema=schema, index_params=index_params)
    client.insert(collection_name, rows)
    client.flush(collection_name)
    fields = [name for name, _, _ in input_fields] + [output_name]
    for row in client.query(collection_name, filter="", output_fields=fields, limit=len(rows)):
        print(row)

client = MilvusClient(uri=MILVUS_URI, token=MILVUS_TOKEN)

run_texttransform_example(
    client=client,
    collection_name="simple_ai_multi_modal_generate_image",
    input_fields=[("image_url", DataType.VARCHAR, 4096)],
    output_field=("description", DataType.VARCHAR, 2048),
    function_name="describe_image",
    function_params={"provider": "aliyun_milvus", "model_name": "qwen3.7-plus", "task": "ai_multi_modal_generate", "media_type": "image", "prompt": "Describe the image in one concise sentence.", "temperature": "0"},
    rows=[{"image_url": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20260415/hynnff/wan-video-edit-clothes.webp", "dummy_vector": [0.1, 0.2]}],
)

@id="h0092k7q2t92"

Résultat attendu : Le champ description contient une description objective concise de l'image, qui peut être écrite dans un champ de recherche de produit.

Exemple 3 : Générer des résumés pour la recherche à partir de vidéos conceptuelles (vidéo)

Cet exemple génère des résumés pour la recherche à partir de vidéos conceptuelles afin de permettre la recherche par personnage, action et atmosphère visuelle. Il valide uniquement que le texte n'est pas vide ; aucune formulation fixe n'est supposée.

REST API

@id="tab0098k7q2t01"

REST API

@id="c0101k7q2t04"

#!/usr/bin/env bash
set -euo pipefail

MILVUS_REST_BASE_URL="http://c-xxxx.milvus.aliyuncs.com:19530"
MILVUS_AUTH_TOKEN="<yourUsername>:<yourPassword>"

post_json() {
  local path="$1"
  local body="$2"
  curl -X POST \
    "$MILVUS_REST_BASE_URL$path" \
    -H "Authorization: Bearer $MILVUS_AUTH_TOKEN" \
    -H "Content-Type: application/json" \
    -d "$body"
}

BODY=$(cat <<JSON
{
  "model_name": "qwen3.7-plus",
  "texts": ["https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20260409/dozxak/Wan_Video_Edit_33_1.mp4"],
  "params": {"media_type": "video", "prompt": "Describe the video in one concise sentence.", "temperature": 0}
}
JSON
)
RESPONSE_BODY="$(post_json "/v2/vectordb/ai/multi_modal_generate" "$BODY")"
if command -v jq >/dev/null 2>&1; then
  echo "$RESPONSE_BODY" | jq .
  [ "$(echo "$RESPONSE_BODY" | jq -r '.code // -1')" = "0" ] || exit 1
else
  echo "$RESPONSE_BODY"
fi

@id="rt0099k7q2t02"

@id="h0100k7q2t03"

Python

@id="pt0102k7q2t05"

Python

@id="c0104k7q2t07"

from __future__ import annotations

from typing import Any

from pymilvus import DataType, Function, FunctionType, MilvusClient

MILVUS_URI = "http://c-xxxx.milvus.aliyuncs.com:19530"
MILVUS_TOKEN = "<yourUsername>:<yourPassword>"

DUMMY_VECTOR_DIM = 2
TEXTTRANSFORM_FUNCTION_TYPE = 9

def texttransform_function_type() -> Any:
    for type_name in ("TEXTTRANSFORM", "TEXT_TRANSFORM", "TextTransform"):
        function_type = getattr(FunctionType, type_name, None)
        if function_type is not None:
            return function_type
    # Alibaba Cloud Milvus provides TEXTTRANSFORM as a managed extension (function type value 9);
    # some pymilvus versions do not yet have this enum member built-in, and Function(...) validates via FunctionType(...).
    existing = getattr(FunctionType, "_value2member_map_", {}).get(TEXTTRANSFORM_FUNCTION_TYPE)
    if existing is not None:
        return existing
    extension = int.__new__(FunctionType, TEXTTRANSFORM_FUNCTION_TYPE)
    extension._name_ = "TEXTTRANSFORM"
    extension._value_ = TEXTTRANSFORM_FUNCTION_TYPE
    FunctionType._value2member_map_[TEXTTRANSFORM_FUNCTION_TYPE] = extension
    FunctionType._member_map_["TEXTTRANSFORM"] = extension
    return extension

def add_id(schema: Any) -> None:
    schema.add_field("id", DataType.INT64, is_primary=True)

def add_dummy_vector(schema: Any) -> None:
    schema.add_field("dummy_vector", DataType.FLOAT_VECTOR, dim=DUMMY_VECTOR_DIM)

def run_texttransform_example(*, client, collection_name, input_fields, output_field, function_name, function_params, rows) -> None:
    if client.has_collection(collection_name):
        client.drop_collection(collection_name)
    schema = MilvusClient.create_schema(auto_id=True, enable_dynamic_field=False)
    add_id(schema)
    for name, data_type, max_length in input_fields:
        field_params = {"max_length": max_length} if max_length is not None else {}
        schema.add_field(name, data_type, **field_params)
    output_name, output_data_type, output_max_length = output_field
    output_params = {"max_length": output_max_length} if output_max_length is not None else {}
    schema.add_field(output_name, output_data_type, **output_params)
    add_dummy_vector(schema)
    schema.add_function(
        Function(
            name=function_name,
            function_type=texttransform_function_type(),
            input_field_names=[name for name, _, _ in input_fields],
            output_field_names=[output_name],
            params=function_params,
        )
    )
    index_params = client.prepare_index_params()
    index_params.add_index(field_name="dummy_vector", index_type="AUTOINDEX", metric_type="COSINE")
    client.create_collection(collection_name=collection_name, schema=schema, index_params=index_params)
    client.insert(collection_name, rows)
    client.flush(collection_name)
    fields = [name for name, _, _ in input_fields] + [output_name]
    for row in client.query(collection_name, filter="", output_fields=fields, limit=len(rows)):
        print(row)

client = MilvusClient(uri=MILVUS_URI, token=MILVUS_TOKEN)

run_texttransform_example(
    client=client,
    collection_name="simple_ai_multi_modal_generate_video",
    input_fields=[("video_url", DataType.VARCHAR, 4096)],
    output_field=("description", DataType.VARCHAR, 2048),
    function_name="describe_video",
    function_params={"provider": "aliyun_milvus", "model_name": "qwen3.7-plus", "task": "ai_multi_modal_generate", "media_type": "video", "prompt": "Describe the video in one concise sentence.", "temperature": "0"},
    rows=[{"video_url": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20260409/dozxak/Wan_Video_Edit_33_1.mp4", "dummy_vector": [0.1, 0.2]}],
)

@id="h0103k7q2t06"

Résultat attendu : Le champ description contient un résumé en une phrase du contenu vidéo, utilisable comme champ de recherche de matériel créatif.

Exemple 4 : Transcrire et archiver un message d'accueil téléphonique (audio)

Cet exemple transcrit un message d'accueil téléphonique en texte consultable pour archivage. Les requêtes audio ne nécessitent pas de prompt. L'exemple bash nécessite jq. L'audio d'exemple est un message d'accueil public ; la langue de transcription dépend du contenu audio réel.

REST API

@id="tab0109k7q2t12"

REST API

@id="c0112k7q2t15"

#!/usr/bin/env bash
set -euo pipefail

MILVUS_REST_BASE_URL="http://c-xxxx.milvus.aliyuncs.com:19530"
MILVUS_AUTH_TOKEN="<yourUsername>:<yourPassword>"

post_json() {
  local path="$1"
  local body="$2"
  curl -X POST \
    "$MILVUS_REST_BASE_URL$path" \
    -H "Authorization: Bearer $MILVUS_AUTH_TOKEN" \
    -H "Content-Type: application/json" \
    -d "$body"
}

BODY=$(cat <<JSON
{
  "model_name": "qwen3-asr-flash",
  "texts": ["https://dashscope.oss-cn-beijing.aliyuncs.com/audios/welcome.mp3"],
  "params": {"media_type": "audio", "language": "en", "enable_itn": true}
}
JSON
)
RESPONSE_BODY="$(post_json "/v2/vectordb/ai/multi_modal_generate" "$BODY")"
if command -v jq >/dev/null 2>&1; then
  echo "$RESPONSE_BODY" | jq .
  [ "$(echo "$RESPONSE_BODY" | jq -r '.code // -1')" = "0" ] || exit 1
else
  echo "$RESPONSE_BODY"
fi

@id="rt0110k7q2t13"

@id="h0111k7q2t14"

Python

@id="pt0113k7q2t16"

Python

@id="c0115k7q2t18"

from __future__ import annotations

from typing import Any

from pymilvus import DataType, Function, FunctionType, MilvusClient

MILVUS_URI = "http://c-xxxx.milvus.aliyuncs.com:19530"
MILVUS_TOKEN = "<yourUsername>:<yourPassword>"

DUMMY_VECTOR_DIM = 2
TEXTTRANSFORM_FUNCTION_TYPE = 9

def texttransform_function_type() -> Any:
    for type_name in ("TEXTTRANSFORM", "TEXT_TRANSFORM", "TextTransform"):
        function_type = getattr(FunctionType, type_name, None)
        if function_type is not None:
            return function_type
    # Alibaba Cloud Milvus provides TEXTTRANSFORM as a managed extension (function type value 9);
    # some pymilvus versions do not yet have this enum member built-in, and Function(...) validates via FunctionType(...).
    existing = getattr(FunctionType, "_value2member_map_", {}).get(TEXTTRANSFORM_FUNCTION_TYPE)
    if existing is not None:
        return existing
    extension = int.__new__(FunctionType, TEXTTRANSFORM_FUNCTION_TYPE)
    extension._name_ = "TEXTTRANSFORM"
    extension._value_ = TEXTTRANSFORM_FUNCTION_TYPE
    FunctionType._value2member_map_[TEXTTRANSFORM_FUNCTION_TYPE] = extension
    FunctionType._member_map_["TEXTTRANSFORM"] = extension
    return extension

def add_id(schema: Any) -> None:
    schema.add_field("id", DataType.INT64, is_primary=True)

def add_dummy_vector(schema: Any) -> None:
    schema.add_field("dummy_vector", DataType.FLOAT_VECTOR, dim=DUMMY_VECTOR_DIM)

def run_texttransform_example(*, client, collection_name, input_fields, output_field, function_name, function_params, rows) -> None:
    if client.has_collection(collection_name):
        client.drop_collection(collection_name)
    schema = MilvusClient.create_schema(auto_id=True, enable_dynamic_field=False)
    add_id(schema)
    for name, data_type, max_length in input_fields:
        field_params = {"max_length": max_length} if max_length is not None else {}
        schema.add_field(name, data_type, **field_params)
    output_name, output_data_type, output_max_length = output_field
    output_params = {"max_length": output_max_length} if output_max_length is not None else {}
    schema.add_field(output_name, output_data_type, **output_params)
    add_dummy_vector(schema)
    schema.add_function(
        Function(
            name=function_name,
            function_type=texttransform_function_type(),
            input_field_names=[name for name, _, _ in input_fields],
            output_field_names=[output_name],
            params=function_params,
        )
    )
    index_params = client.prepare_index_params()
    index_params.add_index(field_name="dummy_vector", index_type="AUTOINDEX", metric_type="COSINE")
    client.create_collection(collection_name=collection_name, schema=schema, index_params=index_params)
    client.insert(collection_name, rows)
    client.flush(collection_name)
    fields = [name for name, _, _ in input_fields] + [output_name]
    for row in client.query(collection_name, filter="", output_fields=fields, limit=len(rows)):
        print(row)

client = MilvusClient(uri=MILVUS_URI, token=MILVUS_TOKEN)

run_texttransform_example(
    client=client,
    collection_name="simple_ai_multi_modal_generate_audio",
    input_fields=[("audio_url", DataType.VARCHAR, 4096)],
    output_field=("transcript", DataType.VARCHAR, 4096),
    function_name="transcribe_audio",
    function_params={"provider": "aliyun_milvus", "model_name": "qwen3-asr-flash", "task": "ai_multi_modal_generate", "media_type": "audio", "language": "en", "enable_itn": "true"},
    rows=[{"audio_url": "https://dashscope.oss-cn-beijing.aliyuncs.com/audios/welcome.mp3", "dummy_vector": [0.1, 0.2]}],
)

@id="h0114k7q2t17"

Résultat attendu : Le champ transcript contient un texte de transcription non vide. L'audio d'exemple se transcrit en Welcome to Alibaba Cloud., rendu en anglais ; l'API renvoie la transcription dans la langue source de l'audio. La transcription peut être archivée, mais n'utilisez pas l'exemple de code pour tirer automatiquement des conclusions en matière de conformité.

Sécurité et conservation des médias

Avant d'utiliser vos propres images, vidéos ou fichiers audio, assurez-vous d'avoir obtenu les autorisations nécessaires pour le contenu, ainsi que pour les droits à l'image et les droits vocaux. Utilisez des URLs avec privilèges minimaux et date d'expiration, et supprimez les informations personnelles contenues dans les transcriptions et les textes dérivés conformément à votre politique de conservation des données.