Modification d'images - Vector Retrieval Service for Milvus - Centre de documentation Alibaba Cloud

La fonction AI_IMAGE_EDIT génère des URL d'images modifiées à partir d'une ou plusieurs images sources, sur la base d'instructions textuelles. Utilisez cette fonction pour créer des images sur fond blanc, remplacer des arrière-plans, appliquer des transferts de style ou fusionner des images de référence.

Syntaxe

Appelez AI_IMAGE_EDIT directement via l'API REST ou intégrez-la en tant que Collection Function dans le schéma d'une collection Milvus.

REST API

@id="cmdtabbimg001"

REST API

@id="c0067k7q2t67"

{
  "model_name": "wan2.7-image-pro",
  "texts": ["<image_url>"],
  "params": {"prompt": "<edit_instruction>", "n": 1}
}

@id="p0068k7q2t68"

Pour une entrée multi-images, utilisez image_inputs, où chaque tableau interne correspond à une tâche de modification distincte :

@id="c0069k7q2t69"

{
  "model_name": "wan2.7-image-pro",
  "image_inputs": [["<base_image_url>", "<reference_image_url>"]],
  "params": {"prompt": "<edit_instruction>", "n": 2}
}

@id="cmdrestsimg01"

@id="cmdresthimg01"

Python

@id="cmdpysecimg01"

Python

@id="cmdpycodimg01"

schema = MilvusClient.create_schema(auto_id=True, enable_dynamic_field=False)
schema.add_field("id", DataType.INT64, is_primary=True)
schema.add_field("image_url", DataType.VARCHAR, max_length=4096)
schema.add_field("instruction", DataType.VARCHAR, max_length=512)
schema.add_field("edited_image", DataType.VARCHAR, max_length=8192)
schema.add_field("dummy_vector", DataType.FLOAT_VECTOR, dim=2)
schema.add_function(
    Function(
        name="edit_image",
        function_type=texttransform_function_type(),
        input_field_names=["image_url", "instruction"],
        output_field_names=["edited_image"],
        params={
            "provider": "aliyun_milvus",
            "model_name": "wan2.7-image-pro",
            "task": "ai_image_edit",
            "image_fields": "image_url",
            "prompt": "${instruction}",
            "n": "1",
            "size": "1024*1024",
            "watermark": "false",
            "timeout_sec": "180",
        },
    )
)

@id="cmdpyhimg0001"

Paramètres

@id="p0077k7q2t77" Paramètre	@id="p0079k7q2t79" Description
@id="p0003k7q2t03" `model_name`	@id="p0005k7q2t05" Obligatoire. Modèle qui retourne les URL d'images de manière synchrone. Utilisez `wan2.7-image-pro` (recommandé), `wan2.7-image` ou tout autre modèle compatible.
@id="p0008k7q2t08" `prompt`	@id="p0010k7q2t10" Obligatoire. Décrit la modification souhaitée. Dans le schéma d'une Collection Function, référencez les autres champs d'entrée au moyen de `${field_name}`.
@id="p0013k7q2t13" `texts`	@id="p0015k7q2t15" Tableau d'URL d'images pour une entrée mono-image. Chaque élément représente une tâche de modification. Incompatible avec `image_inputs`.
@id="p0018k7q2t18" `image_inputs`	@id="p0020k7q2t20" Tableau bidimensionnel pour les entrées multi-images. Chaque sous-tableau regroupe les images d'une même tâche. Le nombre d'images par tâche doit rester constant au sein d'une requête.
@id="p0023k7q2t23" `image_fields`	@id="p0025k7q2t25" Réservé aux Collection Functions. Facultatif. Indique les champs image ; par défaut, le premier champ d'entrée est utilisé. Accepte une chaîne séparée par des virgules, telle que `base_image,reference_image`, ou un tableau de chaînes JSON.
@id="p0028k7q2t28" `n`	@id="p0030k7q2t30" Facultatif. Valeur par défaut : `1`. Plage valide : 1–6. Si `n=1`, une seule URL d'image est retournée. Pour `n > 1`, une liste d'URL est produite et le champ de sortie de la Collection Function doit adopter le type `JSON` plutôt que `VARCHAR`.
@id="p0033k7q2t33" `size` / `negative_prompt` / `seed` / `bbox_list` / `watermark`	@id="p0035k7q2t35" Facultatif. Transmis au modèle selon ses capacités.
@id="p0038k7q2t38" `timeout_sec` / `max_concurrency`	@id="p0040k7q2t40" Facultatif. Définit respectivement le délai d'expiration par requête et la concurrence des traitements par lot.
@id="p0043k7q2t43" `provider` / `task`	@id="p0045k7q2t45" Obligatoire uniquement pour les Collection Functions. Définissez `provider` sur `aliyun_milvus` et `task` sur `ai_image_edit`.

Valeurs de retour

Si n=1, chaque élément de data.output.outputs correspond à une URL d'image. Lorsque n > 1, chaque élément constitue une chaîne JSON contenant un tableau images. L'objet data.usage peut inclure les champs image_tokens et total_tokens.

Exemple 1 : Remplacement d'arrière-plan sur une image unique (texts, n=1)

Remplacez l'arrière-plan d'une photo représentant une personne et son animal par un fond blanc uni, tout en conservant les sujets principaux. La ressource par défaut est une photographie d'une personne accompagnée d'un chien. L'exemple curl nécessite l'outil jq.

REST API

@id="tab0086k7q2t86"

REST API

@id="c0089k7q2t89"

#!/usr/bin/env bash
set -euo pipefail

MILVUS_REST_BASE_URL="http://c-xxxx.milvus.aliyuncs.com:19530"
MILVUS_AUTH_TOKEN="<yourUsername>:<yourPassword>"

post_json() {
  local path="$1"
  local body="$2"
  curl -X POST \
    "$MILVUS_REST_BASE_URL$path" \
    -H "Authorization: Bearer $MILVUS_AUTH_TOKEN" \
    -H "Content-Type: application/json" \
    -d "$body"
}

BODY=$(cat <<JSON
{
  "model_name": "wan2.7-image-pro",
  "texts": ["https://dashscope.oss-cn-beijing.aliyuncs.com/images/dog_and_girl.jpeg"],
  "params": {
    "prompt": "Keep the main subject and replace the background with a clean white background.",
    "n": 1,
    "size": "1024*1024",
    "watermark": false,
    "timeout_sec": 180
  }
}
JSON
)
RESPONSE_BODY="$(post_json "/v2/vectordb/ai/image_edit" "$BODY")"
if command -v jq >/dev/null 2>&1; then
  echo "$RESPONSE_BODY" | jq .
  [ "$(echo "$RESPONSE_BODY" | jq -r '.code // -1')" = "0" ] || exit 1
else
  echo "$RESPONSE_BODY"
fi

@id="rt0087k7q2t87"

@id="h0088k7q2t88"

Python

@id="pt0090k7q2t90"

Python

@id="c0092k7q2t92"

from __future__ import annotations

from typing import Any

from pymilvus import DataType, Function, FunctionType, MilvusClient

MILVUS_URI = "http://c-xxxx.milvus.aliyuncs.com:19530"
MILVUS_TOKEN = "<yourUsername>:<yourPassword>"

DUMMY_VECTOR_DIM = 2
TEXTTRANSFORM_FUNCTION_TYPE = 9

def texttransform_function_type() -> Any:
    for type_name in ("TEXTTRANSFORM", "TEXT_TRANSFORM", "TextTransform"):
        function_type = getattr(FunctionType, type_name, None)
        if function_type is not None:
            return function_type
    # Alibaba Cloud Milvus provides TEXTTRANSFORM as a hosted extension (function type value 9);
    # some pymilvus versions do not have this enum member built in, while Function(...) validates via FunctionType(...).
    existing = getattr(FunctionType, "_value2member_map_", {}).get(TEXTTRANSFORM_FUNCTION_TYPE)
    if existing is not None:
        return existing
    extension = int.__new__(FunctionType, TEXTTRANSFORM_FUNCTION_TYPE)
    extension._name_ = "TEXTTRANSFORM"
    extension._value_ = TEXTTRANSFORM_FUNCTION_TYPE
    FunctionType._value2member_map_[TEXTTRANSFORM_FUNCTION_TYPE] = extension
    FunctionType._member_map_["TEXTTRANSFORM"] = extension
    return extension

def add_id(schema: Any) -> None:
    schema.add_field("id", DataType.INT64, is_primary=True)

def add_dummy_vector(schema: Any) -> None:
    schema.add_field("dummy_vector", DataType.FLOAT_VECTOR, dim=DUMMY_VECTOR_DIM)

def run_texttransform_example(*, client, collection_name, input_fields, output_field, function_name, function_params, rows) -> None:
    if client.has_collection(collection_name):
        client.drop_collection(collection_name)
    schema = MilvusClient.create_schema(auto_id=True, enable_dynamic_field=False)
    add_id(schema)
    for name, data_type, max_length in input_fields:
        field_params = {"max_length": max_length} if max_length is not None else {}
        schema.add_field(name, data_type, **field_params)
    output_name, output_data_type, output_max_length = output_field
    output_params = {"max_length": output_max_length} if output_max_length is not None else {}
    schema.add_field(output_name, output_data_type, **output_params)
    add_dummy_vector(schema)
    schema.add_function(
        Function(
            name=function_name,
            function_type=texttransform_function_type(),
            input_field_names=[name for name, _, _ in input_fields],
            output_field_names=[output_name],
            params=function_params,
        )
    )
    index_params = client.prepare_index_params()
    index_params.add_index(field_name="dummy_vector", index_type="AUTOINDEX", metric_type="COSINE")
    client.create_collection(collection_name=collection_name, schema=schema, index_params=index_params)
    client.insert(collection_name, rows)
    client.flush(collection_name)
    fields = [name for name, _, _ in input_fields] + [output_name]
    for row in client.query(collection_name, filter="", output_fields=fields, limit=len(rows)):
        print(row)

MODEL_NAME = "wan2.7-image-pro"
client = MilvusClient(uri=MILVUS_URI, token=MILVUS_TOKEN)

run_texttransform_example(
    client=client,
    collection_name="simple_ai_image_edit_schema",
    input_fields=[("image_url", DataType.VARCHAR, 4096), ("instruction", DataType.VARCHAR, 512)],
    output_field=("edited_image", DataType.VARCHAR, 8192),
    function_name="edit_image",
    function_params={"provider": "aliyun_milvus", "model_name": MODEL_NAME, "task": "ai_image_edit", "image_fields": "image_url", "prompt": "${instruction}", "n": "1", "size": "1024*1024", "watermark": "false", "timeout_sec": "180"},
    rows=[{"image_url": "https://dashscope.oss-cn-beijing.aliyuncs.com/images/dog_and_girl.jpeg", "instruction": "Keep the main subject and replace the background with a clean white background.", "dummy_vector": [0.1, 0.2]}],
)

@id="h0091k7q2t91"

Résultat attendu : data.output.outputs[0] contient une URL d'image modifiée non vide, stockée dans le champ edited_image (VARCHAR). L'image générée étant non déterministe, ne la comparez pas à un rendu visuel fixe.

Exemple 2 : Génération d'une image conceptuelle de marque à partir d'une photo personne-animal et d'une référence vestimentaire

Utilisez une photo d'une personne avec son animal comme référence de sujet et un tissu vestimentaire comme référence de couleur et de texture pour produire deux images conceptuelles destinées aux réseaux sociaux en une seule requête, en vue d'une validation manuelle.

REST API

@id="tab0096k7q2t96"

REST API

@id="c0099k7q2t02"

#!/usr/bin/env bash
set -euo pipefail

MILVUS_REST_BASE_URL="http://c-xxxx.milvus.aliyuncs.com:19530"
MILVUS_AUTH_TOKEN="<yourUsername>:<yourPassword>"

post_json() {
  local path="$1"
  local body="$2"
  curl -X POST \
    "$MILVUS_REST_BASE_URL$path" \
    -H "Authorization: Bearer $MILVUS_AUTH_TOKEN" \
    -H "Content-Type: application/json" \
    -d "$body"
}

BODY=$(cat <<JSON
{
  "model_name": "wan2.7-image-pro",
  "image_inputs": [["https://dashscope.oss-cn-beijing.aliyuncs.com/images/dog_and_girl.jpeg", "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20260415/hynnff/wan-video-edit-clothes.webp"]],
  "params": {
    "prompt": "Keep the person, dog, and seaside composition from the first image. Use the second image only as clothing color and texture reference. Generate brand-safe social media concept images.",
    "n": 2,
    "size": "1024*1024",
    "watermark": false,
    "timeout_sec": 180
  }
}
JSON
)
RESPONSE_BODY="$(post_json "/v2/vectordb/ai/image_edit" "$BODY")"
if command -v jq >/dev/null 2>&1; then
  echo "$RESPONSE_BODY" | jq .
  [ "$(echo "$RESPONSE_BODY" | jq -r '.code // -1')" = "0" ] || exit 1
else
  echo "$RESPONSE_BODY"
fi

@id="rt0097k7q2t00"

@id="h0098k7q2t01"

Python

@id="pt0100k7q2t03"

Python

@id="c0102k7q2t05"

from __future__ import annotations

from typing import Any

from pymilvus import DataType, Function, FunctionType, MilvusClient

MILVUS_URI = "http://c-xxxx.milvus.aliyuncs.com:19530"
MILVUS_TOKEN = "<yourUsername>:<yourPassword>"

DUMMY_VECTOR_DIM = 2
TEXTTRANSFORM_FUNCTION_TYPE = 9

def texttransform_function_type() -> Any:
    for type_name in ("TEXTTRANSFORM", "TEXT_TRANSFORM", "TextTransform"):
        function_type = getattr(FunctionType, type_name, None)
        if function_type is not None:
            return function_type
    # Alibaba Cloud Milvus provides TEXTTRANSFORM as a hosted extension (function type value 9);
    # some pymilvus versions do not have this enum member built in, while Function(...) validates via FunctionType(...).
    existing = getattr(FunctionType, "_value2member_map_", {}).get(TEXTTRANSFORM_FUNCTION_TYPE)
    if existing is not None:
        return existing
    extension = int.__new__(FunctionType, TEXTTRANSFORM_FUNCTION_TYPE)
    extension._name_ = "TEXTTRANSFORM"
    extension._value_ = TEXTTRANSFORM_FUNCTION_TYPE
    FunctionType._value2member_map_[TEXTTRANSFORM_FUNCTION_TYPE] = extension
    FunctionType._member_map_["TEXTTRANSFORM"] = extension
    return extension

def add_id(schema: Any) -> None:
    schema.add_field("id", DataType.INT64, is_primary=True)

def add_dummy_vector(schema: Any) -> None:
    schema.add_field("dummy_vector", DataType.FLOAT_VECTOR, dim=DUMMY_VECTOR_DIM)

def run_texttransform_example(*, client, collection_name, input_fields, output_field, function_name, function_params, rows) -> None:
    if client.has_collection(collection_name):
        client.drop_collection(collection_name)
    schema = MilvusClient.create_schema(auto_id=True, enable_dynamic_field=False)
    add_id(schema)
    for name, data_type, max_length in input_fields:
        field_params = {"max_length": max_length} if max_length is not None else {}
        schema.add_field(name, data_type, **field_params)
    output_name, output_data_type, output_max_length = output_field
    output_params = {"max_length": output_max_length} if output_max_length is not None else {}
    schema.add_field(output_name, output_data_type, **output_params)
    add_dummy_vector(schema)
    schema.add_function(
        Function(
            name=function_name,
            function_type=texttransform_function_type(),
            input_field_names=[name for name, _, _ in input_fields],
            output_field_names=[output_name],
            params=function_params,
        )
    )
    index_params = client.prepare_index_params()
    index_params.add_index(field_name="dummy_vector", index_type="AUTOINDEX", metric_type="COSINE")
    client.create_collection(collection_name=collection_name, schema=schema, index_params=index_params)
    client.insert(collection_name, rows)
    client.flush(collection_name)
    fields = [name for name, _, _ in input_fields] + [output_name]
    for row in client.query(collection_name, filter="", output_fields=fields, limit=len(rows)):
        print(row)

MODEL_NAME = "wan2.7-image-pro"
client = MilvusClient(uri=MILVUS_URI, token=MILVUS_TOKEN)

# Each item in image_inputs contains a base image and a reference image; n=2 returns two candidate results.
# Note: for ai_image_edit, when n > 1, the output field must be DataType.JSON (VARCHAR is not supported).
run_texttransform_example(
    client=client,
    collection_name="simple_ai_image_edit_multi_schema",
    input_fields=[
        ("base_image_url", DataType.VARCHAR, 4096),
        ("reference_image_url", DataType.VARCHAR, 4096),
        ("instruction", DataType.VARCHAR, 1024),
    ],
    output_field=("edited_images", DataType.JSON, None),
    function_name="edit_images_with_reference",
    function_params={
        "provider": "aliyun_milvus",
        "model_name": MODEL_NAME,
        "task": "ai_image_edit",
        "image_fields": '["base_image_url","reference_image_url"]',
        "prompt": "${instruction}",
        "n": "2",
        "size": "1024*1024",
        "watermark": "false",
        "timeout_sec": "180",
    },
    rows=[
        {
            "base_image_url": "https://dashscope.oss-cn-beijing.aliyuncs.com/images/dog_and_girl.jpeg",
            "reference_image_url": "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20260415/hynnff/wan-video-edit-clothes.webp",
            "instruction": "Keep the person, dog, and seaside composition from the base image. Use the reference image only as clothing color and texture guidance. Generate brand-safe social media concept images.",
            "dummy_vector": [0.1, 0.2],
        }
    ],
)

@id="h0101k7q2t04"

Résultat attendu : Comme n=2, data.output.outputs[0] est une chaîne JSON contenant un tableau images avec deux URL d'images non vides.

Important

@id="bq0048k7q2t48" Lorsque n > 1 pour la tâche ai_image_edit, le champ de sortie de la Collection Function doit utiliser le type JSON au lieu de VARCHAR. Sinon, la création de la collection échoue avec l'erreur suivante : output field must be a JSON field for task [ai_image_edit] when n > 1.