Search by image - Alibaba Cloud Model Studio - Centre de documentation Alibaba Cloud

L'outil de recherche par image permet au modèle de rechercher sur Internet des images visuellement similaires à partir d'une image fournie en entrée. Le modèle peut ensuite analyser les résultats et en tirer des déductions. Cette fonctionnalité s'avère utile pour rechercher des produits similaires ou identifier l'origine d'un contenu visuel.

Utilisation

Appelez la fonctionnalité de recherche par image via l'API Responses. Ajoutez l'outil image_search au paramètre tools et transmettez l'image dans le paramètre input au format multimodal.

Le paramètre input doit contenir le contenu de l'image. Transmettez l'URL de l'image avec le type input_image. Vous pouvez également transmettre du texte avec le type input_text pour fournir des informations supplémentaires pour la recherche.

# Import dependencies and create a client...
input_content = [
    {"type": "input_text", "text": "Find landscape images with a style similar to this one"},
    {"type": "input_image", "image_url": "https://img.alicdn.com/imgextra/i4/O1CN01YbrnSS1qtmsAkw0Ud_!!6000000005554-2-tps-788-450.png"}
]
response = client.responses.create(
    model="qwen3.8-max",
    input=[{"role": "user", "content": input_content}],
    tools=[{"type": "image_search"}]
)

print(response.output_text)

Modèles pris en charge

Modèles recommandés

Pour des performances optimales lors de l'appel d'outils, utilisez les modèles suivants :

Qwen-Plus : séries Qwen3,7-Plus, Qwen3,6-Plus, Qwen3,5-Plus

Qwen-Max : série Qwen3,8-Max, qwen3,7-max-2026-06-08

qwen3,8-27b

Autres modèles

Les modèles suivants prennent également en charge cet outil, mais leurs performances sont inférieures à celles des modèles recommandés.

Qwen-Flash : séries Qwen3,7-Flash, Qwen3,6-Flash, Qwen3,5-Flash

Cette fonctionnalité est uniquement accessible via l'API Responses.

Premiers pas

Exécutez le code suivant pour appeler l'outil de recherche par image via l'API Responses et rechercher des images similaires ou connexes à partir de l'image fournie en entrée.

Avant de commencer, obtenez une clé API et configurez la clé API en tant que variable d'environnement .

Remplacez la valeur image_url dans l'exemple de code par une URL d'image accessible publiquement.

python

import os
import json
from openai import OpenAI

client = OpenAI(
    # If you have not configured the environment variable, replace the following line with your Model Studio API key: api_key="sk-xxx",
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    # The following URL is for the Singapore region. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
    base_url="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"
)

input_content = [
    {"type": "input_text", "text": "Find landscape images with a style similar to this one"},
    # Replace image_url with the actual public URL of the image
    {"type": "input_image", "image_url": "https://img.alicdn.com/imgextra/i4/O1CN01YbrnSS1qtmsAkw0Ud_!!6000000005554-2-tps-788-450.png"}
]

response = client.responses.create(
    model="qwen3.8-max",
    input=[{"role": "user", "content": input_content}],
    tools=[
        {
            "type": "image_search"
        }
    ]
)

# Traverse the output to display each step
for item in response.output:
    if item.type == "image_search_call":
        print(f"[Tool Call] Search by image (status: {item.status})")
        # Parse and display the list of searched images
        if item.output:
            images = json.loads(item.output)
            print(f"  Found {len(images)} images:")
            for img in images[:5]:  # Display the first 5 images
                print(f"  [{img['index']}] {img['title']}")
                print(f"      {img['url']}")
            if len(images) > 5:
                print(f"  ... {len(images)} images in total")
    elif item.type == "message":
        print(f"\n[Model Response]")
        print(response.output_text)

# Display token usage and tool call statistics
print(f"\n[Token Usage] Input: {response.usage.input_tokens}, Output: {response.usage.output_tokens}, Total: {response.usage.total_tokens}")
if hasattr(response.usage, 'x_tools') and response.usage.x_tools:
    for tool_name, info in response.usage.x_tools.items():
        print(f"[Tool Statistics] {tool_name} call count: {info.get('count', 0)}")

javascript

import OpenAI from "openai";
import process from 'process';

const openai = new OpenAI({
    // If you have not configured the environment variable, replace the following line with your Model Studio API key: apiKey: "sk-xxx",
    apiKey: process.env.DASHSCOPE_API_KEY,
    // The following URL is for the Singapore region. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
    baseURL: "https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"
});

async function main() {
    const response = await openai.responses.create({
        model: "qwen3.8-max",
        input: [
            {
                role: "user",
                content: [
                    { type: "input_text", text: "Find landscape images with a style similar to this one" },
                    // Replace image_url with the actual public URL of the image
                    { type: "input_image", image_url: "https://img.alicdn.com/imgextra/i4/O1CN01YbrnSS1qtmsAkw0Ud_!!6000000005554-2-tps-788-450.png" }
                ]
            }
        ],
        tools: [
            { type: "image_search" }
        ]
    });

    // Traverse the output to display each step
    for (const item of response.output) {
        if (item.type === "image_search_call") {
            console.log(`[Tool Call] Search by image (status: ${item.status})`);
            // Parse and display the list of searched images
            if (item.output) {
                const images = JSON.parse(item.output);
                console.log(`  Found ${images.length} images:`);
                images.slice(0, 5).forEach(img => {
                    console.log(`  [${img.index}] ${img.title}`);
                    console.log(`      ${img.url}`);
                });
                if (images.length > 5) {
                    console.log(`  ... ${images.length} images in total`);
                }
            }
        } else if (item.type === "message") {
            console.log(`\n[Model Response]`);
            console.log(response.output_text);
        }
    }

    // Display token usage and tool call statistics
    console.log(`\n[Token Usage] Input: ${response.usage.input_tokens}, Output: ${response.usage.output_tokens}, Total: ${response.usage.total_tokens}`);
    if (response.usage && response.usage.x_tools) {
        for (const [toolName, info] of Object.entries(response.usage.x_tools)) {
            console.log(`[Tool Statistics] ${toolName} call count: ${info.count || 0}`);
        }
    }
}

main();

bash

# The following URL is for the Singapore region. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
curl -X POST https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1/responses \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
    "model": "qwen3.8-max",
    "input": [
        {
            "role": "user",
            "content": [
                {"type": "input_text", "text": "Find landscape images with a style similar to this one"},
                {"type": "input_image", "image_url": "https://img.alicdn.com/imgextra/i4/O1CN01YbrnSS1qtmsAkw0Ud_!!6000000005554-2-tps-788-450.png"}
            ]
        }
    ],
    "tools": [
        {"type": "image_search"}
    ]
}'

Après exécution du code, vous recevez une réponse similaire à celle-ci :

[Tool Call] Search by image (status: completed)
  Found 2 images:
  [1] QingMing Festival Holiday Notice 2024
      https://www.healthcabin.net/blog/wp-content/uploads/2024/04/QingMing-Festival-Holiday-Notice-2024.jpg
  [2] Serene Asian Landscape Stone Bridge Reflecting in Misty Water
      https://thumbs.dreamstime.com/b/serene-asian-landscape-stone-bridge-reflecting-misty-water-tranquil-illustration-traditional-arch-spanning-lake-style-376972039.jpg

[Model Response]
OK. I have found several landscape images with a similar style.

These images all display the artistic conception of typical Chinese ink wash paintings or traditional landscape paintings, and they share the following common points:
*   **Traditional architecture**: such as pavilions, towers, and arch bridges.
*   **Natural elements**: such as distant mountains, lakes, weeping willows, and lotus flowers.
*   **Artistic style**: uses elegant colors and soft lines to create a quiet and serene atmosphere.

...

[Token Usage] Input: 2753, Output: 181, Total: 2934
[Tool Statistics] image_search call count: 1

Sortie en streaming

Le traitement de l'outil de recherche par image prend du temps. Activez la sortie en streaming pour récupérer les résultats intermédiaires en temps réel.

python

import os
import json
from openai import OpenAI

client = OpenAI(
    # If you have not configured the environment variable, replace the following line with your Model Studio API key: api_key="sk-xxx",
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    # The following URL is for the Singapore region. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
    base_url="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"
)

input_content = [
    {"type": "input_text", "text": "Find landscape images with a style similar to this one"},
    # Replace image_url with the actual public URL of the image
    {"type": "input_image", "image_url": "https://img.alicdn.com/imgextra/i4/O1CN01YbrnSS1qtmsAkw0Ud_!!6000000005554-2-tps-788-450.png"}
]

stream = client.responses.create(
    model="qwen3.8-max",
    input=[{"role": "user", "content": input_content}],
    tools=[{"type": "image_search"}],
    stream=True
)

for event in stream:
    # Tool call starts
    if event.type == "response.output_item.added":
        if event.item.type == "image_search_call":
            print("[Tool Call] Searching by image...")
    # Tool call is complete. Parse and display the list of searched images.
    elif event.type == "response.output_item.done":
        if event.item.type == "image_search_call":
            print(f"[Tool Call] Search by image complete (status: {event.item.status})")
            if event.item.output:
                images = json.loads(event.item.output)
                print(f"  Found {len(images)} images:")
                for img in images[:5]:  # Display the first 5 images
                    print(f"  [{img['index']}] {img['title']}")
                    print(f"      {img['url']}")
                if len(images) > 5:
                    print(f"  ... {len(images)} images in total")
    # Model response starts
    elif event.type == "response.content_part.added":
        print(f"\n[Model Response]")
    # Streamed text output
    elif event.type == "response.output_text.delta":
        print(event.delta, end="", flush=True)
    # Response is complete. Output the usage.
    elif event.type == "response.completed":
        usage = event.response.usage
        print(f"\n\n[Token Usage] Input: {usage.input_tokens}, Output: {usage.output_tokens}, Total: {usage.total_tokens}")
        if hasattr(usage, 'x_tools') and usage.x_tools:
            for tool_name, info in usage.x_tools.items():
                print(f"[Tool Statistics] {tool_name} call count: {info.get('count', 0)}")

javascript

import OpenAI from "openai";
import process from 'process';

const openai = new OpenAI({
    // If you have not configured the environment variable, replace the following line with your Model Studio API key: apiKey: "sk-xxx",
    apiKey: process.env.DASHSCOPE_API_KEY,
    // The following URL is for the Singapore region. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
    baseURL: "https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"
});

async function main() {
    const stream = await openai.responses.create({
        model: "qwen3.8-max",
        input: [
            {
                role: "user",
                content: [
                    { type: "input_text", text: "Find landscape images with a style similar to this one" },
                    // Replace image_url with the actual public URL of the image
                    { type: "input_image", image_url: "https://img.alicdn.com/imgextra/i4/O1CN01YbrnSS1qtmsAkw0Ud_!!6000000005554-2-tps-788-450.png" }
                ]
            }
        ],
        tools: [{ type: "image_search" }],
        stream: true
    });

    for await (const event of stream) {
        // Tool call starts
        if (event.type === "response.output_item.added") {
            if (event.item.type === "image_search_call") {
                console.log("[Tool Call] Searching by image...");
            }
        }
        // Tool call is complete. Parse and display the list of searched images.
        else if (event.type === "response.output_item.done") {
            if (event.item && event.item.type === "image_search_call") {
                console.log(`[Tool Call] Search by image complete (status: ${event.item.status})`);
                if (event.item.output) {
                    const images = JSON.parse(event.item.output);
                    console.log(`  Found ${images.length} images:`);
                    images.slice(0, 5).forEach(img => {
                        console.log(`  [${img.index}] ${img.title}`);
                        console.log(`      ${img.url}`);
                    });
                    if (images.length > 5) {
                        console.log(`  ... ${images.length} images in total`);
                    }
                }
            }
        }
        // Model response starts
        else if (event.type === "response.content_part.added") {
            console.log(`\n[Model Response]`);
        }
        // Streamed text output
        else if (event.type === "response.output_text.delta") {
            process.stdout.write(event.delta);
        }
        // Response is complete. Output the usage.
        else if (event.type === "response.completed") {
            const usage = event.response.usage;
            console.log(`\n\n[Token Usage] Input: ${usage.input_tokens}, Output: ${usage.output_tokens}, Total: ${usage.total_tokens}`);
            if (usage && usage.x_tools) {
                for (const [toolName, info] of Object.entries(usage.x_tools)) {
                    console.log(`[Tool Statistics] ${toolName} call count: ${info.count || 0}`);
                }
            }
        }
    }
}

main();

bash

# The following URL is for the Singapore region. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
curl -X POST https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1/responses \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
    "model": "qwen3.8-max",
    "input": [
        {
            "role": "user",
            "content": [
                {"type": "input_text", "text": "Find landscape images with a style similar to this one"},
                {"type": "input_image", "image_url": "https://img.alicdn.com/imgextra/i4/O1CN01YbrnSS1qtmsAkw0Ud_!!6000000005554-2-tps-788-450.png"}
            ]
        }
    ],
    "tools": [
        {"type": "image_search"}
    ],
    "stream": true
}'

Après exécution du code, vous recevez une réponse similaire à celle-ci :

[Tool Call] Searching by image...
[Tool Call] Search by image complete (status: completed)
  Found 3 images:
  [1] QingMing Festival Holiday Notice 2024
      https://www.healthcabin.net/blog/wp-content/uploads/2024/04/QingMing-Festival-Holiday-Notice-2024.jpg
  [2] Serene Asian Landscape Stone Bridge Reflecting in Misty Water
      https://thumbs.dreamstime.com/b/serene-asian-landscape-stone-bridge-reflecting-misty-water-...
  [3] ...

[Model Response]
OK. I have found several landscape images with a similar style. These images all display the style of typical Chinese ink wash or fine-brush paintings...

[Token Usage] Input: 5339, Output: 164, Total: 5503
[Tool Statistics] image_search call count: 1

Facturation

La facturation comprend les aspects suivants :

Frais d'appel de modèle : les résultats de la recherche d'images sont ajoutés à l'invite. Cela augmente le nombre de jetons d'entrée pour le modèle. La facturation s'effectue selon le tarif standard du modèle. Pour plus de détails sur les tarifs, consultez la console Model Studio.
Frais d'appel d'outil : la facturation s'effectue par tranche de 1 000 appels. Le tarif est de 8,00 USD pour les déploiements dans la région de Singapour, tandis que le tarif pour les déploiements dans la région Chine du Nord 2 (Pékin) est de 6 881 USD.

FAQ

Q : Quels formats d'image et méthodes d'entrée sont pris en charge ?

R : Pour plus d'informations, consultez les sections Limites des images et Méthodes d'entrée de fichier.

Le SDK OpenAI ne prend pas en charge la transmission de chemins de fichiers locaux.

Q : Combien d'images puis-je transmettre ?

R : Le nombre d'images que vous pouvez transmettre est limité par la longueur d'entrée maximale du modèle. Le nombre total de jetons pour les images et le texte ne doit pas dépasser la valeur maximale prise en charge par le modèle. Le modèle ne recherche qu'une seule image par appel, mais vous pouvez effectuer plusieurs appels pour traiter plusieurs images.

Le modèle décide du nombre d'images à rechercher.

Q : Combien d'images recherchées sont renvoyées ?

R : Le modèle détermine le nombre d'images à renvoyer. La quantité n'est pas fixe, mais le maximum est de 100 images.