Imagen a imagen search_Alibaba Cloud Model Studio(Model Studio) - Alibaba Cloud Model Studio

La herramienta de búsqueda por imagen permite al modelo buscar en Internet imágenes visualmente similares basándose en una imagen de entrada. Luego, el modelo puede analizar los resultados de la búsqueda y realizar inferencias. Esta función es útil para escenarios como encontrar productos similares o rastrear el origen de contenido visual.

Cómo usar

Puede llamar a la función de búsqueda por imagen a través de la API de Responses. Agregue la herramienta image_search al parámetro tools y pase la imagen en el parámetro input en formato multimodal.

El parámetro input debe contener contenido de imagen. Pase la URL de la imagen utilizando el tipo input_image. También puede pasar texto utilizando el tipo input_text para proporcionar información adicional para la búsqueda.

# Import dependencies and create a client...
input_content = [
    {"type": "input_text", "text": "Find landscape images with a style similar to this one"},
    {"type": "input_image", "image_url": "https://img.alicdn.com/imgextra/i4/O1CN01YbrnSS1qtmsAkw0Ud_!!6000000005554-2-tps-788-450.png"}
]
response = client.responses.create(
    model="qwen3.8-max",
    input=[{"role": "user", "content": input_content}],
    tools=[{"type": "image_search"}]
)

print(response.output_text)

Modelos admitidos

Modelos recomendados

Para un rendimiento óptimo de llamadas a herramientas, utilice los siguientes modelos:

Qwen-Plus: serie Qwen3.7-Plus, serie Qwen3.6-Plus, serie Qwen3.5-Plus

Qwen-Max: serie Qwen3.8-Max, qwen3.7-max-2026-06-08

qwen3.8-27b

Otros modelos

Los siguientes modelos también admiten esta herramienta, pero su rendimiento no es tan alto como el de los modelos recomendados.

Qwen-Flash: serie Qwen3.7-Flash, serie Qwen3.6-Flash, serie Qwen3.5-Flash

Esta función solo se puede invocar a través de la Responses API.

Introducción

Puede ejecutar el siguiente código para llamar a la herramienta de búsqueda por imagen a través de la API de Responses y buscar imágenes similares o relacionadas basándose en la imagen de entrada.

Antes de comenzar, obtenga una clave de API y configure la clave de API como variable de entorno.

Reemplace image_url en el código de ejemplo con una URL de imagen accesible públicamente.

python

import os
import json
from openai import OpenAI

client = OpenAI(
    # If you have not configured the environment variable, replace the following line with your Model Studio API key: api_key="sk-xxx",
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    # The following URL is for the Singapore region. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
    base_url="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"
)

input_content = [
    {"type": "input_text", "text": "Find landscape images with a style similar to this one"},
    # Replace image_url with the actual public URL of the image
    {"type": "input_image", "image_url": "https://img.alicdn.com/imgextra/i4/O1CN01YbrnSS1qtmsAkw0Ud_!!6000000005554-2-tps-788-450.png"}
]

response = client.responses.create(
    model="qwen3.8-max",
    input=[{"role": "user", "content": input_content}],
    tools=[
        {
            "type": "image_search"
        }
    ]
)

# Traverse the output to display each step
for item in response.output:
    if item.type == "image_search_call":
        print(f"[Tool Call] Search by image (status: {item.status})")
        # Parse and display the list of searched images
        if item.output:
            images = json.loads(item.output)
            print(f"  Found {len(images)} images:")
            for img in images[:5]:  # Display the first 5 images
                print(f"  [{img['index']}] {img['title']}")
                print(f"      {img['url']}")
            if len(images) > 5:
                print(f"  ... {len(images)} images in total")
    elif item.type == "message":
        print(f"\n[Model Response]")
        print(response.output_text)

# Display token usage and tool call statistics
print(f"\n[Token Usage] Input: {response.usage.input_tokens}, Output: {response.usage.output_tokens}, Total: {response.usage.total_tokens}")
if hasattr(response.usage, 'x_tools') and response.usage.x_tools:
    for tool_name, info in response.usage.x_tools.items():
        print(f"[Tool Statistics] {tool_name} call count: {info.get('count', 0)}")

javascript

import OpenAI from "openai";
import process from 'process';

const openai = new OpenAI({
    // If you have not configured the environment variable, replace the following line with your Model Studio API key: apiKey: "sk-xxx",
    apiKey: process.env.DASHSCOPE_API_KEY,
    // The following URL is for the Singapore region. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
    baseURL: "https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"
});

async function main() {
    const response = await openai.responses.create({
        model: "qwen3.8-max",
        input: [
            {
                role: "user",
                content: [
                    { type: "input_text", text: "Find landscape images with a style similar to this one" },
                    // Replace image_url with the actual public URL of the image
                    { type: "input_image", image_url: "https://img.alicdn.com/imgextra/i4/O1CN01YbrnSS1qtmsAkw0Ud_!!6000000005554-2-tps-788-450.png" }
                ]
            }
        ],
        tools: [
            { type: "image_search" }
        ]
    });

    // Traverse the output to display each step
    for (const item of response.output) {
        if (item.type === "image_search_call") {
            console.log(`[Tool Call] Search by image (status: ${item.status})`);
            // Parse and display the list of searched images
            if (item.output) {
                const images = JSON.parse(item.output);
                console.log(`  Found ${images.length} images:`);
                images.slice(0, 5).forEach(img => {
                    console.log(`  [${img.index}] ${img.title}`);
                    console.log(`      ${img.url}`);
                });
                if (images.length > 5) {
                    console.log(`  ... ${images.length} images in total`);
                }
            }
        } else if (item.type === "message") {
            console.log(`\n[Model Response]`);
            console.log(response.output_text);
        }
    }

    // Display token usage and tool call statistics
    console.log(`\n[Token Usage] Input: ${response.usage.input_tokens}, Output: ${response.usage.output_tokens}, Total: ${response.usage.total_tokens}`);
    if (response.usage && response.usage.x_tools) {
        for (const [toolName, info] of Object.entries(response.usage.x_tools)) {
            console.log(`[Tool Statistics] ${toolName} call count: ${info.count || 0}`);
        }
    }
}

main();

bash

# The following URL is for the Singapore region. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
curl -X POST https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1/responses \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
    "model": "qwen3.8-max",
    "input": [
        {
            "role": "user",
            "content": [
                {"type": "input_text", "text": "Find landscape images with a style similar to this one"},
                {"type": "input_image", "image_url": "https://img.alicdn.com/imgextra/i4/O1CN01YbrnSS1qtmsAkw0Ud_!!6000000005554-2-tps-788-450.png"}
            ]
        }
    ],
    "tools": [
        {"type": "image_search"}
    ]
}'

Después de ejecutar el código, recibirá una respuesta similar a la siguiente:

[Tool Call] Search by image (status: completed)
  Found 2 images:
  [1] QingMing Festival Holiday Notice 2024
      https://www.healthcabin.net/blog/wp-content/uploads/2024/04/QingMing-Festival-Holiday-Notice-2024.jpg
  [2] Serene Asian Landscape Stone Bridge Reflecting in Misty Water
      https://thumbs.dreamstime.com/b/serene-asian-landscape-stone-bridge-reflecting-misty-water-tranquil-illustration-traditional-arch-spanning-lake-style-376972039.jpg

[Model Response]
OK. I have found several landscape images with a similar style.

These images all display the artistic conception of typical Chinese ink wash paintings or traditional landscape paintings, and they share the following common points:
*   **Traditional architecture**: such as pavilions, towers, and arch bridges.
*   **Natural elements**: such as distant mountains, lakes, weeping willows, and lotus flowers.
*   **Artistic style**: uses elegant colors and soft lines to create a quiet and serene atmosphere.

...

[Token Usage] Input: 2753, Output: 181, Total: 2934
[Tool Statistics] image_search call count: 1

Salida por secuencias

La herramienta de búsqueda por imagen tiene un tiempo de procesamiento prolongado. Puede habilitar la salida en streaming para recuperar resultados intermedios en tiempo real.

python

import os
import json
from openai import OpenAI

client = OpenAI(
    # If you have not configured the environment variable, replace the following line with your Model Studio API key: api_key="sk-xxx",
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    # The following URL is for the Singapore region. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
    base_url="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"
)

input_content = [
    {"type": "input_text", "text": "Find landscape images with a style similar to this one"},
    # Replace image_url with the actual public URL of the image
    {"type": "input_image", "image_url": "https://img.alicdn.com/imgextra/i4/O1CN01YbrnSS1qtmsAkw0Ud_!!6000000005554-2-tps-788-450.png"}
]

stream = client.responses.create(
    model="qwen3.8-max",
    input=[{"role": "user", "content": input_content}],
    tools=[{"type": "image_search"}],
    stream=True
)

for event in stream:
    # Tool call starts
    if event.type == "response.output_item.added":
        if event.item.type == "image_search_call":
            print("[Tool Call] Searching by image...")
    # Tool call is complete. Parse and display the list of searched images.
    elif event.type == "response.output_item.done":
        if event.item.type == "image_search_call":
            print(f"[Tool Call] Search by image complete (status: {event.item.status})")
            if event.item.output:
                images = json.loads(event.item.output)
                print(f"  Found {len(images)} images:")
                for img in images[:5]:  # Display the first 5 images
                    print(f"  [{img['index']}] {img['title']}")
                    print(f"      {img['url']}")
                if len(images) > 5:
                    print(f"  ... {len(images)} images in total")
    # Model response starts
    elif event.type == "response.content_part.added":
        print(f"\n[Model Response]")
    # Streamed text output
    elif event.type == "response.output_text.delta":
        print(event.delta, end="", flush=True)
    # Response is complete. Output the usage.
    elif event.type == "response.completed":
        usage = event.response.usage
        print(f"\n\n[Token Usage] Input: {usage.input_tokens}, Output: {usage.output_tokens}, Total: {usage.total_tokens}")
        if hasattr(usage, 'x_tools') and usage.x_tools:
            for tool_name, info in usage.x_tools.items():
                print(f"[Tool Statistics] {tool_name} call count: {info.get('count', 0)}")

javascript

import OpenAI from "openai";
import process from 'process';

const openai = new OpenAI({
    // If you have not configured the environment variable, replace the following line with your Model Studio API key: apiKey: "sk-xxx",
    apiKey: process.env.DASHSCOPE_API_KEY,
    // The following URL is for the Singapore region. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
    baseURL: "https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1"
});

async function main() {
    const stream = await openai.responses.create({
        model: "qwen3.8-max",
        input: [
            {
                role: "user",
                content: [
                    { type: "input_text", text: "Find landscape images with a style similar to this one" },
                    // Replace image_url with the actual public URL of the image
                    { type: "input_image", image_url: "https://img.alicdn.com/imgextra/i4/O1CN01YbrnSS1qtmsAkw0Ud_!!6000000005554-2-tps-788-450.png" }
                ]
            }
        ],
        tools: [{ type: "image_search" }],
        stream: true
    });

    for await (const event of stream) {
        // Tool call starts
        if (event.type === "response.output_item.added") {
            if (event.item.type === "image_search_call") {
                console.log("[Tool Call] Searching by image...");
            }
        }
        // Tool call is complete. Parse and display the list of searched images.
        else if (event.type === "response.output_item.done") {
            if (event.item && event.item.type === "image_search_call") {
                console.log(`[Tool Call] Search by image complete (status: ${event.item.status})`);
                if (event.item.output) {
                    const images = JSON.parse(event.item.output);
                    console.log(`  Found ${images.length} images:`);
                    images.slice(0, 5).forEach(img => {
                        console.log(`  [${img.index}] ${img.title}`);
                        console.log(`      ${img.url}`);
                    });
                    if (images.length > 5) {
                        console.log(`  ... ${images.length} images in total`);
                    }
                }
            }
        }
        // Model response starts
        else if (event.type === "response.content_part.added") {
            console.log(`\n[Model Response]`);
        }
        // Streamed text output
        else if (event.type === "response.output_text.delta") {
            process.stdout.write(event.delta);
        }
        // Response is complete. Output the usage.
        else if (event.type === "response.completed") {
            const usage = event.response.usage;
            console.log(`\n\n[Token Usage] Input: ${usage.input_tokens}, Output: ${usage.output_tokens}, Total: ${usage.total_tokens}`);
            if (usage && usage.x_tools) {
                for (const [toolName, info] of Object.entries(usage.x_tools)) {
                    console.log(`[Tool Statistics] ${toolName} call count: ${info.count || 0}`);
                }
            }
        }
    }
}

main();

bash

# The following URL is for the Singapore region. Replace {WorkspaceId} with your actual workspace ID. URLs vary by region.
curl -X POST https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1/responses \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
-H "Content-Type: application/json" \
-d '{
    "model": "qwen3.8-max",
    "input": [
        {
            "role": "user",
            "content": [
                {"type": "input_text", "text": "Find landscape images with a style similar to this one"},
                {"type": "input_image", "image_url": "https://img.alicdn.com/imgextra/i4/O1CN01YbrnSS1qtmsAkw0Ud_!!6000000005554-2-tps-788-450.png"}
            ]
        }
    ],
    "tools": [
        {"type": "image_search"}
    ],
    "stream": true
}'

Después de ejecutar el código, recibirá una respuesta similar a la siguiente:

[Tool Call] Searching by image...
[Tool Call] Search by image complete (status: completed)
  Found 3 images:
  [1] QingMing Festival Holiday Notice 2024
      https://www.healthcabin.net/blog/wp-content/uploads/2024/04/QingMing-Festival-Holiday-Notice-2024.jpg
  [2] Serene Asian Landscape Stone Bridge Reflecting in Misty Water
      https://thumbs.dreamstime.com/b/serene-asian-landscape-stone-bridge-reflecting-misty-water-...
  [3] ...

[Model Response]
OK. I have found several landscape images with a similar style. These images all display the style of typical Chinese ink wash or fine-brush paintings...

[Token Usage] Input: 5339, Output: 164, Total: 5503
[Tool Statistics] image_search call count: 1

Facturación

La facturación implica los siguientes aspectos:

Tarifas de llamada al modelo: Los resultados de la búsqueda de imágenes se añaden al prompt. Esto aumenta el número de tokens de entrada para el modelo. Se le cobra según la tarifa estándar del modelo. Para obtener detalles sobre los precios, consulte la consola de Model Studio.
Tarifas de llamada a herramientas: Se le cobra por cada 1,000 llamadas. La tarifa es de $8.00 para implementaciones en la región de Singapore, mientras que la tarifa para implementaciones en Norte de China 2 (Beijing) es de $6.881.

FAQ

P: ¿Qué formatos de imagen y métodos de entrada se admiten?

R: Para obtener más información, consulte Límites de imagen y Métodos de entrada de archivos.

El SDK de OpenAI no admite pasar rutas de archivos locales.

P: ¿Cuántas imágenes se pueden pasar?

R: El número de imágenes que puede pasar está limitado por la longitud máxima de entrada del modelo. El número total de tokens tanto para las imágenes como para el texto no debe exceder el valor máximo que admite el modelo. El modelo busca solo una imagen por llamada, pero puede realizar múltiples llamadas para procesar varias imágenes.

El modelo decide el número de imágenes a buscar.

P: ¿Cuántas imágenes buscadas se devuelven?

R: El modelo determina el número de imágenes a devolver. La cantidad no es fija, pero el máximo es de 100 imágenes.