重排序 - 向量检索服务 Milvus 版

命令格式

REST 接口

POST /v2/vectordb/ai/rerank
Content-Type: application/json

{
  "model_name": "qwen3-rerank",
  "query": "<查询文本>",
  "documents": ["<候选内容>"],
  "params": {"timeout_sec": 10}
}

Python

schema = MilvusClient.create_schema(auto_id=True, enable_dynamic_field=False)
schema.add_field("id", DataType.INT64, is_primary=True)
schema.add_field("content", DataType.VARCHAR, max_length=4096)
schema.add_field("embedding", DataType.FLOAT_VECTOR, dim=1024)
schema.add_function(
    Function(
        name="embed_content",
        function_type=FunctionType.TEXTEMBEDDING,
        input_field_names=["content"],
        output_field_names=["embedding"],
        params={
            "provider": "aliyun_milvus",
            "model_name": "text-embedding-v4",
            "dim": 1024,
        },
    )
)

reranker = Function(
    name="rerank_content",
    function_type=FunctionType.RERANK,
    input_field_names=["content"],
    params={
        "reranker": "model",
        "provider": "aliyun_milvus",
        "model_name": "qwen3-rerank",
        "queries": ["<查询文本>"],
        "timeout_sec": 10,
    },
)

Collection Search 场景使用 FunctionType.RERANK，需设置 reranker="model"、provider="aliyun_milvus"、model_name、queries 和候选内容字段。文本候选使用 qwen3-rerank；图片或视频候选使用 qwen3-vl-rerank。

多模态输入组合

qwen3-vl-rerank 不仅支持以文本查询图片或视频，也支持将图片 URL 直接作为 Query，实现以图搜图或以图搜视频。支持的输入组合如下。

Query	候选内容	模型	典型场景
文本	文本	`qwen3-rerank`	RAG 文档重排
文本	图片 URL	`qwen3-vl-rerank`	以文搜图
图片 URL	图片 URL	`qwen3-vl-rerank`	以图搜图、相似商品推荐
文本	视频 URL	`qwen3-vl-rerank`	以文搜视频
图片 URL	视频 URL	`qwen3-vl-rerank`	以图搜视频、素材匹配

说明

REST 接口中，文本 Query 和媒体 URL 均通过 query 字符串传入；Collection Search 中通过 queries 数组传入。媒体候选需以可访问的 URL 字符串形式保存到 documents 或 Collection 的候选字段中。

参数说明

参数	说明
`model_name`	REST 必填；模型重排 Function 也必填。文本候选使用 `qwen3-rerank`，图片或视频等多模态候选使用 `qwen3-vl-rerank`。
`query`	REST 必填。可以是非空查询文本；多模态重排时也可以是可访问的图片 URL。
`documents`	REST 必填。至少一条候选内容；返回结果的 `index` 对应该数组下标。
`provider`	仅模型 Rerank Function 必填，固定为 `aliyun_milvus`。
`reranker`	仅 Function 使用。调用模型时设为 `model`；`weighted`、`rrf`、`decay`、`boost` 用于内置排序阶段。
`queries`	仅 Search Function 必填。Query 数组，元素可以是查询文本或图片 URL；单 Query 搜索传一个元素。
`max_client_batch_size`	可选。单次发送给模型的最大候选数，默认 `128`。
`max_concurrency`	可选。并发数，范围 `1`~`64`，默认 `1`。
`timeout_sec`	可选。单次模型调用超时秒数，范围 `1`~`300`，默认 `30`。
`on_error`	仅 Search Function 可选。模型异常时可选 `fail`、`fallback_previous`、`fallback_original` 或 `skip_stage`。
`is_multimodal`	可选。图片或视频作为 Query 或候选内容参与重排时设为 `true`。REST 接口使用 JSON 布尔值 `true`，Function 参数使用字符串 `"true"`。
`instruct`	可选。排序指令，用于明确模型的排序关注点，多模态重排时建议使用英文指令。例如：图片重排可强调主体一致性、外观、构图和细粒度视觉特征；视频重排可强调主体、动作、场景和外观。

返回值说明

REST 调用成功时，data.output.results 返回每个候选的 index 和 relevance_score。结果与输入下标对应，业务侧应按 relevance_score 降序排序；分数没有固定阈值，不应依赖示例中的具体数值。响应还包含 usage（如 total_tokens）与 request_id 字段。

{"code":0,"data":{"output":{"results":[{"index":0,"relevance_score":0.97}]}}}

示例一：RAG 问答的候选文档重排（文本）

知识库已召回三条候选文档，用户询问“向量数据库的典型应用场景”。使用 RERANK 重新打分后，应用取分数最高的文档交给大模型生成回答。关键参数为 query、documents 和 timeout_sec。

REST 接口

#!/usr/bin/env bash
set -euo pipefail

MILVUS_REST_BASE_URL="http://c-xxxx.milvus.aliyuncs.com:19530"
MILVUS_AUTH_TOKEN="<yourUsername>:<yourPassword>"

post_json() {
  local path="$1"
  local body="$2"
  curl -X POST "$MILVUS_REST_BASE_URL$path" \
    -H "Authorization: Bearer $MILVUS_AUTH_TOKEN" \
    -H "Content-Type: application/json" \
    -d "$body"
}

MODEL_NAME="qwen3-rerank"

BODY=$(cat <<JSON
{
  "model_name": "$MODEL_NAME",
  "query": "Typical use cases of vector databases",
  "documents": [
    "Milvus is an open-source vector database for RAG, recommendation, and image search.",
    "MySQL is a relational database for transactional applications.",
    "Vector retrieval can be combined with inverted indexes to improve recall."
  ],
  "params": {"max_concurrency": 2, "timeout_sec": 10}
}
JSON
)

RESPONSE_BODY="$(post_json "/v2/vectordb/ai/rerank" "$BODY")"
if command -v jq >/dev/null 2>&1; then
  echo "$RESPONSE_BODY" | jq .
  [ "$(echo "$RESPONSE_BODY" | jq -r '.code // -1')" = "0" ] || exit 1
else
  echo "$RESPONSE_BODY"
fi

Python

from __future__ import annotations
import os
from typing import Any
from pymilvus import DataType, Function, FunctionType, MilvusClient

MILVUS_URI = "http://c-xxxx.milvus.aliyuncs.com:19530"
MILVUS_TOKEN = "<yourUsername>:<yourPassword>"

def add_id(schema: Any) -> None:
    schema.add_field("id", DataType.INT64, is_primary=True)

MODEL_NAME = "qwen3-rerank"
EMBEDDING_MODEL = os.getenv("AIFUNC_RERANK_EMBEDDING_MODEL", "text-embedding-v4")
client = MilvusClient(uri=MILVUS_URI, token=MILVUS_TOKEN)

collection_name = "simple_ai_rerank_text"
if client.has_collection(collection_name):
    client.drop_collection(collection_name)

schema = MilvusClient.create_schema(auto_id=True, enable_dynamic_field=False)
add_id(schema)
schema.add_field("content", DataType.VARCHAR, max_length=4096)
schema.add_field("embedding", DataType.FLOAT_VECTOR, dim=1024)
schema.add_function(Function(name="embed_content", function_type=FunctionType.TEXTEMBEDDING, input_field_names=["content"], output_field_names=["embedding"], params={"provider": "aliyun_milvus", "model_name": EMBEDDING_MODEL, "dim": 1024}))
index_params = client.prepare_index_params()
index_params.add_index(field_name="embedding", index_type="AUTOINDEX", metric_type="COSINE")
client.create_collection(collection_name=collection_name, schema=schema, index_params=index_params)
client.insert(collection_name, [{"content": "Milvus is an open-source vector database for RAG and recommendation."}, {"content": "MySQL is a relational database for transactional applications."}])
client.flush(collection_name)
client.load_collection(collection_name)

QUERY = "Typical use cases of vector databases"
reranker = Function(name="rerank_content", function_type=FunctionType.RERANK, input_field_names=["content"], params={"reranker": "model", "provider": "aliyun_milvus", "model_name": MODEL_NAME, "queries": [QUERY], "max_concurrency": 2, "timeout_sec": 10})
for hit in client.search(collection_name=collection_name, data=[QUERY], anns_field="embedding", limit=2, output_fields=["content"], ranker=reranker)[0]:
    print(f"score={hit['distance']:.4f} content={hit['entity']['content']}")

第 0 和第 2 条文档通常比 MySQL 文档更相关；Python 会按实际 relevance_score 从高到低打印候选，应用可取前两条作为 RAG 上下文。

示例二：商品图片多模态重排（以文搜图与以图搜图）

向量检索先召回候选商品图片，再使用 qwen3-vl-rerank 做精排。本示例对同一批候选图片连续执行两种 Query：一是文本 Query，二是直接传入参考图片 URL 作为 Query，实现以图搜图。建议先从向量检索结果中截取最多 40 张候选图片，再交给模型重排；instruct 使用英文指令，明确要求优先比较主体身份、外观、构图和细粒度视觉特征。

REST 接口

#!/usr/bin/env bash
set -euo pipefail

MILVUS_REST_BASE_URL="http://c-xxxx.milvus.aliyuncs.com:19530"
MILVUS_AUTH_TOKEN="<yourUsername>:<yourPassword>"

post_json() {
  local path="$1"
  local body="$2"
  curl -X POST "$MILVUS_REST_BASE_URL$path" \
    -H "Authorization: Bearer $MILVUS_AUTH_TOKEN" \
    -H "Content-Type: application/json" \
    -d "$body"
}

MODEL_NAME="qwen3-vl-rerank"
QUERY_TEXT="a fashion product matching the query"
QUERY_IMAGE_URL="https://img.alicdn.com/imgextra/i3/O1CN01rdstgY1uiZWt8gqSL_!!6000000006071-0-tps-1970-356.jpg"
CANDIDATE_IMAGE_URL_1="$QUERY_IMAGE_URL"
CANDIDATE_IMAGE_URL_2="https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20260415/hynnff/wan-video-edit-clothes.webp"
CANDIDATE_IMAGE_URL_3="https://dashscope.oss-cn-beijing.aliyuncs.com/images/dog_and_girl.jpeg"
INSTRUCT="Rank the candidate images by relevance to the query, prioritizing subject identity, appearance, composition, and fine-grained visual details."

run_rerank() {
  local label="$1"
  local query="$2"
  local body

  body=$(cat <<JSON
{
  "model_name": "$MODEL_NAME",
  "query": "$query",
  "documents": [
    "$CANDIDATE_IMAGE_URL_1",
    "$CANDIDATE_IMAGE_URL_2",
    "$CANDIDATE_IMAGE_URL_3"
  ],
  "params": {
    "is_multimodal": true,
    "instruct": "$INSTRUCT",
    "max_client_batch_size": 40,
    "timeout_sec": 10
  }
}
JSON
  )

  echo "=== $label ==="
  post_json "/v2/vectordb/ai/rerank" "$body"
}

run_rerank "text query" "$QUERY_TEXT"
run_rerank "image query" "$QUERY_IMAGE_URL"

Python

from __future__ import annotations
import os
from typing import Any
from pymilvus import DataType, Function, FunctionType, MilvusClient

MILVUS_URI = "http://c-xxxx.milvus.aliyuncs.com:19530"
MILVUS_TOKEN = "<yourUsername>:<yourPassword>"

def add_id(schema: Any) -> None:
    schema.add_field("id", DataType.INT64, is_primary=True)

MODEL_NAME = "qwen3-vl-rerank"
EMBEDDING_MODEL = os.getenv("AIFUNC_RERANK_EMBEDDING_MODEL", "qwen3-vl-embedding")
VECTOR_DIM = int(os.getenv("AIFUNC_RERANK_EMBEDDING_DIM", "2560"))
QUERY_TEXT = "a fashion product matching the query"
QUERY_IMAGE_URL = "https://img.alicdn.com/imgextra/i3/O1CN01rdstgY1uiZWt8gqSL_!!6000000006071-0-tps-1970-356.jpg"
CANDIDATE_IMAGE_URLS = [
    QUERY_IMAGE_URL,
    "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20260415/hynnff/wan-video-edit-clothes.webp",
    "https://dashscope.oss-cn-beijing.aliyuncs.com/images/dog_and_girl.jpeg",
]
INSTRUCT = (
    "Rank the candidate images by relevance to the query, prioritizing "
    "subject identity, appearance, composition, and fine-grained visual details."
)
client = MilvusClient(uri=MILVUS_URI, token=MILVUS_TOKEN)

collection_name = "simple_ai_rerank_image"
if client.has_collection(collection_name):
    client.drop_collection(collection_name)

schema = MilvusClient.create_schema(auto_id=True, enable_dynamic_field=False)
add_id(schema)
schema.add_field("content", DataType.VARCHAR, max_length=4096)
schema.add_field("embedding", DataType.FLOAT_VECTOR, dim=VECTOR_DIM)
schema.add_function(Function(name="embed_image", function_type=FunctionType.TEXTEMBEDDING, input_field_names=["content"], output_field_names=["embedding"], params={"provider": "aliyun_milvus", "model_name": EMBEDDING_MODEL, "dim": VECTOR_DIM, "is_multimodal": "true"}))
index_params = client.prepare_index_params()
index_params.add_index(field_name="embedding", index_type="AUTOINDEX", metric_type="COSINE")
client.create_collection(collection_name=collection_name, schema=schema, index_params=index_params)
client.insert(collection_name, [{"content": url} for url in CANDIDATE_IMAGE_URLS[:40]])
client.flush(collection_name)
client.load_collection(collection_name)

for query_label, query in (("text", QUERY_TEXT), ("image", QUERY_IMAGE_URL)):
    reranker = Function(name=f"rerank_image_{query_label}_query", function_type=FunctionType.RERANK, input_field_names=["content"], params={"reranker": "model", "provider": "aliyun_milvus", "model_name": MODEL_NAME, "queries": [query], "is_multimodal": "true", "instruct": INSTRUCT, "max_client_batch_size": 40, "timeout_sec": 10})
    for hit in client.search(collection_name=collection_name, data=[query], anns_field="embedding", limit=len(CANDIDATE_IMAGE_URLS[:40]), output_fields=["content"], ranker=reranker)[0]:
        print(f"{query_label} score={hit['distance']:.4f} content={hit['entity']['content']}")

预期结果如下：

文本 Query 按文字描述与候选图片之间的语义、视觉相关性排序。
图片 Query 执行以图搜图，与 Query 相同的图片通常排在最前。
REST 返回的 index 对应原始 documents 数组的下标，在映射回候选内容前不要改变原数组顺序。
Collection Search 返回的 hit['distance'] 为重排后的相关性分数，结果已按分数排序。

示例三：视频素材多模态重排（以文搜视频与以图搜视频）

视频候选同样使用 qwen3-vl-rerank。除文本 Query 外，还可以传入参考图片 URL 作为 Query，让模型从候选视频中找出主体、服饰、动作或场景最匹配的素材。

REST 接口

#!/usr/bin/env bash
set -euo pipefail

MILVUS_REST_BASE_URL="http://c-xxxx.milvus.aliyuncs.com:19530"
MILVUS_AUTH_TOKEN="<yourUsername>:<yourPassword>"

post_json() {
  local path="$1"
  local body="$2"
  curl -X POST "$MILVUS_REST_BASE_URL$path" \
    -H "Authorization: Bearer $MILVUS_AUTH_TOKEN" \
    -H "Content-Type: application/json" \
    -d "$body"
}

MODEL_NAME="qwen3-vl-rerank"
QUERY_TEXT="person wearing a striped sweater"
QUERY_IMAGE_URL="https://img.alicdn.com/imgextra/i3/O1CN01rdstgY1uiZWt8gqSL_!!6000000006071-0-tps-1970-356.jpg"
VIDEO_URL="https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20260409/dozxak/Wan_Video_Edit_33_1.mp4"
INSTRUCT="Rank the candidate videos by relevance to the query, prioritizing matching subjects, actions, scenes, appearance, and fine-grained visual details."

run_rerank() {
  local label="$1"
  local query="$2"
  local body

  body=$(cat <<JSON
{
  "model_name": "$MODEL_NAME",
  "query": "$query",
  "documents": ["$VIDEO_URL"],
  "params": {
    "is_multimodal": true,
    "instruct": "$INSTRUCT",
    "timeout_sec": 10
  }
}
JSON
  )

  echo "=== $label ==="
  post_json "/v2/vectordb/ai/rerank" "$body"
}

run_rerank "text query" "$QUERY_TEXT"
run_rerank "image query" "$QUERY_IMAGE_URL"

Python

from __future__ import annotations
import os
from typing import Any
from pymilvus import DataType, Function, FunctionType, MilvusClient

MILVUS_URI = "http://c-xxxx.milvus.aliyuncs.com:19530"
MILVUS_TOKEN = "<yourUsername>:<yourPassword>"

def add_id(schema: Any) -> None:
    schema.add_field("id", DataType.INT64, is_primary=True)

MODEL_NAME = "qwen3-vl-rerank"
EMBEDDING_MODEL = os.getenv("AIFUNC_RERANK_EMBEDDING_MODEL", "qwen3-vl-embedding")
VECTOR_DIM = int(os.getenv("AIFUNC_RERANK_EMBEDDING_DIM", "2560"))
QUERY_TEXT = "person wearing a striped sweater"
QUERY_IMAGE_URL = "https://img.alicdn.com/imgextra/i3/O1CN01rdstgY1uiZWt8gqSL_!!6000000006071-0-tps-1970-356.jpg"
VIDEO_URL = "https://help-static-aliyun-doc.aliyuncs.com/file-manage-files/zh-CN/20260409/dozxak/Wan_Video_Edit_33_1.mp4"
INSTRUCT = (
    "Rank the candidate videos by relevance to the query, prioritizing matching "
    "subjects, actions, scenes, appearance, and fine-grained visual details."
)
client = MilvusClient(uri=MILVUS_URI, token=MILVUS_TOKEN)

collection_name = "simple_ai_rerank_video"
if client.has_collection(collection_name):
    client.drop_collection(collection_name)

schema = MilvusClient.create_schema(auto_id=True, enable_dynamic_field=False)
add_id(schema)
schema.add_field("content", DataType.VARCHAR, max_length=4096)
schema.add_field("embedding", DataType.FLOAT_VECTOR, dim=VECTOR_DIM)
schema.add_function(Function(name="embed_video", function_type=FunctionType.TEXTEMBEDDING, input_field_names=["content"], output_field_names=["embedding"], params={"provider": "aliyun_milvus", "model_name": EMBEDDING_MODEL, "dim": VECTOR_DIM, "is_multimodal": "true"}))
index_params = client.prepare_index_params()
index_params.add_index(field_name="embedding", index_type="AUTOINDEX", metric_type="COSINE")
client.create_collection(collection_name=collection_name, schema=schema, index_params=index_params)
client.insert(collection_name, [{"content": VIDEO_URL}])
client.flush(collection_name)
client.load_collection(collection_name)

for query_label, query in (("text", QUERY_TEXT), ("image", QUERY_IMAGE_URL)):
    reranker = Function(name=f"rerank_video_{query_label}_query", function_type=FunctionType.RERANK, input_field_names=["content"], params={"reranker": "model", "provider": "aliyun_milvus", "model_name": MODEL_NAME, "queries": [query], "is_multimodal": "true", "instruct": INSTRUCT, "timeout_sec": 10})
    for hit in client.search(collection_name=collection_name, data=[query], anns_field="embedding", limit=1, output_fields=["content"], ranker=reranker)[0]:
        print(f"{query_label} score={hit['distance']:.4f} content={hit['entity']['content']}")

预期结果如下：

文本 Query 根据文本描述对视频候选重排。
图片 Query 根据参考图中的主体、外观和场景对视频候选重排。
生产环境建议先对向量召回结果做截断再重排，并保持候选 URL 与业务主键之间的稳定映射。