The AI_SIMILARITY function compares the semantic similarity of two text segments and returns a score from 0 to 1. Use this function for scenarios such as duplicate ticket identification, Q&A deduplication, and similar product matching.
Prerequisites
A Milvus instance with its endpoint URL and authentication credentials.
A configured text model, such as
qwen3.7-max.(For Python) PyMilvus installed.
Command format
The AI_SIMILARITY function supports two usage patterns:
REST interface — Send a direct API request to compare text pairs on demand.
Collection Function — Define a function in the collection schema to compute similarity automatically during data insertion.
REST interface
REST interface
POST /v2/vectordb/ai/similarity
Content-Type: application/json
{
"model_name": "<model_name>",
"texts": ["<text_A>", "<text_B>"],
"params": {}
}Python
Python
schema = MilvusClient.create_schema(auto_id=True, enable_dynamic_field=False)
schema.add_field("id", DataType.INT64, is_primary=True)
schema.add_field("left_text", DataType.VARCHAR, max_length=4096)
schema.add_field("right_text", DataType.VARCHAR, max_length=4096)
schema.add_field("similarity", DataType.DOUBLE)
schema.add_field("dummy_vector", DataType.FLOAT_VECTOR, dim=2)
schema.add_function(
Function(
name="compare_texts",
function_type=texttransform_function_type(),
input_field_names=["left_text", "right_text"],
output_field_names=["similarity"],
params={
"provider": "aliyun_milvus",
"model_name": "<model_name>",
"task": "ai_similarity",
"temperature": "0",
},
)
)Parameters
| Parameter | Description |
model_name | Required. The name of the configured text model. REST also accepts model. |
texts | Required for REST. The length must be even. [A,B,C,D] means comparing (A,B) and (C,D). |
temperature | Optional. Set to 0 for more stable scoring. |
max_concurrency/timeout_sec | Optional. Controls the concurrency and timeout for a single call. |
provider/task | Required only for Collection Function. Fixed to aliyun_milvus and ai_similarity. |
Return value
The number of elements in data.output.outputs equals the number of text pairs. Each element is a score string. When writing to a collection, the score is parsed as FLOAT or DOUBLE. Values that cannot be parsed or fall outside the 0–1 range are written as 0.
Usage example: Identifying duplicate customer service issues
A customer service system needs to determine whether two questions can be processed together. The following example uses the REST interface to compare two pairs of texts, and the Collection Function to compute similarity during data insertion.
REST interface
REST interface
Replace the MILVUS_REST_BASE_URL and MILVUS_AUTH_TOKEN placeholders in the script with the actual cluster endpoint and credentials before running.
#!/usr/bin/env bash
set -euo pipefail
MILVUS_REST_BASE_URL="http://c-xxxx.milvus.aliyuncs.com:19530"
MILVUS_AUTH_TOKEN="<yourUsername>:<yourPassword>"
post_json() {
local path="$1"
local body="$2"
curl -X POST \
"$MILVUS_REST_BASE_URL$path" \
-H "Authorization: Bearer $MILVUS_AUTH_TOKEN" \
-H "Content-Type: application/json" \
-d "$body"
}
MODEL_NAME="qwen3.7-max"
BODY=$(cat <<JSON
{
"model_name": "$MODEL_NAME",
"texts": [
"A cat is a mammal.", "Cats belong to mammals.",
"An apple is a fruit.", "Python is a programming language."
],
"params": {"max_concurrency": 2, "temperature": 0}
}
JSON
)
RESPONSE_BODY="$(post_json "/v2/vectordb/ai/similarity" "$BODY")"
if command -v jq >/dev/null 2>&1; then
echo "$RESPONSE_BODY" | jq .
[ "$(echo "$RESPONSE_BODY" | jq -r '.code // -1')" = "0" ] || exit 1
else
echo "$RESPONSE_BODY"
fi
# Expected: number of outputs equals number of text pairs, e.g., ["1","0.0"] (similar pairs score high, dissimilar pairs close to 0)Python
Python
After installing PyMilvus, replace the MILVUS_URI and MILVUS_TOKEN placeholders with the actual cluster address and token before running. The example includes inlined common helper functions.
from __future__ import annotations
from typing import Any
from pymilvus import DataType, Function, FunctionType, MilvusClient
MILVUS_URI = "http://c-xxxx.milvus.aliyuncs.com:19530"
MILVUS_TOKEN = "<yourUsername>:<yourPassword>"
DUMMY_VECTOR_DIM = 2
TEXTTRANSFORM_FUNCTION_TYPE = 9
def texttransform_function_type() -> Any:
for type_name in ("TEXTTRANSFORM", "TEXT_TRANSFORM", "TextTransform"):
function_type = getattr(FunctionType, type_name, None)
if function_type is not None:
return function_type
# Alibaba Cloud Milvus provides TEXTTRANSFORM as a managed extension (function type value 9);
# some pymilvus versions do not include this enum member yet, and Function(...) validates through FunctionType(...).
existing = getattr(FunctionType, "_value2member_map_", {}).get(TEXTTRANSFORM_FUNCTION_TYPE)
if existing is not None:
return existing
extension = int.__new__(FunctionType, TEXTTRANSFORM_FUNCTION_TYPE)
extension._name_ = "TEXTTRANSFORM"
extension._value_ = TEXTTRANSFORM_FUNCTION_TYPE
FunctionType._value2member_map_[TEXTTRANSFORM_FUNCTION_TYPE] = extension
FunctionType._member_map_["TEXTTRANSFORM"] = extension
return extension
def add_id(schema: Any) -> None:
schema.add_field("id", DataType.INT64, is_primary=True)
def add_dummy_vector(schema: Any) -> None:
schema.add_field("dummy_vector", DataType.FLOAT_VECTOR, dim=DUMMY_VECTOR_DIM)
def run_texttransform_example(*, client, collection_name, input_fields, output_field, function_name, function_params, rows) -> None:
if client.has_collection(collection_name):
client.drop_collection(collection_name)
schema = MilvusClient.create_schema(auto_id=True, enable_dynamic_field=False)
add_id(schema)
for name, data_type, max_length in input_fields:
field_params = {"max_length": max_length} if max_length is not None else {}
schema.add_field(name, data_type, **field_params)
output_name, output_data_type, output_max_length = output_field
output_params = {"max_length": output_max_length} if output_max_length is not None else {}
schema.add_field(output_name, output_data_type, **output_params)
add_dummy_vector(schema)
schema.add_function(
Function(
name=function_name,
function_type=texttransform_function_type(),
input_field_names=[name for name, _, _ in input_fields],
output_field_names=[output_name],
params=function_params,
)
)
index_params = client.prepare_index_params()
index_params.add_index(field_name="dummy_vector", index_type="AUTOINDEX", metric_type="COSINE")
client.create_collection(collection_name=collection_name, schema=schema, index_params=index_params)
client.insert(collection_name, rows)
client.flush(collection_name)
fields = [name for name, _, _ in input_fields] + [output_name]
for row in client.query(collection_name, filter="", output_fields=fields, limit=len(rows)):
print(row)
MODEL_NAME = "qwen3.7-max"
client = MilvusClient(uri=MILVUS_URI, token=MILVUS_TOKEN)
run_texttransform_example(
client=client,
collection_name="simple_ai_similarity",
input_fields=[("left_text", DataType.VARCHAR, 4096), ("right_text", DataType.VARCHAR, 4096)],
output_field=("similarity", DataType.DOUBLE, None),
function_name="compare_texts",
function_params={"provider": "aliyun_milvus", "model_name": MODEL_NAME, "task": "ai_similarity", "temperature": "0"},
rows=[{"left_text": "A cat is a mammal.", "right_text": "Cats belong to mammals.", "dummy_vector": [0.1, 0.2]}],
)
# Expected: similarity receives a score close to 1 (highly similar)Similar pairs (such as the cat/mammal example) receive a score close to 1, while dissimilar pairs receive a score close to 0. You can route questions with scores above 0.85 to the same processing queue.