Alibaba Cloud Model Studio:Embedding

Multimodal embedding

This model generates embeddings from text, image, or video inputs. You can use these embeddings for tasks like video and image classification, image-text retrieval, and text-to-image or text-to-video search.

The API accepts single text, image, or video inputs, as well as combinations like text and images. Some models support multiple inputs of the same type, such as multiple images. For details, see the limitations for each model.

Input and language restrictions

Customize vector dimensions

The text-embedding-v4, text-embedding-v3, tongyi-embedding-vision-plus, tongyi-embedding-vision-flash, qwen3-vl-embedding models support custom vector dimensions. Higher dimensions preserve more semantic information but increase storage and compute costs.

• General use cases (Recommended): A dimension of 1024 provides an optimal balance between performance and cost, making it ideal for most semantic search tasks.

• High-precision scenarios: For applications that require high precision, you can select a dimension of 1536 or 2048. This improves precision but significantly increases storage and compute overhead.

• Resource-constrained environments: In cost-sensitive scenarios, select a dimension of 768 or lower. This significantly reduces resource consumption at the cost of some semantic information.

import os
from openai import OpenAI

client = OpenAI(
    # API Keys are region-specific. To get an API Key, see https://www.alibabacloud.com/help/en/model-studio/get-api-key
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    # This is the base URL for the Singapore region. Replace {WorkspaceId} with your Workspace ID. URLs are region-specific.
    base_url="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1",
)

resp = client.embeddings.create(
    model="text-embedding-v4",
    input=["I like it and will buy from here again"],
    # Set the vector dimension to 256
    dimensions=256
)
print(f"Vector dimension: {len(resp.data[0].embedding)}")

import dashscope

# If you use a model in the China (Beijing) region, replace base_http_api_url with: https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1
dashscope.base_http_api_url = 'https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1'

resp = dashscope.TextEmbedding.call(
    model="text-embedding-v4",
    input=["I like it and will buy from here again"],
    # Set the vector dimension to 256
    dimension=256
)

print(f"Vector dimension: {len(resp.output['embeddings'][0]['embedding'])}")

Query vs. document text (text_type)

This parameter is only available through the DashScope SDK and API.

To achieve optimal results in search tasks, you should vectorize content differently based on its role. The text_type parameter is designed for this purpose:

• text_type: 'query': Use for user-provided query text. The model generates a "title-like" vector that is more directional and optimized for information retrieval.

• text_type: 'document' (default): Use for the document text stored in your knowledge base. The model generates a "body-like" vector that contains more comprehensive information and is optimized for matching.

When matching short text against long text, you should distinguish between query and document. However, for tasks such as clustering or classification where all texts have the same role, you do not need to set this parameter.

Task instructions (instruct)

This parameter is only available through the DashScope SDK and API.

You can provide a clear English task instruction to guide the text-embedding-v4 model in optimizing vector quality for specific retrieval scenarios, improving precision. When using this feature, you must set the text_type parameter to query.

# Example: Add an instruction to optimize retrieval quality when building document vectors.
resp = dashscope.TextEmbedding.call(
    model="text-embedding-v4",
    input="Research papers on machine learning",
    text_type="query",
    instruct="Given a research paper query, retrieve relevant research paper"
)

Dense and sparse vectors

This parameter is only available through the DashScope SDK and API.

The text-embedding-v4 and text-embedding-v3 models support three vector output types to accommodate different retrieval strategies.

Semantic search

Perform precise semantic matching by calculating the similarity between the query embedding and the document embeddings.

import dashscope
import numpy as np
from dashscope import TextEmbedding

# To use a model from the China (Beijing) region, change base_http_api_url to: https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1
dashscope.base_http_api_url = 'https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1'

def cosine_similarity(a, b):
    """Calculate cosine similarity."""
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

def semantic_search(query, documents, top_k=5):
    """Perform semantic search."""
    # Generate the query embedding.
    query_resp = TextEmbedding.call(
        model="text-embedding-v4",
        input=query,
        dimension=1024
    )
    query_embedding = query_resp.output['embeddings'][0]['embedding']

    # Generate the document embeddings.
    doc_resp = TextEmbedding.call(
        model="text-embedding-v4",
        input=documents,
        dimension=1024
    )

    # Calculate similarities.
    similarities = []
    for i, doc_emb in enumerate(doc_resp.output['embeddings']):
        similarity = cosine_similarity(query_embedding, doc_emb['embedding'])
        similarities.append((i, similarity))

    # Sort and return the top-k results.
    similarities.sort(key=lambda x: x[1], reverse=True)
    return [(documents[i], sim) for i, sim in similarities[:top_k]]

# Example usage
documents = [
    "Artificial intelligence is a branch of computer science",
    "Machine learning is an important method for achieving artificial intelligence",
    "Deep learning is a subfield of machine learning"
]
query = "What is AI?"
results = semantic_search(query, documents, top_k=2)
for doc, sim in results:
    print(f"Similarity: {sim:.3f}, Document: {doc}")

Recommendation system

Analyze a user's behavioral history embeddings to identify their interests and recommend similar items.

import dashscope
import numpy as np
from dashscope import TextEmbedding

# To use a model from the China (Beijing) region, change base_http_api_url to: https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1
dashscope.base_http_api_url = 'https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1'

def cosine_similarity(a, b):
    """Calculate cosine similarity."""
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))
def build_recommendation_system(user_history, all_items, top_k=10):
    """Build a recommendation system."""
    # Generate user history embeddings.
    history_resp = TextEmbedding.call(
        model="text-embedding-v4",
        input=user_history,
        dimension=1024
    )

    # Calculate the user preference embedding by averaging.
    user_embedding = np.mean([
        emb['embedding'] for emb in history_resp.output['embeddings']
    ], axis=0)

    # Generate all item embeddings.
    items_resp = TextEmbedding.call(
        model="text-embedding-v4",
        input=all_items,
        dimension=1024
    )

    # Calculate recommendation scores.
    recommendations = []
    for i, item_emb in enumerate(items_resp.output['embeddings']):
        score = cosine_similarity(user_embedding, item_emb['embedding'])
        recommendations.append((all_items[i], score))

    # Sort and return the recommendation results.
    recommendations.sort(key=lambda x: x[1], reverse=True)
    return recommendations[:top_k]

# Example usage
user_history = ["Science Fiction", "Action", "Suspense"]
all_movies = ["Future World", "Space Adventure", "Ancient War", "Romantic Journey", "Superhero"]
recommendations = build_recommendation_system(user_history, all_movies)
for movie, score in recommendations:
    print(f"Recommendation Score: {score:.3f}, Movie: {movie}")

Text clustering

Group similar texts by analyzing the distances between their embeddings.

# scikit-learn is required: pip install scikit-learn
import dashscope
import numpy as np
from sklearn.cluster import KMeans

# To use a model from the China (Beijing) region, change base_http_api_url to: https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1
dashscope.base_http_api_url = 'https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1'

def cluster_texts(texts, n_clusters=2):
    """Cluster a set of texts."""
    # 1. Get the embeddings for all texts.
    resp = dashscope.TextEmbedding.call(
        model="text-embedding-v4",
        input=texts,
        dimension=1024
    )
    embeddings = np.array([item['embedding'] for item in resp.output['embeddings']])

    # 2. Use the KMeans algorithm for clustering.
    kmeans = KMeans(n_clusters=n_clusters, random_state=0, n_init='auto').fit(embeddings)

    # 3. Organize and return the results.
    clusters = {i: [] for i in range(n_clusters)}
    for i, label in enumerate(kmeans.labels_):
        clusters[label].append(texts[i])
    return clusters


# Example usage
documents_to_cluster = [
    "Mobile phone company A releases a new phone",
    "Search engine company B launches a new system",
    "World Cup final: Argentina vs. France",
    "China wins another gold medal at the Olympics",
    "A company releases its latest AI chip",
    "European Cup match report"
]
clusters = cluster_texts(documents_to_cluster, n_clusters=2)
for cluster_id, docs in clusters.items():
    print(f"--- Cluster {cluster_id} ---")
    for doc in docs:
        print(f"- {doc}")

Text classification

Perform zero-shot text classification by calculating the similarity between an input text's embedding and predefined label embeddings. This process classifies text into new categories without requiring pre-labeled examples.

import dashscope
import numpy as np

# To use a model from the China (Beijing) region, change base_http_api_url to: https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1
dashscope.base_http_api_url = 'https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1'

def cosine_similarity(a, b):
    """Calculate cosine similarity."""
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))


def classify_text_zero_shot(text, labels):
    """Perform zero-shot text classification."""
    # 1. Get the embeddings for the input text and all labels.
    resp = dashscope.TextEmbedding.call(
        model="text-embedding-v4",
        input=[text] + labels,
        dimension=1024
    )
    embeddings = resp.output['embeddings']
    text_embedding = embeddings[0]['embedding']
    label_embeddings = [emb['embedding'] for emb in embeddings[1:]]

    # 2. Calculate the similarity with each label.
    scores = [cosine_similarity(text_embedding, label_emb) for label_emb in label_embeddings]

    # 3. Return the label with the highest similarity.
    best_match_index = np.argmax(scores)
    return labels[best_match_index], scores[best_match_index]


# Example usage
text_to_classify = "The fabric of this dress is comfortable, and the style is nice too"
possible_labels = ["Digital Products", "Apparel & Accessories", "Food & Beverage", "Home & Living"]

label, score = classify_text_zero_shot(text_to_classify, possible_labels)
print(f"Input text: '{text_to_classify}'")
print(f"Best matching category: '{label}' (Similarity: {score:.3f})")

Anomaly detection

Identify anomalous data by calculating the similarity between a text's embedding and the central embedding of normal samples. Data that significantly deviates from this pattern is considered an anomaly.

The threshold in the example is for demonstration purposes. The ideal value varies based on data content and distribution, so you must calibrate it using your own dataset.

import dashscope
import numpy as np

# To use a model from the China (Beijing) region, change base_http_api_url to: https://{WorkspaceId}.cn-beijing.maas.aliyuncs.com/api/v1
dashscope.base_http_api_url = "https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/api/v1"


def cosine_similarity(a, b):
    """Calculate cosine similarity."""
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))


def detect_anomaly(new_comment, normal_comments, threshold=0.6):
    # 1. Generate embeddings for all normal comments and the new comment.
    all_texts = normal_comments + [new_comment]
    resp = dashscope.TextEmbedding.call(
        model="text-embedding-v4",
        input=all_texts,
        dimension=1024
    )
    embeddings = [item['embedding'] for item in resp.output['embeddings']]

    # 2. Calculate the center embedding (average) of the normal comments.
    normal_embeddings = np.array(embeddings[:-1])
    normal_center_vector = np.mean(normal_embeddings, axis=0)

    # 3. Calculate the similarity between the new comment's embedding and the center embedding.
    new_comment_embedding = np.array(embeddings[-1])
    similarity = cosine_similarity(new_comment_embedding, normal_center_vector)

    # 4. Determine if it is an anomaly.
    is_anomaly = similarity < threshold
    return is_anomaly, similarity


# Example usage
normal_user_comments = [
    "Today's meeting was productive",
    "The project is progressing smoothly",
    "The new version will be released next week",
    "User feedback is positive"
]

test_comments = {
    "Normal comment": "The feature works as expected",
    "Anomaly - meaningless garbled text": "asdfghjkl zxcvbnm"
}

print("--- Anomaly Detection Example ---")
for desc, comment in test_comments.items():
    is_anomaly, score = detect_anomaly(comment, normal_user_comments)
    result = "Yes" if is_anomaly else "No"
    print(f"Comment: '{comment}'")
    print(f"Is anomaly: {result} (Similarity to normal samples: {score:.3f})\n")