Use the Text Embedding model of Alibaba Cloud - Alibaba Cloud Model Studio

Text Embedding is a multilingual unified text embedding model developed by Tongyi Lab based on large language models (LLMs). Text Embedding helps developers convert text in multiple mainstream languages into high-quality vectors.

Model	Name	Vector dimension	Maximum number of lines for a request	Maximum input token length for a line	Supported language
Text Embedding	text-embedding-v1	1,536	25	2,048	Chinese, English, Spanish, French, Portuguese, and Indonesian
	text-embedding-async-v1	1,536	100,000	2,048	Chinese, English, Spanish, French, Portuguese, and Indonesian
	text-embedding-v2	1,536	25	2,048	Chinese, English, Spanish, French, Portuguese, Indonesian, Japanese, Korean, German, and Russian
	text-embedding-async-v2	1,536	100,000	2,048	Chinese, English, Spanish, French, Portuguese, Indonesian, Japanese, Korean, German, and Russian
	text-embedding-v3	1,024, 768, or 512	6	8,192	Over 50 languages, including Chinese, English, Spanish, French, Portuguese, Indonesian, Japanese, Korean, German, and Russian.

Note

Currently, only text-embedding-v3 is supported.

The text-embedding-v2 model incorporates the following updates based on text-embedding-v1:

More supported languages : text-embedding-v2 supports Japanese, Korean, German, and Russian.
Performance: Evaluation results from publicly accessible datasets show that the overall performance of text-embedding-v2 is improved by using a pre-trained model as a foundation and by applying Specific Fine-Tuning (SFT) strategies.

Note

The text-embedding-v3 model incorporates the following updates based on text-embedding-v2:

More supported languages : text-embedding-v3 supports more than 50 languages, including Italian, Polish, Vietnamese, and Thai.
Input token length: The maximum token length increases from 2048 to 8192.
Variable dense vector dimension: text-embedding-v3 allows you to select the dimension of dense vectors from 512, 768, or 1024. To reduce the cost of downstream tasks and maintain high performance, the maximum vector dimension of text-embedding-v3 is reduced to 1024.
Unified treatment of query and document types: text-embedding-v3 does not differentiate between the types of input text and maintains high performance. You do not need to specify query or document for the text_type parameter.
Support for sparse vectors: text-embedding-v3 supports dense and sparse vectors. You can specify the output_type parameter to control whether the output is dense vectors, sparse vectors, or both.
Performance: Evaluation results from publicly accessible datasets show that the overall performance of text-embedding-v3 is improved by using a pre-trained model as a foundation and by applying SFT strategies.

	MTEB	MTEB (retrieval task)	CMTEB	CMTEB (Retrieval task)
text-embedding-v1	58.30	45.47	59.84	56.59
text-embedding-v2	60.13	49.49	62.17	62.78
text-embedding-v3	63.39	55.41	68.92	73.23

Normalization: By default, the text-embedding-v2 model normalizes the output vectors.

Usage notes

Important

Use the same embedding model for offline indexing and online queries. For example, if you use text-embedding-v1 to build your offline index, do not use text-embedding-v2 to vectorize your query requests.

Call the model

Prerequisites

Alibaba Cloud Model Studio is activated and an API key is obtained. For more information, see Create an API key.
The SDK of the latest version is installed. For more information, see Install the SDK.

Sample code

Specify API key

export DASHSCOPE_API_KEY=YOUR_DASHSCOPE_API_KEY

Synchronous call

In a synchronous call, the model processes a single input text and returns the results.

Python

import dashscope
from http import HTTPStatus
dashscope.base_http_api_url = 'https://dashscope-intl.aliyuncs.com/api/v1'

def embed_with_str():
    resp = dashscope.TextEmbedding.call(
        model=dashscope.TextEmbedding.Models.text_embedding_v3,
        input='The clothes are of good quality and look good, definitely worth the wait. I love them and will shop here again.')
    if resp.status_code == HTTPStatus.OK:
        print(resp)
    else:
        print(resp)


if __name__ == '__main__':
    embed_with_str()

Java

import java.util.Arrays;
import java.util.concurrent.Semaphore;
import com.alibaba.dashscope.common.ResultCallback;
import com.alibaba.dashscope.embeddings.TextEmbedding;
import com.alibaba.dashscope.embeddings.TextEmbeddingParam;
import com.alibaba.dashscope.embeddings.TextEmbeddingResult;
import com.alibaba.dashscope.exception.ApiException;
import com.alibaba.dashscope.exception.NoApiKeyException;
import com.alibaba.dashscope.utils.Constants;

public final class Main {
    static {
        Constants.baseHttpApiUrl="https://dashscope-intl.aliyuncs.com/api/v1";
    }
    public static void basicCall() throws ApiException, NoApiKeyException{
        TextEmbeddingParam param = TextEmbeddingParam
        .builder()
        .model("text-embedding-v3")
        .texts(Arrays.asList("Shall I compare thee to a summer's day", "Thou art more lovely and more temperate", "Rough winds do shake the darling buds of May", "And summer's lease hath all too short a date")).build();
        TextEmbedding textEmbedding = new TextEmbedding();
        TextEmbeddingResult result = textEmbedding.call(param);
        System.out.println(result);
    }

  public static void main(String[] args){
      try {
        basicCall();
    } catch (ApiException | NoApiKeyException e) {
        System.out.println(e.getMessage());
    }
    System.exit(0);
  }
}

Sample response

{
    "status_code": 200,
    "request_id": "617b3670-6f9e-9f47-ad57-997ed8aeba6a",
    "code": "",
    "message": "",
    "output": {
        "embeddings": [
            {
                "embedding": [
                    0.05821693316102028,
                    -0.48538774251937866,
                    -2.3512048721313477
                    .,
                    .,
                    .

                ],
                "text_index": 0
            }
        ]
    },
    "usage": {
        "total_tokens": 24
    }
}