General-purpose embedding models often underperform in domain-specific retrieval tasks because their training data does not cover your business content. At the same time, high-dimensional vectors drive up storage and compute costs as data volumes grow. AI Search Open Platform lets you address both problems through model customization:
Text embedding: Fine-tune a base text embedding model with your own data to improve retrieval accuracy in intelligent search and retrieval-augmented generation (RAG) scenarios.
Embedding dimensionality reduction: Train a custom model that compresses high-dimensional vectors into lower-dimensional ones, reducing storage and compute costs without significantly degrading retrieval quality.
In a typical workflow, you first fine-tune a text embedding model, then train a dimensionality reduction model to compress the resulting embeddings.
Prerequisites
Before you begin, make sure you have:
Access to the AI Search Open Platform console
Training data stored in MaxCompute or OSS
(If using a Resource Access Management (RAM) user) The required model service permissions granted to the RAM user
Customize the text embedding service
In the AI Search Open Platform console, go to Model Service > Model Customization and click Create.
On the Model Customization page, configure the following parameters.
Parameter Description Model Name A name for the model. Used when you invoke the service. Model Type Select Text Embedding (text-embedding). Base Model The base model used for training, such as ops-text-embedding-001.Dimensionality Reduction Enable to run dimensionality reduction training alongside text embedding training. Base Model for Reduction The model used for dimensionality reduction. Available only when Dimensionality Reduction is enabled. Data Source Select MaxCompute or OSS. Configure the data source parameters.
MaxCompute
Parameter Description Region The region where the MaxCompute project is located. Project Name The name of your MaxCompute project. AccessKey ID The AccessKey ID of the Alibaba Cloud account or RAM user with read and write permissions for MaxCompute. Get one from the AccessKey Management page. Secret The AccessKey secret corresponding to the AccessKey ID. Table Name The name of the table that stores your training data. Table Partition The partition information of the table. Training Fields Grant GetTableFields permission to the RAM user before selecting fields. Then select the primary key field and String-type text data fields. query-doc pair The query-document pairs used for training. See the sample data in the console for the required format. OSS
Parameter Description Region The region where the OSS bucket is located. OSS Bucket The name of your OSS bucket. Doc Data The training data files in OSS. query-doc pair The query-document pairs used for training. See the sample data in the console for the required format. OSS Endpoint Generated automatically after you complete the preceding configurations. Click OK. In the dialog that appears, choose how to start training:
Create and Train: Starts pre-processing immediately. Training begins after pre-processing completes.
Confirm Creation: Adds the model to the list with a To Be Trained status. Start training at any time from the model list.
Once training completes, the model status changes to Active. At that point, you can deploy the model for production use.
Customize the embedding dimensionality reduction service
In the AI Search Open Platform console, go to Model Service > Model Customization and click Create.
On the Model Customization page, configure the following parameters.
Parameter Description Model Name A name for the model. Used when you invoke the dimensionality reduction service. Model Type Select Vector Dimensionality Reduction (embedding-dim-reduction). Base Model The base model used for training, such as ops-embedding-dim-reduction-001.Data Source Select MaxCompute or OSS. Configure the data source parameters.
MaxCompute
Parameter Description Region The region where the MaxCompute project is located. Project Name The name of your MaxCompute project. AccessKey ID The AccessKey ID of the Alibaba Cloud account or RAM user with read and write permissions for MaxCompute. Get one from the AccessKey Management page. Secret The AccessKey secret corresponding to the AccessKey ID. Table Name The name of the table that stores your training data. Table Partition The partition information of the table. Training Fields Grant GetTableFields permission to the RAM user before selecting fields. Then select the primary key field and String-type embedding fields. Embedding fields must have between 1,024 and 4,096 dimensions. OSS
Parameter Description Region The region where the OSS bucket is located. OSS Bucket The name of your OSS bucket. Doc Data The training data files in OSS. OSS Endpoint Generated automatically after you complete the preceding configurations. Click OK. In the dialog that appears, choose how to start training:
Create and Train: Starts pre-processing immediately. Training begins after pre-processing completes.
Confirm Creation: Adds the model to the list with a To Be Trained status. Start training at any time from the model list.
Once training completes, the model status changes to Active. Click Experience to test the fine-tuned embedding model before invoking it.
Training status reference
| Status | Meaning | Next action |
|---|---|---|
| To Be Trained | Model created but training not started. | Start training from the model list. |
| Pre-processing | Training data is being prepared. | Wait for pre-processing to complete. |
| Training | Model training is in progress. | Wait for training to complete. |
| Active | Training complete. Model is ready. | Deploy or invoke the model. |
Invoke the model
After the model status changes to Active, invoke it using the API: