Alibaba Cloud Model Studio:Model tuning

Model tuning

To optimize model performance, model tuning can:

• Improve model performance for specific industries or scenarios

• Reduce model output latency

• Mitigate model hallucination

• Align with human values or preferences

• Replace larger models with fine-tuned, lightweight models

During fine-tuning, a model learns from your training data, adopting characteristics specific to your business or scenario, such as domain knowledge, tone, phrasing, and persona. After fine-tuning on many specific examples, the model often performs better with one-shot or zero-shot prompts than it previously did with few-shot prompts. This lets you use shorter prompts, which saves a significant number of input tokens and reduces model output latency.

Model fine-tuning

For details, see:

Prerequisites

• While model fine-tuning for text generation can deliver excellent results for specific business scenarios, it has the following limitations:

  • Time-consuming, including: having a large-scale (at least 50 million tokens) CPT dataset, building an effective (1000+) SFT dataset, collecting enough (100+) bad cases to build an effective model deployment billing DPO dataset, and slow model optimization and iteration speeds.

  • Expensive: A fine-tuned model must be deployed before use, and the model deployment billing is high.

• Before considering model fine-tuning, Model Studio recommends that you first try to customize your application by using prompt engineering or function calling. Model fine-tuning is often considered a "last resort" to improve model performance for the following reasons:

  1. For many tasks, a model may not perform well initially. However, you can often improve its performance by using the right prompting techniques, which can eliminate the need for model fine-tuning.

  2. Iteratively optimizing prompts and plugins is more agile and cost-effective than model fine-tuning. Fine-tuning iterations can require resource-intensive steps, such as re-collecting, cleaning, and optimizing data; gathering bad cases; and conducting user surveys.

  3. Even if you ultimately decide that model fine-tuning is necessary, your initial work on prompt engineering and plugin optimization is not wasted. You can fully reuse this preliminary work to build fine-tuning datasets, as it provides their required inputs.

Quick start

Fine-tuning on the console

Tuning data format

# A single training example in JSON format, which has the following structure when expanded:
{"messages": [
  {"role": "system", "content": "System Input 1"}, 
  {"role": "user", "content": "User Input 1"}, 
  {"role": "assistant", "content": "Expected Model Output 1"}, 
  {"role": "user", "content": "User Input 2"}, 
  {"role": "assistant", "content": "Expected Model Output 2"}
  ...
]}

 {"role": "assistant", "content": "Expected Model Output 1", "loss_weight": 1.0}, 
 {"role": "assistant", "content": "Expected Model Output 2", "loss_weight": 0.5}

Dataset building tips

Data diversity and balance

Model tuning approaches vary by scenario. Specific business scenarios require more domain expertise, while question-answering scenarios require more generalization. Design data use cases to match the model's intended business module or usage scenario. Consequently, strong training performance depends less on sheer data volume and more on the data's scenario-specific relevance and diversity.

The following AI conversation scenario illustrates the types of business scenarios to include in a professional, diverse dataset:

Ensure the data for each scenario and business is balanced according to real-world proportions. This prevents the model from becoming biased toward the features of a particular data type, which impairs its generalization ability.

Splitting training and validation sets

For model tuning, the console supports the following:

• Automatically split a training dataset and use a random sample to create a validation set.

• Select a separate dataset.

During training, the console displays real-time validation set loss and token accuracy.

Frequently asked questions

Fine-tuning a custom model

You cannot use Model Studio to fine-tune, upload, export, or download models.