Fine-tune models with the API or CLI | Model Studio - Alibaba Cloud Model Studio

Tune Qwen models in Model Studio through the API (HTTP) or CLI (shell). Three tuning methods are supported: supervised fine-tuning (SFT), continual pre-training (CPT), and direct preference optimization (DPO).

Prerequisites

Review Introduction to model fine-tuning to understand its concepts, process, and data requirements.
Activate the service and obtain an API key. Get an API key.
Grant the RAM user (RAM user) the necessary invocation, training, and deployment permissions.

NoteThe API supports only token-based billing. To use model training units (prepaid or postpaid), create the job in the console.

Upload tuning files

Preparing fine-tuning files

SFT training set

SFT ChatML (Chat Markup Language) format training data supports multi-turn conversations and various role settings.

The OpenAI name and weight parameters are not supported. All assistant outputs will be trained.

# A single line of training data (in JSON format) has the following typical structure when expanded:
{"messages": [
  {"role": "system", "content": "System input 1"},
  {"role": "user", "content": "User input 1"},
  {"role": "assistant", "content": "Expected model output 1"},
  {"role": "user", "content": "User input 2"},
  {"role": "assistant", "content": "Expected model output 2"}
  ...
]}

For information about the differences between system, user, and assistant, see Overview. Sample training datasets: SFT-ChatML_format_example.jsonl, SFT-ChatML_format_example.xlsx. XLS and XLSX formats support only single-turn conversations.

All assistant lines in a single training data entry support the "loss_weight" parameter, which sets the relative importance of that line during training. (Range: 0.0 to 1.0. A larger value indicates higher importance.)

This parameter is available for invitational preview. To use it, contact your account manager.

 {"role": "assistant", "content": "Expected model output 1", "loss_weight": 1.0},
 {"role": "assistant", "content": "Expected model output 2", "loss_weight": 0.5}

You can also download a data template from the Model Studio console.

Below the file upload area on the Create Dataset page, you can find the data template download link.

Upload fine-tuning fileto Model Studio

OpenAI-compatible Files API

python

import os
from pathlib import Path
from openai import OpenAI

client = OpenAI(
    # If you have not configured an environment variable, replace the following line with api_key="sk-xxx" and use your Model Studio API key.
    # API keys for the Singapore and China (Beijing) regions are different. Get an API key: https://www.alibabacloud.com/help/model-studio/get-api-key
    api_key=os.getenv("DASHSCOPE_API_KEY"),
    # The following is the URL for the Singapore region. If you use a service in the China (Beijing) region, replace the URL with: https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1
    base_url="https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1",
)

# test.jsonl is a local sample file.
file_object = client.files.create(file=Path("test.jsonl"), purpose="fine-tune")

print(file_object.model_dump_json())

java

import com.openai.client.OpenAIClient;
import com.openai.client.okhttp.OpenAIOkHttpClient;
import com.openai.models.files.*;

import java.nio.file.Path;
import java.nio.file.Paths;

public class Main {
    public static void main(String[] args) {
        // Create a client and use the API key from the environment variable.
        OpenAIClient client = OpenAIOkHttpClient.builder()
                // API keys for the Singapore and China (Beijing) regions are different. Get an API key: https://www.alibabacloud.com/help/model-studio/get-api-key
                .apiKey(System.getenv("DASHSCOPE_API_KEY"))
                // The following is the URL for the Singapore region. If you use a service in the China (Beijing) region, replace the URL with: https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1
                .baseUrl("https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1")
                .build();
        // Set the file path. Modify the path and filename as needed.
        Path filePath = Paths.get("src/main/java/org/example/test.txt");
        // Create file upload parameters.
        FileCreateParams params = FileCreateParams.builder()
                .file(filePath)
                .purpose(FilePurpose.of("fine-tune"))
                .build();

        // Upload the file.
        FileObject fileObject = client.files().create(params);
        System.out.println(fileObject);
    }
}

bash

# ======= Important =======
# API keys for the Singapore and China (Beijing) regions are different. Get an API key: https://www.alibabacloud.com/help/model-studio/get-api-key
# The following is the URL for the Singapore region. If you use a service in the China (Beijing) region, replace the URL with: https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1/files
# === Delete this comment before running ===

curl -X POST https://{WorkspaceId}.ap-southeast-1.maas.aliyuncs.com/compatible-mode/v1/files \
-H "Authorization: Bearer $DASHSCOPE_API_KEY" \
--form 'file=@"test.jsonl"' \
--form 'purpose="fine-tune"'

NoteLimitations:

The maximum size of a single file is 300 MB.
The total size of all non-deleted files is limited to 5 GB.
You can store a maximum of 100 non-deleted files.
Files are stored indefinitely.

Model fine-tuning

Creating, querying, logging, checkpoint management, and listing/canceling/deleting fine-tuning jobs are all done via API. Each interface's complete request/response parameters and examples are on the corresponding API page; this section only outlines the workflow.

Create a fine-tuning job

Call the Create a fine-tuning job interface (POST /api/v1/fine-tunes) to start training. See that page for the request body (model / training_datasets / hyper_parameters / training_type, etc.) and response examples.

Retrieve a fine-tuning job

Use the job_id to query the task status. See Query fine-tuning task details.

Get fine-tuning job logs

Retrieve training logs via the Get fine-tuning task logs interface (GET /api/v1/fine-tunes/{job_id}/logs).

Query and publish model checkpoints

Checkpoints produced by fine-tuning can be listed and exported (published):

List checkpoints: see List Checkpoint (GET /api/v1/fine-tunes/{job_id}/checkpoints).
Export (publish) a specific checkpoint: see Export Checkpoint (GET /api/v1/fine-tunes/{job_id}/export/{checkpoint}?model_name=...), asynchronous; poll status via the list interface.

More fine-tuning operations

Model deployment and invocation

Model deployment

To deploy the model, go to the Dedicated Deployment console.

Model invocation

Once deployment status is RUNNING, invoke the fine-tuned model like any other model.

You can also get the Model Code from the Dedicated Deployment console.

Usage and parameters: DashScope API Reference.

curl 'https://dashscope.aliyuncs.com/api/v1/services/aigc/text-generation/generation' \
--header 'Authorization: Bearer '${DASHSCOPE_API_KEY}  \
--header 'Content-Type: application/json' \
--data '{
    "model": "<your_model_instance_id>",
    "input":{
        "messages":[
            {
                "role": "user",
                "content": "Who are you?"
            }
        ]
    },
    "parameters": {
        "result_format": "message"
    }
}'