Alibaba Cloud Model Studio:Fine-tune with the API or CLI

Parameter

Recommended setting

Type

Description

n_epochs

(Number of epochs) [Required]

Data size < 10,000: 3~5

Data size > 10,000: 1~2

Integer

The number of times the model iterates through the training data. Adjust based on your experience with model fine-tuning.

More epochs lead to longer training times and higher costs.

learning_rate

(Learning rate)

Use the recommended default value.

Float

Controls the intensity of model weight adjustments.

• A high learning rate causes drastic changes in model parameters, which may lead to poor or worse performance.

• A low learning rate results in minimal changes to the model's performance.

freeze_vit

(Freeze visual backbone)

Adjust as needed

Boolean

Freezes the parameters of the visual backbone so its weights are not updated during training. This applies only to Qwen-VL models.

batch_size

(Batch size) [Required]

Use the recommended default value.

Integer

The number of data entries fed into the model for training at one time. A small value can significantly extend training time. Default values vary by model; check the console.

eval_steps

(Evaluation steps)

Adjust as needed

Integer

The step interval for model validation during training, used for periodic evaluation of training accuracy and loss.

This parameter affects the display frequency of Validation Loss and Validation Token Accuracy during model fine-tuning.

logging_steps

(Logging steps)

Adjust as needed

Integer

The step interval for logging fine-tuning progress.

lr_scheduler_type

(Learning rate scheduler)

Recommended: linear or Inverse_sqrt

String

The strategy for dynamically adjusting the learning rate during training.

For details on each strategy, see Fine-tune a model in the console.

max_length

(Sequence length) [Required]

8192

Integer

The maximum token length supported for a single training data entry. If a data entry exceeds this value, it will be discarded and not used for training.

For the relationship between characters and tokens, see How to convert between tokens and characters.

max_split_val_dataset_sample

(Max validation set samples)

Use the recommended default value.

Integer

If "validation_file_ids" is not set, the automatically split validation set will have a maximum of 1,000 samples.

This parameter is ignored if "validation_file_ids" is set.

split

(Training set ratio)

Use the recommended default value.

Float

If "validation_file_ids" is not set, 80% of the training file is automatically used as the training set and 20% as the validation set.

This parameter is ignored if "validation_file_ids" is set.

warmup_ratio

(Warm-up ratio)

Use the recommended default value.

Float

The proportion of the total training process dedicated to learning rate warm-up, where the learning rate linearly increases from a small value to the set value.

This parameter helps stabilize training by limiting parameter changes in the initial phase.

A high ratio has the same effect as a low learning rate, leading to minimal changes. A low ratio has the same effect as a high learning rate, potentially degrading performance.

A ratio that is too small has the same effect as a learning rate that is too high, which can cause the performance of the tuned model to not improve or even to degrade.

This parameter has no effect on the "Constant" learning rate scheduler.

weight_decay

(Weight decay)

Use the recommended default value.

Float

The strength of L2 regularization, which helps maintain the model's generalizability. An overly large value can diminish the effect of fine-tuning.

Efficient fine-tuning (efficient_sft or dpo_lora) parameters

lora_rank

(LoRA rank)

64

Integer

The rank of the low-rank matrices in LoRA training. A higher rank improves fine-tuning effectiveness but slightly slows down training.

lora_alpha

(LoRA alpha)

Use the recommended default value.

Integer

The scaling factor that controls the combination of original model weights and the LoRA low-rank correction term.

A larger alpha gives more weight to the LoRA correction, making the model rely more on task-specific information.

A smaller alpha makes the model retain more knowledge from the original pre-trained model.

lora_dropout

(LoRA dropout)

Use the recommended default value.

Float

The dropout rate for the values in the low-rank matrices during LoRA training.

Using the recommended value enhances the model's generalization capabilities.

An overly large value can diminish the fine-tuning effect.

Data augmentation (supports efficient_sft and sft) parameters

data_augmentation

(Enable mixed training)

Mix based on the model's use case.

Boolean

When enabled, your training data is mixed with general-purpose datasets (multi-domain/industry/scenario):

- Effect: Improves training results and prevents catastrophic forgetting.

- Billing: Mixed data is included in the total training tokens and billed at the standard rate.

augmentation_ratio

(Mixing ratio)

Mix based on the model's use case.

String

• Format: Must correspond to augmentation_types.

• Example: "0.1,0.05,0.15" (The values correspond to the three datasets listed in augmentation_types, respectively.)

• Description: Randomly sample and mix 10%/5%/15% of the training data.

• Range: 0.0 ~ 2.0

Model Parameter Snapshot Publishing (for efficient_sft or sft only) Parameters

save_strategy

(Snapshot save strategy)

You can set this to epoch or steps.

• When set to steps, you can adjust the save interval by setting the save_steps parameter.

String

Sets the save interval and maximum number of model parameter snapshots (checkpoints) to save during fine-tuning.

save_steps

(Save steps)

If you need to manually modify the value, you should set it to an integer multiple of the eval_steps parameter.

Integer

Sets the step interval for saving model parameter snapshots (checkpoints).

save_total_limit

(Snapshot save limit)

10

Integer

Limits the maximum number of model parameter snapshots (checkpoints) that can be saved for publishing.