Platform For AI:Deploy, fine-tune, and evaluate QwQ-32B

Introduction

On March 6, Alibaba Cloud open-sourced QwQ-32B, an inference model from the Qwen family. Based on large-scale reinforcement learning, it delivers breakthrough gains in mathematics, coding, and general capabilities. Its overall performance is comparable to DeepSeek-R1, while significantly reducing deployment costs.

• On the AIME24 benchmark for math and the LiveCodeBench for coding, QwQ-32B performs on par with DeepSeek-R1 and significantly outperforms o1-mini and R1-distilled models of similar size.

• On LiveBench (the "hardest LLM evaluation leaderboard" led by Meta Chief Scientist Yann LeCun), IFEval for instruction-following (proposed by Google and others), and the BFCL test for function-calling accuracy (proposed by UC Berkeley and others), QwQ-32B scores higher than DeepSeek-R1.

• QwQ-32B integrates agent capabilities, enabling it to think critically while using tools and adjust its reasoning process based on feedback from the environment.

PAI-Model Gallery now fully supports one-click deployment, fine-tuning, and evaluation for QwQ-32B, which requires a GPU with 96 GB of GPU memory for deployment. Quantized versions, such as QwQ-32B-GGUF and QwQ-32B-AWQ, are also supported and can be deployed on lower-cost GPUs, such as a single A10 card.

Model deployment

1. Go to the PAI-Model Gallery page.

   1. Log in to the PAI console and select a region in the upper-left corner. You can switch regions to find available computing resources.

   2. In the left navigation pane, choose Workspaces, and then click the name of a workspace to enter it.

   3. In the left navigation pane, choose QuickStart > Model Gallery.

2. On the PAI-Model Gallery page, find the QwQ-32B model card and click it to open the model details page.

3. Click Deploy in the upper-right corner. Select a deployment framework, configure a name for the inference service, and specify the resources for deployment. This deploys the model to Elastic Algorithm Service (EAS). The platform supports various deployment frameworks, including SGLang, vLLM, and BladeLLM (a high-performance inference framework developed by PAI).

   For the deployment method, you can select Single-Node - Standard Instance or Single-Node - GP7V Instance. For service creation, you can either create a New Service or Update Existing Service, and configure its group and tags. Supported resource types include Public Resources, EAS Resource Group, and Resource Quota.

4. Use the inference service. After the deployment succeeds, go to the service page and click View Call Information to get the endpoint and token. To learn how to call the service, click the pre-trained model link to return to the model details page and view the instructions.

   During service creation, you can click View Deployment Events next to the Status field to monitor the deployment progress. The status message shows Waiting task server to be ready.

   You can also debug the deployed QwQ-32B model service online in Elastic Algorithm Service (EAS).

   On the Online Debugging page, select the Chat Tab. The HTTP method is POST, and the request path is /api/predict/quickstart_qwq32b/v1/chat/completions. In the JSON example in the Body, model is set to QwQ-32B, messages contains role: user and content: Please introduce yourself, and max_tokens is set to 1024. After you click Send Request, a status code of 200 is returned, the response body contains an inference result of the chat.completion type, and the content for the assistant role contains a thinking process enclosed in <think> tags.

Model fine-tuning

PAI-Model Gallery provides a supervised fine-tuning (SFT) algorithm for QwQ-32B. It supports both LoRA fine-tuning and full-parameter fine-tuning, so you can fine-tune the model out of the box.

1. Prepare training data. The SFT algorithm supports training datasets in formats such as JSONL. For more information about format requirements, see the model documentation. The following is an example of a training dataset in JSONL format:

   {"messages": [{"role": "system", "content": "You are a helpful and harmless assistant."}, {"role": "user", "content": "Who are you?"}, {"role": "assistant", "content": "I am Xiaopai, an AI assistant trained by PAI. My goal is to provide users with useful, accurate, and timely information and to help them communicate effectively in various ways. How can I help you?"}]}
   {"messages": [{"role": "system", "content": "You are a helpful and harmless assistant."}, {"role": "user", "content": "Who are you!"}, {"role": "assistant", "content": "Hello! I am an AI language model developed by PAI, named Xiaopai. I can answer your questions, provide information, engage in conversation, and help you solve problems. If you have any questions or need assistance, please feel free to let me know!"}]}

2. Configure training parameters. After you prepare the data, upload it to a bucket in Object Storage Service (OSS). Because the 32B model is large, the algorithm requires GPU resources with at least 96 GB of GPU memory. Ensure that your resource quota provides sufficient computing resources.

   The algorithm supports the following hyperparameters. You can adjust them based on your dataset and computing resources, or use the default values.

   Parameter	Description	More information
   learning_rate	The learning rate, which controls the magnitude of model weight adjustments.	A learning rate that is too high can cause training instability, with the loss value fluctuating wildly and failing to converge. A rate that is too low causes the loss to decrease slowly, requiring a long time to converge. An appropriate learning rate allows the model to converge to an optimal solution quickly and stably.
   num_train_epochs	The number of passes over the training dataset.	Too few epochs can cause underfitting, while too many can cause overfitting. If the sample size is small, you can increase the number of epochs to avoid underfitting. A smaller learning rate typically requires more epochs.
   per_device_train_batch_size	The number of samples processed by each GPU in a single training iteration.	A larger batch size can increase training speed but also increases GPU memory demand. The ideal batch size is typically the maximum value that does not cause GPU memory overflow. You can check GPU memory usage on the task monitoring page in the training details.
   gradient_accumulation_steps	The number of gradient accumulation steps.	A small batch size increases the variance of the gradient estimate, affecting convergence speed. Gradient accumulation allows the model to update weights after accumulating gradients from a specified number of batches. Ensure that gradient_accumulation_steps is a multiple of the number of GPUs.
   max_length	The maximum token length of the input data processed by the model in a single training step.	A tokenizer processes the training data to produce a token sequence. You can use a token estimation tool to estimate the text length in your training data.
   lora_rank	The LoRA dimension.
   lora_alpha	The LoRA weight.	The LoRA scaling factor, typically set to lora_rank * 2.
   lora_dropout	The dropout rate for LoRA training. It helps prevent overfitting by randomly deactivating neurons during training.
   lorap_lr_ratio	The LoRA+ learning rate ratio (λ = ηB/ηA), where ηA and ηB are the learning rates for adapter matrices A and B, respectively.	Compared to LoRA, LoRA+ can achieve better performance and faster fine-tuning by using different learning rates for key parts of the process, without increasing computational requirements. When lorap_lr_ratio is set to 0, the training uses standard LoRA instead of LoRA+.
   advanced_settings	In addition to the parameters above, you can customize other parameters in this field using the --key1 value1 --key2 value2 format. If not needed, leave this field blank.	save_strategy: The model checkpoint saving strategy. Options are steps, epoch, and no. Default: steps. save_steps: The model saving interval. Default: 500. save_total_limit: The maximum number of checkpoints to save. The system deletes older checkpoints to stay within this limit. Default: 2. If None, the system saves all checkpoints. warmup_ratio: A hyperparameter that controls the learning rate warm-up phase. During this initial phase, the learning rate gradually increases from a small value to its initial set value. The warmup_ratio determines the proportion of this phase in the entire training process. Default: 0.

3. Click the Train button to start training. You can view the training job status and logs. You can also deploy the fine-tuned model as an online service.

Model evaluation

PAI-Model Gallery includes built-in evaluation algorithms, allowing you to evaluate pre-trained and fine-tuned models out of the box. Evaluation helps you assess model performance and supports side-by-side comparisons to help you select the most suitable model for your use case.

You can start a model evaluation from two places:

Model evaluation supports both custom and public datasets.

On the New Evaluation Task page, click Switch to professional mode next to the page title to enter professional mode, which supports advanced features like judge model evaluation. In the form, configure the Task Name, Model, Output Path, Dataset Source, Resource Type, Resource Group Type, Resource Configuration Method, Task Resources, VPC Configuration, and Internet gateway parameters.

• Custom dataset evaluation

  Model evaluation supports common NLP text-matching metrics, such as BLEU and ROUGE, as well as judge model evaluation. This feature, available only in professional mode, uses a judge LLM to evaluate another LLM, providing a score and reasoning. You can use data unique to your scenario to determine if a selected model is suitable.

  The evaluation requires a JSONL-formatted evaluation set file, where each line is a JSON object. Use question to identify the question column and answer for the answer column. Example file: evaluation_test.jsonl.

• Public dataset evaluation

  Provides a comprehensive assessment of large models by using open-source evaluation datasets grouped by domain. PAI currently maintains datasets such as CMMLU, GSM8K, TriviaQA, MMLU, C-Eval, TruthfulQA, and HellaSwag, covering multiple domains including mathematics, knowledge, and reasoning. More public datasets are added regularly. Note: Evaluations on the GSM8K, TriviaQA, and HellaSwag datasets are time-consuming. Select them only as needed.

Next, select an output path for the evaluation results, choose the recommended computing resources, and submit the evaluation task. After the task is complete, view the evaluation results on the task page. If you select multiple datasets, the model runs evaluations for them sequentially, which may increase the wait time. You can check the logs to see the task's progress.

View the evaluation report: The following images show example evaluation results for custom and public datasets.

Contact us

We invite you to follow and use PAI-Model Gallery. We continuously update the platform with SOTA models. If you have any model requests, feel free to contact us. You can join the PAI-Model Gallery user community by searching for the DingTalk group number 79680024618.