Chain-of-thought reasoning distillation - Platform For AI

The chain-of-thought (CoT) reasoning distillation template generates complete reasoning traces from a teacher model, scores them by reasoning verbosity and cognitive difficulty, and produces a balanced SFT dataset through difficulty-aware bin sampling — ready for student model fine-tuning.

Overview

The CoT reasoning distillation template is designed for domains that require multi-step reasoning, such as mathematics, logic, and code generation. A teacher model generates full reasoning traces for your question set. Each trace is then scored on reasoning verbosity (RV) and cognitive difficulty (CD). Bin sampling by difficulty produces a dataset with a balanced difficulty distribution, preventing over-concentration on easy questions. The output is a chain-of-thought SFT dataset ready for student model fine-tuning.

Note

This template supports an optional two-stage workflow: Stage 1 builds the distillation data, and Stage 2 performs student model training (optional). You can run Stage 1 alone or execute both stages sequentially.

The CoT reasoning distillation pipeline consists of the following steps:

Step	Name	Description
1	Generate chain-of-thought	Calls the teacher model to generate a response with a complete reasoning trace for each input question.
2	RV/CD scoring	Scores each generated trace on reasoning verbosity (RV), cognitive difficulty (CD), and logical correctness to support downstream filtering.
3	Difficulty bin mixing	Samples by cognitive difficulty bins and matches reasoning verbosity to produce a balanced dataset, preventing over-concentration on easy questions.
4	Build SFT dataset	Filters out empty and oversized samples, and outputs a standardized SFT training dataset.

Procedure

Log on to the PAI console. In the left-side navigation pane, choose Model Application> Model Distillation (ModelDistill).
Click Create task to go to the Distillation Template tab, or navigate directly to the Distillation Template tab.
Select Chain-of-thought reasoning distillation and click Use Template to go to the Create Distillation Task page. Configure the key parameters described below and keep the default values for other settings.
Configure the Basic Configuration:
- Task name: Auto-filled by the system.
- Output Path: Specify an OSS path that you have permission to access, for example, oss://mybucket.oss-cn-hangzhou-internal.aliyuncs.com/cot-distill/.
  Note
  Create a separate output directory for each distillation task to prevent files from being overwritten. The OSS bucket must be in the same region as the PAI service.
Click Next to go to the Build distillation data page:
- Teacher model: For Teacher Model Source, select Token Service and choose a model with strong reasoning capabilities, such as qwen3.8-max.
- Dataset: Select an OSS directory or file that you have permission to access. You can download and use the sample dataset.
- Distillation Configuration: Default values are pre-filled. Modify them as needed.
  - dataset.input_path: If a file is already selected for the dataset, the filename configured here doesn't take effect.
  - For more CoT reasoning distillation template-specific parameters, such as RV/CD scoring dimensions and bin mixing strategies, see the Configuration parameters section below.
- Compute Resource: Set Resource Type to General Computing, set Source to Public Resources, and select an appropriate Instance Type from Job Resource, such as ecs.c6.large.
Click Next: Student Model Training to continue configuring, or select Run Only Distillation Data Build to skip:
- Student Model: Select Public Model and choose a smaller model from the drop-down list, such as Qwen3-4B.
- Training Mode: Select LoRA.
- Compute Resource: Set Resource Type to General Computing, set Source to Public Resources, and select an appropriate Instance Type from Job Resource.
Click Create, and then click OK in the billing confirmation dialog box. The page redirects to the basic configuration page of the task details, where you can track the task status.

Configuration parameters

This section describes the key parameters specific to the CoT reasoning distillation template. Basic configuration and resource configuration are the same as general distillation. Only template-specific parameter differences are listed here.

Basic configuration

Basic configuration includes the task name and output path. The output path is the root directory for all task outputs. The distillation dataset is also written under this directory — no need to specify a separate dataset output path.

Distillation data

Teacher model

The CoT reasoning distillation template requires only one teacher model, which generates complete reasoning traces for each input question. Choose a model with strong reasoning capabilities. The teacher model source supports only PAI Token Service.

Dataset

Select an input directory or file. The data format requirements are as follows:

Format: JSONL, one question per line.
Each entry must contain the instruction field.
Manual answer annotation isn't required. The teacher model generates both the reasoning trace and the answer.

Sample data:

{"instruction": "Prove that the square root of 2 is an irrational number."}
{"instruction": "If 4 machines can produce 800 parts in 5 hours, how many parts can 6 machines produce in 8 hours?"}

Distillation configuration

The distillation configuration is provided in YAML format with recommended default values from the algorithm team. You can adjust them as needed. The following table describes key parameters.

Configuration section	Description
dataset	`output_path` (default: `distillation_dataset.json`): Output path for distillation artifacts. You can change it to another .json file in a relative path, or place it in a subdirectory (for example, `sft_dataset.json` or `output/sft_dataset.json`).
backend	`timeout` (default: `1200.0`): Maximum wait time per request in seconds. Responses that use the full `max_tokens` (16384) budget require more than 400 seconds at observed throughput of 38–58 tokens/s.
generation	`system_prompt` (default: "You are a helpful assistant. Think step by step and provide clear reasoning."): System prompt used when the teacher model generates reasoning traces. `temperature` (default: `0.7`): Controls generation diversity. `max_tokens` (default: `16384`): Maximum number of tokens per generation.
sft	`skip_empty` (default: `true`): Skips empty samples. `min_length` (default: `10`): Filters out samples that are too short. `max_length` (default: `131072`): Filters out samples that are too long.
eval	Scoring dimensions (`metrics`): reasoning_verbosity, cognitive_difficulty, and logical_correctness. Uses a dedicated CoT evaluation prompt file (`configs/prompts/default_cot_eval_prompts.yaml`).
pipeline	Chain-of-thought generation stage (`cot_distill`): Uses the CoT generation prompt template. RV/CD scoring stage (`cot_rvcd_score`): `max_workers` defaults to 4 (concurrent scoring threads). Difficulty bin mixing stage (`cot_mix_by_rv_cd`): `cd_bins` defaults to [0, 3, 6, 10] (cognitive difficulty bin boundaries — 4 boundaries create 3 bins) `rv_target` defaults to matched (matches reasoning verbosity to cognitive difficulty) `samples_per_bin` defaults to 0 (maximum samples per difficulty bin; 0 means no limit; default output cap is 300) `min_correctness` defaults to 1 (minimum logical correctness threshold)