Platform For AI:Quick start: Deploy, fine-tune, and evaluate Qwen3 models

Deploy the model

Deploy the Qwen3-235B-A22B model with SGLang.

1. Go to the Model Gallery page.

   1. Log on to the PAI console and select a region. Switch regions if the current region lacks computing resources.

   2. In the navigation pane, click Workspace Management and click the target workspace name.

   3. In the left navigation pane, choose QuickStart > Model Gallery.

2. On the Model Gallery page, click the Qwen3-235B-A22B model card to open the model details page.

3. Click Deploy. Configure the following parameters and keep defaults for others.

   • Deployment Method: Set Inference Engine to SGLang and Deployment Template to Single-Node.

   • Resource Information: Set Resource Type to Public Resources. The system automatically recommends an instance type. For minimum required configuration, see Required computing power & supported token count.

   • Important

     If no instance types are available, the public resource inventory in this region is insufficient. Try the following:

     • Switch regions. China (Ulanqab) has larger inventory of Lingjun preemptible resources, such as ml.gu7ef.8xlarge-gu100, ml.gu7xf.8xlarge-gu108, ml.gu8xf.8xlarge-gu108, and ml.gu8tf.8.40xlarge. Preemptible resources can be reclaimed, so be mindful of your bid.

     • Use an EAS resource group. Purchase dedicated EAS resources from EAS Dedicated Resources Subscription.

   

Debug online

On the Service Details page, click Online Debugging.

Call the API

1. Obtain the service endpoint and token.

   1. In Model Gallery > Job Management > Deployment Jobs, click the name of the deployed service to open the service details page.

   2. Click View Invocation Method to view the Internet Endpoint and token.

      

2. Call the /v1/chat/completions endpoint for an SGLang deployment.

   curl

   curl -X POST \
       -H "Content-Type: application/json" \
       -H "Authorization: <EAS_TOKEN>" \
       -d '{
           "model": "<model_name, obtained from /v1/models API>",
           "messages": [
           {
               "role": "system",
               "content": "You are a helpful assistant."
           },
           {
               "role": "user",
               "content": "hello!"
           }
           ]
       }' \
       <EAS_ENDPOINT>/v1/chat/completions

   Python

   from openai import OpenAI
   
   ##### API configuration #####
   # Replace <EAS_ENDPOINT> with service endpoint and <EAS_TOKEN> with service token.
   openai_api_key = "<EAS_TOKEN>"
   openai_api_base = "<EAS_ENDPOINT>/v1"
   
   client = OpenAI(
       api_key=openai_api_key,
       base_url=openai_api_base,
   )
   
   models = client.models.list()
   model = models.data[0].id
   print(model)
   
   stream = True
   chat_completion = client.chat.completions.create(
       messages=[
           {"role": "user", "content": "Hello, please introduce yourself."}
       ],
       model=model,
       max_completion_tokens=2048,
       stream=stream,
   )
   
   if stream:
       for chunk in chat_completion:
           print(chunk.choices[0].delta.content, end="")
   else:
       result = chat_completion.choices[0].message.content
       print(result)

   Replace <EAS_ENDPOINT> with your service endpoint and <EAS_TOKEN> with your service token.

Invocation methods vary by deployment type. For more examples, see Deploy large language models and call APIs.

Integrate third-party applications

To connect to Chatbox, Dify, or Cherry Studio, see Integrate third-party clients.

Modify the token limit

Qwen3 models natively support 32,768 tokens. Use RoPE scaling to extend this to 131,072 tokens, though this may cause slight performance degradation. Modify the containers.script field in the service configuration JSON:

• vLLM:

  vllm serve ... --rope-scaling '{"rope_type":"yarn","factor":4.0,"original_max_position_embeddings":32768}' --max-model-len 131072

• SGLang:

  python -m sglang.launch_server ... --json-model-override-args '{"rope_scaling":{"rope_type":"yarn","factor":4.0,"original_max_position_embeddings":32768}}'

Parse tool calls

vLLM and SGLang support parsing tool calling output into structured messages. Modify the containers.script field in the service configuration JSON:

• vLLM:

  vllm serve ... --enable-auto-tool-choice --tool-call-parser hermes

• SGLang:

  python -m sglang.launch_server ... --tool-call-parser qwen25

Control the thinking mode

Qwen3 uses thinking mode by default. Control this behavior with a hard switch (completely disable thinking) or a soft switch (model follows user instruction on whether to think).

{
  "model": "<MODEL_NAME>",
  "messages": [
    {
      "role": "user",
      "content": "/no_think Hello!"
    }
  ],
  "max_tokens": 1024
}

Parse thinking content

To output thinking content separately, modify the containers.script field in the service configuration JSON:

• vLLM:

  vllm serve ... --enable-reasoning --reasoning-parser qwen3

• SGLang:

  python -m sglang.launch_server ... --reasoning-parser deepseek-r1

Training data format

SFT accepts JSON-formatted input. Each record contains an instruction and its corresponding output:

[
  {
    "instruction": "Summarize the key features of Qwen3 models.",
    "output": "Qwen3 models feature a hybrid thinking mode that can be toggled on or off, support for tool calling, and a Mixture-of-Experts (MoE) architecture in the 235B-A22B and 30B-A3B variants for efficient inference."
  },
  {
    "instruction": "What is the difference between LoRA and QLoRA?",
    "output": "LoRA adds low-rank adapters to the model weights for efficient fine-tuning. QLoRA combines LoRA with 4-bit quantization, further reducing memory usage while maintaining comparable training quality."
  }
]

Fine-tuning procedure

1. On the model details page in Model Gallery, click Train. Configure the following parameters:

   • Algorithm: Select SFT or GRPO.

   • Dataset Configuration: Upload training data to OSS, or select data from NAS or CPFS. PAI also provides public datasets for testing.

   • Compute Resource Configuration: A10 GPUs (24 GB) or higher are recommended. For 32B models, use GU100 GPUs (80 GB) or higher.

   • Model Output Path: The fine-tuned model is saved to OSS for download or deployment.

   

2. Configure hyperparameters. The following table describes key parameters:

   Parameter	Default	Required	Description
   training_strategy	sft	Yes	Training strategy. Set to sft for supervised fine-tuning or grpo for preference optimization.
   learning_rate	5e-5	Yes	Controls weight adjustment magnitude per training step.
   num_train_epochs	1	Yes	Number of passes over the training dataset.
   per_device_train_batch_size	1	Yes	Samples processed per GPU per step. Larger values improve efficiency but increase VRAM usage.
   lora_dim	32	No	LoRA adapter rank. When set to > 0, enables LoRA or QLoRA training. Set to 0 for full-parameter training.
   load_in_4bit	false	No	Load model in 4-bit precision. When lora_dim > 0 and load_in_4bit is true, uses QLoRA training.

3. Click Train to start the job. Monitor status and view logs on the training page.

   

4. After training completes, click Deploy to deploy the fine-tuned model as an online service.

How do I maintain conversation context across multiple API calls?

PAI model services are stateless. Each API call is independent — the server does not retain context between requests.

To implement multi-turn conversation, manage conversation history on the client side. Pass the entire conversation history in the messages payload with each API call. For an example, see Implement multi-turn conversation