Platform For AI:Deploy, fine-tune, and evaluate Qwen3 models in PAI

Deploy a model

This example deploys the Qwen3-235B-A22B model with SGLang.

1. Go to the Model Gallery page.

   1. Log in to the PAI console and select a region. Switch regions if needed to find available computing resources.

   2. In the left-side navigation pane, choose Workspaces and click your target workspace.

   3. In the left-side navigation pane, choose QuickStart > Model Gallery.

2. On the Model Gallery page, click the Qwen3-235B-A22B model card.

3. In the upper-right corner, click Deploy. Configure the following parameters. Use the defaults for others.

   • Deployment Method: Set Inference Engine to SGLang and Deployment Template to Single-Node.

   • Resource Information: For Resource Type, select public resource. The system recommends a suitable instance type. For minimum configurations, see Appendix: Computing resources and token limits.

     Important

     If no specifications are available, the region's public resource pool is insufficient. Try:

     • Switch regions. China (Ulanqab) has more Lingjun preemptible instances (ml.gu7ef.8xlarge-gu100, ml.gu7xf.8xlarge-gu108, ml.gu8xf.8xlarge-gu108, ml.gu8tf.8.40xlarge). Preemptible resources can be reclaimed — set your bid carefully.

     • Use an EAS resource group. Purchase dedicated EAS resources on the EAS Dedicated Resources Subscription page.

Debug online

On the Service details page, click online debugging. Example:

Invoke the API

1. Obtain the service endpoint and token:

   1. In the left-side navigation pane, choose Model Gallery > Job Management > Deployment Jobs and click the service name to open the service details page.

   2. Click View Call Information to find the internet endpoint and token.

      

2. Example: call the /v1/chat/completions API for an SGLang-deployed service.

   curl

   curl -X POST \
       -H "Content-Type: application/json" \
       -H "Authorization: <EAS_TOKEN>" \
       -d '{
           "model": "<model_name, obtained from the /v1/models API>",
           "messages": [
           {
               "role": "system",
               "content": "You are a helpful assistant."
           },
           {
               "role": "user",
               "content": "hello!"
           }
           ]
       }' \
       <EAS_ENDPOINT>/v1/chat/completions

   Python

   from openai import OpenAI
   
   ##### API configuration #####
   # Replace <EAS_ENDPOINT> with the service endpoint and <EAS_TOKEN> with the service token.
   openai_api_key = "<EAS_TOKEN>"
   openai_api_base = "<EAS_ENDPOINT>/v1"
   
   client = OpenAI(
       api_key=openai_api_key,
       base_url=openai_api_base,
   )
   
   models = client.models.list()
   model = models.data[0].id
   print(model)
   
   stream = True
   chat_completion = client.chat.completions.create(
       messages=[
           {"role": "user", "content": "Hello, could you please introduce yourself"}
       ],
       model=model,
       max_completion_tokens=2048,
       stream=stream,
   )
   
   if stream:
       for chunk in chat_completion:
           print(chunk.choices[0].delta.content, end="")
   else:
       result = chat_completion.choices[0].message.content
       print(result)

   Replace <EAS_ENDPOINT> with the service endpoint and <EAS_TOKEN> with the service token.

Other invocation methods are covered in Invoke an API for a deployed LLM service.

Third-party integration

To connect to Chatbox, Dify, or Cherry Studio, see Integrate with third-party clients.

Modify token limit

Qwen3 models natively support 32,768 tokens. RoPE scaling extends this to 131,072, though performance may degrade slightly. Modify the containers.script field in the service configuration JSON:

• vLLM:

  vllm serve ... --rope-scaling '{"rope_type":"yarn","factor":4.0,"original_max_position_embeddings":32768}' --max-model-len 131072

• SGLang:

  python -m sglang.launch_server ... --json-model-override-args '{"rope_scaling":{"rope_type":"yarn","factor":4.0,"original_max_position_embeddings":32768}}'

Parse tool calls

vLLM and SGLang can parse tool calls into structured messages. Modify the containers.script field in the service configuration JSON:

• vLLM:

  vllm serve ... --enable-auto-tool-choice --tool-call-parser hermes

• SGLang:

  python -m sglang.launch_server ... --tool-call-parser qwen25

Control thinking mode

Qwen3 uses thinking mode by default. A hard switch disables thinking entirely; a soft switch lets the model decide based on user instructions.

{
  "model": "<MODEL_NAME>",
  "messages": [
    {
      "role": "user",
      "content": "/no_think Hello!"
    }
  ],
  "max_tokens": 1024
}

Parse thinking content

To output thinking content separately, modify the containers.script field:

• vLLM:

  vllm serve ... --enable-reasoning --reasoning-parser qwen3

• SGLang:

  python -m sglang.launch_server ... --reasoning-parser deepseek-r1

Q: Do PAI-deployed services support session context?

No. PAI model service APIs are stateless — each call is independent with no server-side context retained.

To implement multi-turn conversations, the client must include conversation history in each request. How do I implement a multi-turn conversation?