Platform For AI:Deploy, fine-tune, and evaluate Qwen3 models in PAI

Deploy a model

This example deploys the Qwen3-235B-A22B model with SGLang.

1. Go to the Model Gallery page.

   1. Log in to the PAI console and select a region in the upper-left corner. Switch regions to find available compute resources.

   2. In the left-side navigation pane, choose Workspaces and click the name of your target workspace to open it.

   3. In the left-side navigation pane, choose QuickStart > Model Gallery.

2. In the model list on the right side of the Model Gallery page, click the Qwen3-235B-A22B model card to open the model details page.

3. In the upper-right corner, click Deploy. Configure the following parameters and keep the defaults for the others to deploy the model to the EAS inference service platform.

   • Deployment Method: Set Inference Engine to SGLang and Deployment Template to Single-Node.

   • Resource Information: For Resource Type, select public resources. The system recommends a suitable instance type. For the minimum configuration of each model, see Appendix: Computing resources and token limits.

     Important

     If no instance specifications are available, public resources in that region are out of stock. Consider the following options:

     • Switch regions. China (Ulanqab) provides more Lingjun preemptible resources (ml.gu7ef.8xlarge-gu100, ml.gu7xf.8xlarge-gu108, ml.gu8xf.8xlarge-gu108, ml.gu8tf.8.40xlarge). Preemptible resources can be reclaimed, so set your bid carefully.

     • Use an EAS resource group. Go to the EAS dedicated resources subscription page to purchase EAS dedicated resources.

Debug online

At the bottom of the Service details page, click online debugging. The following figure shows an example.

Invoke the API

1. Obtain the service endpoint and token.

   1. In Model Gallery > Job Management > Deployment Jobs, click the service name of the deployed service to open the service details page.

   2. Click View Call Information to obtain the Internet Endpoint and token.

      

2. Example call to the /v1/chat/completions chat API for a service deployed with SGLang.

   curl

   curl -X POST \
       -H "Content-Type: application/json" \
       -H "Authorization: <EAS_TOKEN>" \
       -d '{
           "model": "<model_name, obtained from the /v1/models API>",
           "messages": [
           {
               "role": "system",
               "content": "You are a helpful assistant."
           },
           {
               "role": "user",
               "content": "hello!"
           }
           ]
       }' \
       <EAS_ENDPOINT>/v1/chat/completions

   Python

   from openai import OpenAI
   
   ##### API configuration #####
   # Replace <EAS_ENDPOINT> with the service endpoint and <EAS_TOKEN> with the service token.
   openai_api_key = "<EAS_TOKEN>"
   openai_api_base = "<EAS_ENDPOINT>/v1"
   
   client = OpenAI(
       api_key=openai_api_key,
       base_url=openai_api_base,
   )
   
   models = client.models.list()
   model = models.data[0].id
   print(model)
   
   stream = True
   chat_completion = client.chat.completions.create(
       messages=[
           {"role": "user", "content": "Hello, could you please introduce yourself"}
       ],
       model=model,
       max_completion_tokens=2048,
       stream=stream,
   )
   
   if stream:
       for chunk in chat_completion:
           print(chunk.choices[0].delta.content, end="")
   else:
       result = chat_completion.choices[0].message.content
       print(result)

   Replace <EAS_ENDPOINT> with the service endpoint and <EAS_TOKEN> with the service token.

Invocation methods vary by deployment method. For more invocation methods, see Invoke an API for a deployed LLM service.

Third-party integration

To connect to Chatbox, Dify, or Cherry Studio, see Integrate with third-party clients.

Modify token limit

Qwen3 models natively support 32,768 tokens. RoPE scaling extends this to 131,072, though performance may degrade slightly. Modify the containers.script field in the service configuration JSON:

• vLLM:

  vllm serve ... --rope-scaling '{"rope_type":"yarn","factor":4.0,"original_max_position_embeddings":32768}' --max-model-len 131072

• SGLang:

  python -m sglang.launch_server ... --json-model-override-args '{"rope_scaling":{"rope_type":"yarn","factor":4.0,"original_max_position_embeddings":32768}}'

Parse tool calls

vLLM and SGLang can parse the tool calling content generated by the model into structured messages. Modify the containers.script field in the service configuration JSON:

• vLLM:

  vllm serve ... --enable-auto-tool-choice --tool-call-parser hermes

• SGLang:

  python -m sglang.launch_server ... --tool-call-parser qwen25

Control thinking mode

Qwen3 uses thinking mode by default. A hard switch disables thinking entirely; a soft switch lets the model follow your instructions about whether to think.

{
  "model": "<MODEL_NAME>",
  "messages": [
    {
      "role": "user",
      "content": "/no_think Hello!"
    }
  ],
  "max_tokens": 1024
}

Parse thinking content

To output the think content separately, modify the containers.script field in the service configuration JSON:

• vLLM:

  vllm serve ... --enable-reasoning --reasoning-parser qwen3

• SGLang:

  python -m sglang.launch_server ... --reasoning-parser deepseek-r1

Q: Do model services deployed on PAI support sessions that retain context across multiple requests?

No. The model service APIs deployed on PAI are stateless. Each call is fully independent, and the server retains no context or session state between requests.

To implement multi-turn conversations, the client must store the conversation history and attach it to each request. For a request example, see How do I implement a multi-turn conversation?