Platform For AI:Quickstart: Deploy, fine-tune, and evaluate Qwen3 models

Deploy the model

This section shows how to deploy the Qwen3-235B-A22B model with SGLang.

1. Go to the Model Gallery page.

   1. Log on to the PAI console and select a region in the upper-left corner. You can switch regions to find one with sufficient computing resources.

   2. In the navigation pane on the left, click Workspace Management and click the name of the target workspace.

   3. In the left navigation pane, choose QuickStart > Model Gallery.

2. On the Model Gallery page, click the Qwen3-235B-A22B model card to open the model details page.

3. Click Deploy in the upper-right corner. Configure the following parameters and use the default values for the others to deploy the model to the Elastic Algorithm Service (EAS).

   • Deployment Method: Set Inference Engine to SGLang and Deployment Template to Single-Node.

   • Resource Information: Set Resource Type to Public Resources. The system automatically recommends an instance type. For the minimum required configuration, see Required computing power & supported token count.

   • Important

     If no instance types are available, it means the public resource inventory in the region is insufficient. Consider the following options:

     • Switch regions. For example, the China (Ulanqab) region has a larger inventory of Lingjun preemptible resources, such as ml.gu7ef.8xlarge-gu100, ml.gu7xf.8xlarge-gu108, ml.gu8xf.8xlarge-gu108, and ml.gu8tf.8.40xlarge. Because preemptible resources can be reclaimed, be mindful of your bid.

     • Use an EAS resource group. You can purchase dedicated EAS resources from EAS Dedicated Resources Subscription.

   

Debug online

On the Service Details page, click Online Debugging. An example is shown in the following figure.

Call the API

1. Obtain the service endpoint and token.

   1. In Model Gallery > Job Management > Deployment Jobs, click the name of the deployed service to open the service details page.

   2. Click View Invocation Method to view the Internet Endpoint and token.

      

2. The following example shows how to call the /v1/chat/completions endpoint for an SGLang deployment.

   curl

   curl -X POST \
       -H "Content-Type: application/json" \
       -H "Authorization: <EAS_TOKEN>" \
       -d '{
           "model": "<model_name, get from the /v1/models API>",
           "messages": [
           {
               "role": "system",
               "content": "You are a helpful assistant."
           },
           {
               "role": "user",
               "content": "hello!"
           }
           ]
       }' \
       <EAS_ENDPOINT>/v1/chat/completions

   Python

   from openai import OpenAI
   
   ##### API configuration #####
   # Replace <EAS_ENDPOINT> with the endpoint of the deployed service and <EAS_TOKEN> with the token of the service.
   openai_api_key = "<EAS_TOKEN>"
   openai_api_base = "<EAS_ENDPOINT>/v1"
   
   client = OpenAI(
       api_key=openai_api_key,
       base_url=openai_api_base,
   )
   
   models = client.models.list()
   model = models.data[0].id
   print(model)
   
   stream = True
   chat_completion = client.chat.completions.create(
       messages=[
           {"role": "user", "content": "Hello, please introduce yourself."}
       ],
       model=model,
       max_completion_tokens=2048,
       stream=stream,
   )
   
   if stream:
       for chunk in chat_completion:
           print(chunk.choices[0].delta.content, end="")
   else:
       result = chat_completion.choices[0].message.content
       print(result)

   Replace <EAS_ENDPOINT> with your service endpoint and <EAS_TOKEN> with your service token.

The invocation method varies by deployment type. For more examples, see Deploy large language models and call APIs.

Integrate third-party applications

To connect to Chatbox, Dify, or Cherry Studio, see Integrate third-party clients.

Modify the token limit

Qwen3 models natively support a token length of 32,768. You can use RoPE scaling technology to extend this to a maximum of 131,072 tokens, though this might cause slight performance degradation. To do this, modify the containers.script field in the service configuration JSON file as follows:

• vLLM:

  vllm serve ... --rope-scaling '{"rope_type":"yarn","factor":4.0,"original_max_position_embeddings":32768}' --max-model-len 131072

• SGLang:

  python -m sglang.launch_server ... --json-model-override-args '{"rope_scaling":{"rope_type":"yarn","factor":4.0,"original_max_position_embeddings":32768}}'

Parse tool calls

vLLM and SGLang support parsing the model's tool calling output into a structured message. To enable this, modify the containers.script field in the service configuration JSON file as follows:

• vLLM:

  vllm serve ... --enable-auto-tool-choice --tool-call-parser hermes

• SGLang:

  python -m sglang.launch_server ... --tool-call-parser qwen25

Control the thinking mode

Qwen3 uses a thinking mode by default. You can control this behavior with a hard switch (to completely disable thinking) or a soft switch (where the model follows the user's instruction on whether to think).

{
  "model": "<MODEL_NAME>",
  "messages": [
    {
      "role": "user",
      "content": "/no_think Hello!"
    }
  ],
  "max_tokens": 1024
}

Parse thinking content

To output the thinking part separately, modify the containers.script field in the service configuration JSON file as follows:

• vLLM:

  vllm serve ... --enable-reasoning --reasoning-parser qwen3

• SGLang:

  python -m sglang.launch_server ... --reasoning-parser deepseek-r1

Q: How can I maintain conversation context across multiple API calls with a model deployed on PAI?

No, model services deployed on PAI are stateless. Each API call is independent, and the server does not retain context between requests.

To implement a multi-turn conversation, you must manage the conversation history on the client side. In each new API call, you need to pass the entire conversation history in the messages payload. For an example, see How do I implement a multi-turn conversation?