:One-click deployment of DeepSeek-V3 and DeepSeek-R1

Use WebUI

Only Transformers standard deployment method supports Web application. Accelerated deployment methods provides WebUI code instead. You can download the code and start the Web application locally.

Online debugging

1. Go to Model Gallery > Job Management > Deployment jobs.

2. Click the name of the deployed service.

3. In the Online Debugging section, find the entry for EAS online debugging.

4. Take SGLang deployment as an example, initiate a POST request using <EAS_ENDPOINT>/v1/chat/completions.

   1. Supplement the request path. The prefilled path is <EAS_ENDPOINT>. Append v1/chat/completions to the end of it.

   2. Construct the request body.

      Suppose your prompt is: What is 3+5?

      For the conversation interface, the format of your request body should include the model parameter, which is the model name retrieved from the Model list interface at <EAS_ENDPOINT>/v1/models. Take DeepSeek-R1-Distill-Qwen-7B as an example:

      {
          "model": "DeepSeek-R1-Distill-Qwen-7B",
          "messages": [
              {
                  "role": "system",
                  "content": "You are a helpful assistant."
              },
              {
                  "role": "user",
                  "content": "What is 3+5?"
              }
          ]
      }

   3. Initiate the request.

      

Here are the request paths and corresponding data samples for other deployment methods:

{
    "model": "<model_name>",
    "messages": [
        {
            "role": "system",
            "content": "You are a helpful assistant."
        },
        {
            "role": "user",
            "content": "What is 3+5?"
        }
    ]
}

{
    "messages": [
        {
            "role": "system",
            "content": "You are a helpful assistant."
        },
        {
            "role": "user",
            "content": "What is 3+5?"
        }
    ],
    "max_tokens": 2000
}

Use API

1. Obtain the endpoint and token of the service.

   1. Go to Model Gallery > Job Management > Deployment Jobs, click the name of the deployed service.

   2. Click View Call Information to obtain the endpoint and token.

      

2. Call examples of chat interface

   Replace <EAS_ENDPOINT> with the endpoint and <EAS_TOKEN> with the token.

   OpenAI SDK

   Note:

   • Add /v1 at the end of the endpoint.

   • BladeLLM and Transformers standard deployment methods do not support using client.models.list() to obtain the model list. You can directly specify the model value as "" for compatibility.

   SGLang or vLLM

   from openai import OpenAI
   
   ##### API Configuration #####
   # <EAS_ENDPOINT> needs to be replaced with the endpoint of the deployed service, and <EAS_TOKEN> needs to be replaced with the token of the deployed service.
   openai_api_key = "<EAS_TOKEN>"
   openai_api_base = "<EAS_ENDPOINT>/v1"
   
   client = OpenAI(
       api_key=openai_api_key,
       base_url=openai_api_base,
   )
   
   models = client.models.list()
   model = models.data[0].id
   print(model)
   
   stream = True
   chat_completion = client.chat.completions.create(
       messages=[
           {"role": "user", "content": "Hello, please introduce yourself."}
       ],
       model=model,
       max_completion_tokens=2048,
       stream=stream,
   )
   
   if stream:
       for chunk in chat_completion:
           print(chunk.choices[0].delta.content, end="")
   else:
       result = chat_completion.choices[0].message.content
       print(result)

   BladeLLM or Transformers standard deployment

   from openai import OpenAI
   
   ##### API Configuration #####
   # <EAS_ENDPOINT> needs to be replaced with the endpoint of the deployed service, and <EAS_TOKEN> needs to be replaced with the token of the deployed service.
   openai_api_key = "<EAS_TOKEN>"
   openai_api_base = "<EAS_ENDPOINT>/v1"
   
   client = OpenAI(
       api_key=openai_api_key,
       base_url=openai_api_base,
   )
   
   # BladeLLM accelerated deployment and Transformers standard deployment currently do not support using client.models.list() to obtain the model name. You can directly specify the model value as "" for compatibility.
   model=""
   stream = True
   
   chat_completion = client.chat.completions.create(
       messages=[
                 {"role": "user", "content": "Hello, please introduce yourself."}
       ],
        model=model,
        max_completion_tokens=2048,
        stream=stream,
       )
   
   if stream:
       for chunk in chat_completion:
           print(chunk.choices[0].delta.content, end="")
   else:
       result = chat_completion.choices[0].message.content
       print(result)

   HTTP

   SGLang/vLLM Accelerated Deployment

   In the example below, <model_name> should be replaced with the model name obtained from the Model List Interface <EAS_ENDPOINT>/v1/models.

   curl

   curl -X POST \
       -H "Content-Type: application/json" \
       -H "Authorization: <EAS_TOKEN>" \
       -d '{
           "model": "<model_name>",
           "messages": [
           {
               "role": "system",
               "content": "You are a helpful assistant."
           },
           {
               "role": "user",
               "content": "hello!"
           }
           ]
       }' \
       <EAS_ENDPOINT>/v1/chat/completions
   

   Python

   import json
   import requests
   
   # <EAS_ENDPOINT> needs to be replaced with the endpoint of the deployed service, and <EAS_TOKEN> needs to be replaced with the token of the deployed service.
   EAS_ENDPOINT = "<EAS_ENDPOINT>"
   EAS_TOKEN = "<EAS_TOKEN>"
   
   url = f"{EAS_ENDPOINT}/v1/chat/completions"
   headers = {
       "Content-Type": "application/json",
       "Authorization": EAS_TOKEN,
   }
   
   # <model_name> should be replaced with the model name obtained from the model list interface <EAS_ENDPOINT>/v1/models.
   model = "<model_name>"
   stream = True
   messages = [
       {"role": "system", "content": "You are a helpful assistant."},
       {"role": "user", "content": "Hello, please introduce yourself."},
   ]
   # When using the BladeLLM accelerated deployment method, if the max_tokens parameter is not specified, it will be truncated by default with max_tokens=16. It is recommended to adjust the request parameter max_tokens according to actual needs.
   req = {
       "messages": messages,
       "stream": stream,
       "temperature": 0.0,
       "top_p": 0.5,
       "top_k": 10,
       "max_tokens": 300,
       "model": model,
   }
   response = requests.post(
       url,
       json=req,
       headers=headers,
       stream=stream,
   )
   
   if stream:
       for chunk in response.iter_lines(chunk_size=8192, decode_unicode=False):
           msg = chunk.decode("utf-8")
           if msg.startswith("data"):
               info = msg[6:]
               if info == "[DONE]":
                   break
               else:
                   resp = json.loads(info)
                   print(resp["choices"][0]["delta"]["content"], end="", flush=True)
   else:
       resp = json.loads(response.text)
       print(resp["choices"][0]["message"]["content"])
   

   BladeLLM accelerated deployment/Transformers standard deployment

   curl

   curl -X POST \
       -H "Content-Type: application/json" \
       -H "Authorization: <EAS_TOKEN>" \
       -d '{
           "messages": [
           {
               "role": "system",
               "content": "You are a helpful assistant."
           },
           {
               "role": "user",
               "content": "hello!"
           }
           ]
       }' \
       <EAS_ENDPOINT>/v1/chat/completions
   

   Python

   import json
   import requests
   
   # <EAS_ENDPOINT> needs to be replaced with the endpoint of the deployed service, and <EAS_TOKEN> needs to be replaced with the token of the deployed service.
   EAS_ENDPOINT = "<EAS_ENDPOINT>"
   EAS_TOKEN = "<EAS_TOKEN>"
   
   url = f"{EAS_ENDPOINT}/v1/chat/completions"
   headers = {
       "Content-Type": "application/json",
       "Authorization": EAS_TOKEN,
   }
   
   
   stream = True
   messages = [
       {"role": "system", "content": "You are a helpful assistant."},
       {"role": "user", "content": "Hello, please introduce yourself."},
   ]
   # When using the BladeLLM accelerated deployment method, if the max_tokens parameter is not specified, it will be truncated by default with max_tokens=16. It is recommended to adjust the request parameter max_tokens according to actual needs.
   req = {
       "messages": messages,
       "stream": stream,
       "temperature": 0.2,
       "top_p": 0.5,
       "top_k": 10,
       "max_tokens": 300,
   }
   response = requests.post(
       url,
       json=req,
       headers=headers,
       stream=stream,
   )
   
   if stream:
       for chunk in response.iter_lines(chunk_size=8192, decode_unicode=False):
           msg = chunk.decode("utf-8")
           if msg.startswith("data"):
               info = msg[6:]
               if info == "[DONE]":
                   break
               else:
                   resp = json.loads(info)
                   if(resp["choices"][0]["delta"].get("content") is not None):
                         print(resp["choices"][0]["delta"]["content"], end="", flush=True)
   else:
       resp = json.loads(response.text)
       print(resp["choices"][0]["message"]["content"])
   

3. Different models and deployment frameworks may be different when using inference services. For more detailed API reference, refer to the model details page in the Model Gallery.

   

Full-version models

The following table shows the minimum specification required and the maximum token count supported by different methods across machine types.

Distill models

The service waits for a long time after I click Deploy

Possible causes:

• Insufficient resources in the current region.

• The model requires a long time to load due to its size (for models like DeepSeek-R1 and DeepSeek-V3, it can take 20-30 minutes).

If the service still won't start after a long wait, consider the following steps:

1. Go to Job Management > Deployment Jobs.

   1. Click the name of the job to go to the details page.

   2. Click More > More Infor in the upper-right corner.

   3. Check the Instance Status in the Service Instance section.

   

2. Terminate the current service. Switch to another region in upper-left corner of the page and deploy again.

   Note

   Models with large parameter sizes, such as DeepSeek-R1 and DeepSeek-V3, requires at least 8 GPU cores, and the resource inventory is limited. Consider smaller distill models such as DeepSeek-R1-Distill-Qwen-7B.

API request returns 404

Check whether the URL includes the OpenAI API suffix, such as v1/chat/completions. Refer to the overview page of the model for details.

The request is too long and causes EAS gateway timeout

The default timeout of EAS gateway is 180 seconds. To extend the timeout, configure an EAS dedicated gateway and submit a ticket to adjust the dedicated gateway's timeout, to up to 600 seconds.

How to debug the model online

See How do I debug a deployed model service online?

Why is there no "web search"

The web search feature requires building an AI agent, not just deploying the model service.

You can use PAI's LangStudio to build such an AI agent, see Chat With Web Search.

What to do if the model skips thinking

If the deployed DeepSeek-R1 model occasionally skips the thinking process, use the force thinking chat template updated by DeepSeek:

1. Modify the startup command.

   In the Service Configuration section, edit the JSON script and add --chat-template /model_dir/template_force_thinking.jinja to the containers-script field. It can be appended after --served-model-name DeepSeek-R1.

   If the service is already deployed, click the service name in Model Gallery > Job Management > Deployment jobs. Click Update service in the upper-right corner.

   

2. Modify the request body. Append {"role": "assistant", "content": "<think>\n"} to the end of each request message.

How to connect to Chatbox or Dify

Take DeepSeek-R1-Distill-Qwen-7B as an example. SGLang or vLLM accelerated deployment is recommended.

How to implement multi-round conversation?

The model service itself does not save conversation history. Instead, the client must save the conversation history and then add it to the request. Take SGLang as an example.

curl -X POST \
    -H "Content-Type: application/json" \
    -H "Authorization: ZWUyMzU5MmU2NGFiZWU0ZDRhNWVjMWNhMzI2NTM1ZDllMzZkYTAyYQ==" \
    -d '{
        "model": "<model_name>",
        "messages": [
        {
            "role": "system",
            "content": "You are a helpful assistant."
        },
         {
            "role": "user", 
            "content": "Hello"
        },
        {
            "role": "assistant",
            "content": "Hello! Nice to meet you. How can I help you?"
        },
        {
            "role": "user",
            "content": "What was my previous question?"
        }
        ]
    }' \
    <EAS_ENDPOINT>/v1/chat/completions