Platform For AI:One-click deployment of DeepSeek-V3 and DeepSeek-R1 models

1. Choose an inference engine

2. Choose a model and resources

Your choice of model determines the required compute resources and deployment costs. DeepSeek models are available as "full-version" and "distilled" variants, which have vastly different resource requirements.

• Development and testing: Use a distilled model, such as DeepSeek-R1-Distill-Qwen-7B. These models have a smaller resource footprint, typically a single GPU with 24 GB of GPU memory, deploy quickly, and are cost-effective, making them ideal for rapid feature validation.

• Production environment: Evaluate based on a balance of performance and cost. The DeepSeek-R1-Distill-Qwen-32B model strikes a good balance between effectiveness and cost. If you require higher model performance, choose a full-version model. This requires multiple high-end GPUs, such as eight GPUs with 96 GB of GPU memory each, which significantly increases costs.

The following table lists the minimum configurations for different model versions and the maximum number of tokens supported by different instance types and inference engines.

API call

1. Get the service endpoint and token.

   1. On the Model Gallery > Job Management > Deployment Jobs page, click the name of the deployed service to go to the service details page.

   2. Click View Call Information to get the service endpoint and token.

      

2. Use a chat API call example.

   Replace <EAS_ENDPOINT> with the service endpoint and <EAS_TOKEN> with the service token.

   OpenAI SDK

   Note:

   • Append /v1 to the end of the endpoint.

   • BladeLLM-accelerated deployments do not support client.models.list(). As a workaround, set the model parameter to an empty string ("").

   SGLang/vLLM accelerated deployment

   from openai import OpenAI
   
   # 1. Configure the client
   # Replace <EAS_ENDPOINT> and <EAS_TOKEN> with the actual endpoint and token of your service.
   openai_api_key = "<EAS_TOKEN>"
   openai_api_base = "<EAS_ENDPOINT>/v1"
   
   client = OpenAI(
       api_key=openai_api_key,
       base_url=openai_api_base,
   )
   
   # 2. Get the model name 
   try:
       model = client.models.list().data[0].id
       print(model)
   except Exception as e:
       print(f"Failed to get the model list. Check your endpoint and token. Error: {e}")
       
   # 3. Construct and send the request
   stream = True
   
   chat_completion = client.chat.completions.create(
       messages=[
           {"role": "user", "content": "Hello, please introduce yourself."}
       ],
       model=model,
       max_tokens=2048,
       stream=stream,
   )
   
   if stream:
       for chunk in chat_completion:
           print(chunk.choices[0].delta.content, end="")
   else:
       result = chat_completion.choices[0].message.content
       print(result)

   BladeLLM accelerated deployment

   from openai import OpenAI
   
   ##### API Configuration #####
   # Replace <EAS_ENDPOINT> with the endpoint of the deployed service and <EAS_TOKEN> with the token of the deployed service.
   openai_api_key = "<EAS_TOKEN>"
   openai_api_base = "<EAS_ENDPOINT>/v1"
   
   client = OpenAI(
       api_key=openai_api_key,
       base_url=openai_api_base,
   )
   
   # BladeLLM accelerated deployment does not currently support using client.models.list() to get the model name. You can set the value of model to "" for compatibility.
   model=""
   stream = True
   
   chat_completion = client.chat.completions.create(
       messages=[
                 {"role": "user", "content": "Hello, please introduce yourself."}
       ],
        model=model,
        max_tokens=2048,
        stream=stream,
       )
   
   if stream:
       for chunk in chat_completion:
           print(chunk.choices[0].delta.content, end="")
   else:
       result = chat_completion.choices[0].message.content
       print(result)

   HTTP

   SGLang/vLLM accelerated deployment

   Replace <model_name> with the model name obtained from the model list API <EAS_ENDPOINT>/v1/models.

   curl

   curl -X POST \
       -H "Content-Type: application/json" \
       -H "Authorization: <EAS_TOKEN>" \
       -d '{
           "model": "<model_name>",
           "messages": [
           {
               "role": "user",
               "content": "hello!"
           }
           ]
       }' \
       <EAS_ENDPOINT>/v1/chat/completions
   

   Python

   import json
   import requests
   
   # Replace <EAS_ENDPOINT> with the endpoint of the deployed service and <EAS_TOKEN> with the token of the deployed service.
   EAS_ENDPOINT = "<EAS_ENDPOINT>"
   EAS_TOKEN = "<EAS_TOKEN>"
   
   url = f"{EAS_ENDPOINT}/v1/chat/completions"
   headers = {
       "Content-Type": "application/json",
       "Authorization": EAS_TOKEN,
   }
   
   # Replace <model_name> with the model name obtained from the model list API <EAS_ENDPOINT>/v1/models.
   model = "<model_name>"
   stream = True
   messages = [
       {"role": "user", "content": "Hello, please introduce yourself."},
   ]
   
   req = {
       "messages": messages,
       "stream": stream,
       "temperature": 0.6,
       "top_p": 0.5,
       "top_k": 10,
       "max_tokens": 300,
       "model": model,
   }
   response = requests.post(
       url,
       json=req,
       headers=headers,
       stream=stream,
   )
   
   if stream:
       for chunk in response.iter_lines(chunk_size=8192, decode_unicode=False):
           msg = chunk.decode("utf-8")
           if msg.startswith("data"):
               info = msg[6:]
               if info == "[DONE]":
                   break
               else:
                   resp = json.loads(info)
                   print(resp["choices"][0]["delta"]["content"], end="", flush=True)
   else:
       resp = json.loads(response.text)
       print(resp["choices"][0]["message"]["content"])
   

   BladeLLM accelerated deployment

   curl

   curl -X POST \
       -H "Content-Type: application/json" \
       -H "Authorization: <EAS_TOKEN>" \
       -d '{
           "messages": [
           {
               "role": "user",
               "content": "hello!"
           }
           ]
       }' \
       <EAS_ENDPOINT>/v1/chat/completions
   

   Python

   import json
   import requests
   
   # Replace <EAS_ENDPOINT> with the endpoint of the deployed service and <EAS_TOKEN> with the token of the deployed service.
   EAS_ENDPOINT = "<EAS_ENDPOINT>"
   EAS_TOKEN = "<EAS_TOKEN>"
   
   url = f"{EAS_ENDPOINT}/v1/chat/completions"
   headers = {
       "Content-Type": "application/json",
       "Authorization": EAS_TOKEN,
   }
   
   
   stream = True
   messages = [
       {"role": "user", "content": "Hello, please introduce yourself."},
   ]
   # When you use BladeLLM for accelerated deployment, if you do not specify the max_tokens parameter, the output is truncated to 16 tokens by default. We recommend that you adjust the max_tokens request parameter as needed.
   req = {
       "messages": messages,
       "stream": stream,
       "temperature": 0.6,
       "top_p": 0.5,
       "top_k": 10,
       "max_tokens": 300,
   }
   response = requests.post(
       url,
       json=req,
       headers=headers,
       stream=stream,
   )
   
   if stream:
       for chunk in response.iter_lines(chunk_size=8192, decode_unicode=False):
           msg = chunk.decode("utf-8")
           if msg.startswith("data"):
               info = msg[6:]
               if info == "[DONE]":
                   break
               else:
                   resp = json.loads(info)
                   if resp["choices"][0]["delta"].get("content") is not None:
                         print(resp["choices"][0]["delta"]["content"], end="", flush=True)
   else:
       resp = json.loads(response.text)
       print(resp["choices"][0]["message"]["content"])

3. Different models and deployment frameworks have distinct behaviors when using the inference service. Find more detailed instructions on API calls on the model's introduction page in Model Gallery.

Build a local WebUI

Gradio is a user-friendly Python library for quickly creating interactive interfaces for machine learning models. Follow these steps to run a Gradio WebUI locally.

1. Download the code: Download the appropriate code based on the inference engine you selected during deployment. Use the GitHub link if you have stable network access to GitHub. Otherwise, use the OSS link.

   • vLLM, SGLang: vLLM/SGLang_github, vLLM/SGLang_oss

   • BladeLLM: BladeLLM_github, BladeLLM_oss

2. Prepare the environment: Python 3.10 or later is required. Install the dependencies: pip install openai gradio.

3. Start the web application: Run the following command in your terminal. Replace <EAS_ENDPOINT> and <EAS_TOKEN> with your service's Endpoint and Token.

   python webui_client.py --eas_endpoint "<EAS_ENDPOINT>" --eas_token "<EAS_TOKEN>"

4. After the application starts successfully, a local URL (usually http://127.0.0.1:7860) is printed to your console. Open this URL in your browser to access the WebUI.

Integrate third-party applications

To connect to Chatbox, Dify, or Cherry Studio, see Integrate third-party clients.

Cost breakdown

Services deployed using Public Resources are billed by the minute from creation (when the status becomes "Running") until they are stopped or deleted, with bills settled hourly. Billing continues even if the service is idle. Stopping the service stops the billing.

For more information, see Billing of Elastic Algorithm Service (EAS).

Cost control recommendations

• Clean up promptly: After development and testing, Stop or Delete the service immediately to control costs.

• Select an appropriate model: In non-production environments, prioritize using lower-cost distilled models.

• Use preemptible resources: For non-production tasks, enable the preemptible mode during deployment. Note that certain conditions must be met for a successful bid, and there is a risk of resource instability.

• Leverage long-term use discounts: For long-running production services, reduce costs by purchasing a savings plan or prepaid resources.

Key risks

• Unexpected costs: Forgetting to stop a service results in continuous billing. Always clean up resources immediately after use.

• Output truncation with BladeLLM: When using the BladeLLM engine, if max_tokens is not specified in the API request, the output is truncated to 16 tokens, which can lead to unexpected behavior or functional failure.

• Incorrect API call specifications:

  • Including a system prompt in messages when calling DeepSeek-R1 series models may lead to unexpected behavior.

  • The API request URL must end with a path like /v1/chat/completions; otherwise, it returns a 404 error.

• Resource availability: When deploying LLMs, especially full-version ones, high-end GPU resources in certain regions may have limited availability, leading to deployment failures or long waits. Try switching to other regions.

The service is stuck in a waiting state after I click deploy

Possible reasons include:

• Insufficient machine resources in the current region.

• The model is large, and loading it takes a long time. For LLMs like DeepSeek-R1 and DeepSeek-V3, this process can take 20-30 minutes.

You can wait and observe for a while. If the service still fails to start after an extended period, try the following steps:

1. Go to the Job Management > Deployment Jobs page to view the deployment task details. In the upper-right corner of the page, click More > More Information to navigate to the PAI-EAS model service details page and check the service instance status.

   

2. Stop the current service and switch to another region in the upper-left corner of the console to redeploy it.

   Note

   Ultra-large-parameter models like DeepSeek-R1 and DeepSeek-V3 require an 8-GPU setup to start the service, and resource availability is limited. Consider deploying smaller distilled models like DeepSeek-R1-Distill-Qwen-7B, for which resources are more readily available.

Why do I get a 404 error when calling the API?

Check if the OpenAI API suffix, such as v1/chat/completions, has been added to the URL you are calling. For details, refer to the API call instructions on the model's homepage.

If you use a vLLM accelerated deployment, check whether the model parameter in the request body of the chat API call is filled with the correct model name. You can get the model name by calling the /v1/models endpoint.

The request is too long and causes a gateway timeout

The default request timeout for the gateway used during deployment is 180 seconds. If you need to extend this timeout, configure a Dedicated Gateway and submit a ticket to adjust the request timeout for the Dedicated Gateway. The maximum adjustable timeout is 600 seconds.

Why is there no "web search" feature?

The "web search" feature cannot be implemented by only deploying a model service. You must build an AI application (Agent) based on the model service.

What should I do if the model skips the thinking process?

If you deploy the DeepSeek-R1 model and encounter situations where the model sometimes skips its thinking process, use the updated chat template from DeepSeek that forces thinking. To use it:

1. Modify the startup command.

   As shown in the figure, edit the service configuration by editing the JSON Configuration. Modify the containers-script field to add "--chat-template /model_dir/template_force_thinking.jinja" (you can add it after "--served-model-name DeepSeek-R1").

   If the service is already deployed, go to Model Gallery > Job Management > Deployment Jobs, click the name of the deployed service, and then click Update service in the upper-right corner of the details page to access the same configuration page.

   

2. Modify the request body. In each request, add {"role": "assistant", "content": "<think>\n"} at the end of the messages array.

Can I disable the thinking mode for DeepSeek-R1?

The DeepSeek-R1 series models do not currently support disabling the thinking process.

How do I implement a multi-turn conversation?

The model service itself does not save conversation history. The client application is responsible for storing the history and including it in subsequent requests to the model. The following is an example for a service deployed with SGLang.

curl -X POST \
    -H "Content-Type: application/json" \
    -H "Authorization: <EAS_TOKEN>" \
    -d '{
        "model": "<model_name>",
        "messages": [
         {
            "role": "user", 
            "content": "Hello"
        },
        {
            "role": "assistant",
            "content": "Hello! I'm glad to see you. What can I help you with?"
        },
        {
            "role": "user",
            "content": "What was my previous question?"
        }
        ]
    }' \
    <EAS_ENDPOINT>/v1/chat/completions