Container Compute Service:Deploy a DeepSeek-R1 model inference service with ACS GPU

ACS provides serverless GPU compute with pay-as-you-go billing, eliminating GPU node management. This guide covers deploying a production-ready DeepSeek-R1 full model inference service on ACS with vLLM and RDMA acceleration.

Background

DeepSeek-R1 model

DeepSeek-R1 is DeepSeek's first-generation reasoning model, trained with large-scale reinforcement learning. It not only outperforms other closed-source models but also matches or exceeds OpenAI-O1 in mathematical reasoning, programming, and other benchmarks. DeepSeek-R1 also performs well in knowledge-based tasks and other task types, including creative writing and general Q&A. Explore the DeepSeek AI GitHub repository for details.

vLLM

vLLM is an efficient, easy-to-use LLM inference framework supporting popular models including Qwen. It accelerates inference through PagedAttention, continuous batching, and model quantization. Explore the vLLM GitHub repository for details.

ACS

ACS is a serverless, Kubernetes-based container service delivering general-purpose and GPU compute without node or cluster management. Integrated scheduling, container runtime, storage, and networking reduce operational complexity, while pay-as-you-go billing and elastic scaling lower costs. For LLM inference, ACS data and image acceleration further optimize model startup time and resource costs.

Prerequisites

Complete the following before you start:

1. Create an ACS cluster with the default service role assigned. See Create an ACS cluster.

2. Configure kubectl to connect to your ACS cluster. See Connect to a cluster by using kubectl.

GPU instance specifications and cost estimation

Deploying the DeepSeek-R1 full model on ACS requires 16 GPU hours without acceleration. Recommended single-instance configuration:

For information about selecting instance specifications, see GPU. For billing information, see Billing overview.

Step 1: Prepare the DeepSeek-R1-GPTQ-INT8 model files

LLMs need substantial disk space. Use NAS or OSS storage volumes for persistent model storage. The following steps use OSS as an example.

Note: Submit a ticket to obtain the model files and YAML deployment configuration, including:

• Model files: DeepSeek-R1-GPTQ-INT8

• GPU model: Replace the label alibabacloud.com/gpu-model-series with the actual GPU model supported by ACS

• Base image: Replace the container image with the actual image address

• Image pull secret: Create a Secret and replace the imagePullSecrets name with the actual Secret name

(Optional) Upload the model to OSS

If you downloaded the model files locally, create a directory in OSS and upload the model:

ossutil mkdir oss://<your-bucket-name>/models/DeepSeek-R1-GPTQ-INT8
ossutil cp -r /mnt/models/DeepSeek-R1-GPTQ-INT8 oss://<your-bucket-name>/models/DeepSeek-R1-GPTQ-INT8

Install ossutil.

Step 2: Deploy the model using ACS GPU

Deploy the DeepSeek-R1-GPTQ-INT8 inference service using vLLM with RDMA acceleration. The service exposes an OpenAI-compatible API.

Note:

• The max-model-len parameter controls the maximum token length. Higher values improve conversation quality but require more GPU memory. For DeepSeek-R1-GPTQ-INT8, the recommended maximum is around 128,000 tokens.

• RDMA (Remote Direct Memory Access) provides zero-copy, kernel-bypass networking with lower latency, higher throughput, and lower CPU usage compared to TCP/IP. Add the label alibabacloud.com/hpn-type: rdma to enable it. Submit a ticket to confirm GPU model support for RDMA.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: deepseek-r1
  namespace: default
  labels:
    app: deepseek-r1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: deepseek-r1
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
  template:
    metadata:
      labels:
        app: deepseek-r1
        alibabacloud.com/compute-class: gpu
        alibabacloud.com/gpu-model-series: <example-model>
        alibabacloud.com/hpn-type: "rdma"
    spec:
      imagePullSecrets:
        - name: <your-secret-name>
      containers:
        - name: llm-ds-r1
          image: <your-image-address>
          imagePullPolicy: IfNotPresent
          command:
            - sh
            - -c
            - "vllm serve /data/DeepSeek-R1-GPTQ-INT8 --port 8000 --trust-remote-code --served-model-name ds --max-model-len 128000 --quantization moe_wna16 --gpu-memory-utilization 0.98 --tensor-parallel-size 16"
          resources:
            limits:
              alibabacloud.com/gpu: "16"
              cpu: "64"
              memory: 512Gi
            requests:
              alibabacloud.com/gpu: "16"
              cpu: "64"
              memory: 512Gi
          volumeMounts:
            - name: llm-model
              mountPath: /data/DeepSeek-R1-GPTQ-INT8
            - name: shm
              mountPath: /dev/shm
      restartPolicy: Always
      terminationGracePeriodSeconds: 30
      volumes:
        - name: llm-model
          persistentVolumeClaim:
            claimName: llm-model
        - name: shm
          emptyDir:
            medium: Memory
            sizeLimit: 32Gi
---
apiVersion: v1
kind: Service
metadata:
  name: deepseek-r1
spec:
  type: ClusterIP
  selector:
    app: deepseek-r1
  ports:
    - protocol: TCP
      port: 8000
      targetPort: 8000

Apply the configuration:

kubectl apply -f deepseek-r1-deployment.yaml

Step 3: Verify the inference service

1. Set up port forwarding to the inference service:

   kubectl port-forward svc/deepseek-r1 8000:8000

   Expected output:

   Forwarding from 127.0.0.1:8000 -> 8000
   Forwarding from [::1]:8000 -> 8000

   Important: kubectl port forwarding suits development and debugging only. It does not provide production-level reliability, security, or scalability. For production environments, see Get started with ALB Ingress.

2. Send a test inference request:

   curl http://localhost:8000/v1/chat/completions \
     -H "Content-Type: application/json" \
     -d '{
       "model": "ds",
       "messages": [
         {
           "role": "user",
           "content": "Explain the concept of reinforcement learning in simple terms."
         }
       ],
       "max_tokens": 1024,
       "temperature": 0.7,
       "top_p": 0.9,
       "seed": 10
     }'

   A successful response confirms the service is running.

Result

You have deployed a DeepSeek-R1 full model inference service on ACS with GPU acceleration. The service exposes an OpenAI-compatible API ready for application integration.