Container Compute Service:Deploy a distributed DeepSeek-R1 inference service with ACS GPU

Model splitting

DeepSeek-R1 has 671 billion parameters, exceeding the 96 GB memory of a single GPU. This guide splits the model across two GPU container instances with model parallelism (PP=2) and data parallelism (TP=8).

Model parallelism (PP=2) splits the model into two stages, each running on one GPU container instance. The first stage processes input and passes intermediate results to the second.

Data parallelism (TP=8) distributes computation within each stage across 8 GPUs, processing data in parallel and merging results.

Distributed deployment architecture

This solution deploys a distributed inference service for the full version of DeepSeek on ACS using vLLM and Ray. LWS manages the leader-worker deployment pattern, and Fluid provides distributed caching to accelerate model loading. vLLM runs on two ACS GPU pod instances with 8 GPUs each. Each pod acts as a Ray Group (a Ray head and a Ray worker) for model splitting. Different deployment architectures affect variables such as tensor-parallel-size and LWS_GROUP_SIZE in the YAML file.

Step 1: Prepare DeepSeek-R1 model files

Large language models require substantial disk space. Create a NAS or OSS volume for persistent model file storage. This guide uses OSS.

1. Run the following command to download the DeepSeek-R1 model from ModelScope.

   Note

   Ensure git-lfs is installed. If not, run yum install git-lfs or apt-get install git-lfs. Additional methods are in Installing Git Large File Storage.

   git lfs install
   GIT_LFS_SKIP_SMUDGE=1 git clone https://www.modelscope.cn/deepseek-ai/DeepSeek-R1.git
   cd DeepSeek-R1/
   git lfs pull

2. Create a directory in OSS and upload the model to it.

   Note

   To install and configure ossutil, follow Install ossutil.

   ossutil mkdir oss://<your-bucket-name>/models/DeepSeek-R1
   ossutil cp -r ./DeepSeek-R1 oss://<your-bucket-name>/models/DeepSeek-R1

3. After storing the model in OSS, you can load it in one of two ways.

   1. Mount the model directly by using a PVC and PV: Suitable for smaller models without strict startup or loading speed requirements.

      Console

      The following are the basic configuration details for the example PV:

      Parameter	Description
      Storage volume type	OSS
      Name	llm-model
      Access key	Configure the access key ID and access key secret for accessing OSS.
      Bucket ID	Select the OSS bucket you created in the previous step.
      OSS path	Select the path where the model is located, such as /models/DeepSeek-R1.

      The following are the basic configuration details for the example PVC:

      Parameter	Description
      Storage claim type	OSS
      Name	llm-model
      Allocation mode	Select Existing Volume.
      Existing volume	Click the Select Existing Volume link and select the PV you created.

      kubectl

      The following is an example YAML file:

      apiVersion: v1
      kind: Secret
      metadata:
        name: oss-secret
      stringData:
        akId: <your-oss-ak> # Your AccessKey ID for accessing OSS.
        akSecret: <your-oss-sk> # Your AccessKey Secret for accessing OSS.
      ---
      apiVersion: v1
      kind: PersistentVolume
      metadata:
        name: llm-model
        labels:
          alicloud-pvname: llm-model
      spec:
        capacity:
          storage: 30Gi 
        accessModes:
          - ReadOnlyMany
        persistentVolumeReclaimPolicy: Retain
        csi:
          driver: ossplugin.csi.alibabacloud.com
          volumeHandle: llm-model
          nodePublishSecretRef:
            name: oss-secret
            namespace: default
          volumeAttributes:
            bucket: <your-bucket-name> # The bucket name.
            url: <your-bucket-endpoint> # The endpoint, such as oss-cn-hangzhou-internal.aliyuncs.com.
            otherOpts: "-o umask=022 -o max_stat_cache_size=0 -o allow_other"
            path: <your-model-path> # In this example, the path is /models/DeepSeek-R1/.
      ---
      apiVersion: v1
      kind: PersistentVolumeClaim
      metadata:
        name: llm-model
      spec:
        accessModes:
          - ReadOnlyMany
        resources:
          requests:
            storage: 30Gi
        selector:
          matchLabels:
            alicloud-pvname: llm-model

   2. Accelerate model loading with Fluid: Recommended for larger models that require fast startup and loading. Use Fluid to accelerate data access.

      1. Install the ack-fluid component (version 1.0.11-* or later) from the ACS App Marketplace by using Helm. Create an application by using Helm.

      2. Enable privileged mode for the ACS pod. You can enable it by submitting a ticket.

      3. Create a Secret to access OSS.

         apiVersion: v1
         kind: Secret
         metadata:
           name: mysecret
         stringData:
           fs.oss.accessKeyId: xxx
           fs.oss.accessKeySecret: xxx

         Set fs.oss.accessKeyId and fs.oss.accessKeySecret to the credentials for the OSS bucket created above.

      4. Create a Dataset and a JindoRuntime.

         apiVersion: data.fluid.io/v1alpha1
         kind: Dataset
         metadata:
           name: deepseek
         spec:
           mounts:
             - mountPoint:  oss://<your-bucket-name>       # Replace with your actual bucket name.
               options:
                 fs.oss.endpoint: <your-bucket-endpoint>    # Replace with your actual bucket endpoint.
               name: deepseek
               path: "/"
               encryptOptions:
                 - name: fs.oss.accessKeyId
                   valueFrom:
                     secretKeyRef:
                       name: mysecret
                       key: fs.oss.accessKeyId
                 - name: fs.oss.accessKeySecret
                   valueFrom:
                     secretKeyRef:
                       name: mysecret
                       key: fs.oss.accessKeySecret
         ---
         apiVersion: data.fluid.io/v1alpha1
         kind: JindoRuntime
         metadata:
           name: deepseek
         spec:
           replicas: 16    # Adjust as needed.
           master:
             podMetadata:
               labels:
                 alibabacloud.com/compute-class: performance
                 alibabacloud.com/compute-qos: default
           worker:
             podMetadata:
               labels:
                 alibabacloud.com/compute-class: performance
                 alibabacloud.com/compute-qos: default
               annotations:
                 kubernetes.io/resource-type: serverless
             resources:
               requests:
                 cpu: 16
                 memory: 128Gi
               limits:
                 cpu: 16
                 memory: 128Gi
           tieredstore:
             levels:
               - mediumtype: MEM
                 path: /dev/shm
                 volumeType: emptyDir
                 ## Adjust as needed.
                 quota: 128Gi
                 high: "0.99"
                 low: "0.95"

         After creation, run the kubectl get pod | grep jindo command to check if the pods are in the Running state. Expected output:

         deepseek-jindofs-master-0    1/1     Running   0          3m29s
         deepseek-jindofs-worker-0    1/1     Running   0          2m52s
         deepseek-jindofs-worker-1    1/1     Running   0          2m52s
         ...

      5. Cache the model by creating a DataLoad resource.

         apiVersion: data.fluid.io/v1alpha1
         kind: DataLoad
         metadata:
           name: deepseek
         spec:
           dataset:
             name: deepseek
             namespace: default
           loadMetadata: true

      6. Run the following command to check the cache status.

         kubectl get dataload

         Expected output:

         NAME       DATASET    PHASE       AGE     DURATION
         deepseek   deepseek   Executing   4m30s   Unfinished

         PHASE shows Executing while caching is in progress. Wait about 20 minutes and re-run the command. Complete indicates success. Track progress with kubectl logs $(kubectl get pods --selector=job-name=deepseek-loader-job -o jsonpath='{.items[0].metadata.name}') | grep progress.

         Fluid DataLoad resource parameters

         Parameter	Description	Example
         Name	The name of the data loading task.	deepseek
         Dataset	The name of the associated dataset.	deepseek
         Phase	The status of the data loading task, such as Complete.	Executing, Complete
         Age	The creation time of the data loading task.	4m30s
         Duration	The time taken by the data loading task.	Unfinished, 16m29s

      7. Run the following command to check the Dataset resource.

         kubectl get datasets

         Expected output:

         NAME       UFS TOTAL SIZE   CACHED    CACHE CAPACITY   CACHED PERCENTAGE   PHASE   AGE
         deepseek   1.25TiB          1.25TiB   2.00TiB          100.0%              Bound   21h

         Fluid Dataset resource parameters

         Parameter	Description	Example
         Name	The name of the dataset.	deepseek
         UFS total size	The total size of the dataset in the underlying storage.	1.25TiB
         Cached	The amount of data currently cached.	1.25TiB
         Cache capacity	The total capacity of the cache.	2.00TiB
         Cached %	The percentage of caching completed.	100.0%
         Phase	The status of the dataset, such as Bound.	Bound
         Age	The creation time of the dataset resource.	21h

Step 2: Deploy the model with ACS GPU

1. Install the lws component from the ACS App Marketplace by using Helm. Create an application by using Helm.

2. Deploy the model by using a LeaderWorkerSet.

   Note

   • In the YAML file, replace alibabacloud.com/gpu-model-series: <example-model> with a specific GPU model supported by ACS. For a list of currently supported GPU models, consult your PDSA or submit a ticket.

   • Compared to TCP/IP, high-performance RDMA networking uses zero-copy and kernel bypass to avoid data copying and frequent context switches, resulting in lower latency, higher throughput, and reduced CPU usage. ACS supports RDMA by configuring the label alibabacloud.com/hpn-type: "rdma" in the YAML file. For a list of GPU models that support RDMA, consult your PDSA or submit a ticket.

   • If you use Fluid to load the model, change the claimName in both PVC sections to the name of the Fluid dataset.

   • Different distributed deployment architectures affect the values of variables such as tensor-parallel-size and LWS_GROUP_SIZE in the YAML file.

   Standard deployment

   apiVersion: leaderworkerset.x-k8s.io/v1
   kind: LeaderWorkerSet
   metadata:
     name: deepseek-r1-671b-fp8-distrubution
   spec:
     replicas: 1
     leaderWorkerTemplate:
       size: 2 # Total number of leader and worker pods.
       restartPolicy: RecreateGroupOnPodRestart
       leaderTemplate:
         metadata:
           labels: 
             role: leader
             alibabacloud.com/compute-class: gpu  # Specify the GPU type.
             alibabacloud.com/compute-qos: default # Specify the ACS QoS level.
             alibabacloud.com/gpu-model-series: <example-model> ## Specify the GPU model.
         spec:
           volumes:
             - name: llm-model
               persistentVolumeClaim:
                 ## If using Fluid, enter the Fluid dataset name here, for example: deepseek
                 claimName: llm-model
             - name: shm
               emptyDir:
                 medium: Memory
                 sizeLimit: 32Gi
           containers:
             - name: deepseek-r1-671b-leader
               image: registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/vllm:v0.7.2
               env:
                 - name: NCCL_SOCKET_IFNAME # Specify the network interface card (NIC).
                   value: eth0
               command:
                 - sh
                 - -c
                 - "/vllm-workspace/ray_init.sh leader --ray_cluster_size=$(LWS_GROUP_SIZE);vllm serve /models/DeepSeek-R1/ --port 8000 --trust-remote-code --served-model-name ds --max-model-len 2048 --gpu-memory-utilization 0.95 --tensor-parallel-size 8 --pipeline-parallel-size 2 --enforce-eager"
   # Set tensor-parallel-size to the number of GPUs per pod.
               resources:
                 limits:
                   nvidia.com/gpu: "8"
                   cpu: "64"
                   memory: 512G
                 requests:
                   nvidia.com/gpu: "8"
                   cpu: "64"
                   memory: 512G           
               ports:
                 - containerPort: 8000
               volumeMounts:
                 - mountPath: /models/DeepSeek-R1
                   name: llm-model
                 - mountPath: /dev/shm
                   name: shm
       workerTemplate:
         metadata:
           labels: 
             alibabacloud.com/compute-class: gpu  # Specify the GPU type.
             alibabacloud.com/compute-qos: default # Specify the ACS QoS level.
             alibabacloud.com/gpu-model-series: <example-model> ## Specify the GPU model.
         spec:
           volumes:
             - name: llm-model
               persistentVolumeClaim:
                 ## If using Fluid, enter the Fluid dataset name here, for example: deepseek
                 claimName: llm-model
             - name: shm
               emptyDir:
                 medium: Memory
                 sizeLimit: 32Gi
           containers:
             - name: deepseek-r1-671b-worker
               image: registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/vllm:v0.7.2
               env:
                 - name: NCCL_SOCKET_IFNAME # Specify the network interface card (NIC).
                   value: eth0
               command:
                 - sh
                 - -c
                 - "/vllm-workspace/ray_init.sh worker --ray_address=$(LWS_LEADER_ADDRESS)"
               resources:
                 limits:
                   nvidia.com/gpu: "8"
                   cpu: "64"
                   memory: 512G
                 requests:
                   nvidia.com/gpu: "8"
                   cpu: "64"
                   memory: 512G
               ports:
                 - containerPort: 8000
               volumeMounts:
                 - mountPath: /models/DeepSeek-R1
                   name: llm-model
                 - mountPath: /dev/shm
                   name: shm

   RDMA acceleration

   If you use an open source base image (such as vLLM), add the following environment variables to the YAML file:

   Name	Value
   NCCL_SOCKET_IFNAME	eth0
   NCCL_IB_TC	136
   NCCL_IB_SL	5
   NCCL_IB_GID_INDEX	3
   NCCL_DEBUG	INFO
   NCCL_IB_HCA	mlx5
   NCCL_NET_PLUGIN	none

   apiVersion: leaderworkerset.x-k8s.io/v1
   kind: LeaderWorkerSet
   metadata:
     name: deepseek-r1-671b-fp8-distrubution
   spec:
     replicas: 1
     leaderWorkerTemplate:
       size: 2 # Total number of leader and worker pods.
       restartPolicy: RecreateGroupOnPodRestart
       leaderTemplate:
         metadata:
           labels: 
             role: leader
             alibabacloud.com/compute-class: gpu  # Specify the GPU type.
             alibabacloud.com/compute-qos: default # Specify the ACS QoS level.
             alibabacloud.com/gpu-model-series: <example-model> ## Specify the GPU model.
             # Specify that the application runs in a high-performance RDMA network. Submit a ticket for information on supported GPU models.
             alibabacloud.com/hpn-type: "rdma"
         spec:
           volumes:
             - name: llm-model
               persistentVolumeClaim:
                 ## If using Fluid, enter the Fluid dataset name here, for example: deepseek
                 claimName: llm-model
             - name: shm
               emptyDir:
                 medium: Memory
                 sizeLimit: 32Gi
           containers:
             - name: deepseek-r1-671b-leader
               image: registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/vllm:v0.7.2
               env:
                 - name: NCCL_SOCKET_IFNAME # Specify the network interface card (NIC).
                   value: eth0
                 - name: NCCL_IB_TC
                   value: "136"
                 - name: NCCL_IB_SL
                   value: "5"
                 - name: NCCL_IB_GID_INDEX
                   value: "3"
                 - name: NCCL_DEBUG
                   value: "INFO"
                 - name: NCCL_IB_HCA
                   value: "mlx5"
                 - name: NCCL_NET_PLUGIN
                   value: "none"                
               command:
                 - sh
                 - -c
                 - "/vllm-workspace/ray_init.sh leader --ray_cluster_size=$(LWS_GROUP_SIZE);vllm serve /models/DeepSeek-R1/ --port 8000 --trust-remote-code --served-model-name ds --max-model-len 2048 --gpu-memory-utilization 0.95 --tensor-parallel-size 8 --pipeline-parallel-size 2 --enforce-eager"
   # Set tensor-parallel-size to the number of GPUs per pod.
               resources:
                 limits:
                   nvidia.com/gpu: "8"
                   cpu: "64"
                   memory: 512G
                 requests:
                   nvidia.com/gpu: "8"
                   cpu: "64"
                   memory: 512G           
               ports:
                 - containerPort: 8000
               volumeMounts:
                 - mountPath: /models/DeepSeek-R1
                   name: llm-model
                 - mountPath: /dev/shm
                   name: shm
       workerTemplate:
         metadata:
           labels: 
             alibabacloud.com/compute-class: gpu  # Specify the GPU type.
             alibabacloud.com/compute-qos: default # Specify the ACS QoS level.
             alibabacloud.com/gpu-model-series: <example-model> ## Specify the GPU model.
             # Specify that the application runs in a high-performance RDMA network. Submit a ticket for information on supported GPU models.
             alibabacloud.com/hpn-type: "rdma"
         spec:
           volumes:
             - name: llm-model
               persistentVolumeClaim:
                 ## If using Fluid, enter the Fluid dataset name here, for example: deepseek
                 claimName: llm-model
             - name: shm
               emptyDir:
                 medium: Memory
                 sizeLimit: 32Gi
           containers:
             - name: deepseek-r1-671b-worker
               image: registry-cn-hangzhou.ack.aliyuncs.com/ack-demo/vllm:v0.7.2
               env:
                 - name: NCCL_SOCKET_IFNAME # Specify the network interface card (NIC).
                   value: eth0
                 - name: NCCL_IB_TC
                   value: "136"
                 - name: NCCL_IB_SL
                   value: "5"
                 - name: NCCL_IB_GID_INDEX
                   value: "3"
                 - name: NCCL_DEBUG
                   value: "INFO"
                 - name: NCCL_IB_HCA
                   value: "mlx5"
                 - name: NCCL_NET_PLUGIN
                   value: "none"      
               command:
                 - sh
                 - -c
                 - "/vllm-workspace/ray_init.sh worker --ray_address=$(LWS_LEADER_ADDRESS)"
               resources:
                 limits:
                   nvidia.com/gpu: "8"
                   cpu: "64"
                   memory: 512G
                 requests:
                   nvidia.com/gpu: "8"
                   cpu: "64"
                   memory: 512G
               ports:
                 - containerPort: 8000
               volumeMounts:
                 - mountPath: /models/DeepSeek-R1
                   name: llm-model
                 - mountPath: /dev/shm
                   name: shm

3. Expose the inference service with a Service resource.

   apiVersion: v1
   kind: Service
   metadata:
     name: ds-leader
   spec:
     ports:
       - name: http
         port: 8000
         protocol: TCP
         targetPort: 8000
     selector:
       leaderworkerset.sigs.k8s.io/name: deepseek-r1-671b-fp8-distrubution
       role: leader
     type: ClusterIP

Step 3: Verify the inference service

1. Forward a local port to the inference service.

   Note

   kubectl port-forward is for development and debugging only. For production networking, use Manage Ingresses.

   kubectl port-forward svc/ds-leader 8000:8000

   Expected output:

   Forwarding from 127.0.0.1:8000 -> 8000
   Forwarding from [::1]:8000 -> 8000

2. Send a model inference request.

   curl http://localhost:8000/v1/chat/completions \
     -H "Content-Type: application/json" \
     -d '{
       "model": "ds",
       "messages": [
         {
           "role": "system", 
           "content": "You are a friendly AI assistant."
         },
         {
           "role": "user",
           "content": "Introduce deep learning."
         }
       ],
       "max_tokens": 1024,
       "temperature": 0.7,
       "top_p": 0.9,
       "seed": 10
     }'

   Expected output:

   {"id":"chatcmpl-4bc78b66e2a4439f8362bd434a60be57","object":"chat.completion","created":1739501401,"model":"ds","choices":[{"index":0,"message":{"role":"assistant","reasoning_content":null,"content":"<think>\nOkay, the user wants me to introduce deep learning. I need to think about how to answer this well. First, I should clarify the basic definition of deep learning—it's a branch of machine learning, right? Then I should compare it to traditional machine learning methods to highlight its advantages, like automatic feature extraction. I'll probably need to mention neural networks, especially the structure of deep neural networks with multiple hidden layers.\n\nNext, I should talk about the core components of deep learning, like activation functions, loss functions, and optimizers. The user might not be familiar with these terms, so I'll need to briefly explain the role of each part. I can use examples like ReLU as an activation function, Adam as an optimizer, and cross-entropy loss.\n\nThen, I should cover application areas. Computer vision and natural language processing are common examples. I should provide some concrete examples like image recognition and machine translation to make it easier for the user to understand. I could also mention industry applications, like in healthcare and finance, as the user might be interested in these practical use cases.\n\nI should also mention popular frameworks like TensorFlow and PyTorch, which make it easier to implement deep learning. And the importance of hardware acceleration, like GPUs, which explains why deep learning has developed so rapidly.\n\nAdditionally, I should probably discuss the challenges and limitations of deep learning, such as its dependency on data, high demand for computational resources, and poor interpretability. This will give the user a well-rounded view of its pros and cons. I might also mention future development directions, like efficient training algorithms and research into interpretability.\n\nThe user likely wants a basic understanding of deep learning. They might have some technical background, but not a deep one. They probably want to quickly grasp the key points and applications, so the answer needs to be well-structured and highlight the main points without getting too deep into technical details, but it can't be too brief either. I need to balance technical accuracy with readability.\n\nI should avoid using too much jargon, or explain it briefly when I do, for terms like \"neural network\" or \"convolutional neural network.\" The user could be a student or someone new to the field, so I should use plain language. I'll also use examples to help them connect the concepts to real-world applications, which makes them easier to remember.\n\nI also need to be clear about the relationship between deep learning and machine learning, explaining that deep learning is a subfield of machine learning, but it's deeper and handles more complex problems. I might also mention the backpropagation algorithm as one of the key training techniques.\n\nFinally, I'll provide a summary, emphasizing the impact and potential of deep learning and its future directions, to give the user a comprehensive overview. I'll check if I've missed any important points, like common model architectures such as CNNs and RNNs, which might be worth a brief mention to show the diversity.\n\nSometimes users are interested in the principles, but it's important to keep it concise here. The focus should be on an overview rather than in-depth technical details. I'll ensure the answer flows logically, starting from the definition, then moving to core components, applications, frameworks, challenges, and future directions. That structure seems reasonable.\n</think>\n\nDeep learning is a subfield of machine learning that aims to simulate the learning mechanism of the human brain by building multi-layered neural networks (known as \"deep\" networks). It automatically learns complex features and patterns from large amounts of data and is widely used in fields like image recognition, speech processing, and natural language processing.\n\n### Key concepts\n1.  **Artificial Neural Networks (ANNs)**:\n    *   Composed of an input layer, multiple hidden layers, and an output layer, with each layer containing multiple neurons.\n    *   Processes information by simulating the activation and transmission of signals between neurons.\n\n2.  **Automatic Feature Extraction**:\n    *   Traditional machine learning relies on manually engineered features. In contrast, deep learning uses multi-layer networks to automatically extract abstract features from data (such as edges and shapes from pixels in an image).\n\n3.  **Core Components**:\n    *   **Activation Function** (e.g., ReLU, Sigmoid): Introduces non-linearity, enhancing the model's expressive power.\n    *   **Loss Function** (e.g., Cross-Entropy, Mean Squared Error): Measures the difference between the model's predictions and the actual values.\n    *   **Optimizer** (e.g., SGD, Adam): Optimizes the network's parameters through backpropagation to minimize the loss.\n\n---\n\n### Typical models\n- **Convolutional Neural Network (CNN)**：\n  Designed for images, it uses convolutional kernels to extract spatial features. Classic models include ResNet and VGG.\n- **Recurrent Neural Network (RNN)**：\n  Processes sequential data (text, speech) by introducing a memory mechanism. Improved versions include LSTM and GRU.\n- **Transformer**：\n  Based on a self-attention mechanism, it has significantly advanced natural language processing performance (e.g., BERT, GPT series).\n\n---\n\n### Use cases\n- **Computer Vision**: Face recognition, medical image analysis (e.g., detecting lesions in lung CT scans).\n- **Natural Language Processing**: Intelligent chatbots, document summarization, translation (e.g., DeepL).\n- **Speech Technology**: Voice assistants (e.g., Siri), real-time captioning.\n- **Reinforcement Learning**: Game AI (AlphaGo), robotics control.\n\n---\n\n### Advantages and challenges\n- **Advantages**:\n  - Automatically learns complex features, reducing the need for manual intervention.\n  - Far outperforms traditional methods when given large amounts of data and high computing power.\n- **Challenges**:\n  - Relies on massive amounts of labeled data (e.g., requires tens of thousands of labeled medical images).\n  - High model training costs (e.g., training GPT-3 cost over ten million USD).\n  - \"Black box\" nature leads to poor interpretability, limiting its application in high-risk fields like medicine.\n\n---\n\n### Tools and trends\n- **Mainstream Frameworks**: TensorFlow (industry-deployment friendly), PyTorch (preferred for research).\n- **Research Directions**:\n  - Lightweight models (e.g., MobileNet for mobile devices).\n  - Self-supervised learning (reducing dependency on labeled data).\n  - Enhanced interpretability (e.g., visualizing the model's decision-making basis).\n\nDeep learning is pushing the boundaries of artificial intelligence, from generative AI (like Stable Diffusion for image generation) to autonomous driving, and continues to transform the tech ecosystem. Future advancements will likely focus on reducing computational costs and improving efficiency and interpretability.","tool_calls":[]},"logprobs":null,"finish_reason":"stop","stop_reason":null}],"usage":{"prompt_tokens":17,"total_tokens":1131,"completion_tokens":1114,"prompt_tokens_details":null},"prompt_logprobs":null}

   Model parameters

   This section covers only the parameters used for verification. The full parameter reference is in the DeepSeek API documentation.

   Parameter	Description	Example
   model	The model to use.	ds
   messages	A list of messages in the conversation. role: The role of the message author. content: The content of the message.	-
   max_tokens	The maximum number of tokens per request. Valid values: An integer between 1 and 8192. If the max_tokens parameter is not specified, the default value is 4096.	1024
   temperature	Controls output randomness. Higher values (e.g., 1.0) increase randomness; lower values (e.g., 0.2) increase determinism. Range: 0 to 2.	0.7