All Products
Search

This Product

    Container Compute Service:inference-nv-pytorch 25.02

    Document Center

    Container Compute Service:inference-nv-pytorch 25.02

    Last Updated:Jun 20, 2026

    This document provides the release notes for inference-nv-pytorch 25.02.

    Main features and bug fixes

    Main features

    • vLLM is updated to v0.7.2.

    • Adds support for SGLang v0.4.3.post2.

    • Adds support for DeepSeek models.

    Bug fixes

    None for this release.

    Contents

    Use cases

    LLM inference

    Framework

    PyTorch

    Requirements

    NVIDIA Driver release >= 550

    System components

    • Ubuntu 22.04

    • Python 3.10

    • Torch 2.5.1

    • CUDA 12.4

    • transformers 4.48.3

    • triton 3.1.0

    • ray 2.42.1

    • vllm 0.7.2

    • sgl-kernel 0.0.3.post6

    • sglang 0.4.3.post2

    • flashinfer-python 0.2.1.post2

    • ACCL-N 2.23.4.11

    Assets

    Public image

    • egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-nv-pytorch:25.02-vllm0.7.2-sglang0.4.3.post2-pytorch2.5-cuda12.4-20250305-serverless

    VPC image

    acs-registry-vpc.{region-id}.cr.aliyuncs.com/egslingjun/{image:tag}
    Replace {region-id} with the region where your Alibaba Cloud Container Compute Service (ACS) is activated (for example, cn-beijing or cn-wulanchabu).
    Replace {image:tag} with the name and tag of the image.
    Note

    The inference-nv-pytorch:25.03-vllm0.8.2-pytorch2.6-cu124-20250328-serverless and inference-nv-pytorch:25.03-sglang0.4.4.post1-pytorch2.5-cu124-20250327-serverless images are compatible with ACS and Lingjun multi-tenant deployments only. It is not compatible with Lingjun single-tenant deployments.

    Note

    • The inference-nv-pytorch:25.02-vllm0.7.2-sglang0.4.3.post2-pytorch2.5-cuda12.4-20250305-serverless image is designed for ACS and multi-tenant Lingjun environments. It is not suitable for single-tenant Lingjun environments.

    • The inference-nv-pytorch:25.02-vllm0.7.2-sglang0.4.3.post2-pytorch2.5-cuda12.4-20250305 image is designed for single-tenant Lingjun scenarios.

    Driver requirements

    NVIDIA Driver release >= 550

    Quick start

    This example shows how to pull the inference-nv-pytorch image with Docker and test the inference service on the Qwen2.5-7B-Instruct model.

    Note

    To use the inference-nv-pytorch image in ACS, select it from the artifact center page on the console when you create a workload, or specify the image reference in a YAML file. For more information, see the following topics about building a DeepSeek model inference service by using ACS GPU compute power:

    1. Pull the inference container image.

      docker pull egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-nv-pytorch:[tag]

    2. Download the open-source model in ModelScope format.

      pip install modelscope
cd /mnt
modelscope download --model Qwen/Qwen2.5-7B-Instruct --local_dir ./Qwen2.5-7B-Instruct

    3. Run the following command to start and enter the container.

      docker run -it --rm --gpus all --network=host --privileged --init --ipc=host \
--ulimit memlock=-1 --ulimit stack=67108864  \
-v /mnt/:/mnt/ \
egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-nv-pytorch:[tag]

    4. Test the vLLM conversational inference feature.

      1. Start the server.

        python3 -m vllm.entrypoints.openai.api_server \
--model /mnt/Qwen2.5-7B-Instruct \
--trust-remote-code --disable-custom-all-reduce \
--tensor-parallel-size 1

      2. Run a test from the client.

        curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
    "model": "/mnt/Qwen2.5-7B-Instruct",  
    "messages": [
    {"role": "system", "content": "You are a friendly AI assistant."},
    {"role": "user", "content": "Please introduce deep learning."}
    ]}'

        For more information about using vLLM, see vLLM.

    Known issues