inference-nv-pytorch 25.09 - Container Compute Service - Alibaba Cloud Documentation Center

This topic describes the release notes for inference-nv-pytorch 25.09.

Main features and bug fixes

Main features

PyTorch is upgraded to 2.8.0.
vLLM is upgraded to v0.10.2.
SGLang is upgraded to v0.5.2.
deepgpu-comfyui is upgraded to 1.2.1, and the deepgpu-torch optimization component is upgraded to 0.1.1+torch2.8.0cu128.

Bug fixes

None

	inference-nv-pytorch	inference-nv-pytorch
Tag	25.09-vllm0.10.2-pytorch2.8-cu128-20250922-serverless	25.09-sglang0.5.2-pytorch2.8-cu128-20250917-serverless
Scenarios	Large model inference	Large model inference
Framework	pytorch	pytorch
Requirements	NVIDIA Driver release >= 570	NVIDIA Driver release >= 570
System components	Ubuntu 24.04 Python 3.12 Torch 2.8.0 CUDA 12.8 diffusers 0.35.1 deepgpu-comfyui 1.2.1 deepgpu-torch 0.1.1+torch2.8.0cu128 flash_attn 2.8.3 flashinfer-python 0.3.1 imageio 2.37.0 imageio-ffmpeg 0.6.0 ray 2.49.1 transformers 4.56.1 triton 3.4.0 vllm 0.10.2 xformers 0.0.32.post1 xfuser 0.4.4 xgrammar 0.1.23 ljperf 0.1.0+477686c5	Ubuntu 24.04 Python 3.12 Torch 2.8.0 CUDA 12.8 decord 0.6.0 diffusers 0.35.1 deepgpu-comfyui 1.2.1 deepgpu-torch 0.1.1+torch2.8.0cu128 flash_attn 2.8.3 flash_mla 1.0.0+261330b flashinfer-python 0.3.1 imageio 2.37.0 imageio-ffmpeg 0.6.0 transformers 4.56.1 sgl-kernel 0.3.9 sglang 0.5.2 xgrammar 0.1.24 triton 3.4.0 torchao 0.9.0 torchaudio 2.8.0 xfuser 0.4.4 ljperf 0.1.0+477686c5

Assets

Public image

egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-nv-pytorch:25.09-vllm0.10.2-pytorch2.8-cu128-20250922-serverless
egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-nv-pytorch:25.09-sglang0.5.2-pytorch2.8-cu128-20250917-serverless

VPC image

acs-registry-vpc.{region-id}.cr.aliyuncs.com/egslingjun/{image:tag}
{region-id} indicates the region where your ACS is activated, such as cn-beijing and cn-wulanchabu.
{image:tag} indicates the name and tag of the image.

Important

Currently, you can pull only images in the China (Beijing) region over a VPC.

Note

The inference-nv-pytorch:25.09-vllm0.10.2-pytorch2.8-cu128-20250922-serverless and inference-nv-pytorch:25.09-sglang0.5.2-pytorch2.8-cu128-20250917-serverless images apply to ACS products and Lingjun multi-tenant products. They do not apply to Lingjun single-tenant products.

Driver requirements

NVIDIA Driver release >= 570

Quick start

This example shows how to pull the inference-nv-pytorch image using Docker and test the inference service with the Qwen2.5-7B-Instruct model.

Note

To use the inference-nv-pytorch image in ACS, you can select it on the Artifacts page when you create a workload in the console, or specify the image reference in a YAML file. For more information about building model inference services using ACS GPU computing power, see the following topics:

Build a DeepSeek distilled model inference service with ACS GPU computing power
Build a full-featured DeepSeek model inference service with ACS GPU computing power
Build a distributed full-featured DeepSeek inference service with ACS GPU computing power
Accelerate Wan2.1 video generation with DeepGPU

Pull the inference container image.

docker pull egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-nv-pytorch:[tag]

Download the open source model in ModelScope format.

pip install modelscope
cd /mnt
modelscope download --model Qwen/Qwen2.5-7B-Instruct --local_dir ./Qwen2.5-7B-Instruct

Run the following command to enter the container.

docker run -d -t --network=host --privileged --init --ipc=host \
--ulimit memlock=-1 --ulimit stack=67108864  \
-v /mnt/:/mnt/ \
egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-nv-pytorch:[tag]

Test the vLLM conversational inference feature.

Start the server.

python3 -m vllm.entrypoints.openai.api_server \
--model /mnt/Qwen2.5-7B-Instruct \
--trust-remote-code --disable-custom-all-reduce \
--tensor-parallel-size 1

Test on the client.

curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
    "model": "/mnt/Qwen2.5-7B-Instruct",  
    "messages": [
    {"role": "system", "content": "You are a friendly AI assistant."},
    {"role": "user", "content": "Tell me about deep learning."}
    ]}'

For more information about how to use vLLM, see vLLM.

Known issues

The deepgpu-comfyui plugin, which accelerates Wanx model video generation, currently supports only GN8IS and G49E.

Container Compute Service:inference-nv-pytorch 25.09