inference-nv-pytorch 25.11 - Container Compute Service - Alibaba Cloud Documentation Center

These are the release notes for the inference-nv-pytorch 25.11 container images.

Key features and bug fixes

Key features

This release includes container images for both CUDA 12.8 and CUDA 13.0.
- The CUDA 12.8 image supports only the amd64 architecture.
- The CUDA 13.0 image supports both the amd64 and aarch64 architectures.
PyTorch is upgraded to 2.9.0.
For the CUDA 12.8 image, deepgpu-comfyui is upgraded to 1.3.2 and the deepgpu-torch optimization component is upgraded to 0.1.12+torch2.9.0cu128.
vLLM is upgraded to v0.11.2 for the CUDA 13.0 (amd64) image, and SGLang is upgraded to v0.5.5.post3.

Bug fixes

No bug fixes are included in this release.

Image name	inference-nv-pytorch
Image tag	25.11-vllm0.11.1-pytorch2.9-cu128-20251120-serverless	25.11-sglang0.5.5.post3-pytorch2.9-cu128-20251121-serverless	25.11-vllm0.11.1-pytorch2.9-cu130-20251120-serverless		25.11-sglang0.5.5.post3-pytorch2.9-cu130-20251121-serverless
Architecture	amd64	amd64	amd64	aarch64	amd64	aarch64
Use case	Large model inference	Large model inference	Large model inference	Large model inference	Large model inference	Large model inference
Framework	pytorch	pytorch	pytorch	pytorch	pytorch	pytorch
Requirements	NVIDIA driver release >= 570	NVIDIA driver release >= 570	NVIDIA driver release >= 580	NVIDIA driver release >= 580	NVIDIA driver release >= 580	NVIDIA driver release >= 580
System components	Ubuntu 24.04 Python 3.12 Torch 2.9.0+cu128 CUDA 12.8 diffusers 0.35.2 deepgpu-comfyui 1.3.2 deepgpu-torch 0.1.12+torch2.9.0cu128 flash_attn 2.8.3 flashinfer-python 0.5.2 imageio 2.37.2 imageio-ffmpeg 0.6.0 ray 2.51.1 transformers 4.57.1 triton 3.4.0 torchaudio 2.8.0+cu128 torchvision 0.24.0+cu128 vllm 0.11.1 xfuser 0.4.5 xgrammar 0.1.25 ljperf 0.1.0+477686c5	Ubuntu 24.04 Python 3.12 Torch 2.9.0+cu128 CUDA 12.8 diffusers 0.35.2 decord 0.6.0 decord2 2.0.0 deepgpu-comfyui 1.3.2 deepgpu-torch 0.1.12+torch2.9.0cu128 flash_attn 2.8.3 flash_mla 1.0.0+1408756 flashinfer-python 0.5.2 imageio 2.37.2 imageio-ffmpeg 0.6.0 ray 2.51.1 transformers 4.57.1 sgl-kernel 0.3.17.post1 sglang 0.5.5.post3 xgrammar 0.1.25 triton 3.5.0 torchao 0.9.0 torchaudio 2.8.0+cu128 torchvision 0.24.0+cu128 xfuser 0.4.5 ljperf 0.1.0+477686c5	Ubuntu 24.04 Python 3.12 Torch 2.9.0+cu130 CUDA 13.0.2 diffusers 0.35.2 flash_attn 2.8.3 flashinfer-python 0.5.2 imageio 2.37.2 imageio-ffmpeg 0.6.0 ray 2.51.1 transformers 4.57.1 triton 3.5.0 torchaudio 2.9.0+cu130 torchvision 0.24.0+cu130 vllm 0.11.2 xfuser 0.4.5 xgrammar 0.1.25 ljperf 0.1.0+d0e4a408	Ubuntu 24.04 Python 3.12 Torch 2.9.0+cu130 CUDA 13.0.2 diffusers 0.35.2 flash_attn 2.8.3 flashinfer-python 0.5.2 transformers 4.57.1 Ray 2.51.1 vllm 0.11.1 Triton 3.5.0 torchaudio 2.9.0 torchvision 0.24.0 xfuser 0.4.5 xgrammar 0.1.25	Ubuntu 24.04 Python 3.12 Torch 2.9.0+cu130 CUDA 13.0.2 diffusers 0.35.2 decord 0.6.0 decord2 2.0.0 flash_attn 2.8.3 flashinfer-python 0.5.2 imageio 2.37.2 imageio-ffmpeg 0.6.0 ray 2.51.1 transformers 4.57.1 sgl-kernel 0.3.17.post1 sglang 0.5.5.post3 xgrammar 0.1.25 triton 3.5.0 torchao 0.9.0 torchaudio 2.9.0 torchvision 0.24.0 xfuser 0.4.5 ljperf 0.1.0+d0e4a408	Ubuntu 24.04 Python 3.12 Torch 2.9.0+cu130 CUDA 13.0.2 diffusers 0.35.2 decord2 2.0.0 flashinfer-python 0.5.2 imageio 2.37.2 flash_attn 2.8.3 imageio-ffmpeg 0.6.0 transformers 4.57.1 sgl-kernel 0.3.17.post1 sglang 0.5.5.post3 xgrammar 0.1.25 triton 3.5.0 torchao 0.9.0 torchaudio 2.9.0 torchvision 0.24.0 xfuser 0.4.5

Assets

Public images

CUDA 12.8:

egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-nv-pytorch:25.11-vllm0.11.1-pytorch2.9-cu128-20251120-serverless
egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-nv-pytorch:25.11-sglang0.5.5.post3-pytorch2.9-cu128-20251121-serverless

CUDA 13.0:

egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-nv-pytorch:25.11-vllm0.11.1-pytorch2.9-cu130-20251120-serverless
egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-nv-pytorch:25.11-sglang0.5.5.post3-pytorch2.9-cu130-20251121-serverless

VPC images

To accelerate image pulls from a Virtual Private Cloud (VPC), replace the public URI egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/{image:tag} with the VPC-specific URI acs-registry-vpc.{region-id}.cr.aliyuncs.com/egslingjun/{image:tag}.

{region-id}: The ID of the region where your ACS service is available. Examples include cn-beijing and cn-wulanchabu.
{image:tag}: The name and tag of the image. Examples include inference-nv-pytorch:25.10-vllm0.11.0-pytorch2.8-cu128-20251028-serverless and training-nv-pytorch:25.10-serverless.

Note

These images are intended for ACS and multi-tenant Lingjun environments. Do not use them in single-tenant Lingjun environments.

Driver requirements

CUDA 12.8: NVIDIA driver release >= 570
CUDA 13.0: NVIDIA driver release >= 580

Quick start

This example shows how to pull an inference-nv-pytorch image by using Docker and test an inference service with the Qwen2.5-7B-Instruct model.

Note

For more information about building model inference services with ACS GPU computing power, see the following topics:

Pull the inference image.

docker pull egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-nv-pytorch:[tag]

Download the open-source model from ModelScope.

pip install modelscope
cd /mnt
modelscope download --model Qwen/Qwen2.5-7B-Instruct --local_dir ./Qwen2.5-7B-Instruct

Run the following command to enter the container.

docker run -it --rm --gpus all --network=host --privileged --init --ipc=host \
--ulimit memlock=-1 --ulimit stack=67108864  \
-v /mnt/:/mnt/ \
egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-nv-pytorch:[tag]

Run an inference test to check the conversational feature of vLLM.

Start the API server.

python3 -m vllm.entrypoints.openai.api_server \
--model /mnt/Qwen2.5-7B-Instruct \
--trust-remote-code --disable-custom-all-reduce \
--tensor-parallel-size 1

Send a test request from the client.

curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
    "model": "/mnt/Qwen2.5-7B-Instruct",  
    "messages": [
    {"role": "system", "content": "You are a friendly AI assistant."},
    {"role": "user", "content": "Introduce deep learning."}
    ]}'

For more information about how to use vLLM, see vLLM.

Known issues

The deepgpu-comfyui plugin for accelerating Wanx model video generation currently supports only the GN8IS, G49E, and G59 instance types.

Container Compute Service:inference-nv-pytorch 25.11