inference-nv-pytorch 26.03 - Container Compute Service - Alibaba Cloud Documentation Center

This document provides the release notes for inference-nv-pytorch 26.03.

Main features and bug fixes

Main features

This release provides images for CUDA 12.8 and CUDA 13.0:
- The CUDA 12.8 image supports only the amd64 architecture.
- The CUDA 13.0 image supports the amd64 and aarch64 architectures.
This release upgrades vLLM to v0.17.1, adding support for the Qwen3.5 model.

Bug fixes

None.

Image name	inference-nv-pytorch
Tag	26.03-vllm0.17.1-pytorch2.10-cu128-20260317-serverless	26.03-vllm0.17.1-pytorch2.10-cu130-20260317-serverless
Supported architecture	amd64	amd64	aarch64
Use case	large model inference	large model inference	large model inference
Framework	PyTorch	PyTorch	PyTorch
Requirements	NVIDIA Driver release >= 570	NVIDIA Driver release >= 580	NVIDIA Driver release >= 580
System components	Ubuntu 24.04 Python 3.12 Torch 2.10.0 CUDA 12.8 NCCL 2.29.7 diffusers 0.37.0 flash_attn 2.8.4 flash_attn_3 3.0.0 flashinfer-python 0.6.4 imageio-ffmpeg 0.6.0 ray 2.54.0 transformers 4.57.6 triton 3.6.0 torchaudio 2.10.0 torchvision 0.25.0 vllm 0.17.1 xfuser 0.4.5 xgrammar 0.1.29	Ubuntu 24.04 Python 3.12 Torch 2.10.0+cu130 CUDA 13.0.2 NCCL 2.29.7 diffusers 0.37.0 flash_attn 2.8.4 flash_attn_3 3.0.0 flashinfer-python 0.6.4 imageio-ffmpeg 0.6.0 ray 2.54.0 transformers 4.57.6 triton 3.6.0 torchaudio 2.10.0+cu130 torchvision 0.25.0+cu130 vllm 0.17.1 xfuser 0.4.5 xgrammar 0.1.29 ljperf 0.1.0+d0e4a408	Ubuntu 24.04 Python 3.12 Torch 2.10.0+cu130 CUDA 13.0.2 NCCL 2.29.7 flash_attn 2.8.4 flashinfer-python 0.6.4 transformers 4.57.6 ray 2.54.0 vllm 0.17.1 triton 3.6.0 torchaudio 2.10.0+cu130 torchvision 0.25.0+cu130 xgrammar 0.1.29 ljperf 0.1.0+477686c5

Assets

Public image

CUDA 12.8 asset

egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-nv-pytorch:26.03-vllm0.17.1-pytorch2.10-cu128-20260317-serverless

CUDA 13.0 asset

egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-nv-pytorch:26.03-vllm0.17.1-pytorch2.10-cu130-20260317-serverless

VPC image

All existing and new ACS AI container images in the egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun repository support IN-VPC pulling.

Replace the public network registry host in your image URI with the region-specific VPC endpoint.

URI component	Public network	IN-VPC
Registry host	`egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com`	`acs-registry-vpc.{region-id}.cr.aliyuncs.com`
Repository	`egslingjun`	`egslingjun` (unchanged)
Image and tag	`{image:tag}`	`{image:tag}` (unchanged)

Replace {region-id} with the ID of the region where your ACS service runs. For example:

Region	Region ID
China (Beijing)	`cn-beijing`
China (Ulanqab)	`cn-wulanchabu`

For the full list of supported regions, see Regions.

Example {image:tag} values:

inference-nv-pytorch:25.10-vllm0.11.0-pytorch2.8-cu128-20251028-serverless
training-nv-pytorch:25.10-serverless

Note

These images are for AI Computing Service (ACS) and Lingjun multi-tenant environments, and are not supported in Lingjun single-tenant environments.

Driver requirements

CUDA 12.8: NVIDIA Driver release >= 570
CUDA 13.0: NVIDIA Driver release >= 580

Quick start

The following example shows how to pull the inference-nv-pytorch image using Docker and test the inference service with the Qwen2.5-7B-Instruct model.

Note

To use the inference-nv-pytorch image in AI Computing Service (ACS), select it from the artifact repository page when you create a workload in the console, or specify the image reference in a YAML file. For more information, see the following topics on building model inference services with ACS GPU computing power:

Pull the inference container image.

docker pull egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-nv-pytorch:[tag]

Download the open-source model from ModelScope.

pip install modelscope
cd /mnt
modelscope download --model Qwen/Qwen2.5-7B-Instruct --local_dir ./Qwen2.5-7B-Instruct

Run the following command to enter the container.

docker run -it --rm --gpus all --network=host --privileged --init --ipc=host \
--ulimit memlock=-1 --ulimit stack=67108864  \
-v /mnt/:/mnt/ \
egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-nv-pytorch:[tag]

Run an inference test to verify the vLLM chat completion feature.

Start the server-side service.

python3 -m vllm.entrypoints.openai.api_server \
--model /mnt/Qwen2.5-7B-Instruct \
--trust-remote-code --disable-custom-all-reduce \
--tensor-parallel-size 1

Run a test on the client side.

curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
    "model": "/mnt/Qwen2.5-7B-Instruct",  
    "messages": [
    {"role": "system", "content": "You are a friendly AI assistant."},
    {"role": "user", "content": "Introduce deep learning."}
    ]}'

For more information, see the vLLM documentation.

Known issues

This image does not support the deepgpu-comfyui plugin.

Container Compute Service:inference-nv-pytorch 26.03