inference-nv-pytorch 26.03 release notes - Container Compute Service

What's new

Main features

Provides images for CUDA 12.8 and CUDA 13.0:
- CUDA 12.8 — supports amd64 only
- CUDA 13.0 — supports amd64 and aarch64
Upgrades vLLM to v0.17.1 with Qwen3.5 model support.

Bug fixes

None.

Image details

Image name	inference-nv-pytorch
Tag	26.03-vllm0.17.1-pytorch2.10-cu128-20260317-serverless	26.03-vllm0.17.1-pytorch2.10-cu130-20260317-serverless
Supported architecture	amd64	amd64	aarch64
Use case	large model inference	large model inference	large model inference
Framework	PyTorch	PyTorch	PyTorch
Requirements	NVIDIA Driver release ≥ 570	NVIDIA Driver release ≥ 580	NVIDIA Driver release ≥ 580
System components	Ubuntu 24.04 Python 3.12 Torch 2.10.0 CUDA 12.8 NCCL 2.29.7 diffusers 0.37.0 flash_attn 2.8.4 flash_attn_3 3.0.0 flashinfer-python 0.6.4 imageio-ffmpeg 0.6.0 ray 2.54.0 transformers 4.57.6 triton 3.6.0 torchaudio 2.10.0 torchvision 0.25.0 vllm 0.17.1 xfuser 0.4.5 xgrammar 0.1.29	Ubuntu 24.04 Python 3.12 Torch 2.10.0+cu130 CUDA 13.0.2 NCCL 2.29.7 diffusers 0.37.0 flash_attn 2.8.4 flash_attn_3 3.0.0 flashinfer-python 0.6.4 imageio-ffmpeg 0.6.0 ray 2.54.0 transformers 4.57.6 triton 3.6.0 torchaudio 2.10.0+cu130 torchvision 0.25.0+cu130 vllm 0.17.1 xfuser 0.4.5 xgrammar 0.1.29 ljperf 0.1.0+d0e4a408	Ubuntu 24.04 Python 3.12 Torch 2.10.0+cu130 CUDA 13.0.2 NCCL 2.29.7 flash_attn 2.8.4 flashinfer-python 0.6.4 transformers 4.57.6 ray 2.54.0 vllm 0.17.1 triton 3.6.0 torchaudio 2.10.0+cu130 torchvision 0.25.0+cu130 xgrammar 0.1.29 ljperf 0.1.0+477686c5

Assets

Public images

CUDA 12.8

egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-nv-pytorch:26.03-vllm0.17.1-pytorch2.10-cu128-20260317-serverless

CUDA 13.0

egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-nv-pytorch:26.03-vllm0.17.1-pytorch2.10-cu130-20260317-serverless

VPC images

To speed up image pulls within a VPC, replace the registry hostname with a region-specific VPC endpoint.

Change the image path from:

egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/{image:tag}

To:

acs-registry-vpc.{region-id}.cr.aliyuncs.com/egslingjun/{image:tag}

Placeholder	Description	Example
`{region-id}`	Region ID of your ACS cluster	`cn-beijing`, `cn-wulanchabu`
`{image:tag}`	Image name and tag	`inference-nv-pytorch:26.03-vllm0.17.1-pytorch2.10-cu128-20260317-serverless`

Note

Compatible with standard ACS products and multi-tenant Lingjun environments. Not supported in single-tenant Lingjun environments.

Driver requirements

CUDA 12.8: NVIDIA Driver release ≥ 570
CUDA 13.0: NVIDIA Driver release ≥ 580

Quick start

This example uses Docker only to pull the inference-nv-pytorch image and test the inference service with the Qwen2.5-7B-Instruct model.

Note

To use this image in ACS, select it from the Artifact Center when you create a workload in the console, or specify the image reference in a YAML file. For more information, see the following topics about building model inference services with ACS GPU compute:

Pull the image.

docker pull egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-nv-pytorch:[tag]

Download the model from ModelScope.

pip install modelscope
cd /mnt
modelscope download --model Qwen/Qwen2.5-7B-Instruct --local_dir ./Qwen2.5-7B-Instruct

Enter the container.

docker run -it --rm --gpus all --network=host --privileged --init --ipc=host \
--ulimit memlock=-1 --ulimit stack=67108864  \
-v /mnt/:/mnt/ \
egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-nv-pytorch:[tag]

Verify vLLM chat completions.

Start the server.

python3 -m vllm.entrypoints.openai.api_server \
--model /mnt/Qwen2.5-7B-Instruct \
--trust-remote-code --disable-custom-all-reduce \
--tensor-parallel-size 1

Run a client-side test.

curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
    "model": "/mnt/Qwen2.5-7B-Instruct",  
    "messages": [
    {"role": "system", "content": "You are a friendly AI assistant."},
    {"role": "user", "content": "Introduce deep learning."}
    ]}'

The vLLM documentation covers additional features and configuration.

Known issues

The 26.03 image does not support the deepgpu-comfyui plugin.