All Products
Search

This Product

    Container Compute Service:inference-nv-pytorch 25.09

    Document Center

    Container Compute Service:inference-nv-pytorch 25.09

    Last Updated:Jun 18, 2026

    This document provides the release notes for inference-nv-pytorch 25.09.

    Main features and bug fixes

    Main features

    • This release upgrades PyTorch to 2.8.0.

    • This release upgrades vLLM to v0.10.2.

    • This release upgrades SGLang to v0.5.2.

    • This release upgrades deepgpu-comfyui to 1.2.1 and the deepgpu-torch optimization component to 0.1.1+torch2.8.0cu128.

    Bug fixes

    None.

    Contents

    inference-nv-pytorch

    inference-nv-pytorch

    Tag

    25.09-vllm0.10.2-pytorch2.8-cu128-20250922-serverless

    25.09-sglang0.5.2-pytorch2.8-cu128-20250917-serverless

    Scenarios

    large model inference

    large model inference

    Framework

    PyTorch

    PyTorch

    Requirements

    NVIDIA Driver release >= 570

    NVIDIA Driver release >= 570

    System components

    • Ubuntu 24.04

    • Python 3.12

    • PyTorch 2.8.0

    • CUDA 12.8

    • diffusers 0.35.1

    • deepgpu-comfyui 1.2.1

    • deepgpu-torch 0.1.1+torch2.8.0cu128

    • flash_attn 2.8.3

    • flashinfer-python 0.3.1

    • imageio 2.37.0

    • imageio-ffmpeg 0.6.0

    • ray 2.49.1

    • transformers 4.56.1

    • triton 3.4.0

    • vllm 0.10.2

    • xformers 0.0.32.post1

    • xfuser 0.4.4

    • xgrammar 0.1.23

    • ljperf 0.1.0+477686c5

    • Ubuntu 24.04

    • Python 3.12

    • PyTorch 2.8.0

    • CUDA 12.8

    • decord 0.6.0

    • diffusers 0.35.1

    • deepgpu-comfyui 1.2.1

    • deepgpu-torch 0.1.1+torch2.8.0cu128

    • flash_attn 2.8.3

    • flash_mla 1.0.0+261330b

    • flashinfer-python 0.3.1

    • imageio 2.37.0

    • imageio-ffmpeg 0.6.0

    • transformers 4.56.1

    • sgl-kernel 0.3.9

    • sglang 0.5.2

    • xgrammar 0.1.24

    • triton 3.4.0

    • torchao 0.9.0

    • torchaudio 2.8.0

    • xfuser 0.4.4

    • ljperf 0.1.0+477686c5

    Asset

    Public image

    • egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-nv-pytorch:25.09-vllm0.10.2-pytorch2.8-cu128-20250922-serverless

    • egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-nv-pytorch:25.09-sglang0.5.2-pytorch2.8-cu128-20250917-serverless

    VPC image

    acs-registry-vpc.{region-id}.cr.aliyuncs.com/egslingjun/{image:tag}
    Replace {region-id} with the region where your Alibaba Cloud Container Compute Service (ACS) is activated (for example, cn-beijing or cn-wulanchabu).
    Replace {image:tag} with the name and tag of the image.
    Note

    The inference-nv-pytorch:25.03-vllm0.8.2-pytorch2.6-cu124-20250328-serverless and inference-nv-pytorch:25.03-sglang0.4.4.post1-pytorch2.5-cu124-20250327-serverless images are compatible with ACS and Lingjun multi-tenant deployments only. It is not compatible with Lingjun single-tenant deployments.

    Note

    The inference-nv-pytorch:25.09-vllm0.10.2-pytorch2.8-cu128-20250922-serverless and inference-nv-pytorch:25.09-sglang0.5.2-pytorch2.8-cu128-20250917-serverless images are for ACS and Lingjun multi-tenant products. They are not applicable to Lingjun single-tenant products.

    Driver requirements

    NVIDIA Driver release >= 570

    Quick start

    The following example shows how to pull the inference-nv-pytorch image with Docker and test the inference service with the Qwen2.5-7B-Instruct model.

    Note

    To use the inference-nv-pytorch image in AI Computing Service (ACS), select it from the Artifact Center page when creating a workload, or specify the image reference in a YAML file. For more information, see the following topics on building model inference services with ACS GPU compute:

    1. Pull the inference container image.

      docker pull egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-nv-pytorch:[tag]

    2. Download the open-source model in ModelScope format.

      pip install modelscope
cd /mnt
modelscope download --model Qwen/Qwen2.5-7B-Instruct --local_dir ./Qwen2.5-7B-Instruct

    3. Run the following command to enter the container.

      docker run -it --rm --gpus all --network=host --privileged --init --ipc=host \
--ulimit memlock=-1 --ulimit stack=67108864  \
-v /mnt/:/mnt/ \
egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-nv-pytorch:[tag]

    4. Run an inference test to verify vLLM's chat functionality.

      1. Start the server.

        python3 -m vllm.entrypoints.openai.api_server \
--model /mnt/Qwen2.5-7B-Instruct \
--trust-remote-code --disable-custom-all-reduce \
--tensor-parallel-size 1

      2. Run a test on the client.

        curl http://localhost:8000/v1/chat/completions \
    -H "Content-Type: application/json" \
    -d '{
    "model": "/mnt/Qwen2.5-7B-Instruct",  
    "messages": [
    {"role": "system", "content": "You are a friendly AI assistant."},
    {"role": "user", "content": "Tell me about deep learning."}
    ]}'

        For more information on using vLLM, see the vLLM documentation.

    Known issues

    • The deepgpu-comfyui plugin, which accelerates video generation for Wanx models, only supports GN8IS and G49E instances.