Container Compute Service:inference-nv-pytorch 25.09

Announcements

The inference-nv-pytorch:25.09-vllm0.10.2-pytorch2.8-cu128-20250922-serverless and inference-nv-pytorch:25.09-sglang0.5.2-pytorch2.8-cu128-20250917-serverless images apply to ACS (Alibaba Cloud Container Service for Kubernetes) and Lingjun multi-tenant products only. They do not apply to Lingjun single-tenant products.

What's new

Image contents

The following tables list the system components for each image tag.

Assets

egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-nv-pytorch:25.09-vllm0.10.2-pytorch2.8-cu128-20250922-serverless

egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-nv-pytorch:25.09-sglang0.5.2-pytorch2.8-cu128-20250917-serverless

acs-registry-vpc.{region-id}.cr.aliyuncs.com/egslingjun/{image:tag}

Driver requirements

NVIDIA Driver release >= 570

Quick start

This example shows how to pull the inference-nv-pytorch image with Docker and test an inference service using the Qwen2.5-7B-Instruct model.

To use the inference-nv-pytorch image in ACS, select it on the Artifacts page when creating a workload in the console, or specify the image reference in a YAML file. For more information about building model inference services with ACS GPU computing power, see:

Build a DeepSeek distilled model inference service with ACS GPU computing power

Build a full-featured DeepSeek model inference service with ACS GPU computing power

Build a distributed full-featured DeepSeek inference service with ACS GPU computing power

Accelerate Wan2.1 video generation with DeepGPU

1. Pull the inference container image.

   docker pull egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-nv-pytorch:[tag]

2. Download the Qwen2.5-7B-Instruct model from ModelScope.

   pip install modelscope
   cd /mnt
   modelscope download --model Qwen/Qwen2.5-7B-Instruct --local_dir ./Qwen2.5-7B-Instruct

3. Start the container.

   docker run -d -t --network=host --privileged --init --ipc=host \
   --ulimit memlock=-1 --ulimit stack=67108864  \
   -v /mnt/:/mnt/ \
   egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/inference-nv-pytorch:[tag]

4. Test the vLLM conversational inference.

   1. Start the vLLM server.

      python3 -m vllm.entrypoints.openai.api_server \
      --model /mnt/Qwen2.5-7B-Instruct \
      --trust-remote-code --disable-custom-all-reduce \
      --tensor-parallel-size 1

   2. Send a test request.

      curl http://localhost:8000/v1/chat/completions \
          -H "Content-Type: application/json" \
          -d '{
          "model": "/mnt/Qwen2.5-7B-Instruct",
          "messages": [
          {"role": "system", "content": "You are a friendly AI assistant."},
          {"role": "user", "content": "Tell me about deep learning."}
          ]}'

      For more information about vLLM, see the vLLM documentation.

Known issues

The deepgpu-comfyui plugin for Wanx model video generation acceleration currently supports only GN8IS and G49E GPU types.