Container Compute Service:training-nv-pytorch 25.12

New features

• vLLM upgraded to 0.12.0

• flashinfer-python upgraded to 0.5.3

Bug fixes

None.

Public images

egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/training-nv-pytorch:25.12-cu130-serverless

egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/training-nv-pytorch:25.12-cu128-serverless

VPC images

To pull Alibaba Cloud Container Compute Service (ACS) AI container images faster within a VPC, replace the image URI prefix:

• Replace: egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/{image:tag}

• With: acs-registry-vpc.{region-id}.cr.aliyuncs.com/egslingjun/{image:tag}

This image is compatible with ACS products and Lingjun multi-tenant products. It is not compatible with Lingjun single-tenant products.

PyTorch compiling optimization

torch.compile(), introduced in PyTorch 2.0, is effective for single-GPU training but provides limited or negative benefit for large language model (LLM) training, which depends on GPU memory optimization and distributed frameworks such as Fully Sharded Data Parallel (FSDP) or DeepSpeed.

This release improves torch.compile() for distributed LLM training through two optimizations:

• Communication granularity control in DeepSpeed: Controlling communication granularity gives the compiler a complete compute graph, enabling wider compiling optimization.

• Frontend improvements: The PyTorch compiler frontend now compiles even when a graph break occurs, with enhanced mode matching and dynamic shape capabilities.

Result: ~20% higher end-to-end throughput in 8B-parameter LLM training.

GPU memory optimization for recomputation

Based on performance tests across different clusters and parameter configurations, this release integrates the optimal number of activation recomputation layers directly into PyTorch. Enable it with a single environment variable—no manual tuning required.

This feature is available in the DeepSpeed framework.

1. Pull the image

docker pull egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/training-nv-pytorch:[tag]

Replace [tag] with the target image tag, such as 25.12-cu130-serverless or 25.12-cu128-serverless.

2. Enable compiler and memory optimizations

Enable compiler optimization

Use the Hugging Face Transformers Trainer API:

Enable GPU memory optimization for recomputation

export CHECKPOINT_OPTIMIZATION=true

3. Start the container

The image includes a built-in model training tool, ljperf. The following example starts the container and runs a training job.

LLM workloads

# Start the container
docker run --rm -it --ipc=host --net=host --privileged egslingjun-registry.cn-wulanchabu.cr.aliyuncs.com/egslingjun/training-nv-pytorch:[tag]

# Run the training demo
ljperf benchmark --model deepspeed/llama3-8b

4. Configuration recommendations

Follow these recommendations when using this image:

• Do not reinstall pre-bundled libraries such as PyTorch and DeepSpeed. The image is tuned with specific library versions; reinstalling them may break optimizations.

• Leave zero_optimization.stage3_prefetch_bucket_size in your DeepSpeed configuration blank or set it to auto.

• Set NCCL_SOCKET_IFNAME based on the number of GPUs requested per pod:

  GPU count per pod	Setting
  1, 2, 4, or 8 GPUs	NCCL_SOCKET_IFNAME=eth0 (default)
  16 GPUs (all GPUs on the machine, using High-Performance Network (HPN))	NCCL_SOCKET_IFNAME=hpn0

Compiling fa3 on CUDA 13.0.2 fails

Condition: Using the CUDA 13.0.2 (25.12-cu130-serverless) image and compiling flash-attention 3 (fa3) directly inside the container.

Impact: The compilation fails with an error.

Workaround: This is a known community issue. Do not compile fa3 directly on the CUDA 13.0.2 image. Use the CUDA 12.8 (25.12-cu128-serverless) image if fa3 compilation is required.