AI acceleration overview - Platform For AI - Alibaba Cloud Documentation Center

PAI accelerates AI training and inference through dataset caching, distributed training, and model optimization tools.

Acceleration tools

PAI provides acceleration tools for each stage of the AI workflow:

Tool	Stage	Description
EPL	Training	Large-scale distributed training framework for TensorFlow. Supports data, operator-splitting, pipeline, and automatic parallelism.
Pai-Megatron-Patch	Training	Accelerates PyTorch Transformer training by integrating optimization techniques with Transformer model libraries.
PAI-Blade	Inference	General-purpose inference optimization for TensorFlow and PyTorch across GPUs, CPUs, and edge devices. Applies graph optimization, AI compiler optimization, mixed precision, and automatic compression via a single API call.
BladeLLM	Inference	High-performance inference engine for large language models (LLMs). Optimizes deployment with model quantization, efficient batching, and GPU memory management.
Local cache acceleration	Data loading	Caches datasets on Lingjun compute nodes to eliminate repeated remote storage reads during multi-epoch training. Supports OSS and CPFS storage.

Choose a tool based on your optimization goal.

EPL

Distributed training framework for TensorFlow with automatic parallelism. EPL for distributed training acceleration.
Pai-Megatron-Patch

Optimizes PyTorch Transformer training with acceleration switches. Transformer training acceleration (Pai-Megatron-Patch).
PAI-Blade

General-purpose inference optimization combining graph optimization, mixed precision, and compression. Inference acceleration (Blade) overview.
BladeLLM

High-performance LLM inference engine with quantization and efficient batching. LLM inference engine (BladeLLM).
Local cache acceleration

Caches datasets on Lingjun compute nodes for faster multi-epoch training. Local cache acceleration.