PAI provides acceleration tools for AI training and inference, covering dataset caching, distributed training, and model optimization.
Acceleration tools
PAI offers the following acceleration tools for different stages of the AI workflow:
|
Tool |
Stage |
Description |
|
EPL |
Training |
Large-scale distributed training framework for TensorFlow. Supports data parallelism, operator splitting, pipeline parallelism, and automatic parallelism strategies. |
|
Pai-Megatron-Patch |
Training |
Accelerates Transformer model training in PyTorch by integrating multiple optimization techniques with Transformer model libraries. |
|
PAI-Blade |
Inference |
General-purpose inference optimization for TensorFlow and PyTorch. Supports GPUs, CPUs, and edge devices. Applies graph optimization, AI compiler optimization, mixed precision, and automatic compression through a single API call. |
|
BladeLLM |
Inference |
High-performance inference engine for large language models (LLMs). Optimizes deployment with model quantization, efficient batching, and GPU memory management. |
|
Local cache acceleration |
Data loading |
Caches datasets on Lingjun compute nodes to eliminate repeated remote storage reads during multi-epoch training. Supports OSS and CPFS storage. |
Get started
Select the tool that matches your optimization goal.
-
EPL
Distributed training framework for TensorFlow with automatic parallelism strategies. See EPL for distributed training acceleration.
-
Pai-Megatron-Patch
Training optimization tool for PyTorch Transformer models. Improves training speed through acceleration switches. See Transformer training acceleration (Pai-Megatron-Patch).
-
PAI-Blade
General-purpose inference optimization tool that integrates graph optimization, mixed precision, and compression techniques. See Inference acceleration (Blade) overview.
-
BladeLLM
High-performance inference engine for LLM deployment with quantization and efficient batching. See LLM inference engine (BladeLLM).
-
Local cache acceleration
Caches datasets on Lingjun compute nodes for faster multi-epoch training. See Local cache acceleration.
For hands-on tutorials, see AI acceleration use cases.