Learning about AIACC-ACSpeed | AIACC-ACSpeed Performance Data

This article describes the performance data of AIAAC 2.0-AIACC Communication Speeding (AIACC-ACSpeed) in training models. Compared to training performed by using native PyTorch DDP, ACSpeed has a significantly improved performance.

Background Information

This article uses the performance data of multi-instance training with AIACC-ACSpeed (ACSpeed) V1.0.2 enabled on an eight-GPU ECS instance as an example. This example tests the performance of ACSpeed when training models in different scenarios.

Tested Versions

ACSpeed: ACSpeed V1.0.2
CUDA: CUDA V11.1
Torch: Torch 1.8.1 + cu111
Instance type: an eight-GPU instance

Test Results

ACSpeed shows significant performance improvements in multiple model trainings, with performance improvements ranging from 5% to 200%. As shown in the results, the improved performance of ACSpeed is more notable when compared to the poor scalability of PyTorch DDP. The performance of ACSpeed is not affected by scaling. The following figure shows the results of the test.Performance improvement

The following table describes the terms used in this test:

Term	Description
ddp_acc (x-axis)	Indicates the scalability of PyTorch DDP for multiple multi-GPU instances. The scalability of PyTorch DDP for multiple multi-GPU instances is indicated by multi-instance linearity. A smaller value of the linearity means a poorer scalability. The linearity is calculated based on the following formula: Linearity = multi-instance performance / single-instance performance / the number of clusters.
acc_ratio (y-axis)	Indicates the improvement ratio of ACSpeed over PyTorch DDP measured by performance metrics. For example, 1.25 indicates that the performance of ACSpeed is 1.25 times that of PyTorch DDP, which means the performance is improved by 25%.
Dots	Indicates different sizes of clusters. • Blue dot: indicates that the number of clusters is 1. • Orange dot: indicates that the number of clusters is 2. • Red dot: indicates that the number of clusters is 4. • Green dot: indicates that the number of clusters is 8.

Performance Data of Example Models

This section shows only the performance data of the example models that have been tested. The performance improvements vary across different scenarios, which is caused by the different proportions of communication computing. The following section shows the performance data of specific test models.

Scenario 1: Training an alexnet model
- Model: alexnet
- Domain: COMPUTER_VISION
- Subdomain: CLASSIFICATION
- Batch size: 128
- Precision: Automatic mixed precision (AMP)