Automatically install AIACC-Inference

AIACC-Inference can optimize models based on TensorFlow and exportable frameworks in the Open Neural Network Exchange (ONNX) format to improve inference performance. This topic describes how to automatically install AIACC-Inference and test the demo.

Background information

Conda is an open source package management system and environment management system that can run on various platforms. Miniconda is a small installer of conda. When you create a GPU-accelerated instance, you can configure a conda environment that contains AIACC-Inference that is automatically installed. You can use Miniconda to select a conda environment and use AIACC-Inference to improve inference performance.

ONNX is an open source format in which trained models are stored. You can convert data of models of different frameworks such as PyTorch and MXNet to the ONNX format. This way, the test for models of different frameworks in the same environment becomes easy.

AIACC-Inference depends on the GPU driver, CUDA, and cuDNN. When you create a GPU-accelerated instance, select Auto-install GPU Driver and Auto-install AIACC-Inference. Then, select a CUDA version, a GPU driver, and a cuDNN version. After the instance is created, you can configure a conda environment that contains AIACC-Inference based on the CUDA version. For more information about how to create a GPU-accelerated instance, see Create an NVIDIA GPU-accelerated instance.

Test demo

Connect to the instance. For more information, see Overview.
Select a conda environment.
1. Initialize Miniconda.
```
. /root/miniconda/etc/profile.d/conda.sh
```
2. View all conda environments.
```
conda env list
```
  The following figure shows an example command output.
3. Select a conda environment.
```
conda activate [environments_name]
```
  The following figure shows an example command output.
Test the demo.
By default, the aiacc_inference_demo.tgz demo file is located in the /root directory. In this example, the ONNX demo is tested.
1. Decompress the demo test package.
```
tar -xvf aiacc_inference_demo.tgz
```
2. Go to the ONNX demo directory.
```
cd /root/aiacc_inference_demo/aiacc_inference_onnx/resnet50v1
```
3. Run the test script in the directory.
  The following command is used in the example:
```
python3 test.py
```
  The test script executes inference tasks and randomly generates and classifies images based on a ResNet50 model. This reduces the amount of time that is required for a task to perform inference from 6.4 ms to less than 1.5 ms. The following figure shows an example inference result.

Delete Miniconda

if you no longer use AIACC-Inference, you can delete Miniconda. By default, the root user can install and delete Miniconda.

Delete the miniconda folder.
```
rm -rf /root/miniconda
```
Delete relevant environment variables and output.
1. Modify the /root/.bashrc file and comment out the environment variables and output that are related to Miniconda and AIACC-Inference.
  The following figure shows an example command output.
2. Make the changes to the environment variables take effect.
```
source /root/.bashrc
```