How to deploy Yolov5 model to OpenVINO

Related Tags：1.PyTorch
2. Alluxio Deep Learning Practices - 1: Running PyTorch Framework on HDFS


Abstract: OpenVINO is an AI tool suite launched by Intel, which can be used to rapidly deploy AI applications and solutions. There are more than 150 CNN network structures supporting computer vision.

Introduction: OpenVINO is an AI tool suite launched by Intel, which can be used to rapidly deploy AI applications and solutions. There are more than 150 CNN network structures supporting computer vision. It is compatible with a variety of mainstream open source frameworks such as PyTorch, Tensorflow, Keras, mxnet, Caffe and ONNX, and can easily convert the models trained by the above open source frameworks to the OpenVINO framework through tools, and support the deployment of pre-trained models to Intel's CPUs, GPUs, and VPUs.

1:Overview of OpenVINO

According to Baidu Encyclopedia and other information, OpenVINO is an AI tool suite launched by Intel, which can be used to rapidly deploy AI applications and solutions. There are more than 150 CNN network structures supporting computer vision. It is compatible with a variety of mainstream open source frameworks such as PyTorch, Tensorflow, Keras, mxnet, Caffe and ONNX, and can easily convert the models trained by the above open source frameworks to the OpenVINO framework through tools, and support the deployment of pre-trained models to Intel's CPUs, GPUs, and VPUs.

For developers who want to experience it initially, it is very convenient for OpenVINO to deploy AI models to Intel's CPUs. The OpenVINO suite also includes the image processing package OpenCV and the video processing toolkit Media SDK, which are used to process image and video decoding, pre-processing and post-processing of inference results. Generally speaking, when doing model inference, pre-processing and post-processing are required in most cases. Pre-processing includes channel transformation, size change, averaging and normalization, etc.; and post-processing needs to draw features such as target detection frames to the On images, these operations can be done using OpenVINO and OpenCV.

In addition, we need to note that different hardware environments, supported models and the precision of input and output are different, and the model formats supported by different hardware are given below.

As can be seen from the above figure, the VPU currently only supports the FP16 model format. And optimized on the following hardware (Optimized for processors):
• 6th to 12th generation Intel® Core™ processors and Intel® Xeon® processors
• 3rd generation Intel® Xeon® Scalable processor (formerly code named Cooper Lake)
• Intel® Xeon® Scalable processor (formerly Skylake and Cascade Lake)
• Intel Atom® processor with support for Intel® Streaming SIMD Extensions 4.1 (Intel® SSE4.1)
• Intel Pentium® processor N4200/5, N3350/5, or N3450/5 with Intel® HD Graphics
• Intel® Iris® Xe MAX Graphics
• Intel® Neural Compute Stick 2
• Intel® Vision Accelerator Design with Intel® Movidius™ VPUs

2: Yolov5 overview

Yolov5 is the fifth version of the YOLO algorithm (You Only Look Once). YOLO can be said to be the leader in the current real-time object detection algorithm. It was named by Redmon et al. in a paper in 2016 and is currently widely used in In a variety of AI target object detection scenarios, it is very convenient to detect object categories in images, such as people, cars, animals, etc. Although the inference accuracy of yolov4 is better than the current yolov5 in some scenarios, some new features of yolov5 are more attractive. For example, yolov5 has the characteristics of smaller inference model file, shorter training time and inference speed on the basis of not reducing the average accuracy of detection. In this way, in the case of low accuracy requirements, yolov5 is used in model construction and model deployment. etc. will be more convenient, and the inference speed will be faster.

The source address path of the official website is: , and different pre-trained weight models (YOLOv5s, YOLOv5m, YOLOv5n, etc.) of yolov5 (6.0 releases) are also given on the official website. In order to adapt to OpenVINO, choose the yolov5 v5.0 version here. Here you can download the weight model file and source code yolov5-5.0 from https://github.com/ultralytics/yolov5/releases/tag/v5.0.

3:Overview of the adaptation environment

• Operating System: Windows 10 (Intel CPU)
• Python version: Python 3.7.7
• OpenVINO Python Package : openvino 2021.4.2
• Yolov : yolov5-5.0
• netron 5.5.3
• onnx 1.10.2
• opencv-python: 4.5.4.60
About the OpenVINO suite, you can download it from the official website.

In addition, openvino_toolkit also needs to install other dependencies (C++ environment), where cmake-3.22.1-windows-x86_64.msi and Visual Studio 2019 with MSBuild are installed, which can be done by installing Visual Studio Community Edition, which needs to be checked C++ desktop development environment and other components. For details, please refer to the OpenVINO official website documentation:
https://docs.openvino.ai/latest/openvino_docs_install_guides_installing_openvino_windows.html. After the environment is set up, you can execute the following CMD command to verify whether the C++ environment is set up successfully:

#Temporarily set environment variables
# switch directory
# install dependencies
install_prerequisites.bat
#Compile and run Demo
demo_security_barrier_camera.bat