BuildingCode

Build OpenVINO™ Inference Engine

Contents

• Introduction
• Building for different OSes
  • Windows* systems
  • Linux* Systems
  • macOS* Systems with Intel CPU
  • macOS* Systems with Apple Silicon
  • Android* Systems
  • Raspbian Stretch* OS
• Installing
• Adding OpenVINO Runtime (Inference Engine) to Your Project
• Building in a Docker image
• Use Custom OpenCV Builds
• Next Steps
• Additional Resources

Introduction

The OpenVINO Runtime can infer models in different formats with various input and output formats.

The open source version of Inference Engine includes the following plugins:

PLUGIN	DEVICE TYPES
CPU plugin	Intel® Xeon® with Intel® AVX2 and AVX512, Intel® Core™ Processors with Intel® AVX2, Intel® Atom® Processors with Intel® SSE
GPU plugin	Intel® Processor Graphics, including Intel® HD Graphics and Intel® Iris® Graphics
GNA plugin	Intel® Speech Enabling Developer Kit, Amazon Alexa* Premium Far-Field Developer Kit, Intel® Pentium® Silver processor J5005, Intel® Celeron® processor J4005, Intel® Core™ i3-8121U processor
MYRIAD plugin	Intel® Neural Compute Stick 2 powered by the Intel® Movidius™ Myriad™ X
Heterogeneous plugin	Heterogeneous execution enables computing for inference on one network on several Intel® devices.
MULTI plugin	Automatic inference on multiple devices simultaneously
AUTO plugin	Automatic device selection
BATCH plugin	Automatic batching

Building for different OSes

• Windows* systems
• Linux* Systems
• macOS* Systems
• Android* Systems
• Raspbian Stretch* OS

NOTE: If you need a custom build, refer to a dedicated guide with CMake options which control OpenVINO build if you need a custom build.

Installing

Once the project is built you can install OpenVINO™ Runtime into custom location:

cmake --install <BUILDDIR> --prefix <INSTALLDIR>

Checking your installation:

For versions prior to 2022.1

1. Obtaining Open Model Zoo tools and models

To have the ability to run samples and demos, you need to clone the Open Model Zoo repository and copy the folder under ./deployment_tools to your install directory:

git clone https://github.com/openvinotoolkit/open_model_zoo.git
cmake -E copy_directory ./open_model_zoo/ <INSTALLDIR>/deployment_tools/open_model_zoo/

2. Adding OpenCV to your environment

You can find more info on OpenCV custom builds here.

Open Model Zoo samples use OpenCV functionality to load images. To use it for demo builds you need to provide the path to your OpenCV custom build by setting OpenCV_DIR environment variable and add path OpenCV libraries to the LD_LIBRARY_PATH (Linux) or PATH (Windows) variable before running demos.

Linux:

export LD_LIBRARY_PATH=/path/to/opencv_install/lib/:$LD_LIBRARY_PATH
export OpenCV_DIR=/path/to/opencv_install/cmake

Windows:

set PATH=\path\to\opencv_install\bin\;%PATH%
set OpenCV_DIR=\path\to\opencv_install\cmake

3. Running demo

To check your installation go to the demo directory and run Classification Demo:

Linux and macOS:

cd <INSTALLDIR>/deployment_tools/demo
./demo_squeezenet_download_convert_run.sh

Windows:

cd <INSTALLDIR>\deployment_tools\demo
demo_squeezenet_download_convert_run.bat

Result:

Top 10 results:

Image <INSTALLDIR>/deployment_tools/demo/car.png

classid probability label
------- ----------- -----
817     0.6853030   sports car, sport car
479     0.1835197   car wheel
511     0.0917197   convertible
436     0.0200694   beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon
751     0.0069604   racer, race car, racing car
656     0.0044177   minivan
717     0.0024739   pickup, pickup truck
581     0.0017788   grille, radiator grille
468     0.0013083   cab, hack, taxi, taxicab
661     0.0007443   Model T

[ INFO ] Execution successful

For 2022.1 and after

1. Build samples

To build C++ sample applications, run the following commands:

Linux and macOS:

cd <INSTALLDIR>/samples/cpp
./build_samples.sh

Windows:

cd <INSTALLDIR>\samples\cpp
build_samples_msvc.bat

2. Install OpenVINO Development Tools

NOTE: To build OpenVINO Development Tools (Model Optimizer, Post-Training Optimization Tool, Model Downloader, and Open Model Zoo tools) wheel package locally you are required to use CMake option: -DENABLE_WHEEL=ON.

To install OpenVINO Development Tools to work with Caffe models, execute the following commands:

Linux and macOS:

#setup virtual envrinment
python3 -m venv openvino_env
source openvino_env/bin/activate
pip install pip --upgrade

#install local package from install directory
pip install openvino_dev-<version>-py3-none-any.whl[caffe]  --find-links=<INSTALLDIR>/tools

Windows:

rem setup virtual envrinment
python -m venv openvino_env
openvino_env\Scripts\activate.bat
pip install pip --upgrade

rem install local package from install directory
cd <INSTALLDIR>\tools
pip install openvino_dev-<version>-py3-none-any.whl[caffe] --find-links=<INSTALLDIR>\tools

3. Download the Models

Download the following model to run the Image Classification Sample:

Linux and macOS:

omz_downloader --name googlenet-v1 --output_dir ~/models

Windows:

omz_downloader --name googlenet-v1 --output_dir %USERPROFILE%\Documents\models

4. Convert the Model with Model Optimizer

Linux and macOS:

mkdir ~/ir
mo --input_model ~/models/public/googlenet-v1/googlenet-v1.caffemodel --data_type FP16 --output_dir ~/ir

Windows:

mkdir %USERPROFILE%\Documents\ir
mo --input_model %USERPROFILE%\Documents\models\public\googlenet-v1\googlenet-v1.caffemodel --data_type FP16 --output_dir %USERPROFILE%\Documents\ir

5. Run Inference on the Sample

Set up the OpenVINO environment variables:

Linux and macOS:

source <INSTALLDIR>/setupvars.sh

Windows:

<INSTALLDIR>\setupvars.bat

The following commands run the Image Classification Code Sample using the dog.bmp file as an input image, the model in IR format from the ir directory, and on different hardware devices:

Linux and macOS:

cd ~/inference_engine_cpp_samples_build/intel64/Release
./classification_sample_async -i ~/Downloads/dog.bmp -m ~/ir/googlenet-v1.xml -d CPU

Windows:

cd  %USERPROFILE%\Documents\Intel\OpenVINO\inference_engine_samples_build\intel64\Release
.\classification_sample_async.exe -i %USERPROFILE%\Downloads\dog.bmp -m %USERPROFILE%\Documents\ir\googlenet-v1.xml -d CPU

When the sample application is complete, you see the label and confidence data for the top 10 categories on the display:

Top 10 results:

Image dog.bmp

classid probability
------- -----------
156     0.6875963
215     0.0868125
218     0.0784114
212     0.0597296
217     0.0212105
219     0.0194193
247     0.0086272
157     0.0058511
216     0.0057589
154     0.0052615

Adding OpenVINO Runtime (Inference Engine) to Your Project

For versions prior to 2022.1

For CMake projects, set the InferenceEngine_DIR and when you run CMake tool:

cmake -DInferenceEngine_DIR=/path/to/openvino/build/ .

Then you can find Inference Engine by find_package:

find_package(InferenceEngine REQUIRED)
target_link_libraries(${PROJECT_NAME} PRIVATE ${InferenceEngine_LIBRARIES})

For 2022.1 and after

For CMake projects, set the OpenVINO_DIR and when you run CMake tool:

cmake -DOpenVINO_DIR=<INSTALLDIR>/runtime/cmake .

Then you can find OpenVINO Runtime (Inference Engine) by find_package:

find_package(OpenVINO REQUIRED)
add_executable(ov_app main.cpp)
target_link_libraries(ov_app PRIVATE openvino::runtime)

add_executable(ov_c_app main.c)
target_link_libraries(ov_c_app PRIVATE openvino::runtime::c)

Use Custom OpenCV Builds

NOTE: The recommended and tested version of OpenCV is 4.4.0.

The required versions of OpenCV packages are downloaded automatically when building the Inference Engine library. If the build script can not find and download the OpenCV package that is supported on your platform, you can use one of the following options:

• Download the most suitable version from the list of available pre-build packages from https://download.01.org/opencv/2020/openvinotoolkit from the <release_version>/inference_engine directory.

• Use a system-provided OpenCV package (e.g with running the apt install libopencv-dev command). The following modules must be enabled: imgcodecs, videoio, highgui.

• Get the OpenCV package using a package manager: pip, conda, conan etc. The package must have the development components included (header files and CMake scripts).

• Build OpenCV from source using the build instructions on the OpenCV site.

After you get the built OpenCV library, perform the following preparation steps before running the Inference Engine build:

1. Set the OpenCV_DIR environment variable to the directory where the OpenCVConfig.cmake file of you custom OpenCV build is located.
2. Disable automatic downloading of the package, using the -DENABLE_OPENCV=OFF option for CMake-based build script for Inference Engine.

Building in a Docker image

You can also build Intel® Distribution of OpenVINO™ toolkit in a Docker image by following this guide.

Next Steps

Congratulations, you have built the OpenVINO Runtime (Inference Engine). To get started with OpenVINO™, proceed to the Get Started guides:

• Get Started with OpenVINO™ on Linux*

Additional Resources

• OpenVINO™ Release Notes
• Introduction to Intel® OpenVINO™ toolkit
• Inference Engine Samples Overview
• Inference Engine Developer Guide
• Model Optimizer Developer Guide

---

* Other names and brands may be claimed as the property of others.