Add ARM CPU plugin module (openvinotoolkit#19)

* initial commit * 3rd party module added * Delete .gitmodules * Update CODEOWNERS * Update README.md * Updated dockerfiles * Add setup.py for mo_pytorch * Add files via upload * Update CODEOWNERS * Add files via upload * Add files via upload * Update README.md * Update CONTRIBUTING.md * Update README.md * Update CONTRIBUTING.md * Update CONTRIBUTING.md * Update README.md * Update CONTRIBUTING.md * Update CONTRIBUTING.md * Update CONTRIBUTING.md * Update README.md * Update README.md * propagate commits from internal repo (20-28 Jan) * Update README.md * Update README.md * delete decompose_normalizel2_max and normalizel2_fusion * Update README.md * Removed deprecated nGraph API + klocwork fixes * Update README.md * Update README.md * Update README.md * Update README.md * Update README.md * Update CONTRIBUTING.md * fixed copyright year and disabled pytorch failed test * fix make job number clean apt cache remove INTEL_ITT_LIBS dependency fixed docker run command fixed export LD_LIBRARY_PATH command * removed git compilation from Dockerfile.RPi32 added cmake compilation to Dockerfile.RPi64 added a note about Docker cache to README.md changed STATUS to FATAL_ERROR if ARM_COMPUTE_INCLUDE_DIR or ARM_COMPUTE_LIB_DIR is not defined Co-authored-by: Vitaly Tuzov <[email protected]> Co-authored-by: Anton Pankratv <[email protected]>
redradist · Feb 1, 2021 · c25a404 · c25a404
1 parent 8aa91c1
commit c25a404
Show file tree

Hide file tree

Showing 230 changed files with 14,056 additions and 4 deletions.
diff --git a/.gitmodules b/.gitmodules
@@ -0,0 +1,3 @@
+[submodule "modules/arm_plugin/thirdparty/ComputeLibrary"]
+	path = modules/arm_plugin/thirdparty/ComputeLibrary
+	url = https://github.com/ARM-software/ComputeLibrary.git
diff --git a/CODEOWNERS b/CODEOWNERS
@@ -7,3 +7,12 @@ CODEOWNERS  @openvinotoolkit/openvino-admins @openvinotoolkit/openvino_contrib-m
 # CI:
 /.github/  @openvinotoolkit/openvino-admins
 /.ci/  @openvinotoolkit/openvino-admins
+
+# ARM Plugin:
+/modules/arm_plugin/ @apankratovantonp @l-bat
+
+# Inference Engine Java API:
+/modules/java_api/ @dkurt
+
+# PyTorch extension for Model Optimizer:
+/modules/mo_pytorch/ @dkurt
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -0,0 +1,29 @@
+
+# How to Contribute to OpenVINO&trade; Contrib repository
+
+We appreciate your intention to contribute to already existed modules or to propose a new one.
+Contrib modules are licensed under the Apache\* License, Version 2.0. By contributing to the project, you agree to the license and copyright terms therein and release your contribution under these terms. 
+
+## Pull Request Requirements
+
+To contribute to `openvino_contrib` repository, please do the following steps:
+
+* Make sure you can build the module and pass all functional tests with your patch.
+* Submit a pull request (PR) to `master` branch at https://github.com/openvinotoolkit/openvino_contrib/pulls
+  * Add a prefix "[module name]" to PR title, e.g. `[java_api] Versioning support`
+
+All guidelines for contributing to the OpenVINO repositories can be found [here](https://github.com/openvinotoolkit/openvino/wiki/Contribute).
+
+## Legal Information
+
+[\*] Other names and brands may be claimed as the property of others.
+
+OpenVINO is a trademark of Intel Corporation or its subsidiaries in the U.S. and/or other countries.
+
+Copyright &copy; 2018-2021 Intel Corporation
+
+Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
+```
+http://www.apache.org/licenses/LICENSE-2.0
+```
+Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
diff --git a/README.md b/README.md
@@ -28,6 +28,7 @@ In order to keep a clean overview containing all contributed modules, the follow
 
 * Add a README.md inside your own module folder. This README explains which functionality (separate functions) is available, explains in somewhat more detail what the module is expected to do. If any extra requirements are needed to build the module without problems, add them here also.
 
-## Contributing guidelines
+## How to Contribute
 
-All guidelines for contributing to the OpenVINO repositories can be found at [How to contribute guideline](https://github.com/openvinotoolkit/openvino/wiki/Contribute).
+We welcome community contributions to the `openvino_contrib` repository. If you have an idea how to improve the modules, please share it with us.
+All guidelines for contributing to the repository can be found [here](CONTRIBUTING.md).
diff --git a/modules/README.md b/modules/README.md
@@ -6,5 +6,7 @@ This list gives an overview of all modules available inside the contrib reposito
 $ cmake -DIE_EXTRA_MODULES=<openvino_contrib>/modules -DBUILD_<module name>=OFF <openvino_source_directory>
 ```
 
+
+* [**arm_plugin**](./arm_plugin): ARM CPU Plugin -- allows to perform deep neural networks inference on ARM CPUs, using OpenVINO API.
 * [**java_api**](./java_api): Inference Engine Java API -- provides Java wrappers for Inference Engine public API.
 * [**mo_pytorch**](./mo_pytorch): PyTorch extensions for Model Optimizer -- native PyTorch to OpenVINO IR converter
diff --git a/modules/arm_plugin/CMakeLists.txt b/modules/arm_plugin/CMakeLists.txt
@@ -0,0 +1,35 @@
+# Copyright (C) 2020-2021 Intel Corporation
+# SPDX-License-Identifier: Apache-2.0
+#
+
+cmake_minimum_required(VERSION 3.13 FATAL_ERROR)
+
+project(InferenceEngineArmPlugin)
+
+set(IE_MAIN_ARM_PLUGIN_SOURCE_DIR ${InferenceEngineArmPlugin_SOURCE_DIR})
+
+find_package(InferenceEngineDeveloperPackage REQUIRED)
+
+if(NOT ARM AND NOT AARCH64)
+    message(WARNING "ARM CPU plugin can be built only for ARM or ARM64 target. Exiting..")
+    return()
+endif()
+
+add_subdirectory(thirdparty)
+add_subdirectory(src)
+
+if(ENABLE_TESTS)
+    include(CTest)
+    enable_testing()
+
+    disable_deprecated_warnings()
+
+    if(ENABLE_FUNCTIONAL_TESTS)
+        add_subdirectory(tests/functional)
+    endif()
+endif()
+
+# install
+
+# ATTENTION: uncomment to install component
+# ie_cpack(template)
diff --git a/modules/arm_plugin/Dockerfile.RPi32 b/modules/arm_plugin/Dockerfile.RPi32
@@ -0,0 +1,131 @@
+#This Dockerfile is for x86 and should be used for OpenVINO ARM plugin cross-compilation
+#https://github.com/openvinotoolkit/openvino_contrib/tree/master/modules/arm_plugin#how-to-build
+
+FROM debian:9
+
+ENV DEBIAN_FRONTEND noninteractive
+ENV PAKAGE_UPDATES_OF 20210120_11
+
+RUN apt-get update && apt-get -y upgrade
+
+#Prerequisite installation
+RUN apt-get update && \
+    apt-get install -y --no-install-recommends \
+        ca-certificates \
+        lsb-release \
+        nano \
+        wget \
+        curl \
+        nano \
+        tar \
+        bzip2 \
+        unzip \
+        cpio \
+        cifs-utils \
+        locales \
+        rsync \
+        apt-transport-https \
+        debian-archive-keyring \
+        sudo \
+        openssh-client \
+        default-jre \
+        p7zip-full \
+        software-properties-common \
+        dirmngr \
+        gnupg \
+        netcat-openbsd \
+        dh-autoreconf \
+        libcurl4-gnutls-dev \
+        libexpat1-dev \
+        gettext \
+        libz-dev \
+        libssl-dev \
+        build-essential \
+    && \
+    locale-gen en_US.UTF-8 && \
+    rm -rf /var/lib/apt/lists/*
+
+RUN dpkg --add-architecture armhf && \
+    apt-get update && \
+    apt-get install -y --no-install-recommends -f -o Dpkg::Options::="--force-confnew" \
+        build-essential \
+        libusb-1.0-0-dev:armhf \
+        software-properties-common \
+        crossbuild-essential-armhf \
+        zlib1g-dev \
+        libffi-dev \
+        libssl-dev \
+        chrpath \
+        libssl-dev \
+        libprotobuf-dev \
+        libprotoc-dev \
+        protobuf-compiler \
+        # For ARM CPU plugin
+        scons && \
+        rm -rf /var/lib/apt/lists/*
+
+#cmake 3.13 or higher is required to build OpenVINO
+RUN wget https://www.cmake.org/files/v3.13/cmake-3.13.3.tar.gz && \
+    tar xf cmake-3.13.3.tar.gz && \
+    (cd cmake-3.13.3 && ./bootstrap --parallel=$(nproc --all) && make --jobs=$(nproc --all) && make install) && \
+    rm -rf cmake-3.13.3 cmake-3.13.3.tar.gz
+
+RUN curl -sSf "https://packagecloud.io/install/repositories/github/git-lfs/config_file.list?os=debian&dist=stretch&source=script" > "/etc/apt/sources.list.d/github_git-lfs.list"
+RUN curl -L "https://packagecloud.io/github/git-lfs/gpgkey" 2> /dev/null | apt-key add - &>/dev/null && \
+    apt-get update && \
+    apt-get install -y --no-install-recommends  --allow-unauthenticated \
+        git-lfs \
+        && \
+    rm -rf /var/lib/apt/lists/*
+
+#configure paths
+RUN mkdir -p /armcpu_plugin/armcpu_debian32_rpi
+WORKDIR /armcpu_plugin
+ENV DEV_HOME /armcpu_plugin
+ENV OPENCV_HOME=$DEV_HOME/opencv
+ENV OPENVINO_HOME $DEV_HOME/openvino
+ENV OPENVINO_CONTRIB $DEV_HOME/openvino_contrib
+ENV ARM_PLUGIN_HOME $OPENVINO_CONTRIB/modules/arm_plugin
+ENV STAGING_DIR $DEV_HOME/armcpu_debian32_rpi
+
+ENV SOURCE_UPDATES_OF 20210120_11
+
+#OpenCV
+RUN git clone https://github.com/opencv/opencv.git $OPENCV_HOME && \
+    mkdir $OPENCV_HOME/build && \
+    cd $OPENCV_HOME/build && \
+    cmake .. -DCMAKE_BUILD_TYPE=Release -DBUILD_LIST=imgcodecs,videoio,highgui \
+             -DCMAKE_TOOLCHAIN_FILE=$OPENCV_HOME/platforms/linux/arm-gnueabi.toolchain.cmake \
+             -DCMAKE_STAGING_PREFIX=$STAGING_DIR && \
+    make -j$(nproc) && \
+    make install && \
+    cd $DEV_HOME
+
+#OpenVINO
+RUN git lfs clone --recurse-submodules https://github.com/openvinotoolkit/openvino.git $OPENVINO_HOME && \
+    mkdir $OPENVINO_HOME/build && \
+    cd $OPENVINO_HOME/build && \
+    cmake -DOpenCV_DIR=$STAGING_DIR/lib/cmake/opencv4 -DENABLE_OPENCV=OFF \
+          -DENABLE_TESTS=ON -DENABLE_BEH_TESTS=ON -DENABLE_FUNCTIONAL_TESTS=ON \
+          -DENABLE_SSE42=OFF -DENABLE_MYRIAD=OFF -DENABLE_GNA=OFF -DCMAKE_BUILD_TYPE=Release \
+          -DENABLE_VALIDATION_SET=OFF -DENABLE_MODELS=OFF \
+          -DCMAKE_TOOLCHAIN_FILE="$OPENVINO_HOME/cmake/arm.toolchain.cmake" \
+          -DCMAKE_STAGING_PREFIX=$STAGING_DIR \
+          $OPENVINO_HOME && \
+    make -j$(nproc) && \
+    cd $DEV_HOME
+
+#ArmCPU plugin
+RUN git clone --recurse-submodules --single-branch --branch=master https://github.com/openvinotoolkit/openvino_contrib.git $OPENVINO_CONTRIB && \
+    mkdir $ARM_PLUGIN_HOME/build && \
+    cd $ARM_PLUGIN_HOME/build && \
+    cmake -DCMAKE_BUILD_TYPE=Release -DInferenceEngineDeveloperPackage_DIR=$OPENVINO_HOME/build \
+          -DENABLE_TESTS=ON -DENABLE_BEH_TESTS=ON -DENABLE_FUNCTIONAL_TESTS=ON \
+          -DCMAKE_TOOLCHAIN_FILE="$OPENVINO_HOME/cmake/arm.toolchain.cmake" \
+          -DCMAKE_STAGING_PREFIX=$STAGING_DIR \
+          $ARM_PLUGIN_HOME && \
+    make -j$(nproc)
+
+RUN cp -vr $OPENVINO_HOME/bin/armv7l/Release/* $STAGING_DIR/ && \
+    cd $STAGING_DIR/ && \
+    tar -czvf ../OV_ARM_package.tar.gz ./*
diff --git a/modules/arm_plugin/Dockerfile.RPi64 b/modules/arm_plugin/Dockerfile.RPi64
@@ -0,0 +1,71 @@
+#This Dockerfile is for x86 and should be used for OpenVINO ARM plugin cross-compilation
+#https://github.com/openvinotoolkit/openvino_contrib/tree/master/modules/arm_plugin#how-to-build
+
+FROM ubuntu:18.04
+
+ENV DEBIAN_FRONTEND noninteractive
+ENV PAKAGE_UPDATES_OF 20210120_11
+
+RUN apt-get update && apt-get -y upgrade
+
+#Prerequisite installation
+RUN apt-get install -y wget git git-lfs scons cmake build-essential crossbuild-essential-arm64 python libprotoc-dev protobuf-compiler && apt-get clean
+
+#cmake 3.13 or higher is required to build OpenVINO
+RUN wget https://www.cmake.org/files/v3.13/cmake-3.13.3.tar.gz && \
+    tar xf cmake-3.13.3.tar.gz && \
+    (cd cmake-3.13.3 && ./bootstrap --parallel=$(nproc --all) && make --jobs=$(nproc --all) && make install) && \
+    rm -rf cmake-3.13.3 cmake-3.13.3.tar.gz
+
+#configure paths
+RUN mkdir -p /armcpu_plugin/armcpu_debian64_rpi
+WORKDIR /armcpu_plugin
+ENV DEV_HOME /armcpu_plugin
+ENV OPENCV_HOME=$DEV_HOME/opencv
+ENV OPENVINO_HOME $DEV_HOME/openvino
+ENV OPENVINO_CONTRIB $DEV_HOME/openvino_contrib
+ENV ARM_PLUGIN_HOME $OPENVINO_CONTRIB/modules/arm_plugin
+ENV STAGING_DIR $DEV_HOME/armcpu_debian64_rpi
+
+ENV SOURCE_UPDATES_OF 20210120_11
+
+#OpenCV
+RUN git clone https://github.com/opencv/opencv.git $OPENCV_HOME && \
+    mkdir $OPENCV_HOME/build && \
+    cd $OPENCV_HOME/build && \
+    cmake .. -DCMAKE_BUILD_TYPE=Release -DBUILD_LIST=imgcodecs,videoio,highgui \
+             -DCMAKE_TOOLCHAIN_FILE=$OPENCV_HOME/platforms/linux/aarch64-gnu.toolchain.cmake \
+             -DCMAKE_STAGING_PREFIX=$STAGING_DIR && \
+    make -j$(nproc) && \
+    make install && \
+    cd $DEV_HOME
+
+#OpenVINO
+RUN git clone --recurse-submodules https://github.com/openvinotoolkit/openvino.git $OPENVINO_HOME && \
+    mkdir $OPENVINO_HOME/build && \
+    cd $OPENVINO_HOME/build && \
+    cmake -DOpenCV_DIR=$STAGING_DIR/lib/cmake/opencv4 -DENABLE_OPENCV=OFF \
+          -DENABLE_TESTS=ON -DENABLE_BEH_TESTS=ON -DENABLE_FUNCTIONAL_TESTS=ON \
+          -DENABLE_SSE42=OFF -DENABLE_MYRIAD=OFF -DENABLE_GNA=OFF -DCMAKE_BUILD_TYPE=Release \
+          -DENABLE_VALIDATION_SET=OFF -DENABLE_MODELS=OFF \
+          -DCMAKE_TOOLCHAIN_FILE="$OPENVINO_HOME/cmake/arm64.toolchain.cmake" \
+          -DCMAKE_STAGING_PREFIX=$STAGING_DIR \
+          $OPENVINO_HOME && \
+    make -j$(nproc) && \
+    cd $DEV_HOME
+
+#ArmCPU plugin
+RUN git clone --recurse-submodules --single-branch --branch=master https://github.com/openvinotoolkit/openvino_contrib.git $OPENVINO_CONTRIB && \
+    mkdir $ARM_PLUGIN_HOME/build && \
+    cd $ARM_PLUGIN_HOME/build && \
+    cmake -DCMAKE_BUILD_TYPE=Release -DInferenceEngineDeveloperPackage_DIR=$OPENVINO_HOME/build \
+          -DENABLE_TESTS=ON -DENABLE_BEH_TESTS=ON -DENABLE_FUNCTIONAL_TESTS=ON \
+          -DARM_COMPUTE_TARGET_ARCH=arm64-v8a \
+          -DCMAKE_TOOLCHAIN_FILE="$OPENVINO_HOME/cmake/arm64.toolchain.cmake" \
+          -DCMAKE_STAGING_PREFIX=$STAGING_DIR \
+          $ARM_PLUGIN_HOME && \
+    make -j$(nproc)
+
+RUN cp -vr $OPENVINO_HOME/bin/aarch64/Release/* $STAGING_DIR/ && \
+    cd $STAGING_DIR/ && \
+    tar -czvf ../OV_ARM_package.tar.gz ./*