Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CMake based build for pyngraph module #3080

Merged
Show file tree
Hide file tree
Changes from 29 commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
072a6ab
[MO] Add CMake install for Model Optimizer
slyubimt Oct 26, 2020
8fc1bad
[MO] Update test for version.py
slyubimt Oct 27, 2020
19793b3
[MO] Add CMake install for Model Optimizer
slyubimt Oct 26, 2020
3cb829d
[MO] Update test for version.py
slyubimt Oct 27, 2020
bac9036
Merge branch 'features/cmake_install' of https://github.com/slyubimt/…
slyubimt Oct 27, 2020
e99c831
[MO] Add CMake install for Model Optimizer
slyubimt Oct 26, 2020
023f8b9
[MO] Update test for version.py
slyubimt Oct 27, 2020
771d4b8
Merge branch 'features/cmake_install' of https://github.com/slyubimt/…
slyubimt Oct 28, 2020
0fc6f8d
Merge branch 'master' into features/cmake_install
slyubimt Oct 28, 2020
0311cc3
Merge branch 'features/cmake_install' of https://github.com/slyubimt/…
slyubimt Oct 28, 2020
74e9698
Merge remote-tracking branch 'upstream/master' into master
slyubimt Oct 30, 2020
30161c8
Merge branch 'master' of https://github.com/slyubimt/openvino into ma…
slyubimt Nov 2, 2020
dafb9ef
Merge remote-tracking branch 'upstream/master' into master
slyubimt Nov 3, 2020
28867d4
Merge remote-tracking branch 'upstream/master' into master
slyubimt Nov 3, 2020
ee54fa3
Merge remote-tracking branch 'upstream/master' into master
slyubimt Nov 6, 2020
e71385f
Merge remote-tracking branch 'upstream/master' into master
slyubimt Nov 11, 2020
78ec374
[nGraph] Python API should be compiled and installed via CMake (41857)
slyubimt Nov 11, 2020
784f1e4
Merge remote-tracking branch 'upstream/master' into master
slyubimt Nov 17, 2020
838cb7f
Merge remote-tracking branch 'upstream/master' into features/cmake_in…
slyubimt Nov 17, 2020
c94dc6f
Refactored wheel setup script to build module using CMake
slyubimt Nov 17, 2020
1393618
Update build instructions
slyubimt Nov 17, 2020
3831a7d
Merge remote-tracking branch 'upstream/master' into master
slyubimt Nov 18, 2020
867dd97
Merge remote-tracking branch 'upstream/master' into features/cmake_in…
slyubimt Nov 18, 2020
bdd2051
Added USE_SOURCE_PERMISSIONS to cmake install
slyubimt Nov 18, 2020
dfb3dd2
Adjust CMake compiler flags conditions
slyubimt Nov 19, 2020
873203f
Merge remote-tracking branch 'upstream/master' into master
slyubimt Nov 19, 2020
baf28cd
Merge branch 'master' of https://github.com/slyubimt/openvino into ma…
slyubimt Nov 19, 2020
2e14744
fix CPack issue for CI build pipeline
slyubimt Nov 20, 2020
6bb548d
case insensitive option check
slyubimt Nov 20, 2020
79a6033
build only python API if ngraph_DIR provided
slyubimt Nov 23, 2020
b91fddd
fix lib extension for macOS
slyubimt Nov 24, 2020
7422f3d
-fixed style (flake8)
slyubimt Nov 24, 2020
09fe1f4
fix flake8 B006 check
slyubimt Nov 24, 2020
b43b426
add ngraph_DIR & remove unsed env. variables.
slyubimt Nov 25, 2020
d0ac91f
Reworked build & test instructions to make it more straightforward
slyubimt Nov 26, 2020
5379d33
remove unused CMake arguments for setup.py
slyubimt Nov 26, 2020
bf30710
make source dir condition more general
slyubimt Nov 26, 2020
c1a8700
Merge remote-tracking branch 'upstream/master' into features/cmake_in…
slyubimt Nov 26, 2020
9299e34
Update BUILDING.md
andrew-zaytsev Nov 27, 2020
16b9451
Merge remote-tracking branch 'upstream/master' into features/cmake_in…
slyubimt Nov 27, 2020
60d956c
Merge branch 'features/cmake_install_pyngraph' of https://github.com/…
slyubimt Nov 27, 2020
db3b8aa
Update BUILDING.md
andrew-zaytsev Nov 27, 2020
b24f5cc
Update BUILDING.md
andrew-zaytsev Nov 27, 2020
5bdb3cb
beautified instructions wording
slyubimt Nov 30, 2020
72a3373
Merge remote-tracking branch 'upstream/master' into master
slyubimt Nov 30, 2020
5b1618d
Merge remote-tracking branch 'upstream/master' into master
slyubimt Dec 1, 2020
80db2cf
Merge remote-tracking branch 'upstream/master' into master
slyubimt Dec 2, 2020
0291edf
Merge remote-tracking branch 'upstream/master' into master
slyubimt Dec 10, 2020
edaf16b
Merge branch 'master' into features/cmake_install_pyngraph
slyubimt Dec 10, 2020
cae9057
fix wheel build issue after sourcing setupvars
slyubimt Dec 11, 2020
26ed1c5
Merge remote-tracking branch 'upstream/master' into master
slyubimt Dec 14, 2020
4388a0b
Extend user options to build, install and develop commands
slyubimt Dec 22, 2020
d0787ff
Merge remote-tracking branch 'upstream/master' into master
slyubimt Dec 22, 2020
de4cdbf
Merge branch 'master' into features/cmake_install_pyngraph
slyubimt Dec 22, 2020
e06ec8d
Merge branch 'features/cmake_install_pyngraph' of https://github.com/…
slyubimt Dec 22, 2020
2dea25c
Merge remote-tracking branch 'upstream/master' into master
slyubimt Dec 24, 2020
67f3255
Merge branch 'master' into features/cmake_install_pyngraph
slyubimt Dec 24, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 34 additions & 38 deletions ngraph/python/BUILDING.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,29 @@ Install Cython in the Python installation, or virtualenv which you are planning

pip3 install cython

### Configure, build and install OpenVINO
### Build nGraph Python wheel

We can build the Python wheel by issuing the following command:

cd "${MY_OPENVINO_BASEDIR}/openvino/ngraph/python"
python3 setup.py bdist_wheel

Once completed, the wheel package should be located under the following path:

$ ls "${MY_OPENVINO_BASEDIR}/openvino/ngraph/python/dist/"
ngraph_core-0.0.0-cp38-cp38-linux_x86_64.whl

You can now install the wheel in your Python environment:

cd "${MY_OPENVINO_BASEDIR}/openvino/ngraph/python/dist/"
pip3 install ngraph_core-0.0.0-cp38-cp38-linux_x86_64.whl

#### What does `python3 setup.py bdist_wheel` do?

The `python3 setup.py bdist_wheel` use CMake to build ngraph python bindings and pack module into a wheel package.
You can recreate the process manually using OpenVINO build instructions:

### Configure, build and install OpenVINO

The following section will illustrate how to download, build and install OpenVINO in a workspace directory specified
by the `${MY_OPENVINO_BASEDIR}` variable. Let's start by setting this variable to a directory of your choice:
Expand Down Expand Up @@ -62,42 +84,26 @@ variable. Examples:
```
-DPYTHON_EXECUTABLE=/path/to/venv/bin/python
-DPYTHON_EXECUTABLE=$(which python3.8)
```
```

### Build nGraph Python wheel

When OpenVINO is built and installed, we can build the Python wheel by issuing the following command:

make python_wheel

Once completed, the wheel package should be located under the following path:

$ ls "${MY_OPENVINO_BASEDIR}/openvino/ngraph/python/dist/"
ngraph_core-0.0.0-cp38-cp38-linux_x86_64.whl
## Build nGraph Python Wheels on Windows

You can now install the wheel in your Python environment:
### Prerequisites

cd "${MY_OPENVINO_BASEDIR}/openvino/ngraph/python/dist/"
pip3 install ngraph_core-0.0.0-cp38-cp38-linux_x86_64.whl
In order to build OpenVINO and the nGraph Python wheel on Windows, you will need to install Visual Studio and Python.

#### What does `make python_wheel` do?
Once Python is installed, you will also need to install Cython using `pip install cython`.

The `python_wheel` target automates a few steps, required to build the wheel package. You can recreate the process
manually by issuing the following commands:
Build the Python wheel package:

cd "${MY_OPENVINO_BASEDIR}/openvino/ngraph/python"
git clone --branch v2.5.0 https://github.com/pybind/pybind11.git
export NGRAPH_CPP_BUILD_PATH="${MY_OPENVINO_BASEDIR}/openvino_dist"
python3 setup.py bdist_wheel


## Build nGraph Python Wheels on Windows

### Prerequisites
python setup.py bdist_wheel

In order to build OpenVINO and the nGraph Python wheel on Windows, you will need to install Visual Studio and Python.
The final wheel should be located in `ngraph\python\dist` directory.

Once Python is installed, you will also need to install Cython using `pip install cython`.
dir openvino\ngraph\python\dist\
10/09/2020 04:06 PM 4,010,943 ngraph_core-0.0.0-cp38-cp38-win_amd64.whl


### Configure, build and install OpenVINO

Expand Down Expand Up @@ -142,16 +148,6 @@ In order to build and install OpenVINO, build the `install` target:
In this step OpenVINO will be built and installed to the directory specified above. You can
adjust the number of threads used in the building process to your machine's capabilities.

Build the Python wheel package:

cmake --build . --target python_wheel --config Release -j 8

The final wheel should be located in `ngraph\python\dist` directory.

dir openvino\ngraph\python\dist\
10/09/2020 04:06 PM 4,010,943 ngraph_core-0.0.0-cp38-cp38-win_amd64.whl


## Run tests

### Using a virtualenv (optional)
Expand Down
81 changes: 60 additions & 21 deletions ngraph/python/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -18,28 +18,67 @@ cmake_minimum_required (VERSION 3.13)

project (pyngraph)

include(ExternalProject)

ExternalProject_Add(
pybind11
GIT_REPOSITORY "https://github.com/pybind/pybind11.git"
GIT_TAG "v2.5.0"
SOURCE_DIR "${CMAKE_CURRENT_BINARY_DIR}/pybind11"
CONFIGURE_COMMAND ""
BUILD_COMMAND ""
INSTALL_COMMAND ""
)
if(NOT DEFINED IE_MAIN_SOURCE_DIR)
find_package(InferenceEngineDeveloperPackage)
find_package(ngraph REQUIRED)
endif()

if(ngraph_FOUND)
message("ngraph version = {${ngraph_VERSION}}")
endif()

set(BUILD_PY_IN "${CMAKE_CURRENT_SOURCE_DIR}/build_wheel.py.in")
set(BUILD_PY "${CMAKE_CURRENT_BINARY_DIR}/build_wheel.py")
configure_file(${BUILD_PY_IN} ${BUILD_PY} @ONLY)
include(FetchContent)

add_custom_command(
DEPENDS pybind11
OUTPUT ${CMAKE_CURRENT_BINARY_DIR}/dist/
POST_BUILD
WORKING_DIR ${CMAKE_CURRENT_BINARY_DIR}
COMMAND python ${BUILD_PY}
FetchContent_Declare(
pybind11
GIT_REPOSITORY "https://github.com/pybind/pybind11.git"
GIT_TAG "v2.5.0"
)

add_custom_target(python_wheel DEPENDS ngraph ${CMAKE_CURRENT_BINARY_DIR}/dist/)
FetchContent_GetProperties(pybind11)
if(NOT pybind11_POPULATED)
FetchContent_Populate(pybind11)
add_subdirectory(${pybind11_SOURCE_DIR} ${pybind11_BINARY_DIR})
endif()

find_package(Python COMPONENTS Interpreter Development REQUIRED)

if(PYTHON_FOUND)
set(PYTHON_VERSION python${PYTHON_VERSION_MAJOR}.${PYTHON_VERSION_MINOR})
message("Python version={${Python_VERSION}}")
else()
message(FATAL_ERROR "Python was not found!")
endif()

if(CMAKE_COMPILER_IS_GNUCXX)
add_compile_options(-fPIC)
endif()

if (APPLE)
add_link_options(-stdlib=libc++)
endif()

if (CMAKE_CXX_COMPILER_ID STREQUAL "MSVC")
#disable warning: This operator was deprecated and will be removed with v0 operation.
add_compile_options(/wd4996)
endif()

file(GLOB_RECURSE SOURCES src/pyngraph/*.cpp)
pybind11_add_module(_${PROJECT_NAME} MODULE ${SOURCES})
target_include_directories(_${PROJECT_NAME} PRIVATE src)
target_link_libraries(_${PROJECT_NAME} PRIVATE ngraph::ngraph ngraph::onnx_importer)

if(InferenceEngineDeveloperPackage_FOUND)
ie_cpack_add_component(pyngraph_${PYTHON_VERSION})

install(TARGETS _${PROJECT_NAME}
DESTINATION python/${PYTHON_VERSION}
COMPONENT pyngraph_${PYTHON_VERSION})

install(DIRECTORY ${CMAKE_CURRENT_LIST_DIR}/src/ngraph
DESTINATION python/${PYTHON_VERSION}
COMPONENT pyngraph_${PYTHON_VERSION}
USE_SOURCE_PERMISSIONS)

ie_cpack(pyngraph_${PYTHON_VERSION})
endif()
81 changes: 0 additions & 81 deletions ngraph/python/build_wheel.py.in

This file was deleted.

Loading