Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] Added macOS pipeline #7

Merged
merged 7 commits into from
Feb 7, 2024
Merged
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
300 changes: 300 additions & 0 deletions .github/workflows/mac.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,300 @@
name: macOS (12, Python 3.11)
on:
workflow_dispatch:
pull_request:
push:
branches:
- master
- 'releases/**'

concurrency:
# github.ref is not unique in post-commit
group: ${{ github.event_name == 'push' && github.run_id || github.ref }}-mac
cancel-in-progress: true

env:
PYTHON_VERSION: '3.11'

jobs:
openvino_build:
timeout-minutes: 150
defaults:
run:
shell: bash
runs-on: 'macos-13-large'
env:
DEBIAN_FRONTEND: noninteractive # to prevent apt-get from waiting user input
CMAKE_BUILD_TYPE: 'Release'
CMAKE_GENERATOR: 'Ninja Multi-Config'
CMAKE_CXX_COMPILER_LAUNCHER: ccache
CMAKE_C_COMPILER_LAUNCHER: ccache
OPENVINO_REPO: ${{ github.workspace }}/openvino
INSTALL_DIR: ${{ github.workspace }}/openvino/install
BUILD_DIR: ${{ github.workspace }}/openvino/build

steps:
- name: Set apt retries
run: echo 'Acquire::Retries "10";' | sudo tee -a /etc/apt/apt.conf.d/80-retries > /dev/null

- name: Install git
run: |
sudo apt-get update
sudo apt-get install --assume-yes --no-install-recommends git ca-certificates

- name: Clone OpenVINO
uses: actions/checkout@v4
with:
repository: 'openvinotoolkit/openvino'
path: ${{ env.OPENVINO_REPO }}
submodules: 'true'
ref: 'master'

#
# Dependencies
#

- name: Install build dependencies
run: |
sudo -E ${OPENVINO_REPO}/install_build_dependencies.sh

- name: Setup Python ${{ env.PYTHON_VERSION }}
uses: actions/setup-python@v5
with:
python-version: ${{ env.PYTHON_VERSION }}
cache: 'pip'

- name: Install python dependencies
run: |
# For Python API: build and wheel packaging
python3 -m pip install -r ${OPENVINO_REPO}/src/bindings/python/wheel/requirements-dev.txt

#
# Build
#

- name: Setup ccache
uses: hendrikmuhs/[email protected]
with:
max-size: "2000M"
# Should save cache only if run in the master branch of the base repo
# github.ref_name is 'ref/PR_#' in case of the PR, and 'branch_name' when executed on push
save: ${{ github.ref_name == 'master' && 'true' || 'false' }}
verbose: 2
key: linux-ubuntu
restore-keys: |
linux-ubuntu

- name: CMake configure - OpenVINO
run: |
cmake \
-G "${{ env.CMAKE_GENERATOR }}" \
-DENABLE_CPPLINT=OFF \
-DENABLE_NCC_STYLE=OFF \
-DENABLE_TESTS=OFF \
-DENABLE_STRICT_DEPENDENCIES=OFF \
-DENABLE_SYSTEM_TBB=ON \
-DENABLE_SYSTEM_OPENCL=ON \
-DCMAKE_VERBOSE_MAKEFILE=ON \
-DCPACK_GENERATOR=TGZ \
-DENABLE_JS=OFF \
-DENABLE_SAMPLES=OFF \
-DENABLE_OV_ONNX_FRONTEND=OFF \
-DENABLE_OV_PADDLE_FRONTEND=OFF \
-DENABLE_OV_PYTORCH_FRONTEND=OFF \
-DENABLE_OV_TF_FRONTEND=OFF \
-DENABLE_OV_TF_LITE_FRONTEND=OFF \
-DENABLE_INTEL_GPU=OFF \
-DCMAKE_COMPILE_WARNING_AS_ERROR=ON \
-DCMAKE_CXX_COMPILER_LAUNCHER=ccache \
-DCMAKE_C_COMPILER_LAUNCHER=ccache \
-S ${OPENVINO_REPO} \
-B ${BUILD_DIR}

- name: Clean ccache stats
run: ccache --zero-stats --show-config

- name: Cmake build - OpenVINO
run: cmake --build ${BUILD_DIR} --parallel --config ${{ env.CMAKE_BUILD_TYPE }}

- name: Show ccache stats
run: ccache --show-stats

- name: Cmake install - OpenVINO
run: |
cmake -DCMAKE_INSTALL_PREFIX=${INSTALL_DIR} -P ${BUILD_DIR}/cmake_install.cmake
cmake -DCMAKE_INSTALL_PREFIX=${INSTALL_DIR} -DCOMPONENT=python_wheels -P ${BUILD_DIR}/cmake_install.cmake

- name: Pack Artifacts
run: |
pushd ${INSTALL_DIR}
tar -czvf ${BUILD_DIR}/openvino_package.tar.gz *
popd

#
# Upload build artifacts and logs
#

- name: Upload openvino package
if: ${{ always() }}
uses: actions/upload-artifact@v3
with:
name: openvino_package
path: ${{ env.BUILD_DIR }}/openvino_package.tar.gz
if-no-files-found: 'error'

openvino_tokenizers:
name: OpenVINO tokenizers extension
needs: [ openvino_build ]
timeout-minutes: 25
defaults:
run:
shell: bash
runs-on: macos-13

env:
OPENVINO_REPO: ${{ github.workspace }}/openvino
INSTALL_DIR: ${{ github.workspace }}/openvino/install
OPENVINO_TOKENIZERS_REPO: ${{ github.workspace }}/openvino_tokenizers
BUILD_DIR: ${{ github.workspace }}/openvino_tokenizers/build

steps:
- name: Clone Openvino tokenizers
uses: actions/checkout@v4
with:
path: ${{ env.OPENVINO_TOKENIZERS_REPO }}

- name: Clone Openvino
uses: actions/checkout@v4
with:
repository: 'openvinotoolkit/openvino'
path: ${{ env.OPENVINO_REPO }}
ref: 'master'
sparse-checkout: |
install_build_dependencies.sh

- name: Setup Python ${{ env.PYTHON_VERSION }}
uses: actions/setup-python@v5
with:
python-version: ${{ env.PYTHON_VERSION }}
cache: 'pip'

- name: Download OpenVINO package
uses: actions/download-artifact@v3
with:
name: openvino_package
path: ${{ env.INSTALL_DIR }}

- name: Extract OpenVINO packages
run: |
pushd ${INSTALL_DIR}
tar -xzf openvino_package.tar.gz -C ${INSTALL_DIR}
popd

#
# Dependencies
#

- name: Install build dependencies
run: sudo ${{ env.OPENVINO_REPO }}/install_build_dependencies.sh

- name: Install python dependencies
run: |
# wheel packaging
python3 -m pip install -r ${OPENVINO_TOKENIZERS_REPO}/requirements-build.txt

#
# Build
#

- name: Build tokenizers wheel
run: |
source ${INSTALL_DIR}/setupvars.sh
python -m build --wheel --outdir ${BUILD_DIR} ${OPENVINO_TOKENIZERS_REPO}
env:
CMAKE_ARGS: '-DBUILD_FAST_TOKENIZERS=OFF'
CMAKE_BUILD_PARALLEL_LEVEL: '4'
CMAKE_GENERATOR: 'Unix Makefiles'
SKBUILD_WHEEL_BUILD_TAG: ${{github.run_attempt}}

#
# Upload build artifacts
#

- name: Upload openvino tokenizers wheel
if: ${{ always() }}
uses: actions/upload-artifact@v3
with:
name: openvino_tokenizers_wheel
path: ${{ env.BUILD_DIR }}/*.whl
if-no-files-found: 'error'

openvino_tokenizers_tests:
name: OpenVINO tokenizers tests
needs: [ openvino_tokenizers ]
timeout-minutes: 25
defaults:
run:
shell: bash
runs-on: macos-13

env:
OPENVINO_REPO: ${{ github.workspace }}/openvino
INSTALL_DIR: ${{ github.workspace }}/openvino/install
OPENVINO_TOKENIZERS_REPO: ${{ github.workspace }}/openvino_tokenizers
BUILD_DIR: ${{ github.workspace }}/openvino_tokenizers/build

steps:
- name: Clone Openvino tokenizers sources and tests
uses: actions/checkout@v4
with:
path: ${{ env.OPENVINO_TOKENIZERS_REPO }}

- name: Setup Python ${{ env.PYTHON_VERSION }}
uses: actions/setup-python@v5
with:
python-version: ${{ env.PYTHON_VERSION }}
cache: 'pip'

- name: Download tokenizers package
uses: actions/download-artifact@v3
with:
name: openvino_tokenizers_wheel
path: ${{ env.INSTALL_DIR }}/ov_tokenizers

- name: Download OpenVINO package
uses: actions/download-artifact@v3
with:
name: openvino_package
path: ${{ env.INSTALL_DIR }}

- name: Extract OpenVINO packages
run: |
pushd ${INSTALL_DIR}
tar -xzf openvino_package.tar.gz -C ${INSTALL_DIR}
popd

- name: Install OpenVINO Python wheel
run: |
# Find and install wheel
pushd ${INSTALL_DIR}/tools
wheel_name=$(find . -name 'openvino-*.whl')
python3 -m pip install $wheel_name
popd

- name: Install OpenVINO tokenizers wheel
run: |
# Find and install wheel
pushd ${INSTALL_DIR}/ov_tokenizers
wheel_name=$(find . -name 'openvino_tokenizers*.whl')
python3 -m pip install $wheel_name[dev]
popd

- name: Tokenizers Bandit tests
run: |
bandit -c pyproject.toml -r python
working-directory: ${{ env.OPENVINO_TOKENIZERS_REPO }}

- name: Tokenizers regression tests
run: |
python3 -m pytest tokenizers_test.py
working-directory: ${{ env.OPENVINO_TOKENIZERS_REPO }}/tests
mryzhov marked this conversation as resolved.
Show resolved Hide resolved
Loading