Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] [GHA] Introduce GHA Linux CUDA Pipeline #19884

Merged
Changes from 3 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
a610ea2
add pipeline
akashchi Sep 15, 2023
6c23fca
Merge commit '0247f4a9abe98b5d6cd6b346a15f44123429b451' into ci/gha/i…
akashchi Sep 18, 2023
690a6df
rm triggers
akashchi Sep 18, 2023
f6b094f
Merge commit '233ae78ff4542f08eac242ff6a65ae96a9b83610' into ci/gha/i…
akashchi Sep 19, 2023
fb12c49
address comments
akashchi Sep 19, 2023
0b6e408
use uninteractive as env
akashchi Sep 19, 2023
f494c40
rm triggers
akashchi Sep 19, 2023
2b8b789
rm unused testdata
akashchi Sep 20, 2023
e62f0d2
Merge commit 'a558ebd4bcfee0972659481ed14ba39dc2e22269' into ci/gha/i…
akashchi Sep 20, 2023
bdd0d41
use better concurrency group
akashchi Sep 27, 2023
b02acb3
Merge commit 'd2ba528993a157a4bea6fb5568bf5972b87f1990' into ci/gha/i…
akashchi Sep 28, 2023
dab511a
use aks runner
akashchi Sep 28, 2023
a836275
correct path
akashchi Sep 28, 2023
39b613c
provide path
akashchi Sep 28, 2023
9215ccb
add missing cmake options; rm unnecessary dir creation
akashchi Sep 28, 2023
16daab7
Merge commit 'bcd331d145e5c1c34190d603e554e1dd6e0535e2' into ci/gha/i…
akashchi Oct 3, 2023
be020ab
use image from private docker
akashchi Oct 3, 2023
60f2b75
split OV and plugin cmake & build; do not fail on warning for plugin …
akashchi Oct 3, 2023
be00a97
use different build_dir for nvidia plugin
akashchi Oct 3, 2023
031af59
add missing options
akashchi Oct 3, 2023
f35aba6
rm unnecessary options; add target for build
akashchi Oct 3, 2023
5bad9d2
Apply suggestions from code review
ilya-lavrenov Oct 4, 2023
ec29458
Apply suggestions from code review
ilya-lavrenov Oct 4, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
204 changes: 204 additions & 0 deletions .github/workflows/linux_cuda.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,204 @@
name: Linux CUDA Plugin (Ubuntu 20.04, Python 3.11)
on:
workflow_dispatch:
schedule:
# run daily at 00:00
- cron: '0 0 * * *'
# pull_request:
# paths-ignore:
# - '**/docs/**'
# - 'docs/**'
# - '**/**.md'
# - '**.md'
# - '**/layer_tests_summary/**'
# - '**/conformance/**'
# push:
# paths-ignore:
# - '**/docs/**'
# - 'docs/**'
# - '**/**.md'
# - '**.md'
# - '**/layer_tests_summary/**'
# - '**/conformance/**'
# branches:
# - master
ilya-lavrenov marked this conversation as resolved.
Show resolved Hide resolved

concurrency:
group: ${{ github.head_ref || github.run_id }}-linux-cuda
akashchi marked this conversation as resolved.
Show resolved Hide resolved
cancel-in-progress: true

jobs:
Build:
# TODO: remove. Temporary measure to prevent the workflow from scheduling on forks.
if: ${{ github.repository_owner == 'openvinotoolkit' }}
ilya-lavrenov marked this conversation as resolved.
Show resolved Hide resolved
defaults:
run:
shell: bash
runs-on: ubuntu-20.04-8-cores
ilya-lavrenov marked this conversation as resolved.
Show resolved Hide resolved
container:
image: nvidia/cuda:11.8.0-runtime-ubuntu20.04
ilya-lavrenov marked this conversation as resolved.
Show resolved Hide resolved
env:
CMAKE_BUILD_TYPE: 'Release'
CMAKE_GENERATOR: 'Ninja'
CMAKE_CXX_COMPILER_LAUNCHER: ccache
ilya-lavrenov marked this conversation as resolved.
Show resolved Hide resolved
CMAKE_C_COMPILER_LAUNCHER: ccache
CMAKE_CXX_LINKER_LAUNCHER: ccache
CMAKE_C_LINKER_LAUNCHER: ccache
ilya-lavrenov marked this conversation as resolved.
Show resolved Hide resolved
BUILD_TYPE: 'Release'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's drop, we already have CMAKE_BUILD_TYPE defined to Release

OPENVINO_REPO: ${{ github.workspace }}/openvino
OPENVINO_CONTRIB_REPO: ${{ github.workspace }}/openvino_contrib
BUILD_DIR: ${{ github.workspace }}/build
INSTALL_DIR: ${{ github.workspace }}/install_pkg
DATA_PATH: ${{ github.workspace }}/testdata
MODELS_PATH: ${{ github.workspace }}/testdata
akashchi marked this conversation as resolved.
Show resolved Hide resolved
DEBIAN_FRONTEND: 'noninteractive'
akashchi marked this conversation as resolved.
Show resolved Hide resolved
steps:

- name: Install Prerequisites
run: |
apt update
apt install -y git curl git git-lfs unzip wget
ilya-lavrenov marked this conversation as resolved.
Show resolved Hide resolved

- name: Clone OpenVINO
uses: actions/checkout@v3
with:
path: 'openvino'
submodules: 'recursive'
akashchi marked this conversation as resolved.
Show resolved Hide resolved

- name: Clone OpenVINO Contrib
uses: actions/checkout@v3
with:
repository: 'openvinotoolkit/openvino_contrib'
ilya-lavrenov marked this conversation as resolved.
Show resolved Hide resolved
path: 'openvino_contrib'
submodules: 'recursive'
akashchi marked this conversation as resolved.
Show resolved Hide resolved

- name: Clone testdata for C API tests
uses: actions/checkout@v3
with:
repository: 'openvinotoolkit/testdata'
path: 'testdata'
submodules: 'recursive'
akashchi marked this conversation as resolved.
Show resolved Hide resolved
lfs: 'true'
akashchi marked this conversation as resolved.
Show resolved Hide resolved

- name: Create Directories
run: |
mkdir -p $GITHUB_WORKSPACE/build
mkdir -p $GITHUB_WORKSPACE/install_pkg

- name: Setup Python 3.11
ilya-lavrenov marked this conversation as resolved.
Show resolved Hide resolved
uses: actions/setup-python@v4
with:
python-version: '3.11'

#
# Dependencies
#

- name: Install build dependencies
run: |
apt update
ilya-lavrenov marked this conversation as resolved.
Show resolved Hide resolved
$GITHUB_WORKSPACE/openvino/install_build_dependencies.sh

apt -y --no-install-recommends install unzip wget software-properties-common
wget https://github.com/ninja-build/ninja/releases/download/v1.10.2/ninja-linux.zip
ilya-lavrenov marked this conversation as resolved.
Show resolved Hide resolved
unzip ninja-linux.zip
cp -v ninja /usr/local/bin/

python3 -m pip install -r $GITHUB_WORKSPACE/openvino/src/bindings/python/requirements.txt

- name: Install CUDA
run: |
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-ubuntu2004.pin
mv cuda-ubuntu2004.pin /etc/apt/preferences.d/cuda-repository-pin-600

apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/3bf863cc.pub
add-apt-repository "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/ /"
apt update
DEBIAN_FRONTEND=noninteractive apt install -y \
gnupg2 \
software-properties-common \
libcudnn8=8.9.4.*-1+cuda11.8 \
libcudnn8-dev=8.9.4.*-1+cuda11.8 \
libcudnn8-samples=8.9.4.*-1+cuda11.8 \
autoconf \
automake \
build-essential \
ca-certificates \
clang-format-9 \
cuda-runtime-11-8 \
cuda-11-8 \
g++-multilib \
gcc-multilib \
gstreamer1.0-plugins-base \
libavcodec-dev \
libavformat-dev \
libboost-regex-dev \
libcairo2-dev \
libcutensor1=1.6.1.5-1 \
libcutensor-dev=1.6.1.5-1 \
libglib2.0-dev \
libgstreamer1.0-0 \
libgtk2.0-dev \
libopenblas-dev \
libpango1.0-dev \
libpng-dev \
libssl-dev \
libswscale-dev \
libtool \
libusb-1.0-0-dev \
libzstd-dev \
pkg-config \
shellcheck \
unzip \
wget \
cuda-drivers=520.61.05-1
akashchi marked this conversation as resolved.
Show resolved Hide resolved

- name: Setup ccache
uses: hendrikmuhs/[email protected]
with:
max-size: "2000M"
# Should save cache only if run in the master branch of the base repo
# github.ref_name is 'ref/PR_#' in case of the PR, and 'branch_name' when executed on push
save: ${{ github.ref_name == 'master' && 'true' || 'false' }}
verbose: 2
key: linux-cuda
restore-keys: |
linux-cuda
ilya-lavrenov marked this conversation as resolved.
Show resolved Hide resolved

#
# Build
#

- name: Get number of CPU cores
uses: SimenB/github-actions-cpu-cores@v1
id: cpu-cores
ilya-lavrenov marked this conversation as resolved.
Show resolved Hide resolved

- name: CMake configure
run: |
cmake \
-GNinja \
-DCMAKE_VERBOSE_MAKEFILE=ON \
akashchi marked this conversation as resolved.
Show resolved Hide resolved
-DENABLE_CPPLINT=OFF \
-DCMAKE_BUILD_TYPE=${{ env.BUILD_TYPE }} \
-DOPENVINO_EXTRA_MODULES=$GITHUB_WORKSPACE/openvino_contrib/modules/nvidia_plugin \
-DENABLE_INTEL_CPU=OFF \
-DENABLE_INTEL_GPU=OFF \
-DENABLE_INTEL_GNA=OFF \
-DENABLE_OV_TF_FRONTEND=OFF \
akashchi marked this conversation as resolved.
Show resolved Hide resolved
-DENABLE_OV_PADDLE_FRONTEND=OFF \
-DENABLE_OV_PYTORCH_FRONTEND=OFF \
-DENABLE_OV_ONNX_FRONTEND=OFF \
-DENABLE_PYTHON=OFF \
-DENABLE_TESTS=ON \
akashchi marked this conversation as resolved.
Show resolved Hide resolved
-S $GITHUB_WORKSPACE/openvino \
-B $GITHUB_WORKSPACE/build

- name: Clean ccache stats
run: ccache --zero-stats --show-config
ilya-lavrenov marked this conversation as resolved.
Show resolved Hide resolved

- name: Build
run: |
cmake --build $GITHUB_WORKSPACE/build --parallel ${{ steps.cpu-cores.outputs.count }} --config ${{ env.BUILD_TYPE }} --verbose -- ov_nvidia_func_tests ov_nvidia_unit_tests

- name: Show ccache stats
run: ccache --show-stats