Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] [GHA] Introduce Debian 10 ARM #26156

Merged
Merged
Show file tree
Hide file tree
Changes from 40 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
d0fd2dc
start with Docker and wf for Debian 10
akashchi Aug 21, 2024
3091b72
Merge remote-tracking branch 'upstream/master' into ci/gha/introduce-…
akashchi Aug 21, 2024
85d74ed
complete Dockerfile; start rewriting wf
akashchi Aug 21, 2024
dd9a532
adapt wf
akashchi Aug 21, 2024
941f609
update docker tag
akashchi Aug 21, 2024
a6635ed
use docker from ACR
akashchi Aug 21, 2024
0933902
fix ACR
akashchi Aug 21, 2024
963d3d4
install py3.11 as default
akashchi Aug 21, 2024
d57742c
use gcc
akashchi Aug 21, 2024
bdf8098
add missing compilers
akashchi Aug 21, 2024
2d384f0
arm sccache
akashchi Aug 21, 2024
367d9f2
rm unnecessary dependencies
akashchi Aug 22, 2024
6146582
Merge remote-tracking branch 'upstream/master' into ci/gha/introduce-…
akashchi Aug 22, 2024
6ee8f22
provide cmake options via input
akashchi Aug 22, 2024
80b5c72
add line bread
akashchi Aug 22, 2024
73afad4
use reusable job for debian workflow
akashchi Aug 23, 2024
7206307
merge
akashchi Aug 23, 2024
2922f19
use arm machine
akashchi Aug 23, 2024
96c8bd8
add missing Py dep
akashchi Aug 23, 2024
bc3bd23
fix image name
akashchi Aug 23, 2024
c044b29
rm ninja as default
akashchi Aug 23, 2024
a3e8aab
try with another option
akashchi Aug 23, 2024
f123f59
copy cmake options
akashchi Aug 23, 2024
d9104c8
off python
akashchi Aug 23, 2024
c6ab219
add missing space
akashchi Aug 23, 2024
e94086d
Merge branch 'master' into ci/gha/introduce-debian-10-arm
akashchi Aug 27, 2024
8b5ae98
try with system python
akashchi Aug 27, 2024
bc9b0dd
add more packages
akashchi Aug 27, 2024
f1976e5
try with hardcoded cmake arg
akashchi Aug 27, 2024
553ec24
full cmake options via input
akashchi Aug 27, 2024
73118d2
keep newlines
akashchi Aug 27, 2024
6aeeaae
add options to build job
akashchi Aug 27, 2024
d1a176b
rm requirements
akashchi Aug 27, 2024
39b9dd2
rm unnecessary packages
akashchi Aug 27, 2024
8e525bc
Merge branch 'master' into ci/gha/introduce-debian-10-arm
akashchi Aug 27, 2024
4092a38
cpplint
akashchi Aug 27, 2024
c9a918d
Merge branch 'ci/gha/introduce-debian-10-arm' of github.com:akashchi/…
akashchi Aug 27, 2024
a1cbbd5
return packages
akashchi Aug 27, 2024
2658ad6
Merge remote-tracking branch 'upstream/master' into ci/gha/introduce-…
akashchi Aug 28, 2024
e873a15
disable some builds for dpcpp
akashchi Aug 28, 2024
76f9802
add arch to product type
akashchi Aug 28, 2024
ef88aff
rm debug print
akashchi Aug 28, 2024
764ca29
rm comments
akashchi Aug 28, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/dockerfiles/docker_tag
Original file line number Diff line number Diff line change
@@ -1 +1 @@
pr-26067
pr-26156
89 changes: 89 additions & 0 deletions .github/dockerfiles/ov_build/debian_10_arm/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
FROM openvinogithubactions.azurecr.io/dockerio/library/debian:10.13

USER root

# APT configuration
RUN echo 'Acquire::Retries "10";' > /etc/apt/apt.conf && \
echo 'APT::Get::Assume-Yes "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::Fix-Broken "true";' >> /etc/apt/apt.conf && \
echo 'APT::Get::no-install-recommends "true";' >> /etc/apt/apt.conf

ENV DEBIAN_FRONTEND="noninteractive" \
TZ="Europe/London"

RUN apt-get update && \
apt-get install \
software-properties-common \
curl \
git \
gpg-agent \
tzdata \
# Pythons \
python3 \
python3-pip \
python3-dev \
python3-venv \
python3-distutils \
libhdf5-dev \
# For building Python from source
build-essential \
libffi-dev \
libgdbm-dev \
libc6-dev \
libssl-dev \
zlib1g-dev \
libbz2-dev \
libreadline-dev \
libsqlite3-dev \
libncurses5-dev \
libncursesw5-dev \
xz-utils \
tk-dev \
libxml2-dev \
libxmlsec1-dev \
liblzma-dev \
wget \
# Compilers
gcc-arm-linux-gnueabihf \
g++-arm-linux-gnueabihf \
&& \
rm -rf /var/lib/apt/lists/*

# Install build dependencies
ADD install_build_dependencies.sh /install_build_dependencies.sh
RUN chmod +x /install_build_dependencies.sh && \
/install_build_dependencies.sh && \
rm -rf /var/lib/apt/lists/*

# Install sscache
ARG SCCACHE_VERSION="v0.7.5"
ENV SCCACHE_HOME="/opt/sccache" \
SCCACHE_PATH="/opt/sccache/sccache"

RUN mkdir ${SCCACHE_HOME} && cd ${SCCACHE_HOME} && \
SCCACHE_ARCHIVE="sccache-${SCCACHE_VERSION}-aarch64-unknown-linux-musl.tar.gz" && \
curl -SLO https://github.com/mozilla/sccache/releases/download/${SCCACHE_VERSION}/${SCCACHE_ARCHIVE} && \
tar -xzf ${SCCACHE_ARCHIVE} --strip-components=1 && rm ${SCCACHE_ARCHIVE}

ENV PATH="$SCCACHE_HOME:$PATH"

# Setup Python
RUN wget https://www.python.org/ftp/python/3.11.9/Python-3.11.9.tar.xz

RUN tar -xf Python-3.11.9.tar.xz && \
cd Python-3.11.9 && \
./configure --enable-optimizations && \
make -j 8 && \
make altinstall

# Setup pip
ENV PIP_VERSION="24.0"
RUN python3 -m pip install --upgrade pip==24.0
RUN python3.11 -m pip install --upgrade pip==24.0

# Use Python 3.11 as default
# Using venv here because other methods to switch the default Python break both system and wheels build
RUN python3.11 -m venv venv
ENV PATH="/venv/bin:$SCCACHE_HOME:$PATH"

ENV PIP_CACHE_DIR=/mount/caches/pip/linux/${PIP_VERSION}
236 changes: 236 additions & 0 deletions .github/workflows/debian_10_arm.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,236 @@
name: Debian 10 ARM
on:
workflow_dispatch:
pull_request:
merge_group:
push:
branches:
- master
- 'releases/**'

concurrency:
# github.ref is not unique in post-commit
group: ${{ github.event_name == 'push' && github.run_id || github.ref }}-debian-10-arm
cancel-in-progress: true

permissions: read-all

jobs:
Smart_CI:
runs-on: ubuntu-latest
outputs:
affected_components: "${{ steps.smart_ci.outputs.affected_components }}"
changed_components: "${{ steps.smart_ci.outputs.changed_components }}"
skip_workflow: "${{ steps.smart_ci.outputs.skip_workflow }}"
steps:
- name: checkout action
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
with:
sparse-checkout: .github/actions/smart-ci

- name: Get affected components
id: smart_ci
uses: ./.github/actions/smart-ci
with:
repository: ${{ github.repository }}
pr: ${{ github.event.number }}
commit_sha: ${{ github.sha }}
ref_name: ${{ github.ref_name }}
component_pattern: "category: (.*)"
repo_token: ${{ secrets.GITHUB_TOKEN }}
skip_when_only_listed_labels_set: 'docs'
skip_when_only_listed_files_changed: '*.md,*.rst,*.png,*.jpg,*.svg,*/layer_tests_summary/*,*/conformance/*'

- name: Show affected components
run: |
echo "${{ toJSON(steps.smart_ci.outputs.affected_components) }}"
shell: bash

Docker:
needs: Smart_CI
runs-on: aks-linux-16-cores-arm-docker-build
container:
image: openvinogithubactions.azurecr.io/docker_build:0.2
volumes:
- /mount:/mount
outputs:
images: "${{ steps.handle_docker.outputs.images }}"
steps:
- name: Checkout
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7

- uses: ./.github/actions/handle_docker
id: handle_docker
with:
images: |
ov_build/debian_10_arm
registry: 'openvinogithubactions.azurecr.io'
dockerfiles_root_dir: '.github/dockerfiles'
changed_components: ${{ needs.smart_ci.outputs.changed_components }}

Build:
needs: [Docker, Smart_CI]
if: "!needs.smart_ci.outputs.skip_workflow"
uses: ./.github/workflows/job_build_linux.yml
with:
runner: 'aks-linux-16-cores-arm'
container: '{"image": "${{ fromJSON(needs.docker.outputs.images).ov_build.debian_10_arm }}", "volumes": ["/mount:/mount"], "options": "-e SCCACHE_AZURE_BLOB_CONTAINER -e SCCACHE_AZURE_CONNECTION_STRING"}'
affected-components: ${{ needs.smart_ci.outputs.affected_components }}
event-name: ${{ github.event_name }}
os: 'debian_10'
arch: 'arm'
build-js: false
build-debian-packages: false
build-contrib: false
cmake-options: |-
-DCMAKE_TOOLCHAIN_FILE=${OPENVINO_REPO}/cmake/arm.toolchain.cmake \
-DTHREADS_PTHREAD_ARG="-pthread" \
-DCMAKE_COMPILE_WARNING_AS_ERROR=OFF \
-DCMAKE_BUILD_TYPE=Release \
-DENABLE_STRICT_DEPENDENCIES=OFF \
-DENABLE_CPPLINT=OFF \
-DENABLE_NCC_STYLE=OFF \
-DCMAKE_VERBOSE_MAKEFILE=ON \
-DENABLE_CONFORMANCE_PGQL=ON \
-DENABLE_LTO=ON \
-DENABLE_TESTS=ON \
-DENABLE_PYTHON=OFF

# Build:
# needs: [ Smart_CI, Docker ]
# timeout-minutes: 150
# defaults:
# run:
# shell: bash
# runs-on: 'aks-linux-16-cores-arm'
# container:
# image: ${{ fromJSON(needs.docker.outputs.images).ov_build.debian_10 }}
# volumes:
# - /mount:/mount
# options: -e SCCACHE_AZURE_BLOB_CONTAINER -e SCCACHE_AZURE_CONNECTION_STRING
# env:
# CMAKE_BUILD_TYPE: 'Release'
# CMAKE_CXX_COMPILER_LAUNCHER: sccache
# CMAKE_C_COMPILER_LAUNCHER: sccache
# SCCACHE_IGNORE_SERVER_IO_ERROR: 1
# SCCACHE_SERVER_PORT: 35555
# SCCACHE_ERROR_LOG: /__w/openvino/sccache_log.txt
# SCCACHE_LOG: warn
# GITHUB_WORKSPACE: '/__w/openvino/openvino'
# OPENVINO_REPO: /__w/openvino/openvino/openvino
# INSTALL_DIR: /__w/openvino/openvino/openvino_install
# INSTALL_TEST_DIR: /__w/openvino/openvino/tests_install
# BUILD_DIR: /__w/openvino/openvino/openvino_build
# SCCACHE_AZURE_KEY_PREFIX: 'debian_10_arm_Release'
# if: "!needs.smart_ci.outputs.skip_workflow"
#
# steps:
# - name: Clone OpenVINO
# uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
# with:
# path: ${{ env.OPENVINO_REPO }}
# submodules: 'true'
#
# #
# # Print system info
# #
#
# - name: System info
# uses: ./openvino/.github/actions/system_info
#
# #
# # Dependencies
# #
#
# - name: Install python dependencies
# run: |
# python3 -m pip install -r ${OPENVINO_REPO}/tools/mo/requirements.txt
# python3 -m pip install -r ${OPENVINO_REPO}/src/bindings/python/requirements_test.txt
# python3 -m pip install -r ${OPENVINO_REPO}/src/bindings/python/wheel/requirements-dev.txt
#
# #
# # Build
# #
#
# - name: CMake configure - OpenVINO
# run: |
# cmake \
# -DCMAKE_TOOLCHAIN_FILE=${OPENVINO_REPO}/cmake/arm.toolchain.cmake \
# -DTHREADS_PTHREAD_ARG="-pthread" \
# -DCMAKE_COMPILE_WARNING_AS_ERROR_DEFAULT=OFF \
# -DCMAKE_BUILD_TYPE=${{ env.CMAKE_BUILD_TYPE }} \
# -DENABLE_STRICT_DEPENDENCIES=OFF \
# -DCMAKE_VERBOSE_MAKEFILE=ON \
# -DENABLE_CONFORMANCE_PGQL=ON \
# -DENABLE_LTO=ON \
# -DENABLE_TESTS=ON \
# -DENABLE_PYTHON=OFF \
# -DCMAKE_CXX_COMPILER_LAUNCHER=${{ env.CMAKE_CXX_COMPILER_LAUNCHER }} \
# -DCMAKE_C_COMPILER_LAUNCHER=${{ env.CMAKE_C_COMPILER_LAUNCHER }} \
# -S ${OPENVINO_REPO} \
# -B ${BUILD_DIR}
#
# - name: Clean sccache stats
# run: ${SCCACHE_PATH} --zero-stats
#
# - name: Cmake build - OpenVINO
# run: cmake --build ${BUILD_DIR} --parallel --config ${{ env.CMAKE_BUILD_TYPE }}
#
# - name: Show sccache stats
# run: ${SCCACHE_PATH} --show-stats
#
# - name: Cmake install - OpenVINO
# run: |
# cmake -DCMAKE_INSTALL_PREFIX=${INSTALL_DIR} -P ${BUILD_DIR}/cmake_install.cmake
# cmake -DCMAKE_INSTALL_PREFIX=${INSTALL_TEST_DIR} -DCOMPONENT=tests -P ${BUILD_DIR}/cmake_install.cmake
#
# - name: Pack Artifacts
# run: |
# pushd ${INSTALL_DIR}
# tar -czvf ${BUILD_DIR}/openvino_package.tar.gz *
# popd
#
# pushd ${INSTALL_TEST_DIR}
# tar -czvf ${BUILD_DIR}/openvino_tests.tar.gz *
# popd
#
# #
# # Upload build artifacts and logs
# #
# - name: Upload build logs
# uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
# if: always()
# with:
# name: build_logs
# path: ${{ env.SCCACHE_ERROR_LOG }}
# if-no-files-found: 'ignore'
#
# - name: Upload openvino package
# if: ${{ always() }}
# uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
# with:
# name: openvino_package
# path: ${{ env.BUILD_DIR }}/openvino_package.tar.gz
# if-no-files-found: 'error'
#
# - name: Upload openvino tests package
# if: ${{ always() }}
# uses: actions/upload-artifact@0b2256b8c012f0828dc542b3febcab082c67f72b # v4.3.4
# with:
# name: openvino_tests
# path: ${{ env.BUILD_DIR }}/openvino_tests.tar.gz
# if-no-files-found: 'error'

Overall_Status:
name: ci/gha_overall_status_debian_10_arm
needs: [Smart_CI, Build]
if: ${{ always() }}
runs-on: ubuntu-latest
steps:
- name: Check status of all jobs
if: >-
${{
contains(needs.*.result, 'failure') ||
contains(needs.*.result, 'cancelled')
}}
run: exit 1
Loading
Loading