Skip to content

Commit

Permalink
GHI #20 Setup pipelines and presets (#24)
Browse files Browse the repository at this point in the history
- created CMake presets for compiling with the following flags (on supported platforms)
  - ansi
  - coverage
  - warning
  - sanitize
- improved tests:
  - tests now reside in their own `kind` directory
  - tests in CMake now have a label corresponding to their kind (to make it easier to run just one kind of test)
  - added a new kind, System Test (ST), currently used to check that sanitisers work
- add pipeline steps that:
  - run tests on all possible platforms, both natively and with QEMU, check that all tests succeed, and display results in GitHub summary and PR
  - check that we have achieved our minimum coverage goals, and display results in GitHub summary and PR
  - check that there are no fixmes/todos in code without an open GitHub issue linked
- update option variables:
  - `PATOMIC_BUILD_TESTING` now works properly, and depends on `PROJECT_IS_TOP_LEVEL`
  - `PATOMIC_INSTALL_CMAKEDIR` is now versioned
  • Loading branch information
doodspav authored Jan 4, 2024
1 parent e15fe7e commit 1d5b0ec
Show file tree
Hide file tree
Showing 29 changed files with 2,096 additions and 258 deletions.
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
* @doodspav
251 changes: 251 additions & 0 deletions .github/workflows/_reusable-test-native.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,251 @@
name: Run Tests on Native Platforms

on:
workflow_call:
inputs:
os:
description: 'Used to set the runs-on attribute, must be one of macos, ubuntu, windows'
required: true
type: string
default: ''
preset:
description: 'CMake preset used to build patomic and googletest, will be conditionally suffixed with one of ansi, coverage, sanitize, warning'
required: true
type: string
default: ''
architecture:
description: 'CPU architecture tests are run on, used for naming (does not affect how tests are run)'
required: true
type: string
default: ''

jobs:
check:
runs-on: ubuntu-latest

steps:
- name: Check No Empty Inputs
shell: bash
run: |
# checks input is not empty
check_input() {
input_name="${1}"
input_value="${2}"
if [[ -z "${input_value}" ]]; then
echo "Error: required input '${input_name}' is empty or was not specified."
exit 1
else
echo "${input_name}: ${input_value}"
fi
}
# apply function to all inputs
check_input "os" "${{ inputs.os }}"
check_input "preset" "${{ inputs.preset }}"
check_input "architecture" "${{ inputs.architecture }}"
run:
needs: check
runs-on: ${{ inputs.os }}-latest
strategy:
fail-fast: false
matrix:
kind:
- ansi
- coverage
- sanitize
- warning
build_shared:
- static
- shared
include:
# hidden single variable
- compiler: ${{ contains(inputs.preset, 'clang') && 'clang' || ( contains(inputs.preset, 'gcc') && 'gcc' || 'msvc' ) }}
# alias variables
- build_shared: static
cmake_build_shared: OFF
- build_shared: shared
cmake_build_shared: ON
env:
# these have to be in env context because they depend on matrix context
UNIQUE_ID: ${{ matrix.kind }}-${{ inputs.architecture }}-${{ inputs.os }}-${{ matrix.compiler }}-${{ matrix.build_shared }}
UNIQUE_NAME: ${{ inputs.architecture }}-${{ inputs.os }}-${{ matrix.compiler }}-${{ matrix.build_shared }}
CMAKE_PRESET: ${{ inputs.preset }}-${{ matrix.kind }}
CMAKE_BUILD_TYPE: ${{ ( matrix.kind == 'ansi' || matrix.kind == 'warning' ) && 'Release' || 'Debug' }}
# currently not possible to skip job using matrix context, so this is the next best thing
SKIP_JOB: >-
${{ (
( matrix.kind == 'ansi' && matrix.compiler == 'msvc' ) ||
( matrix.kind == 'coverage' && inputs.os == 'windows' ) ||
( matrix.kind == 'sanitize' && inputs.os == 'windows' )
) }}
# this job supports multiple os, each with their own root path
# this screws up lcov when it tries to merge the coverage path mappings
# we can't just modify mappings because:
# - GitHub issue #65006 on llvm/llvm-project suggests to NOT use -fcoverage-prefix-map
# - macos runner has AppleClang 14 which doesn't support -fcoverage-compilation-dir
# workaround is to have a fixed root path that everyone works from
# use /Users as root dir because /home is not supported on macos
ROOT_PATH: /Users/${{ matrix.kind }}

steps:
- name: Set Up Root Path
if: env.SKIP_JOB != 'true'
shell: bash
run: |
# macos and ubuntu need sudo, windows does not
prefix=""
if [[ "${{ inputs.os }}" != "windows" ]]; then
prefix="sudo "
fi
${prefix}mkdir -p "${{ env.ROOT_PATH }}"
${prefix}chmod -R a+rwx "${{ env.ROOT_PATH }}"
- name: Checkout patomic
if: env.SKIP_JOB != 'true'
uses: actions/checkout@v4
with:
# cannot prefix with ROOT_PATH here, so copy over in next step
path: patomic

- name: Move patomic to Root Path Location
if: env.SKIP_JOB != 'true'
run: |
cp -R ./patomic/ "${{ env.ROOT_PATH }}/patomic"
- name: Restore Cached GoogleTest
if: env.SKIP_JOB != 'true'
id: cache-googletest
uses: actions/cache@v3
with:
path: ${{ env.ROOT_PATH }}/googletest/build/install
key: googletest-${{ env.UNIQUE_ID }}

- name: Checkout GoogleTest
if: env.SKIP_JOB != 'true' && steps.cache-googletest.outputs.cache-hit != 'true'
uses: actions/checkout@v4
with:
repository: google/googletest
# cannot prefix with ROOT_PATH here, so copy over in next step
path: googletest

- name: Move GoogleTest to Root Path Location
if: env.SKIP_JOB != 'true' && steps.cache-googletest.outputs.cache-hit != 'true'
run: |
cp -R ./googletest/ "${{ env.ROOT_PATH }}/googletest"
- name: Build and Install GoogleTest
if: env.SKIP_JOB != 'true' && steps.cache-googletest.outputs.cache-hit != 'true'
run: |
cd ${{ env.ROOT_PATH }}
cd googletest
cp ../patomic/CMakePresets.json .
mkdir build
cd build
cmake --preset ${{ env.CMAKE_PRESET }} -DBUILD_TESTING=OFF -DCMAKE_CXX_FLAGS="" -DBUILD_SHARED_LIBS=${{ matrix.cmake_build_shared }} -DCMAKE_BUILD_TYPE=${{ env.CMAKE_BUILD_TYPE }} -Dgtest_force_shared_crt=ON -Dgtest_hide_internal_symbols=ON ..
cmake --build . --verbose --config ${{ env.CMAKE_BUILD_TYPE }}
cmake --install . --config ${{ env.CMAKE_BUILD_TYPE }} --prefix install
# TODO: figure out how to cache this for builds where we don't change any source code (GHI #31)
- name: Build patomic
if: env.SKIP_JOB != 'true'
run: |
cd ${{ env.ROOT_PATH }}
cd patomic
mkdir build
cd build
cmake --preset ${{ env.CMAKE_PRESET }} -DBUILD_SHARED_LIBS=${{ matrix.cmake_build_shared }} -DCMAKE_BUILD_TYPE=${{ env.CMAKE_BUILD_TYPE }} -DGTest_ROOT:PATH="../../googletest/build/install" ..
cmake --build . --verbose --config ${{ env.CMAKE_BUILD_TYPE }}
- name: Test patomic
if: env.SKIP_JOB != 'true'
continue-on-error: true
env:
LABEL_REGEX: ${{ matrix.kind == 'coverage' && '^(ut)$' || '^(.*)$' }}
run: |
cd ${{ env.ROOT_PATH }}
cd patomic/build
ctest --label-regex "${{ env.LABEL_REGEX }}" --verbose --output-junit Testing/Temporary/results.xml --build-config ${{ env.CMAKE_BUILD_TYPE }} .
- name: Prepare Test Results
if: env.SKIP_JOB != 'true'
run: |
cd ${{ env.ROOT_PATH }}
mkdir -p upload/test/${{ matrix.kind }}
python3 patomic/test/improve_ctest_xml.py --input patomic/build/Testing/Temporary/results.xml --triple ${{ matrix.kind }}-${{ env.UNIQUE_NAME }} --output upload/test/${{ matrix.kind }}/${{ env.UNIQUE_NAME }}.xml
- name: Upload Test Results
if: env.SKIP_JOB != 'true'
uses: actions/upload-artifact@v3
with:
name: test-results
path: ${{ env.ROOT_PATH }}/upload/test/

- name: Generate Lcov Tracefile and Root Path File (clang)
if: env.SKIP_JOB != 'true' && matrix.kind == 'coverage' && matrix.compiler == 'clang'
shell: bash
run: |
cd ${{ env.ROOT_PATH }}
# set up directory
mkdir -p upload/cov/${{ env.UNIQUE_NAME }}
cd patomic/build
# macos needs xcrun to help use xcode to run llvm tools
# ubuntu needs llvm tools to be installed
prefix=""
if [[ "${{ inputs.os }}" == "macos" ]]; then
prefix="xcrun "
else # [[ "${{ inputs.os }}" == "ubuntu" ]]; then
sudo apt install llvm
fi
# merge coverage output from all tests
# use find because bash on GitHub Actions currently does not support '**'
find test/working -type f -name "*.profraw" -print0 | xargs -0 ${prefix}llvm-profdata merge -output=patomic.profdata
# convert profdata to lcov tracefile, and copy to upload
lib_files=(libpatomic.*) # matches shared/static lib files and symlinks
${prefix}llvm-cov export -format=lcov -instr-profile=patomic.profdata -object="${lib_files[0]}" >> patomic.lcov
cp patomic.lcov ../../upload/cov/${{ env.UNIQUE_NAME }}/patomic.lcov
# we need original source mapping to make use of lcov tracefile
# we can't just modify mapping because:
# - GitHub issue #65006 on llvm/llvm-project suggests to NOT use -fcoverage-prefix-map
# - macos runner has AppleClang 14 which doesn't support -fcoverage-compilation-dir
# workaround is to store root path to help use coverage directory mappings later
cd ../..
echo "$PWD" >> upload/cov/${{ env.UNIQUE_NAME }}/patomic.rootpath
- name: Generate Lcov Tracefile and Root Path File (gcc)
if: env.SKIP_JOB != 'true' && matrix.kind == 'coverage' && matrix.compiler == 'gcc'
shell: bash
run: |
cd ${{ env.ROOT_PATH }}
# install lcov
sudo apt install lcov
# set up directory
mkdir -p upload/cov/${{ env.UNIQUE_NAME }}
# merge all .gcda files into an lcov tracefile, and copy to upload
# all tests have .gcno, but only tests that were executed have .gcda
lcov --directory --rc lcov_branch_coverage=1 patomic/build --capture --output-file patomic.lcov
cp patomic.lcov upload/cov/${{ env.UNIQUE_NAME }}/patomic.lcov
# we need root path file because next job needs it because clang (above) needs it
# we might also need it for gcc, but at the moment i have no idea
echo "$PWD" >> upload/cov/${{ env.UNIQUE_NAME }}/patomic.rootpath
- name: Upload Internal Coverage Results
if: env.SKIP_JOB != 'true' && matrix.kind == 'coverage'
uses: actions/upload-artifact@v3
with:
name: test-coverage-internal
path: ${{ env.ROOT_PATH }}/upload/cov/
Loading

0 comments on commit 1d5b0ec

Please sign in to comment.