Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix CI and update readme #184

Merged
merged 4 commits into from
Sep 30, 2024
Merged

Fix CI and update readme #184

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
298f4a4
Update benchmarks
cschreib Sep 30, 2024
6c3ff9e
Fix autolink in readme TOC
cschreib Sep 30, 2024
ca3dc37
Update upload-artifact
cschreib Sep 30, 2024
61855bf
Update version number
cschreib Sep 30, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 17 additions & 17 deletions .github/workflows/cmake.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,22 +89,22 @@ jobs:
fail-fast: false
matrix:
platform:
- { name: Ubuntu GCC, os: ubuntu-latest, publish: true, compiler: g++, arch: "64", build: "ubuntu64-libstdc++", cmakepp: "", flags: "-DCMAKE_CXX_COMPILER=g++ -DCMAKE_CXX_FLAGS='--coverage' --warn-uninitialized -Wdev -Wno-deprecated -Werror=dev"}
- { name: Ubuntu GCC noexcept, os: ubuntu-latest, publish: false, compiler: g++, arch: "64", build: "ubuntu64-libstdc++", cmakepp: "", flags: "-DCMAKE_CXX_COMPILER=g++ -DCMAKE_CXX_FLAGS='-fno-exceptions' --warn-uninitialized -Wdev -Wno-deprecated -Werror=dev"}
- { name: Ubuntu GCC notime, os: ubuntu-latest, publish: false, compiler: g++, arch: "64", build: "ubuntu64-libstdc++", cmakepp: "", flags: "-DCMAKE_CXX_COMPILER=g++ -DSNITCH_WITH_TIMINGS=0 --warn-uninitialized -Wdev -Wno-deprecated -Werror=dev"}
- { name: Ubuntu GCC nounity, os: ubuntu-latest, publish: false, compiler: g++, arch: "64", build: "ubuntu64-libstdc++", cmakepp: "", flags: "-DCMAKE_CXX_COMPILER=g++ -DSNITCH_UNITY_BUILD=0 --warn-uninitialized -Wdev -Wno-deprecated -Werror=dev"}
- { name: Ubuntu GCC header-only, os: ubuntu-latest, publish: true, compiler: g++, arch: "64", build: "header-only", cmakepp: "", flags: "-DCMAKE_CXX_COMPILER=g++ -DSNITCH_HEADER_ONLY=1 --warn-uninitialized -Wdev -Wno-deprecated -Werror=dev"}
- { name: Ubuntu GCC shared, os: ubuntu-latest, publish: false, compiler: g++, arch: "64", build: "ubuntu64-libstdc++", cmakepp: "", flags: "-DCMAKE_CXX_COMPILER=g++ -DBUILD_SHARED_LIBS=1 --warn-uninitialized -Wdev -Wno-deprecated -Werror=dev"}
- { name: Ubuntu Clang, os: ubuntu-latest, publish: true, compiler: clang++, arch: "64", build: "ubuntu64-libc++", cmakepp: "", flags: "-DCMAKE_CXX_COMPILER=clang++ -DCMAKE_CXX_FLAGS='-stdlib=libc++' --warn-uninitialized -Wdev -Wno-deprecated -Werror=dev"}
- { name: Ubuntu Clang noexcept, os: ubuntu-latest, publish: false, compiler: clang++, arch: "64", build: "ubuntu64-libc++", cmakepp: "", flags: "-DCMAKE_CXX_COMPILER=clang++ -DCMAKE_CXX_FLAGS='-stdlib=libc++ -fno-exceptions' --warn-uninitialized -Wdev -Wno-deprecated -Werror=dev"}
- { name: Windows 32, os: windows-latest, publish: true, compiler: vs2019, arch: "32", build: "win32-vs2019", cmakepp: "", flags: "-A Win32 --warn-uninitialized -Wdev -Wno-deprecated -Werror=dev"}
- { name: Windows 64, os: windows-latest, publish: true, compiler: vs2019, arch: "64", build: "win64-vs2019", cmakepp: "", flags: "-A x64 --warn-uninitialized -Wdev -Wno-deprecated -Werror=dev"}
- { name: Windows 64 noexcept, os: windows-latest, publish: false, compiler: vs2019, arch: "64", build: "win64-vs2019", cmakepp: "", flags: "-A x64 --warn-uninitialized -Wdev -Wno-deprecated -Werror=dev"}
- { name: Windows 64 shared, os: windows-latest, publish: false, compiler: vs2019, arch: "64", build: "win64-vs2019", cmakepp: "", flags: "-A x64 -DBUILD_SHARED_LIBS=1 --warn-uninitialized -Wdev -Wno-deprecated -Werror=dev"}
- { name: MacOS, os: macos-latest, publish: true, compiler: clang++, arch: "64", build: "osx-libc++", cmakepp: "", flags: "--warn-uninitialized -Wdev -Wno-deprecated -Werror=dev"}
- { name: MacOS noexcept, os: macos-latest, publish: false, compiler: clang++, arch: "64", build: "osx-libc++", cmakepp: "", flags: "-DCMAKE_CXX_FLAGS='-fno-exceptions' --warn-uninitialized -Wdev -Wno-deprecated -Werror=dev"}
- { name: WebAssembly, os: ubuntu-latest, publish: true, compiler: em++, arch: "32", build: "wasm32", cmakepp: "emcmake", flags: "-DCMAKE_CXX_FLAGS='-s DISABLE_EXCEPTION_CATCHING=0' -DCMAKE_CROSSCOMPILING_EMULATOR=node"}
- { name: WebAssembly noexcept, os: ubuntu-latest, publish: false, compiler: em++, arch: "32", build: "wasm32", cmakepp: "emcmake", flags: "-DCMAKE_CXX_FLAGS='-fno-exceptions' -DCMAKE_CROSSCOMPILING_EMULATOR=node"}
- { name: Ubuntu GCC, os: ubuntu-latest, publish: true, compiler: g++, arch: "64", build: "ubuntu64-libstdc++-static", cmakepp: "", flags: "-DCMAKE_CXX_COMPILER=g++ -DCMAKE_CXX_FLAGS='--coverage' --warn-uninitialized -Wdev -Wno-deprecated -Werror=dev"}
- { name: Ubuntu GCC noexcept, os: ubuntu-latest, publish: false, compiler: g++, arch: "64", build: "ubuntu64-libstdc++-static-noexcept", cmakepp: "", flags: "-DCMAKE_CXX_COMPILER=g++ -DCMAKE_CXX_FLAGS='-fno-exceptions' --warn-uninitialized -Wdev -Wno-deprecated -Werror=dev"}
- { name: Ubuntu GCC notime, os: ubuntu-latest, publish: false, compiler: g++, arch: "64", build: "ubuntu64-libstdc++-static-notime", cmakepp: "", flags: "-DCMAKE_CXX_COMPILER=g++ -DSNITCH_WITH_TIMINGS=0 --warn-uninitialized -Wdev -Wno-deprecated -Werror=dev"}
- { name: Ubuntu GCC nounity, os: ubuntu-latest, publish: false, compiler: g++, arch: "64", build: "ubuntu64-libstdc++-static-nounity", cmakepp: "", flags: "-DCMAKE_CXX_COMPILER=g++ -DSNITCH_UNITY_BUILD=0 --warn-uninitialized -Wdev -Wno-deprecated -Werror=dev"}
- { name: Ubuntu GCC header-only, os: ubuntu-latest, publish: true, compiler: g++, arch: "64", build: "header-only", cmakepp: "", flags: "-DCMAKE_CXX_COMPILER=g++ -DSNITCH_HEADER_ONLY=1 --warn-uninitialized -Wdev -Wno-deprecated -Werror=dev"}
- { name: Ubuntu GCC shared, os: ubuntu-latest, publish: false, compiler: g++, arch: "64", build: "ubuntu64-libstdc++-shared", cmakepp: "", flags: "-DCMAKE_CXX_COMPILER=g++ -DBUILD_SHARED_LIBS=1 --warn-uninitialized -Wdev -Wno-deprecated -Werror=dev"}
- { name: Ubuntu Clang, os: ubuntu-latest, publish: true, compiler: clang++, arch: "64", build: "ubuntu64-libc++-static", cmakepp: "", flags: "-DCMAKE_CXX_COMPILER=clang++ -DCMAKE_CXX_FLAGS='-stdlib=libc++' --warn-uninitialized -Wdev -Wno-deprecated -Werror=dev"}
- { name: Ubuntu Clang noexcept, os: ubuntu-latest, publish: false, compiler: clang++, arch: "64", build: "ubuntu64-libc++-static-noexcept", cmakepp: "", flags: "-DCMAKE_CXX_COMPILER=clang++ -DCMAKE_CXX_FLAGS='-stdlib=libc++ -fno-exceptions' --warn-uninitialized -Wdev -Wno-deprecated -Werror=dev"}
- { name: Windows 32, os: windows-latest, publish: true, compiler: vs2019, arch: "32", build: "win32-vs2019-static", cmakepp: "", flags: "-A Win32 --warn-uninitialized -Wdev -Wno-deprecated -Werror=dev"}
- { name: Windows 64, os: windows-latest, publish: true, compiler: vs2019, arch: "64", build: "win64-vs2019-static", cmakepp: "", flags: "-A x64 --warn-uninitialized -Wdev -Wno-deprecated -Werror=dev"}
- { name: Windows 64 noexcept, os: windows-latest, publish: false, compiler: vs2019, arch: "64", build: "win64-vs2019-static-noexcept", cmakepp: "", flags: "-A x64 --warn-uninitialized -Wdev -Wno-deprecated -Werror=dev"}
- { name: Windows 64 shared, os: windows-latest, publish: false, compiler: vs2019, arch: "64", build: "win64-vs2019-shared", cmakepp: "", flags: "-A x64 -DBUILD_SHARED_LIBS=1 --warn-uninitialized -Wdev -Wno-deprecated -Werror=dev"}
- { name: MacOS, os: macos-latest, publish: true, compiler: clang++, arch: "64", build: "osx-libc++-static", cmakepp: "", flags: "--warn-uninitialized -Wdev -Wno-deprecated -Werror=dev"}
- { name: MacOS noexcept, os: macos-latest, publish: false, compiler: clang++, arch: "64", build: "osx-libc++-static-noexcept", cmakepp: "", flags: "-DCMAKE_CXX_FLAGS='-fno-exceptions' --warn-uninitialized -Wdev -Wno-deprecated -Werror=dev"}
- { name: WebAssembly, os: ubuntu-latest, publish: true, compiler: em++, arch: "32", build: "wasm32", cmakepp: "emcmake", flags: "-DCMAKE_CXX_FLAGS='-s DISABLE_EXCEPTION_CATCHING=0' -DCMAKE_CROSSCOMPILING_EMULATOR=node"}
- { name: WebAssembly noexcept, os: ubuntu-latest, publish: false, compiler: em++, arch: "32", build: "wasm32-noexcept", cmakepp: "emcmake", flags: "-DCMAKE_CXX_FLAGS='-fno-exceptions' -DCMAKE_CROSSCOMPILING_EMULATOR=node"}
build-type:
- Release
- Debug
Expand Down Expand Up @@ -201,7 +201,7 @@ jobs:

- name: Upload build as an artifact
if: matrix.build-type == 'Release' && matrix.platform.publish && github.ref == 'refs/heads/main'
uses: actions/upload-artifact@v2
uses: actions/upload-artifact@v4
with:
name: snitch-build
path: |
Expand Down
2 changes: 1 addition & 1 deletion CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
cmake_minimum_required(VERSION 3.14)

project(snitch LANGUAGES CXX VERSION 1.2.5)
project(snitch LANGUAGES CXX VERSION 1.2.6)

# Maximum lengths.
set(SNITCH_MAX_TEST_CASES 5000 CACHE STRING "Maximum number of test cases in a test application.")
Expand Down
27 changes: 16 additions & 11 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ The goal of _snitch_ is to be a simple, cheap, non-invasive, and user-friendly t
- [Using _snitch_ as a regular library](#using-snitch-as-a-regular-library)
- [Using _snitch_ as a header-only library](#using-snitch-as-a-header-only-library)
- [Example build configuration with meson](#example-build-configuration-with-meson)
- [Example build configuration with vcpkg](https://github.com/snitch-org/snitch/tree/main/doc/vcpkg-example/README.md)
- [Example build configuration with vcpkg](#example-build-configuration-with-vcpkg)
- [Benchmark](#benchmark)
- [Documentation](#documentation)
- [Detailed comparison with _Catch2_](#detailed-comparison-with-catch2)
Expand Down Expand Up @@ -214,6 +214,11 @@ then you can configure with:
And this disables the build step that generates the single-header file "`snitch_all.hpp`".


## Example build configuration with vcpkg

See [the dedicated page in the docs folder](doc/vcpkg-example/README.md).


## Benchmark

The following benchmarks were done using real-world tests from another library ([_observable_unique_ptr_](https://github.com/cschreib/observable_unique_ptr)), which generates about 4000 test cases and 25000 checks. This library uses "typed" tests almost exclusively, where each test case is instantiated several times, each time with a different tested type (here, 25 types). Building and running the tests was done without parallelism to simplify the comparison. The benchmarks were run on a desktop with the following specs:
Expand Down Expand Up @@ -241,23 +246,23 @@ Results for Debug builds:

| **Debug** | _snitch_ | _Catch2_ | _doctest_ | _Boost UT_ |
|-----------------|----------|----------|-----------|------------|
| Build framework | 3.8s | 42s | 2.1s | 0s |
| Build tests | 68s | 75s | 76s | 117s |
| Build all | 72s | 117s | 78s | 117s |
| Run tests | 42ms | 67ms | 63ms | 14ms |
| Library size | 7.6MB | 33.5MB | 2.8MB | 0MB |
| Executable size | 35.7MB | 47.7MB | 38.6MB | 51.8MB |
| Build framework | 4.0s | 42s | 2.1s | 0s |
| Build tests | 71s | 75s | 76s | 117s |
| Build all | 75s | 117s | 78s | 117s |
| Run tests | 44ms | 67ms | 63ms | 14ms |
| Library size | 8.3MB | 33.5MB | 2.8MB | 0MB |
| Executable size | 36.4MB | 47.7MB | 38.6MB | 51.8MB |

Results for Release builds:

| **Release** | _snitch_ | _Catch2_ | _doctest_ | _Boost UT_ |
|-----------------|----------|----------|-----------|------------|
| Build framework | 5.0s | 48s | 3.7s | 0s |
| Build tests | 145s | 233s | 210s | 289s |
| Build all | 150s | 281s | 214s | 289s |
| Build framework | 5.5s | 48s | 3.7s | 0s |
| Build tests | 146s | 233s | 210s | 289s |
| Build all | 152s | 281s | 214s | 289s |
| Run tests | 26ms | 37ms | 42ms | 5ms |
| Library size | 1.3MB | 2.5MB | 0.39MB | 0MB |
| Executable size | 10.0MB | 17.4MB | 15.5MB | 11.4MB |
| Executable size | 10.2MB | 17.4MB | 15.5MB | 11.4MB |

Notes:
- No attempt was made to optimize each framework's configuration; the defaults were used. C++20 modules were not used.
Expand Down
2 changes: 1 addition & 1 deletion meson.build
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
project('snitch', 'cpp',
default_options: ['cpp_std=c++20', 'default_library=static'],
version: '1.2.5'
version: '1.2.6'
)

cpp_arguments = []
Expand Down
Loading