Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better handling of internal failure modes and fix section bug #81

Merged
merged 24 commits into from
Apr 13, 2023
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
189d55f
Add contribution guidelines
cschreib Apr 9, 2023
fc48052
Use standard interface for default reporter
cschreib Apr 9, 2023
54e06dd
small_function always holds valid function now
cschreib Apr 9, 2023
fb2e783
Updated benchmarks
cschreib Apr 9, 2023
ff8ab07
Apply Lakos Rule for noexcept
cschreib Apr 9, 2023
c5947d9
Update benchmark
cschreib Apr 10, 2023
b8c4215
Add support for non-noexcept small_function
cschreib Apr 10, 2023
babfcbc
Add customizable handler for failed assertions
cschreib Apr 10, 2023
8a3f7fe
Test small_vector error modes
cschreib Apr 10, 2023
d27cdde
Fixed line jump skipped when message is too long
cschreib Apr 10, 2023
a97ac94
Removed inconsistent `noexcept`
cschreib Apr 10, 2023
1443a9f
Updated benchmark
cschreib Apr 10, 2023
509f50a
Remove unnecessary const_cast
cschreib Apr 10, 2023
b564fa3
Add missing tests for coverage
cschreib Apr 10, 2023
662de97
Fixed incorrect link in readme
cschreib Apr 11, 2023
83f003f
Fix typos in docs and add more details
cschreib Apr 11, 2023
f4ee0b3
Remove unused header includes
cschreib Apr 11, 2023
4db0b17
Clarify rules for heap allocations
cschreib Apr 11, 2023
ed8874d
Added more details on current heap allocations on linux gcc
cschreib Apr 12, 2023
581c1b4
Prevent color tags being truncated for #82
cschreib Apr 12, 2023
0954970
Simplify stdout_print using fwrite
cschreib Apr 12, 2023
844ab58
Fix #83
cschreib Apr 13, 2023
38b8c42
More detailed test name for small vector error cases
cschreib Apr 13, 2023
2800a42
Fix readme typos spotted by @tocic
cschreib Apr 13, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 28 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,8 @@ The goal of _snitch_ is to be a simple, cheap, non-invasive, and user-friendly t
- [Exceptions](#exceptions)
- [Header-only build](#header-only-build)
- [`clang-format` support](#clang-format-support)
- [Why _snitch_?](#why-_snitch_)
- [Contributing](#contributing)
- [Why _snitch_?](#why-_snitch_)

<!-- /MarkdownTOC -->

Expand Down Expand Up @@ -240,10 +241,10 @@ Results for Debug builds:
| **Debug** | _snitch_ | _Catch2_ | _doctest_ | _Boost UT_ |
|-----------------|----------|----------|-----------|------------|
| Build framework | 2.0s | 41s | 2.0s | 0s |
| Build tests | 64s | 79s | 73s | 118s |
| Build all | 66s | 120s | 75s | 118s |
| Run tests | 31ms | 76ms | 63ms | 20ms |
| Library size | 3.4MB | 38.6MB | 2.8MB | 0MB |
| Build tests | 65s | 79s | 73s | 118s |
| Build all | 67s | 120s | 75s | 118s |
| Run tests | 34ms | 76ms | 63ms | 20ms |
| Library size | 3.3MB | 38.6MB | 2.8MB | 0MB |
| Executable size | 32.5MB | 49.3MB | 38.6MB | 51.9MB |

Results for Release builds:
Expand All @@ -252,9 +253,9 @@ Results for Release builds:
|-----------------|----------|----------|-----------|------------|
| Build framework | 2.6s | 47s | 3.5s | 0s |
| Build tests | 134s | 254s | 207s | 289s |
| Build all | 136s | 301s | 210s | 289s |
| Build all | 137s | 301s | 210s | 289s |
| Run tests | 24ms | 46ms | 44ms | 5ms |
| Library size | 0.65MB | 2.6MB | 0.39MB | 0MB |
| Library size | 0.63MB | 2.6MB | 0.39MB | 0MB |
| Executable size | 8.9MB | 17.4MB | 15.2MB | 11.3MB |

Notes:
Expand Down Expand Up @@ -939,7 +940,26 @@ IfMacros: ['SECTION', 'SNITCH_SECTION']
SpaceBeforeParens: ControlStatementsExceptControlMacros
```

## Contributing

### Why _snitch_?
Contributions to the source code are always welcome! Simply follow the rules laid out below. If you are not familiar with contributing to an open-source project, feel free to open a [Discussion](https://github.com/cschreib/snitch/discussions/categories/ideas) and ask for guidance. Regardless, you will receive help all the way, and particularly during the code review.

The process:
- If you are considering adding a feature from *Catch2* that *snitch* currently does not support, please check the [*Catch2* support roadmap](doc/comparison_catch2.md) first.
- Please check the [Issue Tracker](https://github.com/cschreib/snitch/issues) for any issue (open or closed) related to the feature you would like to add, or the problem you would like to solve. Read the discussion that has taken place there, if any, and check if any decision was taken that would be incompatible with your planned contribution.
- If the path is clear, fork this repository and commit your changes to your own fork.
- Use "atomic" commits (check that the code compiles before committing) and reasonably clear commit messages (no "WIP"). Linear history is preferred (i.e., avoid merge commits), but will not be enforced.
- Check your code mostly follows the [*snitch* C++ Coding Guidelines](doc/coding_guidelines.md).
- Run `clang-format` on your code before committing. The `.clang-format` file at the root of this repository contains all the formatting rules, and will be used automatically.
- Add tests to cover your new code if applicable (see the `tests` sub-folder).
- Run the *snitch* tests and fix any failure if you can (CMake can run them for you if you ask it to "build" the `snitch_runtime_tests_run` target, otherwise just run manually `build/tests/snitch_runtime_tests`).
- Open a [Pull Request](https://github.com/cschreib/snitch/pulls), with a description of what you are trying to do.
- If there are issues you were unable to solve on your own (e.g., tests failing for reasons you do not understand, or high-impact design decisions), please feel free to open the pull request as a "draft", and highlight the areas that you need help with in the description. Once the issues are addressed, you can take your Pull Request out of draft mode.
- Your code will then be reviewed, and the reviewer(s) may leave comments and suggestions. It is up to you to act on these comments and suggestions, and commit any required code changes. It's OK to push back on a suggestion if you have a good reason; don't always assume the reviewer is right.
- When all comments are addressed, the reviewer(s) should mark the Pull Request as "approved", at which point anyone involved can merge it into the main branch.
- Job done! Congratulations.


## Why _snitch_?

Libraries and programs sometimes do shady or downright illegal stuff (i.e., bugs, crashes). _snitch_ is a library like any other; it may have its own bugs and faults. But it's a snitch! It will tell you when other libraries and programs misbehave, with the hope that you will overlook its own wrongdoings.
53 changes: 53 additions & 0 deletions doc/coding_guidelines.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# Guidelines for writing C++ code for *snitch*

Unless otherwise stated, follow the [C++ Core Guidelines](https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines). Below are exceptions to these guidelines, or more opinionated choices.


## `noexcept`

Follow the [Lakos Rule](https://quuxplusone.github.io/blog/2018/04/25/the-lakos-rule/) (with some tweaks):
- Functions with a wide contract (i.e., with no precondition) should be marked as unconditionally `noexcept`.
- Functions with a narrow contract (i.e., with preconditions), should not be marked as `noexcept`.
- If a template function is conditionally-wide, (e.g., its purpose is only to be a wrapper or adapter over another function), then it may be marked conditionally `noexcept`.

In particular:
- Do not mark a function `noexcept` just because it happens not to throw. The decision should be based on the *interface* of the function (which includes its pre-condition), and not its implementation.

Rationale:
- Easy transition to using contracts when they come to C++.
- Enable testing for pre-condition violations by conditionally throwing.


## Heap allocations

*snitch* code must not directly or indirectly allocate heap (or "free store") memory while running tests. This means that a number of common C++ STL classes cannot be used (at least not with the default allocator):
- `std::string`: use `std::string_view` (for constant strings) or `snitch::small_string` (for variable strings) instead.
- `std::vector`, `std::map`, `std::set`, and their variants: use `std::array` (for fixed size arrays) or `snitch::small_vector` (for variable size arrays) instead.
- `std::function`: use `snitch::small_function` instead.
- `std::unique_ptr`, `std::shared_ptr`: use values on the stack, and raw pointers for non-owning references.

Unfortunately, the standard does not generally specify if a function or class allocates heap memory or not. We can make reasonable guesses for simple cases; in particular the following are fine to use:
- `std::string_view`,
- `std::array`,
- `std::span`,
- `std::variant` with `std::get_if` and `std::visit`,
- `std::optional`,
- `std::tuple`,
- Functions in `<algorithm>` ([except `std::stable_sort`, `std::stable_partition`, and `std::inplace_merge`](https://stackoverflow.com/a/46714875/1565581)).
- Concepts and type traits.

Any type or function not listed above *should* be assumed to use heap memory unless demonstrated otherwise. One way to monitor this is on Linux is to use [valgrind/massif](https://valgrind.org/docs/manual/ms-manual.html).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-One way to monitor this is on Linux is to use
+One way to monitor this on Linux is to use

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good spot, thanks!


Note: If necessary, it is acceptable to allocate/de-allocate heap memory when a test is not running, i.e., either at the start or end of the program, or (if single-threaded) in between test cases. For example, as of writing this, with GCC 10 on Linux and the default reporter, *snitch* performs heap allocations on two occasions:
- several allocations at program startup, generated by [`libstdc++` initialisation](https://gcc.gnu.org/bugzilla/show_bug.cgi?id=68606).
- one at the first console output, generated by `glibc` for its internal I/O buffer.


## Heavy headers and compilation time

One of the advantages of *snitch* over competing testing framework is fast compilation of tests. To preserve this advantage, ["heavy" STL headers](https://artificial-mind.net/projects/compile-health/) should not be included in *snitch* headers unless absolutely necessary. However, they can be included in the *snitch* implementation `*.cpp` files.

Therefore:
- Place as much code as possible in the `*.cpp` files rather than in headers.
- When not possible (templates, constexpr, etc.), consider if you can use a short and clear hand-written alternative instead. For example, `std::max(a, b)` requires `<algorithm>`, but can also be written as `a > b ? a : b`. Some of the simplest algorithms in `<algorithm>`, like `std::copy`, can also be written with an explicit loop.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-and clear hand-written alternative instead
+and clear handwritten alternative instead

ngrams

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The more you know :) Thanks

- Finally, consider if you really need the full feature from the STL, or just a small subset. For example, if you need a metaprogramming type list and don't need to instanciate the types, don't use `std::tuple<...>`: use a custom `template<typename ... Args> struct type_list {}` instead.
Loading