Skip to content

Commit

Permalink
Merge pull request #15 from cschreib/update
Browse files Browse the repository at this point in the history 
Fix check operands evaluated twice
  • Loading branch information
@cschreib
cschreib authored Nov 2, 2022
2 parents 6df0aad + ac7178d commit c008400
Show file tree
Hide file tree
Showing 3 changed files with 167 additions and 59 deletions.
48 changes: 24 additions & 24 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ If you need features that are not in the list above, please use _Catch2_ or _doc
Notable current limitations:

- Test macros (`REQUIRE(...)`, etc.) may only be used inside the test body (or in lambdas defined in the test body), and cannot be used in other functions.
- No set-up/tear-down helpers.
- No fixtures, or set-up/tear-down helpers (`SECTION()` can be used to share set-up/tear-down logic).
- No multi-threaded test execution.


Expand Down Expand Up @@ -108,33 +108,33 @@ The following benchmarks were done using real-world tests from another library (

Description of results below:
- *Build framework*: Time required to build the testing framework library (if any), without any test.
- *Build tests*: Time required to build the tests, assuming the framework library was already built (if any).
- *Build tests*: Time required to build the tests, assuming the framework library (if any) was already built.
- *Build all*: Total time to build the tests and the framework library (if any).
- *Run tests*: Total time require to run the tests.
- *Run tests*: Total time required to run the tests.
- *Library size*: Size of the compiled testing framework library (if any).
- *Executable size*: Size of the compiled test executable, static linking to the testing framework library (if any).

Results for _snatch_:

| | _snatch_ (Debug) | _snatch_ (Release) |
|-----------------|------------------|--------------------|
| Build framework | 1.7s | 2.5s |
| Build tests | 64s | 131s |
| Build all | 65s | 134s |
| Run tests | 16ms | 8ms |
| Library size | 2.70MB | 0.60MB |
| Executable size | 29.5MB | 8.5MB |

Results for alternative testing frameworks:

| | _Catch2_ (Debug) | _Catch2_ (Release) | _doctest_ (Debug) | _doctest_ (Release) | _Boost UT_ (Debug) | _Boost UT_ (Release) |
|-----------------|------------------|--------------------|-------------------|---------------------|--------------------|----------------------|
| Build framework | 41s | 48s | 2.4s | 4.1s | 0s | 0s |
| Build tests | 86s | 310s | 76s | 208s | 113s | 279s |
| Build all | 127s | 358s | 78s | 212s | 113s | 279s |
| Run tests | 74ms | 36ms | 59ms | 35ms | 20ms | 10ms |
| Library size | 34.6MB | 2.5MB | 2.8MB | 0.39MB | 0MB | 0MB |
| Executable size | 51.5MB | 19.1MB | 38.6MB | 15.2MB | 51.7MB | 11.3MB |
Results for Debug builds:

| | _snatch_ (Debug) | _Catch2_ (Debug) | _doctest_ (Debug) | _Boost UT_ (Debug) |
|-----------------|--------------------|--------------------|---------------------|---------------------|
| Build framework | 1.7s | 41s | 2.4s | 0s |
| Build tests | 68s | 86s | 76s | 113s |
| Build all | 70s | 127s | 78s | 113s |
| Run tests | 17ms | 74ms | 59ms | 20ms |
| Library size | 2.70MB | 34.6MB | 2.8MB | 0MB |
| Executable size | 31.4MB | 51.5MB | 38.6MB | 51.7MB |

Results for Release builds:

| | _snatch_ (Release) | _Catch2_ (Release) | _doctest_ (Release)| _Boost UT_ (Release) |
|-----------------|--------------------|--------------------|--------------------|----------------------|
| Build framework | 2.4s | 48s | 4.1s | 0s |
| Build tests | 140s | 310s | 208s | 279s |
| Build all | 142s | 358s | 212s | 279s |
| Run tests | 9ms | 36ms | 35ms | 10ms |
| Library size | 0.60MB | 2.5MB | 0.39MB | 0MB |
| Executable size | 9.3MB | 19.1MB | 15.2MB | 11.3MB |

Notes:
- No attempt was made to optimize each framework's configuration; the defaults were used. C++20 modules were not used.
Expand Down
166 changes: 137 additions & 29 deletions include/snatch/snatch.hpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -773,43 +773,149 @@ struct section_entry_checker {
explicit operator bool() noexcept;
};

struct operator_less {
static constexpr std::string_view inverse = " >= ";
template<typename T, typename U>
constexpr bool operator()(const T& lhs, const U& rhs) const noexcept {
return lhs < rhs;
}
};

struct operator_greater {
static constexpr std::string_view inverse = " <= ";
template<typename T, typename U>
constexpr bool operator()(const T& lhs, const U& rhs) const noexcept {
return lhs > rhs;
}
};

struct operator_less_equal {
static constexpr std::string_view inverse = " > ";
template<typename T, typename U>
constexpr bool operator()(const T& lhs, const U& rhs) const noexcept {
return lhs <= rhs;
}
};

struct operator_greater_equal {
static constexpr std::string_view inverse = " < ";
template<typename T, typename U>
constexpr bool operator()(const T& lhs, const U& rhs) const noexcept {
return lhs >= rhs;
}
};

struct operator_equal {
static constexpr std::string_view inverse = " != ";
template<typename T, typename U>
constexpr bool operator()(const T& lhs, const U& rhs) const noexcept {
return lhs == rhs;
}
};

struct operator_not_equal {
static constexpr std::string_view inverse = " == ";
template<typename T, typename U>
constexpr bool operator()(const T& lhs, const U& rhs) const noexcept {
return lhs != rhs;
}
};

struct expression {
std::string_view content;
small_string<max_expr_length> data;
bool failed = false;
std::string_view expected;
small_string<max_expr_length> actual;

template<string_appendable T>
void append(T&& value) noexcept {
if (!snatch::append(data, std::forward<T>(value))) {
failed = true;
}
[[nodiscard]] bool append(T&& value) noexcept {
return snatch::append(actual, std::forward<T>(value));
}

template<typename T>
void append(T&&) noexcept {
if (!snatch::append(data, "?")) {
failed = true;
[[nodiscard]] bool append(T&&) noexcept {
return snatch::append(actual, "?");
}
};

template<typename T, typename O, typename U>
struct extracted_binary_expression {
expression& expr;
const T& lhs;
const U& rhs;

#define EXPR_OPERATOR(OP) \
template<typename V> \
bool operator OP(const V&) noexcept { \
static_assert( \
!std::is_same_v<V, V>, \
"cannot chain expression in this way; please decompose it into multiple checks"); \
return false; \
}

EXPR_OPERATOR(<=)
EXPR_OPERATOR(<)
EXPR_OPERATOR(>=)
EXPR_OPERATOR(>)
EXPR_OPERATOR(==)
EXPR_OPERATOR(!=)
EXPR_OPERATOR(&&)
EXPR_OPERATOR(||)

#undef EXPR_OPERATOR

explicit operator bool() const noexcept {
if (!O{}(lhs, rhs)) {
if (!expr.append(lhs) || !expr.append(O::inverse) || !expr.append(rhs)) {
expr.actual.clear();
}

return true;
}

return false;
}
};

#define EXPR_OPERATOR(OP, INVERSE_OP) \
template<typename T> \
expression& operator OP(const T& value) noexcept { \
if (!data.empty()) { \
append(" " #INVERSE_OP " "); \
} \
append(value); \
return *this; \
template<typename T>
struct extracted_unary_expression {
expression& expr;
const T& lhs;

#define EXPR_OPERATOR(OP, OP_TYPE) \
template<typename U> \
constexpr extracted_binary_expression<T, OP_TYPE, U> operator OP(const U& rhs) \
const noexcept { \
return {expr, lhs, rhs}; \
}

EXPR_OPERATOR(<=, >)
EXPR_OPERATOR(<, >=)
EXPR_OPERATOR(>=, <)
EXPR_OPERATOR(>, <=)
EXPR_OPERATOR(==, !=)
EXPR_OPERATOR(!=, ==)
EXPR_OPERATOR(<, operator_less)
EXPR_OPERATOR(>, operator_greater)
EXPR_OPERATOR(<=, operator_less_equal)
EXPR_OPERATOR(>=, operator_greater_equal)
EXPR_OPERATOR(==, operator_equal)
EXPR_OPERATOR(!=, operator_not_equal)

#undef EXPR_OPERATOR

explicit operator bool() const noexcept {
if (!static_cast<bool>(lhs)) {
if (!expr.append(lhs)) {
expr.actual.clear();
}

return true;
}

return false;
}
};

struct expression_extractor {
expression& expr;

template<typename T>
constexpr extracted_unary_expression<T> operator<=(const T& lhs) const noexcept {
return {expr, lhs};
}
};

struct scoped_capture {
Expand Down Expand Up @@ -1115,7 +1221,9 @@ struct with_what_contains : private contains_substring {
#define SNATCH_MACRO_CONCAT(x, y) SNATCH_CONCAT_IMPL(x, y)
#define SNATCH_MACRO_DISPATCH2(_1, _2, NAME, ...) NAME

#define SNATCH_EXPR(type, x) snatch::impl::expression{type "(" #x ")", {}, false} <= x
#define SNATCH_EXPR(TYPE, EXP) \
auto SNATCH_CURRENT_EXPRESSION = snatch::impl::expression{TYPE "(" #EXP ")", {}}; \
snatch::impl::expression_extractor{SNATCH_CURRENT_EXPRESSION} <= EXP

// Public test macros.
// -------------------
Expand Down Expand Up @@ -1155,9 +1263,9 @@ struct with_what_contains : private contains_substring {
SNATCH_WARNING_PUSH \
SNATCH_WARNING_DISABLE_PARENTHESES \
SNATCH_WARNING_DISABLE_CONSTANT_COMPARISON \
if (!(EXP)) { \
if (SNATCH_EXPR("REQUIRE", EXP)) { \
snatch::tests.report_failure( \
SNATCH_CURRENT_TEST, {__FILE__, __LINE__}, SNATCH_EXPR("REQUIRE", EXP)); \
SNATCH_CURRENT_TEST, {__FILE__, __LINE__}, SNATCH_CURRENT_EXPRESSION); \
SNATCH_TESTING_ABORT; \
} \
SNATCH_WARNING_POP \
Expand All @@ -1169,9 +1277,9 @@ struct with_what_contains : private contains_substring {
SNATCH_WARNING_PUSH \
SNATCH_WARNING_DISABLE_PARENTHESES \
SNATCH_WARNING_DISABLE_CONSTANT_COMPARISON \
if (!(EXP)) { \
if (SNATCH_EXPR("CHECK", EXP)) { \
snatch::tests.report_failure( \
SNATCH_CURRENT_TEST, {__FILE__, __LINE__}, SNATCH_EXPR("CHECK", EXP)); \
SNATCH_CURRENT_TEST, {__FILE__, __LINE__}, SNATCH_CURRENT_EXPRESSION); \
} \
SNATCH_WARNING_POP \
} while (0)
Expand Down
12 changes: 6 additions & 6 deletions src/snatch.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -586,10 +586,10 @@ void registry::print_details(std::string_view message) const noexcept {
}

void registry::print_details_expr(const expression& exp) const noexcept {
print(" ", make_colored(exp.content, with_color, color::highlight2));
print(" ", make_colored(exp.expected, with_color, color::highlight2));

if (!exp.failed) {
print(", got ", make_colored(exp.data, with_color, color::highlight2));
if (!exp.actual.empty()) {
print(", got ", make_colored(exp.actual, with_color, color::highlight2));
}

print("\n");
Expand Down Expand Up @@ -648,9 +648,9 @@ void registry::report_failure(

if (!report_callback.empty()) {
const auto captures_buffer = make_capture_buffer(state.captures);
if (!exp.failed) {
if (!exp.actual.empty()) {
small_string<max_message_length> message;
append_or_truncate(message, exp.content, ", got ", exp.data);
append_or_truncate(message, exp.expected, ", got ", exp.actual);
report_callback(
*this, event::assertion_failed{
state.test.id, state.sections.current_section, captures_buffer.span(),
Expand All @@ -662,7 +662,7 @@ void registry::report_failure(
state.sections.current_section,
captures_buffer.span(),
location,
{exp.content}});
{exp.expected}});
}
} else {
print_failure();
Expand Down

0 comments on commit c008400

Please sign in to comment.