Add support for STATIC_CHECK to test compile-time/constexpr expressions

[cschreib]

This has been requested on reddit.

Relevant bit of the discussion copied here:

Having reviewed this some more, Catch2 already defines STATIC_CHECK as just static_assert, and they have a pre-processor value to convert that to a runtime check instead. But they won't ever run both the compile-time and run-time check at the same time. I think it would be confusing to have snitch do both, for this reason.

I am thinking however that we could call it something new and different, like CONSTEXPR_CHECK(expr), which would expand to something equivalent to

CHECK(std::bool_constant<(expr)>::value); // compile-time eval, run-time reporting

and CONSTEXPR_REQUIRE(expr) would be

static_assert(expr); // compile-time eval, compile-time reporting
CHECK(expr); // run-time; for recording success

[cschreib]

After thinking about this some more, I was toying with the idea that:

• CONSTEXPR_CHECK(expr) reports a runtime failure if expr is not constexpr, or if expr evaluates to false, then continues the test
• CONSTEXPR_REQUIRE(expr) reports a runtime failure if expr is not constexpr, or if expr evaluates to false, then stops the test

So essentially identical to CHECK and REQUIRE, but with an extra (compile-time-computed but runtime-reported) check that the expression is a constant expression. I thought this would make the reporting nicer: rather than depend on the compiler-specific error messages generated when trying to use a non-constant expression in a constexpr context, we could output our own nice message. That would come at the expense of delayed reporting (runtime instead of compile time), which I wasn't sure I liked.

Regardless, it turned out to be impossible. There does not seem to exist a generic way to obtain a boolean that is true if a general expression is constexpr and false otherwise. Some solutions can be found here and there, but they are either broken (using the test suite from this solution):

• https://www.reddit.com/r/cpp/comments/7c208c/is_constexpr_a_macro_that_check_if_an_expression/
• https://stackoverflow.com/questions/46919582/is-it-possible-to-test-if-a-constexpr-function-is-evaluated-at-compile-time/46920091#46920091

or too limited (apply to functions only, and/or cannot use constexpr variables in the enclosing scope):

• https://quuxplusone.github.io/blog/2022/01/04/test-constexpr-friendliness/
• https://www.reddit.com/r/cpp/comments/7c208c/comment/dpnh75s/?utm_source=share&utm_medium=web2x&context=3
• https://github.com/coryshrmn/cgs/blob/master/include/cgs/meta/constexpr.hpp

We will probably need to wait for reflection (that must be a meme by now) to be able to do this reliably.

So in the end I'm sticking with the original plan:

• CONSTEXPR_CHECK(expr) reports a compile-time failure if expr is not constexpr, or a runtime failure if expr evaluates to false, then continues the test
• CONSTEXPR_REQUIRE(expr) reports a compile-time failure if expr is not constexpr, or a runtime failure if expr evaluates to false, then stops the test

Messages if the expression is not constexpr don't look the greatest but at least they are straight to the point:

GCC 10:

Clang 10:

MSVC 19.33:

<source>(3601): error C2131: expression did not evaluate to a constant
<source>(3601): note: failure was caused by call of undefined function or one not declared 'constexpr'
<source>(3601): note: see usage of 'test_fun_0::not_fully_constexpr::foo'

And perhaps that is the best way, because the compiler may be able to say exactly what wasn't constexpr in the expression, which snitch definitely can't do.

[cschreib]

I was also considering supporting expressions that evaluate to void in CONSTEXPR_CHECK and CONSTEXPR_REQUIRE. Say you had a constexpr_vector class, you might be tempted to write

CONSTEXPR_CHECK(constexpr_vector{10}.clear());

to test that constexpr_vector::clear() is indeed constexpr, without bothering about the return value of the expression. But this has proven impossible to implement. The expression has type void and this really doesn't play nice with the expression decomposer (can't overload operators with void operands). In principle we could use if constexpr based on the decltype of the expression and select what we want to do, but in practice we're not in a dependent context inside CONSTEXPR_CHECK, i.e., not inside a template, so if constexpr branches that don't compile generate errors even if they are not taken... This issue has been a major pain in implementing snitch, and I don't understand why the standard is written that way.

Anyway, perhaps the best way is to use another check macro CHECK_CONSTEXPR that does just this, since the purpose is different. I'm a bit annoyed at the similarity between CONSTEXPR_CHECK and CHECK_CONSTEXPR though. This is even more confusing because CONSTEXPR_CHECK is actually closer to if constexpr (...), while CHECK_CONSTEXPR is closer to if (constexpr(...)) (assuming we had a constexpr operator like noexcept). I might call this CHECK_IS_CONSTEXPR to avoid this issue, but this deviates from precedents like CHECK_FALSE, which should have been CHECK_IS_FALSE.

Nothing is ever easy! I'll open a ticket for this and decide later.