Don't show test output unless tests fail

[brendanlong]

Our code does a lot of logging, which makes it really hard to quickly tell which test failed when looking at our test output. Is there a way ppx_inline_test could hide stdout unless a test fails?

I looked into implementing this myself with Runtime.am_running_inline_test, but I would also need a way to know when a test starts and stops, and also know whether it passed or failed.

(One option is to get rid of all output when tests are running, but then it's really difficult to debug failing tests)

I could also implement my own test runner, but it's not clear to me what I would need to do there since the built-in runner just calls Ppx_inline_test_lib.Runtime.exit ().

[ghost]

I have you tried ppx_expect? The experience is nicer in general for this kind of things.

[brendanlong]

We use it for some things, but I don't think we would want logging to show up in our expect tests.

Right now what we've done to work around this is:

• We had some tests that really don't belong inline, so we moved them to non-inline tests
• I created a wrapper function Logger.in_test that redirects output and re-prints it if a test fails. The main annoyance here is that we have to add it to the top of every test that could potentially log something

[ghost]

Ok. ppx_expect has special support for configuring the test harness. For instance we use that to write expectation tests using the Async library. This is done by tweaking the Expect_test_config module that's in scope:

module Expect_test_config = struct
  include Expect_test_config

  let run = ...
end

Modifying the run function might be enough for your use case. This is the API of the module: https://github.com/janestreet/ppx_expect/blob/master/config/expect_test_config.mli

[brendanlong]

Ah cool, I'll take a look!