[FeatureRequest] Provide better error reporting during provider verification

[will-banked]

Desired behaviour

As an example, you've written a consumer test that's successfully generated a contract. In the contract, you're sending a POST request with a body, and expecting a 201 back. But when you've then validated against your provider, the service has actually returned back a 422, and the response body has details about what's gone wrong.

However, when this happens, all that Pact reports back is that the actual response body doesn't match the expected response body. There's no way to debug this within Pact itself, or to report on what the error was.

Essentially I'd like some way of being able to do so. I'm not really too fussed around the implementation, but it would be good to either be able to interact directly with the response object in some way during the test or in like an afterEach() hook, or for the VerifyProvider function to return more information in the error from the response of the failed test.

[mefellows]

Thanks. The body should be in the DEBUG level logs, but I think you're asking for it to be made more accessible to the API itself?

[will-banked]

Hmm, I think the logs themselves are pretty challenging to decipher. I'm also not 100% sure that setting the logLevel in the test actually works. I tried setting it to NONE at one point, and it made no obvious difference to the logging. But besides that, the DEBUG logs are filled with stuff, and I have no idea what part of it is meant to reflect the request or response. Having it accessible is preferential for me, because then I can have the logs set to NONE and just get out the information I actually require myself if I ever want to.

[mefellows]

Hmm, I think the logs themselves are pretty challenging to decipher.

Agreed, there is work to be done there. But it sounds like even if we get that right, it won't be enough. I'll leave this open as a request. It's doable, but needs consideration (especially given different content-types, interaction modes etc.).

[will-banked]

Yeah no worries, understand it's a different way of doing things. Cleaning up the logs would definitely go a good way to actually fixing the issue though. Or just having a new log mode that specifically just logged out the request and response and nothing else perhaps?