-
-
Notifications
You must be signed in to change notification settings - Fork 689
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cucumber 7 HTML formatter shows scenario as failed even if it passed in retry. #1533
Comments
Thanks for the detailed report @FrankieDcom - I'll take a look at this. |
I just want to add that the same thing happens if we use builtin message formatter. @FrankieDcom, did you find any workaround for that? |
@thiagojacinto could you please clarify what you mean by that? The |
Sure, @aslakhellesoy. If you follow the same steps described by Frankie to reproduce this but adding the message formatter with the test fails on the first run, but passes on a retry, the |
@thiagojacinto the message formatter only reports status at the step level and doesn't aggregate/summarise - this is down to consuming tools. You'll see all the test step results for every attempt, so when there have been retries you probably just want to disregard the data from previous attempts. This is what we're doing now in |
Hello we have recently upgraded to Cucumber 7, and have started using the out of the box HTML formatter for our test reports. We also have retries enabled on our tests.
We are noticing when a test fails on the first run, but passes on a retry, the HTML formatter incorrectly records it as a failed test, while the summary recording it as a success. With a simple scenario:
Running the following generates the following report and summary:
As you can see, the HTML report displays a failure while the summary correctly shows the scenario as passing with a warning.
The text was updated successfully, but these errors were encountered: