-
Notifications
You must be signed in to change notification settings - Fork 26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: Errors thrown no longer fail entire benchmark process #166
Conversation
…marks fix: performance benchmarks failure handling
…marks Feature/collect failing benchmarks
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## main #166 +/- ##
=======================================
Coverage 71.42% 71.42%
=======================================
Files 26 26
Lines 3446 3446
=======================================
Hits 2461 2461
Misses 985 985 see 1 file with indirect coverage changes see 1 file with indirect coverage changes Continue to review full report in Codecov by Sentry.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the PR! Agree it would be nicer to not drop the rest of the results, but the suggested solution would break CI workflows, so needs to be reworked a bit, see comments.
…marks Feature/collect failing benchmarks
…marks feat: refactored error code handling
Co-authored-by: Joakim Hassila <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Super, LGTM.
Thanks for the contribution!
Description
When running benchmarks, if even one failed, we would lose all collected data.
These changes output collected benchmarks and drop the data from failed benchmarks. This helps prevent data loss on benchmarks that aren't failing.
How Has This Been Tested?
Ran locally and checked output
Minimal checklist:
DocC
code-level documentation for any public interfaces exported by the package