Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhance blacklisting: split out blacklisting based on build failures and benchmarks failures #32

Open
yarikoptic opened this issue Jul 3, 2013 · 0 comments

Comments

@yarikoptic
Copy link
Contributor

I am just judging from the code so I could be wrong:
ATM vbench doesn't differentiate either benchmarking failed due to a build failure or actually benchmarks failures. Those are different since if a revision build fails, then for sure we can blacklist that revision right away. The situation with failing benchmarks is different: may be currently provided benchmarks all fail but some added in the future would succeed (e.g. particularly targeting older API). So IMHO revisions should be blacklisted only if build fails, and then additional blacklisting could be done based on both (rev, benchmark).

Related: If I am correct, as it is now -- build failures are simply ignored -- I am about to submit a PR with small refactoring to not even bother running benchmarks if build failed, and then blacklisting it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant