Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Indicate that keccak benchmark has high max-rss variance #800

Closed
tgnottingham opened this issue Nov 25, 2020 · 2 comments
Closed

Indicate that keccak benchmark has high max-rss variance #800

tgnottingham opened this issue Nov 25, 2020 · 2 comments
Labels
A-benchmark Issues related to benchmarks

Comments

@tgnottingham
Copy link
Contributor

This benchmark regularly varies by +/- 15% max-rss on any given perf run. This should be indicated somehow, so that changes aren't regarded as regressions/improvements when it's just variance. If we don't have per-statistic variance indications, perhaps it would be good to add them.

@Mark-Simulacrum
Copy link
Member

All variance is just client side - it should be fairly straightforward to extend the ?? adding logic to support keccak. It might also be interesting to try to figure out why there's such a large variance.

@rylev rylev added the A-benchmark Issues related to benchmarks label Jul 29, 2021
@tgnottingham tgnottingham changed the title Indicate that keccak benchmark has high max-mss variance Indicate that keccak benchmark has high max-rss variance Aug 15, 2021
@tgnottingham
Copy link
Contributor Author

Since rust-lang/rust#83152, the wide swings in max-rss have disappeared. And we're doing more work to track variance in general now. Closing!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-benchmark Issues related to benchmarks
Projects
None yet
Development

No branches or pull requests

3 participants