Add option to not print outliers

[LukasKalbertodt]

Quite honestly, basically all of my benchmarks always have a few outliers (only a couple percent of all measurements), usually just "high". Assuming my benchmarks are not totally garbage, those outputs are just noise to me. It would be awesome to be able to tell the benchmarking utility to not print any outliers. Or maybe only print them when there are more than X outliers. Something like that maybe.

[bheisler]

One of the things I'd like to do is find a better way to detect when outlier values are likely to have meaningfully influenced the measurements and only warn if it matters. You're right that they're not very useful as-is.

[LukasKalbertodt]

That, of course, sounds like a much better solution! And it's good to know that many outlier warnings are in fact not very meaningful (I already felt like a criminal for ignoring outliers).

[bheisler]

Note for later: I like how Criterion (the original Haskell benchmark library) displays outliers. Example taken from https://github.com/bennydictor/haskell-vs-rust-regex-benchmark/:

benchmarking ab/1_000
time                 166.6 μs   (138.8 μs .. 198.7 μs)
                     0.903 R²   (0.863 R² .. 1.000 R²)
mean                 143.6 μs   (139.6 μs .. 153.2 μs)
std dev              26.50 μs   (11.82 μs .. 47.24 μs)
variance introduced by outliers: 95% (severely inflated)

I'm not sure how that percentage is calculated, though.