Benchee Warnings

Fast execution warning

This warning occurs when an individual invocation of the function to benchmark takes less than 10μs. This is warning worthy as measurements occur in microseconds and in integers so measuring it would be rather inaccurate. (Measuring in nanoseconds was tried but was found to be much more inaccurate for #reasons).

When this happens Benchee won't measure individual runs but rather run it a couple of times and report the average back. Measures will still be correct, but the overhead of running it n times goes into the measurement. Also statistical results aren't as good, as they are based on averages now. You can see an example of too fast functions in this sample.

Especially memory measurements may be way off due to the overhead introduced be repeating the function call etc.

To remedy the situation, if possible, increase the input size so that an individual run takes more than 10μs.

Also please note that this isn't a limitation specific to Benchee - all benchmarking tools are affected by it as far as I know and if someone comes up with a counter measure then I'm eager to implement it myself :)