Some annoyances.. and a wrapper

[mralusw]

Once you get a good hammer, you tend to find more nails around. After running a lot of hyperfine's, here are a few usability sore spots. A tool like this is easiest to use when you can run it in your shell, press <up>, change a few characters, and iterate.

• -r1 could mean "just run the commands". Something doesn't work, I need to hit up, add --show-output and change to -r2. And it still gives a lot of output. Or, I need to delete hyperfine and its options, and stick semicolons between the scripts. -r1 could just do the same thing.
• -S 'zsh --emulate ksh': there's an issue for that (--shell 'busybox sh', 'zsh --emulate posix', 'bash --posix' #410)
• iterate in the shell script, because the benchmarked code is too fast. I'm not sure what a "proper" interface would be, but in my gist I used ---n=varname=5
• more generally adding pre- and post- snippets to each script: can be handled with -L pre single-val -L post single-val, but not conveniently (no commas), and it makes it hard to edit the benchmark command back into a regular shell command. I've used ---pre= and ---post=
• a ~/config/hyperfine.rc file, as well as HYPERFINE_OPTS, would be nice
• an option to print just the summary in console mode, no per-script timings. Those are nasty to edit out from the output; if you pipe hyperfine, by default you lose colors

For now, here's my hyperfine wrapper. It handles -S'shell cmd', -r1, and three-dash extensions ---n=cnt=30, ---pre=, ---post=. Everything else is passed to hyperfine. The -- delimiter between options and scripts is mandatory. It can be used like this:

HYF_WR7PPER_OPTS='-w30'  ./hyf_wr7pper -S'busybox sh' -r1 ---n=k=2 -- 'ls /' 'echo /*'

[sharkdp]

Thank you very much for your report. Could you please remove (or somehow separate or hide) the parts that are duplicates of #375, #410, #247?

iterate in the shell script, because the benchmarked code is too fast. I'm not sure what a "proper" interface would be, but in my gist I used ---n=varname=5

I don't understand?

a ~/config/hyperfine.rc file, as well as HYPERFINE_OPTS, would be nice

What would you like to configure in these files?

[sharkdp]

* but not conveniently (no commas)

not your main point, but you can actually have values with commas. you just need to escape them:

-L string "foo,bar,hello\, world"

[mralusw]

Thank you very much for your report. Could you please remove (or somehow separate or hide) the parts that are duplicates of #375, #410, #247?

I've commented on #375, #410 is my own report, and I don't understand #247

iterate in the shell script, because the benchmarked code is too fast. I'm not sure what a "proper" interface would be, but in my gist I used ---n=varname=5

I don't understand?

For example my wrapper adds a varname=<arg>; while $varname -gt 0; do varname=$(( varname - 1 )); ...; done loop around each user-supplied to-be-benchmarked code (with varname and number of iterations configurable, ---n=varname=INT). Because sometimes each hyperfine iteration is too fast to measure ("less than 5ms"), so you need to loop in the shell code to slow it down. hyf_wr7pper does this via its ---pre= / ---post= options, which are also useful more generally (add pre- / post- snippets to all benchmarked scripts)

a ~/config/hyperfine.rc file, as well as HYPERFINE_OPTS, would be nice

What would you like to configure in these files?

For instance in .rc I predefine some prelude code that can then be easily added to any benchmarked script via a ---pre=$MYPRELUDE hyf_wr7pper option. Or in regular hyperfine, it can be prepended to the benchmarked code.

In OPTS, I usually put -w 2 -M 100. The .rc can also set OPTS unless already defined.

Flexibility :)

[sharkdp]

iterate in the shell script, because the benchmarked code is too fast. I'm not sure what a "proper" interface would be, but in my gist I used ---n=varname=5

I don't understand?

For example my wrapper adds a varname=<arg>; while $varname -gt 0; do varname=$(( varname - 1 )); ...; done loop around each user-supplied to-be-benchmarked code (with varname and number of iterations configurable, ---n=varname=INT). Because sometimes each hyperfine iteration is too fast to measure ("less than 5ms"), so you need to loop in the shell code to slow it down. hyf_wr7pper does this via its ---pre= / ---post= options, which are also useful more generally (add pre- / post- snippets to all benchmarked scripts)

This will be resolved by #336.

a ~/config/hyperfine.rc file, as well as HYPERFINE_OPTS, would be nice

What would you like to configure in these files?

For instance in .rc I predefine some prelude code that can then be easily added to any benchmarked script via a ---pre=$MYPRELUDE hyf_wr7pper option. Or in regular hyperfine, it can be prepended to the benchmarked code.

In OPTS, I usually put -w 2 -M 100. The .rc can also set OPTS unless already defined.

Flexibility :)

I think this can be implemented rather easily with shell scripting by power users. I don't want to implement this in hyperfine itself.

Thank you very much for your report. Could you please remove (or somehow separate or hide) the parts that are duplicates of #375, #410, #247?

I've commented on #375

thanks

I would like to close this. The remaining open points are addressed in other tickets.

[sharkdp]

* `-r1` could mean "just run the commands".

This is now supported in #447