Have sh_test directly invoke benchmarks #4552
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
These tests typically take 10-20s, but I'm seeing some timeouts here. This seemed particularly suspicious due to the absence of output (copied below). That got me looking, and maybe the subprocessing tickles a cpu bottleneck, so proposing this approach to remove the exec. Even if this doesn't solve the flakiness, I think it's a simpler implementation.
Note I believe this is intended to work. The
sh
rules rely on shebangs (as noted at https://bazel.build/reference/be/shell#sh_test), and are essentially just subprocessing to the input. Note this could've also hadargs
on acc_test
rule, but I'd expect the same args to be passed torun
where instead the benchmark behavior should be default (and I'm assuming you'd rather not have args there). Fundamentally this becomes a symlink:Copying snippet from timeout below: