Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Passive upload mode #75

Closed
rrnewton opened this issue Jan 21, 2015 · 2 comments
Closed

Passive upload mode #75

rrnewton opened this issue Jan 21, 2015 · 2 comments
Assignees
@rrnewton
Labels
Improvement

Comments

@rrnewton
Copy link
Member

rrnewton commented Jan 21, 2015 •
edited
Loading

Basically, the proposal is to have a passive mode for the command line hsbencher tool. Instead of running the benchmarks it would just read its stdin for benchmark data and upload it. We already have similar things for uploading from files on disk (e.g. criterion reports)... this would do it live from stdin, something like:

  myscript | hsb upload --fromLog --fusion --codespeed ...

And to do that we'd need expand our vocabulary of "tags" to include some kind of start and end markers:

START_BENCHMARK
PROGNAME: foo
VARIANT:  x
SELFTIMED:  9.99
....
END_BENCHMARK

This would also solve our related problem of wanting to run a job via hsbencher, but report multiple, different benchmark results from a given subprocess execution. (Like an entire criterion suite.)
@benjmart
Copy link

Any updates on this one?

@rrnewton rrnewton self-assigned this Apr 26, 2015
@rrnewton
Copy link
Member Author

It's next on my TODO list. We should get to it soon.

rrnewton referenced this issue Apr 29, 2015
@rrnewton
Major overhaul of how HSBencher runs subprocesses and harvests benchm…
0abf242 
…ark results

The idea is that a single process may now issue START_BENCHMARK/END_BENCHMARK tags
and thus report results for distinct benchmarks.  These "sub-benchmarks" inherit all
the settings from the benchmark config that launched the process, but they may
override those settings by outputting tags.

Multilpe trials still work, and when each trial reports on multiple benchmarks, the
results are "zipped" together.

This is the major change that will enable #73.

Note, this commit creates a large amount of dead code.  These disconnected functions
will be harvested in a dedicated future commit.
@rrnewton rrnewton mentioned this issue Apr 29, 2015
Closed
rrnewton added a commit that referenced this issue Apr 11, 2017
@rrnewton
WIP: getting close to completing #75
b4785d3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rrnewton rrnewton

Labels
Improvement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@rrnewton @benjmart