Benchmark downloader script should take the latest score, not the smallest #8707
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Thanks to the observations of @JaroslavTulach, I have noticed that on our benchmark results pages (for both engine and stdlib), we have a lot of nonsensical data.
Problem description
Every benchmark run generates
bench-results.xml
file with a format like this:If the runner is not clean, the
bench-results.xml
file is not rewritten, but rather a new score value is appended, like so:The procedure in the
bench_download.py
script used to choose the lowest of these numbers:https://github.com/enso-org/enso/blob/develop/tools/performance/engine-benchmarks/bench_download.py#L280-L284
This is obviously wrong. This PR fixes this by choosing the latest score, that is the last value.
TL;DR;
All our benchmark data from runners that were not clean are wrong! On this picture, on the right there is the old and wrong value, and on the left is the correct one:
The data was fetched from this snippet: