Ensure that stdlib benchmark reports are collected

[Akirathan]

After #7519 and #7597, let's ensure that the artifacts uploaded by the newly introduced "Standard libraries benchmarks" job are collected and visualized along engine bench results at https://enso-org.github.io/engine-benchmark-results/

Changes to the design of the website

• The Engine bench results and the Stdlib bench results should be visualized on two different pages.
• There are almost 200 stdlib benchmarks, and the number will grow in the future. Let's introduce some kind of side navigation panel to the website for easier navigation.
• Introduce a filtering functionality based on dates for the benchmark results, with a predefined option to filter the last 14 days and last month.

Changes to the bench downloader script

The following are some ideas. Not necessary to implement all of those.

• Remove gh CLI dependency
  • Use some official GH API client instead, e.g., https://pypi.org/project/ghapi/
• Make the script asynchronous
  • Currently, it is synchronous only. And it makes the script very slow, since the number of requests done grow fast.
• Refactor the downloader script to a package and use venv?
  • Easier navigation in IDEs.
  • IDEs should recognize it as a separate Python project.
  • Users should not be forced to install dependencies globally.

Blocked by:

• Ability to invoke all std-benchmarks via JMH #7489
• Create a GitHub action for std-benchmarks #7490

Tasks

Preview Give feedback

1. Ensure that the new job works fine within 2 hours, tweak warmup time if necessary for some benchmarks

[enso-bot]

Pavel Marek reports a new STANDUP for today (2023-08-18):

Progress: - Adding filter by date simple functionality

• Some small design changes to the website It should be finished by 2023-08-25.

[enso-bot]

Pavel Marek reports a new STANDUP for today (2023-08-21):

Progress: - Making sure that the std-libs benchmark job works - #7597

• Fixing warmup and measure configs in benchmark such that the benchmarks are more stable
  • Making sure that the warmup is sufficient.
  • No sophisticated algorithm used, just a rule of thumb - we want the duration of a single invocation to be roughly between 0.5 ms to 100 ms.
• Fix build of the std-benchmarks project - it depends on buildEngineDistribution.
  • Struggled a bit with how to declare this in sbt. It should be finished by 2023-08-25.

Next Day: Next day I will be working on the #7598 task. Will look into the bench results and see if any simple improvements can be made.

[enso-bot]

Pavel Marek reports a new STANDUP for today (2023-08-22):

Progress: - The results seems reasonable: https://github.com/enso-org/enso/actions/runs/5929049160

• Along with the uploaded artifact that has the same format as Engine benchmarks
• Waiting for the approvement from Michal. It should be finished by 2023-08-25.

GitHub

Add std-libs benchmarks GH workflow · enso-org/enso@e8cc78a

Hybrid visual and textual functional programming. Contribute to enso-org/enso development by creating an account on GitHub.

[enso-bot]

Pavel Marek reports a new STANDUP for today (2023-08-23):

Progress: - PR ready for review

• Functional prototype of the new website that contains results for both engine and stdlib benchmarks
• Various improvements to the bench downloader script It should be finished by 2023-08-25.

[enso-bot]

Pavel Marek reports a new 🔴 DELAY for today (2023-09-04):

Summary: There is 14 days delay in implementation of the Ensure that stdlib benchmark reports are collected (#7598) task.
It will cause 14 days delay for the delivery of this weekly plan.

Delay Cause: vacation

[enso-bot]

Pavel Marek reports a new STANDUP for today (2023-09-04):

Progress: - Fixing the script portability.

• Catching on the last week. It should be finished by 2023-09-08.