Add compiletime benchmarks and fix performance regression #14273
Conversation
mbovel
changed the title
|
test performance please |
|
performance test scheduled: 1 job(s) in queue, 0 running. |
|
Performance test finished successfully: Visit https://dotty-bench.epfl.ch/14273/ to see the changes. Benchmarks is based on merging with master (7939ddb) |
|
test performance with #compiletime-ops please |
|
performance test scheduled: 1 job(s) in queue, 1 running. |
|
test performance with #compiletime please |
|
performance test scheduled: 1 job(s) in queue, 0 running. |
|
Performance test finished successfully: Visit https://dotty-bench.epfl.ch/14273/ to see the changes. Benchmarks is based on merging with master (9e14f5f) |
|
test performance with #compiletime please |
|
performance test scheduled: 1 job(s) in queue, 0 running. |
|
Performance test finished successfully: Visit https://dotty-bench.epfl.ch/14273/ to see the changes. Benchmarks is based on merging with master (9e14f5f) |
|
test performance with #compiletime please |
|
performance test scheduled: 1 job(s) in queue, 0 running. |
|
Performance test finished successfully: Visit https://dotty-bench.epfl.ch/14273/ to see the changes. Benchmarks is based on merging with master (9e14f5f) |
|
test performance with #compiletime please |
|
performance test scheduled: 1 job(s) in queue, 0 running. |
|
Performance test finished successfully: Visit https://dotty-bench.epfl.ch/14273/ to see the changes. Benchmarks is based on merging with master (9e14f5f) |
mbovel
force-pushed
the
mb/compiletime-ops-benchmarks
branch
from
c9ba658 to
0b5005d
Compare
|
test performance with #compiletime please |
|
performance test scheduled: 1 job(s) in queue, 1 running. |
|
Performance test finished successfully: Visit https://dotty-bench.epfl.ch/14273/ to see the changes. Benchmarks is based on merging with master (819c2ff) |
The PR adds 5 new benchmarks aimed at monitoring the performance of compile-time operations (
scala.compiletime.ops.*), more particularly those on integers. This is in prevision of my next PRs that will extend the normalization of these operations.The new benchmarks are generated inside a folder
bench/tests-generated. The resulting files are not committed to avoid adding 1.5 MB of repetitive code to the repository. The reason I made these benchmarks that large is because I wanted runtimes of at least 500 ms so that these are one order of magnitude slower than the empty baseline which runs in ~50 ms.I hesitated to generate these benchmarks using SBT's source generators but I thought that having both the generation code and results directly inside
benchwas better for discoverability and navigation. The chosen design also made it simpler to write the profile file.The idea is to not run these benchmarks automatically on every performance test, but only when specifically asking for them using
test performance with #compiletime-ops please. Running these 5 benchmarks with the suggested configuration takes approximately half an hour.This PR also fixes a performance regression introduced in #13400. The regression is documented in #14259 (comment).