Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add compiler perf scripts #1307

Merged
merged 1 commit into from
Jul 1, 2016
Merged

Conversation

dsyme
Copy link
Contributor

@dsyme dsyme commented Jul 1, 2016

I added some very preliminary scripts for automatically checking the performance of the compiler.

It's very early days on this at the moment. If you can contribute any updates or wisdom to these scripts please do so.

  • Right now the PRs and commits to run the tests against are specified manually, but as you can see from the scripts there is scope to automatically fetch these from GitHub
  • The tests do 3x compilation of a set of largish script files taken from the tests\fsharp directory
  • If these files are changed substantially then the times will change
  • I have a dedicated machine to run these on under my desk MSR Cambridge, so the times in compiler-perf-results.txt are for that machine and that machine only....
  • The end result is a file like this:
url ref  sha                                                                                            build  startup-to-parseonly parseonly-to-check   check-to-nooptimize  check-to-optimize    nooptimize-to-debug
https://github.com/forki/visualfsharp.git foreach_optimization d0ab5fec77482e1280578f47e3257cf660d7f1b2 206.56 11.02                30.55                47.72                62.52                52.96               
https://github.com/Microsoft/visualfsharp master               0247247d480340c27ce7f7de9b2fbc3b7c598b03 208.37 10.80                30.96                46.90                64.63                52.79               
  • From time to time I will commit compiler-perf-results.txt to github to keep a record of where we are at against different commits
  • We expect some natural variation in test times, e.g. it can depend which phase a GC kicks in
  • We could also collect more information like the output of --times which shows memory usage and GCs
  • If anyone could contribute a graphing visualization of the output file (taken directly from github) that would be great

@dsyme dsyme merged commit 9931f8d into dotnet:master Jul 1, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants