Time full program runtime

[ElliottKasoar]

Changes to timing method to resolve #33

• The most significant change is to use /usr/bin/time to time the full runtime of each benchmark program (in addition to date as an extra check), for a more reliable value than timing within each loop (or even within the program but outside of loops, due to asynchronous functions etc.).
  • Note: this approach may need further changes for testing on multiple nodes, but this is not currently an issue.
• Removes the module deletion/creation timer, as this is not something that will realistically be called multiple times, and arguably a more meaningful idea of the effect can be obtained by varying N (e.g. running for N=1000 and N=10,000, with the module timings becoming increasingly insignificant).
• Removes (and does not update) the large stride benchmarks from the benchmarking script
• Adds hard-coded (for simplicity) way to use cuda flag in forpy via TorchScript model, allowing GPU comparisons
  • The flag will not work without USETS=1 on compile time, but I'm not sure it's worth investing more time into forpy

To do:

• Tidy up Python e.g. device comments/setting ("cpu" isn't set as the default as described, and more the docstring is unclear)
• Update Python utils to read slurm output file with wall times
• Update notebook with new plots/results (resolve Track benchmarking #16?)
• Update changes to reflect changes for fypp preprocessor

[TomMelt]

Thanks @ElliottKasoar. I will try running this on my machine first, before I review it 👍

[TomMelt]

Code looks good. I am working with @surbhigoel77 to reproduce the tests on CSD3 before we merge

[jatkinson1000]

@surbhigoel77 @TomMelt Is there any update on this?
We are just working through the project board recapping open issues.
Thanks.

[jatkinson1000]

@surbhigoel77 @TomMelt Is there any update on this? We are just working through the project board recapping open issues. Thanks.

@surbhigoel77 @TomMelt
Please could you give us an update here?
This is still listed as 'in progress' on the projects board, does it need reverting or closing?