Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce test_on_push workflow run-time #120

Closed
BradyPlanden opened this issue Nov 28, 2023 · 2 comments · Fixed by #129
Closed

Reduce test_on_push workflow run-time #120

BradyPlanden opened this issue Nov 28, 2023 · 2 comments · Fixed by #129

Comments

@BradyPlanden
Copy link
Member

Currently we are at ~ 9 minutes to run the test_on_push workflow. I think we should keep this around 5-10 minutes, so this should probably be reduced unless others have different thoughts @martinjrobins @NicolaCourtier?

A few thoughts on this:

  • Reduce the number of optimisers that are performance tested (i.e. asserted on the identified parameter)
  • Reduce the maximum number of iterations for each optimiser during performance testing (this may result in more failures)
  • Split the test_on_push workflow so that performance tests are only run for a PR.
@NicolaCourtier
Copy link
Member

I agree that we should reduce the time! On my laptop, reducing the max_iterations from 250 to 100 takes the run time for test_parameterisations.py from 14 minutes down to 6 minutes, and the tests still pass. I think, in future, we could also test the optimisers in isolation from the battery models in order to speed things up further. I would prefer this to reducing the number of optimisers that are tested.

@NicolaCourtier NicolaCourtier linked a pull request Dec 1, 2023 that will close this issue
@BradyPlanden BradyPlanden moved this to Todo in v23.12 Dec 1, 2023
@NicolaCourtier
Copy link
Member

I've created a linked branch with three updates:

  • moved the setting of the default model, parameters and spm_cost to fixtures in test_parameterisations
  • reduced the max_iterations to 100 in test_parameterisations : test_spm_optimisers
  • parameterised test_examples based on the file path so that progress through each example can be seen

I'll leave PR open for further input.

@github-project-automation github-project-automation bot moved this from Todo to Done in v23.12 Dec 6, 2023
@NicolaCourtier NicolaCourtier self-assigned this Dec 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

2 participants