Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make all tests run and show summary table #433

Merged
merged 4 commits into from
Aug 13, 2024
Merged

Conversation

Bill-Becker
Copy link
Collaborator

@Bill-Becker Bill-Becker commented Aug 13, 2024

This update to runtests.jl:

  • Makes sure that all tests are run and displays a summary table at the end of the GitHub Actions step
    • By wrapping/nesting all the tests that we run on GitHub Actions within a high-level @testset, this gives a big summary table of all the nested @testsets after running all of them
    • Before, if there was certain errors in tests, the remaining tests would not run because it was not all nested under one
  • Remove the "Imported Xpress tests" @testset now that we plan to only use HiGHS or other OS solver going forward, and all of those imported tests have been vetted and pass
  • By including verbose=true, we see the results and a time of each testset even if they pass
  • Removes an "inputs" @testset because there are tons of other inputs tests within other @testsets

There is still a bit of inconsistency for when we wrap certain @testsets within a higher-level @testset to categorize and group tests, so there's room to improve the organization of this for clarity. This PR does NOT attempt to take that on.

Bill-Becker and others added 4 commits August 12, 2024 21:37
And remove an unnecessary superset "inputs" which did not include a majority of input-related tests
@Bill-Becker Bill-Becker changed the title Make testsets more verbose to show nice summary table Make all tests run and show summary table Aug 13, 2024
Copy link
Collaborator

@adfarth adfarth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for bringing back the summary table! Did you test this with having a test fail? (e.g. temporarily changing a test value)

@Bill-Becker
Copy link
Collaborator Author

Bill-Becker commented Aug 13, 2024

Thanks for bringing back the summary table! Did you test this with having a test fail? (e.g. temporarily changing a test value)

@adfarth Yes because I had added these commits to another branch where I was developing something else, and I just cherry picked the commits into this branch off of develop. This branch had a couple of tests fail, with these same edits to runtests.jl:
https://github.com/NREL/REopt.jl/actions/runs/10363166826/job/28686315729

@Bill-Becker Bill-Becker merged commit 5084565 into develop Aug 13, 2024
1 check passed
@Bill-Becker Bill-Becker deleted the verbose-testsets branch August 13, 2024 23:56
indu-manogaran pushed a commit that referenced this pull request Sep 16, 2024
Make all tests run and show summary table
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants