Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add single-point transient tests that test against expected subgrid areas #542

Open
billsacks opened this issue Oct 20, 2018 · 1 comment
Labels
testing additions or changes to tests

Comments

@billsacks
Copy link
Member

In order to prevent bugs like #538 in the future: We could have one or more new test types that are meant to be used in a single-point, transient simulation (though we might want to test a non-transient simulation, too): run for some known amount of time (like an SMS test), then compare subgrid areas (PCT_LANDUNIT, PCT_NAT_PFT, PCT_CFT, etc.) against some expected values. These expected values would match the values used in creating this single-point dataset; they can probably simply be hard-coded in the python test script.

This would ensure that, even if there are answer changes in a tag, the expected behavior of subgrid area changes remains correct.

I imagine that this would be implemented by reading in the test's final history file using the python Netcdf4 library and checking the desired variables in python code.

@billsacks billsacks added the testing additions or changes to tests label Oct 20, 2018
@billsacks
Copy link
Member Author

@ekluzek wondered if we would need some changes in CIME to go along with this. I don't think we do: I think we can start off with a custom test type defined in CTSM that does the necessary comparison and uses the existing COMPARE_XXX phase (where XXX can be anything you want) to set the results. For example, we could call the result, COMPARE_EXPECTED.

Long-term we might want a generic test type with some common functionality that can be extended for this purpose, but I'd argue for starting with a couple of specific cases before we try to write something general.

mariuslam pushed a commit to NordicESMhub/ctsm that referenced this issue Aug 26, 2019
Remove unnecessary (and erroneous) calls to treesai and treelai
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
testing additions or changes to tests
Projects
None yet
Development

No branches or pull requests

1 participant