You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In order to prevent bugs like #538 in the future: We could have one or more new test types that are meant to be used in a single-point, transient simulation (though we might want to test a non-transient simulation, too): run for some known amount of time (like an SMS test), then compare subgrid areas (PCT_LANDUNIT, PCT_NAT_PFT, PCT_CFT, etc.) against some expected values. These expected values would match the values used in creating this single-point dataset; they can probably simply be hard-coded in the python test script.
This would ensure that, even if there are answer changes in a tag, the expected behavior of subgrid area changes remains correct.
I imagine that this would be implemented by reading in the test's final history file using the python Netcdf4 library and checking the desired variables in python code.
The text was updated successfully, but these errors were encountered:
@ekluzek wondered if we would need some changes in CIME to go along with this. I don't think we do: I think we can start off with a custom test type defined in CTSM that does the necessary comparison and uses the existing COMPARE_XXX phase (where XXX can be anything you want) to set the results. For example, we could call the result, COMPARE_EXPECTED.
Long-term we might want a generic test type with some common functionality that can be extended for this purpose, but I'd argue for starting with a couple of specific cases before we try to write something general.
mariuslam
pushed a commit
to NordicESMhub/ctsm
that referenced
this issue
Aug 26, 2019
In order to prevent bugs like #538 in the future: We could have one or more new test types that are meant to be used in a single-point, transient simulation (though we might want to test a non-transient simulation, too): run for some known amount of time (like an SMS test), then compare subgrid areas (PCT_LANDUNIT, PCT_NAT_PFT, PCT_CFT, etc.) against some expected values. These expected values would match the values used in creating this single-point dataset; they can probably simply be hard-coded in the python test script.
This would ensure that, even if there are answer changes in a tag, the expected behavior of subgrid area changes remains correct.
I imagine that this would be implemented by reading in the test's final history file using the python Netcdf4 library and checking the desired variables in python code.
The text was updated successfully, but these errors were encountered: