Add long term exact restart test and fixed biogeog + no competition tests to fates suite #1849

glemieux · 2022-09-12T23:05:23Z

Description of changes

Adds two 12 month ERS tests to the fates suite along with a new test mod that combines the no competition and fixed biogeography modes as a 30 day ERS test.

Specific notes

The long term ERS test is being added to avoid missing issues related to NGEET/fates#897 in which mid-year restarts were not bfb.

While we include a fates satellite phenology test that exercises fixed biogeography + no competition modes by definition, we should provide additional test coverage for testing these two modes together without being driven by satellite data. This combination is a valid scientific mode in its own right and exercises different code paths.

Contributors other than yourself, if any:

CTSM Issues Fixed (include github issue #):
resolves #1839
resolves #1817
resolves #1551

Are answers expected to change (and if so in what way)? Answers are not expected to change.

Any User Interface Changes (namelist or namelist defaults changes)? No

Testing performed, if any:

TBD

NOTE: Be sure to check your coding style against the standard
(https://github.com/ESCOMP/ctsm/wiki/CTSM-coding-guidelines) and review
the list of common problems to watch out for
(https://github.com/ESCOMP/CTSM/wiki/List-of-common-problems).

cime_config/testdefs/testlist_clm.xml

cime_config/testdefs/testmods_dirs/clm/FatesColdDefReducedComplexNoCompFbg/user_nl_clm

…ractions

glemieux · 2022-09-13T16:47:05Z

@ekluzek @rgknox what do you think about truncating some of these names? I have run into problems where the length of the testmod name for the reduced complexity modes has caused build errors when I run a single test with a long-ish --testid-base added. The NoComp + FBG testmod name truncation that I came up with is not great 😒

…ests-ers-fbg_nocomp

glemieux · 2022-09-20T19:01:38Z

@ekluzek @rgknox what do you think about truncating some of these names? I have run into problems where the length of the testmod name for the reduced complexity modes has caused build errors when I run a single test with a long-ish --testid-base added. The NoComp + FBG testmod name truncation that I came up with is not great unamused

Per a discussion at the ctsm software meeting a consensus on the truncated naming convention was achieved. ColdDef has been truncated to Cold and ReducedComplex has been removed entirely.

glemieux · 2022-09-20T19:10:15Z

Per @ekluzek suggestion, I've merged in #1827 into my PR branch and run aux_clm testing against the ctsm5.1.dev108 baseline. All non-fates testmods pass as expected on cheyenne and izumi, with one exception: SMS_D_Ld1_PS.f09_g17.I1850Clm50BgcSpinup.cheyenne_intel.clm-cplhist fails during the run. I am trying to resubmit that now.

The fates test mods all have BFAIL and NLCOMP differences due to the name changes in all the test mods. I manually used cprnc to check against the baselines for all the fates testmods and they all have b4b results or only differ in the FIELDLIST. The output for these checks are in the top of the cheyenne directory. Test locations:

cheyenne: /glade/u/home/glemieux/scratch/ctsm-tests/tests_pr1827-1849
izumi: /scratch/cluster/glemieux/ctsm-tests/tests_pr1827-1849-2

glemieux · 2022-09-20T19:12:22Z

I just noticed I missed that two fates tests are failing for other reasons:

ERP_P144x2_Ld30.f45_f45_mg37.I2000Clm50FatesRs.cheyenne_intel.clm-mimicsFatesCold
ERP_P36x2_Ld30.f45_f45_mg37.I2000Clm51FatesSpCruRsGs.cheyenne_intel.clm-FatesColdSatPhen

Looking into these as well.

glemieux · 2022-09-20T21:06:40Z

Per @ekluzek suggestion, I've merged in #1827 into my PR branch and run aux_clm testing against the ctsm5.1.dev108 baseline. All non-fates testmods pass as expected on cheyenne and izumi, with one exception: SMS_D_Ld1_PS.f09_g17.I1850Clm50BgcSpinup.cheyenne_intel.clm-cplhist fails during the run. I am trying to resubmit that now.

I guess I missed that this is actually an expected failure:

CTSM/cime_config/testdefs/ExpectedTestFails.xml

Lines 40 to 45 in 11d589f

    
           <test name="SMS_D_Ld1_PS.f09_g17.I1850Clm50BgcSpinup.cheyenne_intel.clm-cplhist"> 
        
             <phase name="RUN"> 
        
               <status>FAIL</status> 
        
               <issue>#1844</issue> 
        
             </phase> 
        
           </test>

Resubmitting it resulted in it passing however.

glemieux · 2022-09-20T22:31:20Z

I just noticed I missed that two fates tests are failing for other reasons:

ERP_P144x2_Ld30.f45_f45_mg37.I2000Clm50FatesRs.cheyenne_intel.clm-mimicsFatesCold

ERP_P36x2_Ld30.f45_f45_mg37.I2000Clm51FatesSpCruRsGs.cheyenne_intel.clm-FatesColdSatPhen

The ERP SatPhen test is failing the restart in FATES_TVEG and FATES_TVEG24 only starting at t_index 18 an 19 respectively. The reason this is showing up now and not in the ctsm5.1.dev108 baseline is due to the fixes that @adrifoster put in place with #1827. Prior to these fixes, the user_nl_clm for this test would end with clearing the history tapes and only including the variable set in FatesColdDef testmod which doesn't include either of the above variables (whereas the current fix does include them by calling exclude and avoiding clearing the history tapes).

Testing a version of this test as an ERS without the 36x2 shows that only FATES_TVEG is not b4b on the restart. As I'm currently working on another exact restart issue, I'm inclined to think these might be related to NGEET/fates#897. Additionally, this could suggest that there is a problem with FATES_TVEG24 on the threaded restart for some reason. I'm going to test both variation of those (i.e. ERP_Ld30, and ERS_P36x2_Ld30).

glemieux · 2022-09-20T22:55:46Z

Testing a version of this test as an ERS without the 36x2 shows that only FATES_TVEG is not b4b on the restart. ... Additionally, this could suggest that there is a problem with FATES_TVEG24 on the threaded restart for some reason. I'm going to test both variation of those (i.e. ERP_Ld30, and ERS_P36x2_Ld30).

All the other variations result in only FATES_TVEG as being non-b4b. Not sure what it says about FATES_TVEG24 being not b4b in the ERP_P36x2 test only.

glemieux · 2022-09-24T00:02:52Z

The issue causing the non-b4b fates SatPhen testmod in part has been posted at NGEET/fates#908 and is due to a fates-side restart variable not being set properly for no comp modes.

The FATES_TVEG24 variable is still not b4b for this test mod.

Fixes for NEON cases Two sets of fixes for NEON cases: (1) Fixes for leap year handling when there are discrepancies between the model and the DATM forcing data in terms of whether the current year is a leap year. This involved updates to the share and cdeps externals from Jim Edwards. (2) Other fixes for NEON cases from Will Wieder, as documented in ESCOMP#1860

glemieux · 2022-10-05T23:17:15Z

Fix for the non-b4b SatPhen testmod (NGEET/fates#908 and NGEET/fates#911) is forthcoming in NGEET/fates#914. As such, this PR will be updated with a new fates tag once integrated on the fates-side.

glemieux · 2022-10-13T15:50:43Z

Fates-side PR NGEET/fates#914 fix has been integrated. I will update the externals for this PR to point to the fix tag sci.1.59.7_api.24.1.0. As such, this will result in non-b4b fates testsmods, although the name changes in the PR will obscure that fact. Retesting aux_clm.

glemieux · 2022-10-13T21:28:33Z

Testing aux_clm on izumi and cheyenne are complete. All tests are b4b with the exception of expected fails and expected DIFFs due to the fates update. I spot check a few tests that overlap with the fates suite regression tests and the DIFFs appear consistent, particularly the model day output updates from sci.1.59.4_api.24.1.0.

The test locations are here:

izumi: /scratch/cluster/glemieux/ctsm-tests/tests_pr1827-1849-pr914
cheyenne: /glade/u/home/glemieux/scratch/ctsm-tests/tests_pr1827-1849-pr914 and /glade/u/home/glemieux/scratch/ctsm-tests/tests_pr1827-1849-pr914-2

Note that the second cheyenne location is the test for the mimicsFatesColdDef test which initially failed (in the first folder location) to create the test case due to having a mismatch in the testlist and testmod directory names.

@ekluzek will this go before or after #1735?

ekluzek · 2022-10-13T23:06:57Z

@glemieux if you are ready to go we should plan on it coming in now. It can go either way. We have scheduled #1735 as first right now, but this can go first instead. We are at a space where whatever is ready should go next.

glemieux · 2022-10-13T23:15:19Z

Thanks @ekluzek. I'll update the changelog.

glemieux · 2022-10-14T22:11:28Z

The izumi fates suite tests are showing DIFFs consistent and expected with the DIFFs from the aux_clm test mod review (after running cprnc manually against the baseline). That said, issue NGEET/fates#701 has cropped back up in the hydro test. I'm currently planning on simply reopening the issue and adding it back into the expected fails list to be resolved with a future PR.
Folder location: /scratch/cluster/glemieux/ctsm-tests/tests_pr1827-1849-pr914-2

glemieux · 2022-10-15T00:04:07Z

I'm still waiting on two tests to clear the queue for the cheyenne fates test suite. Otherwise all tests are passing with BFAILS and DIFFs as expected.
Folder location: /glade/u/home/glemieux/scratch/ctsm-tests/tests_pr1827-pr1849-pr914-fates-2

glemieux · 2022-10-15T03:30:50Z

The last two test passed through the queue. No unexpected issues. This should be good to go now.

glemieux · 2022-10-15T20:04:50Z

@ekluzek per our google meet conversation here is the test I ran for the mimicsFatesColdDef to enable a baseline comparison with ctsm5.1.dev111: /glade/u/home/glemieux/scratch/ctsm-tests/tests_pr1827-1849-pr914-2. The results are non b4b as I accidentally stated in the call; they do however match expected differences due to the fates externals update. Side note: I had been generating my baselines as ctsm5.1.dev113 just in case #1735 was going to come in prior, and have since renamed the directory to ctsm5.1.dev112.

I've renamed the testmod to match the new name convention via 84f768d. I re-ran the testmod to make sure the naming was consistent and would generate the correct name in the ctsm5.1.dev112 baseline; it built and ran successfully. Folder is here: /glade/u/home/glemieux/scratch/ctsm-tests/tests_pr1827-1849-pr914-mimics

Adrianna Foster and others added 9 commits August 9, 2022 10:37

update SatPhen test defs files

0824a12

fixing some other SP issues

8cde3fe

reverting some changes

f7223e0

update testdef for the one of the FATES SP smoke tests

ee553ef

put back call to fates_sp usermods

f2019f5

add back fates_sp user mods call

2275289

changed order of include mods

b6c239c

update for DAMP

695ee13

add nocomp+fbg test and 12 month ERS fates tests

f6727f2

glemieux mentioned this pull request Sep 12, 2022

Next FATES testing update glemieux/ctsm#3

Open

2 tasks

glemieux requested a review from rgknox September 12, 2022 23:08

rgknox reviewed Sep 13, 2022

View reviewed changes

cime_config/testdefs/testlist_clm.xml Outdated Show resolved Hide resolved

cime_config/testdefs/testmods_dirs/clm/FatesColdDefReducedComplexNoCompFbg/user_nl_clm Outdated Show resolved Hide resolved

update 12 month to 13 month to catch potential start of year clm inte…

f951f8c

…ractions

glemieux marked this pull request as ready for review September 14, 2022 22:07

glemieux added 5 commits September 15, 2022 10:56

rename Fates tests to truncate length

c0d8e08

Merge remote-tracking branch 'adrifoster/fates_test_def' into fates-t…

c19b5c9

…ests-ers-fbg_nocomp

remove duplicate satellite phenology test mods

d37cb52

rename FatesColdDefBasic everywhere

4d90237

rename DryDep and Megan satellite phenology testmods

6567378

glemieux mentioned this pull request Sep 23, 2022

Bareground area_pft not being carried over during restarts NGEET/fates#908

Closed

glemieux mentioned this pull request Oct 3, 2022

24-hr running mean vegetation temperature is not b4b on threaded exact restart SatPhen test NGEET/fates#911

Closed

glemieux added 2 commits October 4, 2022 12:17

fix included directory name to remove complex word

992f113

glemieux mentioned this pull request Oct 5, 2022

Fix mean vegetation temperature output for exact restarts NGEET/fates#914

Merged

5 tasks

glemieux added 2 commits October 13, 2022 09:52

update fates externals to include tveg restart fix

e2593a1

restore clm mimics fates name to use ColdDef suffix

63d8794

glemieux added 3 commits October 13, 2022 16:48

fix testmod name for nocomp + fixed biogeo

f3b28ff

update changelog with test status TBD

e05e50e

add new exact restart tests to known failure list

bb2ecf5

glemieux mentioned this pull request Oct 14, 2022

FATES_ERRH2O_SCPF fails COMPARE_base_rest NGEET/fates#701

Open

glemieux added 4 commits October 14, 2022 15:13

adding fates issue 701 to the expected fail list

74be22b

update compset name to current fates lightning expected diff

389826f

updating changelog and changesum

ec55ade

fix minor grammatical error in changelog

e27e717

ekluzek self-assigned this Oct 15, 2022

ekluzek added the FATES API update Changes to the FATES version that also REQUIRE an API change in CTSM label Oct 15, 2022

ekluzek approved these changes Oct 15, 2022

View reviewed changes

rename mimics fates test to be consistent with other test names

84f768d

Update date on changelog, add note that I ran the build-namelist testing

b098b6b

ekluzek merged commit 616905b into ESCOMP:master Oct 15, 2022

ekluzek deleted the fates-tests-ers-fbg_nocomp branch October 15, 2022 22:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add long term exact restart test and fixed biogeog + no competition tests to fates suite #1849

Add long term exact restart test and fixed biogeog + no competition tests to fates suite #1849

glemieux commented Sep 12, 2022 •

edited by ekluzek

Loading

glemieux commented Sep 13, 2022

glemieux commented Sep 20, 2022

glemieux commented Sep 20, 2022

glemieux commented Sep 20, 2022

glemieux commented Sep 20, 2022

glemieux commented Sep 20, 2022 •

edited

Loading

glemieux commented Sep 20, 2022

glemieux commented Sep 24, 2022 •

edited

Loading

glemieux commented Oct 5, 2022

glemieux commented Oct 13, 2022

glemieux commented Oct 13, 2022

ekluzek commented Oct 13, 2022

glemieux commented Oct 13, 2022

glemieux commented Oct 14, 2022 •

edited

Loading

glemieux commented Oct 15, 2022

glemieux commented Oct 15, 2022

glemieux commented Oct 15, 2022 •

edited

Loading

Add long term exact restart test and fixed biogeog + no competition tests to fates suite #1849

Add long term exact restart test and fixed biogeog + no competition tests to fates suite #1849

Conversation

glemieux commented Sep 12, 2022 • edited by ekluzek Loading

Description of changes

Specific notes

glemieux commented Sep 13, 2022

glemieux commented Sep 20, 2022

glemieux commented Sep 20, 2022

glemieux commented Sep 20, 2022

glemieux commented Sep 20, 2022

glemieux commented Sep 20, 2022 • edited Loading

glemieux commented Sep 20, 2022

glemieux commented Sep 24, 2022 • edited Loading

glemieux commented Oct 5, 2022

glemieux commented Oct 13, 2022

glemieux commented Oct 13, 2022

ekluzek commented Oct 13, 2022

glemieux commented Oct 13, 2022

glemieux commented Oct 14, 2022 • edited Loading

glemieux commented Oct 15, 2022

glemieux commented Oct 15, 2022

glemieux commented Oct 15, 2022 • edited Loading

glemieux commented Sep 12, 2022 •

edited by ekluzek

Loading

glemieux commented Sep 20, 2022 •

edited

Loading

glemieux commented Sep 24, 2022 •

edited

Loading

glemieux commented Oct 14, 2022 •

edited

Loading

glemieux commented Oct 15, 2022 •

edited

Loading