Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fates two stream API #2265

Merged
merged 29 commits into from
Jan 17, 2024
Merged

Fates two stream API #2265

merged 29 commits into from
Jan 17, 2024

Conversation

rgknox
Copy link
Collaborator

@rgknox rgknox commented Nov 28, 2023

Description of changes

This set of changes supports the addition of two-stream radiation in fates. The changes are mostly cosmetic. Radiation drivers in fates have been re-named to use FATES naming conventions instead of ED, and they no longer imply only Norman radiation.

Specific notes

Contributors other than yourself, if any: On the FATES side of code: @mpaiao @adrifoster @rosiealice @ckoven @JessicaNeedham Gordon Bonan and Jennifer Kowalczyk, to name a few

This should be synchronized with: NGEET/fates#1141

CTSM Issues Fixed (include github issue #):

Fixes #2305

Are answers expected to change (and if so in what way)? Some FATES-specific radiation diagnostics have been changed, removed or added.

Any User Interface Changes (namelist or namelist defaults changes)? No, a fates parameter file switch fates_rad_model is already in the parameter file, it had previously been inconsequential.

Testing performed, if any:

I've run numerous multi-decade smoke tests, site level analysis and comparisons have been made with ilamb.

TBD

Copy link
Collaborator

@ekluzek ekluzek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, it looks like the guts of this is just in FATES. These changes are small and moving from a specific ED interface to a generic FATES one. So there's nothing to really see here....

It adds an important capability that we want to bring in right away.

From looking this over I'm guessing you change it from Norman to 2-stream in the FATES parameter file? Is that right? If so we should add some tests for this similar to the seed dispersal tests that modify the FATES parameter file. We should have at least one if not a few tests that run with two stream.

@rosiealice
Copy link
Contributor

Thanks @ekluzek. Just on the tests, it's likely that w will shift to twostream being the default asap, and probably advise against using the Norman scheme, so maybe we don't need to have a test proliferation in the long run...

@ekluzek
Copy link
Collaborator

ekluzek commented Jan 3, 2024

Thanks @ekluzek. Just on the tests, it's likely that w will shift to twostream being the default asap, and probably advise against using the Norman scheme, so maybe we don't need to have a test proliferation in the long run...

Ah, OK so 2-stream is to become the default, and Norman will be deprecated. That's useful to know. I still assume that you want both to work. And certainly now want 2-stream to work and be tested until it does become the default. Sometimes we think those transitions will happen more quickly then they actually do. So having at least one test during the transition seems critical to me.

In the longer run I'm also betting you still want to keep Norman around and ensure it works. First as an option to go back to if 2-stream has a problem. But, second is likely you'll want to reproduce results of previous FATES versions to compare to. So you might want to keep it around longer than you think. And if so you'll want to keep some testing for it. Anything that doesn't continue to get tested is going to get broken is a reliable axiom. But, if you don't care about it much, you can just reverse the testing so that everything is 2-stream, and you have one Norman. Verses in the transition you maybe most of the tests are Norman because that's been the standard, but if 2-stream is becoming the standard you do want to test it enough to not break it.

@ekluzek
Copy link
Collaborator

ekluzek commented Jan 3, 2024

The other suggestion if 2-stream is to become the standard is try running the FATES test suite tests with 2-stream as the default to see if that brings up any problems. The test suite might show a problem in an obscure configuration that you won't see without running it that way.

@rosiealice
Copy link
Contributor

Good point on the backwards compatibility. I hadn't thought of that. But I guess most of the tests should be using twostream in the nearish future with a smaller set for the Norman? Anyway, this isn't really my thing, so I should probably be quiet ;)

@ekluzek
Copy link
Collaborator

ekluzek commented Jan 3, 2024

Yes, based on this we'll want most tests with 2-stream, but some at least for Norman.

But, @rosiealice there is an important partnership between science and SE in regard to testing. You don't need to know some of the finer details about the test list. But, we need to know that the test list is testing all the important science options for the code. If something isn't tested I'm going to want to argue that we remove it as an option. But, the same is true if we are testing something that isn't scientifically important -- it might be a good candidate to remove from the testing as well as from the code. What we test regularly is the true list of what we expect to work we should all agree that the list is correct. If we aren't testing something important it'll end up broken and take more time to fix. But, if we are testing something NOT important we are wasting resources. So getting that list right requires us working together...

rgknox and others added 6 commits January 10, 2024 09:34
Adds capability for cropland soil tillage and post-harvest residue removal.

Tillage: This PR brings in the tillage code written by Sam Levis and Michael Graham and used in Graham et al. (2021, ERL, doi:10.1088/1748-9326/abe6c6). Low- and high-intensity tillage here work by increasing the decomposition rate of different soil carbon pools. These "decomposition multipliers" vary based on soil pool and how long it's been since the crop was planted; they are set with new paramfile variables till_decompk_multipliers and mimics_till_decompk_multipliers. Note that tillage is off by default.

Residue removal: Adds a parameter hat represents what fraction of post-harvest crop residues (stems and leaves) should be removed to the crop product pool rather than being transferred to litter.
@rgknox
Copy link
Collaborator Author

rgknox commented Jan 11, 2024

After running the fates test suite, things are nominal, except I'm generating small diffs with the base on this test:

SMS_Ld3.f09_g17.I2000Clm51FatesSpCruRsGs.derecho_intel.clm-FatesColdSatPhen_prescribed

The diffs are very odd. When I plot variables that are flagged in the cprnc.out file, I can't see any visible differences in the maps on the day of interest with ncview. It is a very dense map (for fates standards) so perhaps its just an issue that my eye can't identify the rogue cell with the diffs. Along with diffs in variables for soil moisture and gpp, I'm also seeing a difference in fates_lai, which is dictated by the satphen, so I just don't see how the changes to the radiation scheme could possibly generate this difference...
/glade/derecho/scratch/rgknox/tests_0110-155529de/SMS_Ld3.f09_g17.I2000Clm51FatesSpCruRsGs.derecho_intel.clm-FatesColdSatPhen_prescribed.GC.0110-155529de_int/

BTRAN   (lon,lat,time)  t_index =      2     2
          2    55296  (   186,   151,     1) (   165,     4,     1) (    81,   144,     1) (    81,   144,     1)
               15272   1.000000000000000E+00   0.000000000000000E+00 1.9E-02  2.527227369338567E-01 5.0E-06  2.527227369338567E-01
               15272   1.000000000000000E+00   0.000000000000000E+00          2.337220545184604E-01          2.337220545184604E-01
               55296  (   186,   151,     1) (   165,     4,     1)
          avg abs field values:    6.315363838015648E-01    rms diff: 1.5E-04   avg rel diff(npos):  5.0E-06
                                   6.315351262813031E-01                        avg decimal digits(ndif):  2.1 worst:  1.1
 RMS BTRAN                            1.5376E-04            NORMALIZED  2.4347E-04

FATES_LAI   (lon,lat,time)  t_index =      2     2
          2    55296  (   165,     4,     1) (   165,     4,     1) (    81,   144,     1) (    47,   144,     1)
               15272   0.000000000000000E+00   0.000000000000000E+00 2.0-154  0.000000000000000E+00 1.3E-04  0.000000000000000E+00
               15272   1.993615028778571-154   0.000000000000000E+00          1.993615028778571-154          1.703015731097354-154
               55296  (    81,   144,     1) (   165,     4,     1)
          avg abs field values:    0.000000000000000E+00    rms diff: 0.0E+00   avg rel diff(npos):  1.3E-04
                                   2.420528260788322-158                        avg decimal digits(ndif):  0.0 worst:  0.0
 RMS FATES_LAI                        0.0000E+00            NORMALIZED  0.0000E+00

@rgknox
Copy link
Collaborator Author

rgknox commented Jan 13, 2024

Since there is evidence that SMS_Ld3.f09_g17.I2000Clm51FatesSpCruRsGs.derecho_intel.clm-FatesColdSatPhen_prescribed has uninitialized variables, I re-ran the test with the gnu compiler and debug flags. This combination checks and fails with unitiatlized variables.

I was able to trigger the failure, in both this branch and with dev163 (base). See #2321.

Note with the base I also found initialized errors in the dust module, see log here:

dec1741.hsn.de.hpc.ucar.edu 769:  Error dstmbl: pft=           69  lnd_frc_mbl(p)=    2216381976734.5981     
dec1741.hsn.de.hpc.ucar.edu 769: iam = 641: local  patch    index = 69
dec1741.hsn.de.hpc.ucar.edu 769: iam = 641: global patch    index = 149382
dec1741.hsn.de.hpc.ucar.edu 769: iam = 641: global column   index = 24438
dec1741.hsn.de.hpc.ucar.edu 769: iam = 641: global landunit index = 15097
dec1741.hsn.de.hpc.ucar.edu 769: iam = 641: global gridcell index = 7810
dec1741.hsn.de.hpc.ucar.edu 769: iam = 641: gridcell longitude    =  135.0000000
dec1741.hsn.de.hpc.ucar.edu 769: iam = 641: gridcell latitude     =  -14.6073298
dec1741.hsn.de.hpc.ucar.edu 769: iam = 641: pft      type         = 0
dec1741.hsn.de.hpc.ucar.edu 769: iam = 641: column   type         = 1
dec1741.hsn.de.hpc.ucar.edu 769: iam = 641: landunit type         = 1
dec1741.hsn.de.hpc.ucar.edu 769:  ENDRUN:
dec1741.hsn.de.hpc.ucar.edu 769:  ERROR: ERROR in /glade/u/home/rgknox/ctsm/src/biogeochem/DUSTMod.F90 at line 336
dec1741.hsn.de.hpc.ucar.edu 769: #0  0x108c820 in __shr_abort_mod_MOD_shr_abort_backtrace
dec1741.hsn.de.hpc.ucar.edu 769: 	at /glade/u/home/rgknox/ctsm/share/src/shr_abort_mod.F90:104
dec1741.hsn.de.hpc.ucar.edu 769: #1  0x108c8e3 in __shr_abort_mod_MOD_shr_abort_abort
dec1741.hsn.de.hpc.ucar.edu 769: 	at /glade/u/home/rgknox/ctsm/share/src/shr_abort_mod.F90:61
dec1741.hsn.de.hpc.ucar.edu 769: #2  0x5b3960 in __abortutils_MOD_endrun_write_point_context
dec1741.hsn.de.hpc.ucar.edu 769: 	at /glade/u/home/rgknox/ctsm/src/main/abortutils.F90:98
dec1741.hsn.de.hpc.ucar.edu 769: #3  0x89f8e9 in __dustmod_MOD_dustemission
dec1741.hsn.de.hpc.ucar.edu 769: 	at /glade/u/home/rgknox/ctsm/src/biogeochem/DUSTMod.F90:336

In light of this, I think we should:

@rgknox
Copy link
Collaborator Author

rgknox commented Jan 16, 2024

Test output:
Izumi:
/scratch/cluster//tests_0116-111700iz OK
Derecho:

  • fates: /glade/derecho/scratch/rgknox/tests_0116-120553de OK
  • aux_clm: /glade/derecho/scratch//tests_0111-123749de OK

@ekluzek
Copy link
Collaborator

ekluzek commented Jan 17, 2024

The FATES test list wasn't run on izumi. So @rgknox is going to run a baseline for ctsm5.1.dev163 and for this update. In the meantime I'm going to go ahead and make the tag as it is. If we see any issues we'll document them as issues to be fixed later, since the testing on Derecho was fine. We'll also talk about if we should run the izumi FATES test list for all tags with fates updates tomorrow....

@ekluzek ekluzek merged commit 5b72315 into ESCOMP:master Jan 17, 2024
2 checks passed
@ekluzek ekluzek deleted the fates-two-stream branch January 17, 2024 19:40
@samsrabin samsrabin added the science Enhancement to or bug impacting science label Aug 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement new capability or improved behavior of existing capability science Enhancement to or bug impacting science
Projects
Status: Ready to eat (Done!)
Development

Successfully merging this pull request may close these issues.

len function is around the wrong variable
5 participants