-
Notifications
You must be signed in to change notification settings - Fork 25
2017 MARBL Dev team meetings
- December 26, 2017 (lots of people out for Christmas / winter travel)
- December 19, 2017 (CGD holiday party)
- (POP listed before MARBL because MARBL updates relate to this new POP workflow)
- Proposed workflow for getting diagnostics from MARBL
-
buildnml
calls uses newMARBL_diagnostics_class
(owned by MARBL) to create default diagnostic recommendations in MARBL-designed format; file will beBuildconf/popconf/marbl_diagnostics
[NOTE: ifSourceMods/src.pop/marbl_diagnostics
exists,build_nml
will copy it topopconf/
instead of running MARBL tools) -
ocn.ecosys.tavg.csh
will be responsible for translatingmarbl_diagnostics
intotavg_contents
form and writing it toecosys_tavg_contents
(I will try to do this in C shell but may end up writing a new python fileparse_MARBL_diagnostics.py
to be called fromocn.ecosys.tavg.csh
...)
-
- Issue with MARBL-generated
marbl_in
file during multi-instance cases (POP is looking formarbl_in
but file is being renamedmarbl_in_####
)- I think this is the result of an incomplete bug-fix -- POP was previously using
marbl_in
on all tasks, which would lead to unexpected behavior if you tried to set up a multi-instance run with differing MARBL configurations. However, it seems like I missed one place where I needed to append the instance string tomarbl_in
(probably inbuild-namelist
) - Taking a one-day break from diagnostics to fix this so Chris F can make the
cesm2_0_beta08
tag - In addition to fixing this bug, I'll also add a multi-instance test to
aux_pop_MARBL
to make sure this doesn't creep back in later
- I think this is the result of an incomplete bug-fix -- POP was previously using
-
For now, we will treat the script to generate the MARBL recommended diagnostics differently than the settings. Namely, MARBL can read
user_nl_marbl
and override values inmarbl_in
accordingly... but there won't be an option to override the diagnostic defaults. To do that, the user needs to rely on the GCM. -
Current format of diagnostics output file (that a POP user would edit in
SourceMods
); note that variables are currently listed alphabetically:# This file contains list of all diagnostics MARBL can compute for a given configuration, # as well a recommended frequency for outputting each diagnostic. The format of this file is: # # DIAGNOSTIC_NAME : frequency # # And fields that should be output at multiple different frequencies will be comma-separated: # # DIAGNOSTIC_NAME : frequency1, frequency2, ..., frequencyN # ATM_CO2 : medium CO2STAR : medium DCO2STAR : medium DpCO2 : medium, high DUST_FLUX : never ECOSYS_ATM_PRESS : medium ECOSYS_IFRAC : medium, high ECOSYS_XKW : medium, high FG_ALT_CO2 : medium FG_CO2 : medium, high IRON_FLUX : medium NHx_SURFACE_EMIS : medium NHy_FLUX : medium NOx_FLUX : medium O2SAT : medium pCO2SURF : medium PH : medium PV_CO2 : never PV_O2 : never SCHMIDT_CO2 : medium SCHMIDT_O2 : medium STF_O2 : medium, high
- Meeting next week? (AGU)
- marbl0.22.0 provides Python / JSON script for generating input settings file
- First bug found in
marbl0.22.0
:user_nl_marbl
is case-sensitive whenuser_nl_pop
wasn't (found because Jessica was settinglvariable_ptoc = .false.
and the Python code expectedlvariable_PtoC = .false.
instead); fix available in #209 - Jessica also pointed out another issue: current computation of tracer count (used by POP as
MARBL_NT
) assumesPFT_defaults = 'CESM2'
and doesn't play nice withPFT_defaults = 'user-specified'
. This will be addressed after the CESM 2 code freeze, but prior to MARBL 1.0 release. - Time for another release like 0.19.1? (Backlog of small fixes to make?)
-
marbl_dev_n67:
build-namelist
is no longer responsible formarbl_in
! -
marbl_dev_n68
: Keith has an update to how we initialize radiocarbon for the 1850 control run. This tag will end up as next POP trunk (I don't want to mergemarbl_dev_n67
onto trunk when I'm the only one who has run with it), likely incesm2_0_beta10
.
- Jessica has a tool to generate size-structured PFT classes
- marbl0.21.2 fixes PAR threshold issue in surface layer
-
marbl0.22.0
will provide python package for generating settings input file, computing tracer count, etc - Keith found a new bug, #207: wrong denominator in bury coefficient computation
-
marbl_dev_n61 brings in Keith's developments through
marbl_dev_klindsay_n94
(phys cycle, refactoring forcing field initialization, CMIP forcings for CISO, etc) - marbl_dev_n62 uses marbl0.21.2 (see above)
-
marbl_dev_n63 brings in Keith's developments through
marbl_dev_klindsay_n97
(split dust flux into fine and coarse; check for NaNs instead of blindly accepting MARBL output) - marbl_dev_n64 and marbl_dev_n65 bring in latest POP trunk (Alper added namelist option for initial condition perturbation and then I added Keith's fix for reading r4 restart history files in DEBUG mode; plus two additional tags fixing bugs Keith and I introduced, which is pretty impressive given how small cesm_pop_2_1_20171115 was)
-
cesm_pop_2_1_20171117b has
marbl_dev_n65
merged in -
marbl_dev_n66 finishes the "zig-zag" merge (it's identical to
marbl_dev_n65
andcesm_pop_2_1_20171117b
) -
marbl_dev_n67
will have my updates to let MARBL generate its own input file (i.e.buildcpp
andbuildnml
will call MARBL code to determine MARBL build-time settings / generatemarbl_in
instead of relying onbuild-namelist
)
- November 21 (canceled due to Thanksgiving)
-
user_nl_marbl
(notuser_settings_marbl
) can go in$CASEROOT
- I need to set up baselines on yellowstone since it seems like cheyenne may be down for a while, but then I'll make a new
marbl_dev
tag with Keith's changes:This merge brings in 1) support for phys_cycle 2) refactoring of D14C atm forcing in abio and ciso 3) introduction and usage of forcing_timeseries_mod 4) new CMIP6 D14C and d13C atm forcing datasets
- Cecile uncovered a bug in reading single-precision restart history files containing data MARBL writes column-by-column; Keith has a fix that I'll get on the trunk (along with adding a test to
aux_pop
to prevent future regressions) - I need to talk to Mariana or Jim about reading
config_cache.xml
inbuildnml
: want to compareMARBL_NT
to what POP was build with and that seems like the best way (this will avoid issues between run-time configurability of MARBL tracer count and POP's requirement that it is known at build-time: e.g. you can not build POP and then change the value ofciso_on
) - I still need to remove the requirement that
ciso
is listed inOCN_TRACER_MODULES
when settingciso_on = .true.
; hold up is matter of priority, not difficulty
-
YAML vs XML
-
PyYAML
is not loaded by default in hobart system version of python, so before users can launch CESM scripts they need to run
(CIME can not load this module because it is running inside a python environment already)module load lang/python/2.7.11
- cheyenne compute nodes ignore the user's environment from the log-in node, so even if users load
python/2.7.13
and can generatemarbl_in
from the head node, submitted jobs fail (unlessenv_mach_specific.xml
is updated)
SOLUTION: provide YAML -> JSON converter until CIME supports PyYAML (not CESM2, but sometime in CESM3 dev)
-
-
Cheyenne testing uncovered an issue in how we initialize burial coefficients in
marbl_particulate_share
-
Current workflow:
- if
ladjust_bury_coeff == .false.
andinit_bury_coeff_opt == 'nml'
, thenmarbl_particulate_share%*_bury_coeff = parm_init_*_bury_coeff
during initialization - if
ladjust_bury_coeff == .true.
thenmarbl_particulate_share%*_bury_coeff
is computed as a running mean; initial value for the running mean either comes fromparm_init_*_bury_coeff
(ifinit_bury_coeff_opt == 'nml'
) or from a POP running mean field written to a restart file (init_bury_coeff_opt == 'restfile'
) [note that eventually MARBL will compute the running mean itself, so this will come from a saved state field rather than being set by POP]
- if
-
Issue:
init_bury_coeff_opt
is a MARBL setting, but its default needs to depend on both another MARBL setting (ladjust_bury_coeff
) and the CESM state (use'nml'
for startup runs but'restfile'
for continuation runs (including hybrids / branches, basically any time the saved state field is available in restart file) -
Potential solutions:
- Take
init_bury_coeff_opt
out of MARBL; if the running mean is available in the POP restart then overwrite theparm_init_*_bury_coeff
initial value otherwise leave it unchanged. I think this will mimic the behavior we see once MARBL computes the running mean, because the GCM will provide the initial running mean via saved state if it exists in a restart file. [Downside: if for some reason running mean isn't in the restart file when user expects it to be, instead of an error MARBL will blindly use the settings file value.] - (Temporary) Introduce
lmarbl_rmeans_in_restfile
in POP's&ecosys_driver_nml
namelist; variable is.false.
by default but set to.true.
ifRUN_TYPE != 'startup'
andladjust_bury_coeff = .true.
(build-namelist
already queriesmarbl_in
for this value!); in the driver, if variable is true callput_settings
to changeinit_bury_coeff_opt
to'restfile'
. [Downside: seems awfully kludgy, we'll probably want to replace it once running means are part of MARBL and these variables are either in saved state (or not).]
- Take
-
Current workflow:
-
marbl_dev_levy
update:- Using
MARBL_tools.generate_settings_file()
instead of POP'sbuild-namelist
utility to createmarbl_in
(Users edit$CASEROOT/user_settings_marbl
) - Cleaned up a lot of MARBL-specific options from
buildcpp
-
OCN_TRACER_MODULES_OPT
no longer containsCISO_NT
,ECOSYS_BASE_NT
, orVARIABLE_PTOC
- Users still add
ciso
toOCN_TRACER_MODULES
, but only until MARBL can generatetavg_contents
(or I addciso
to tracer module list inbuild-namelist
ifciso_on = .true.
inmarbl_in
)
-
- Using
- Up next:
MARBL_NT
set by MARBL (and removed fromOCN_TRACER_MODULES_OPT
)
- Moving tool to generate input file into MARBL repository (lots of development in a one-off repo)
- Instead of just bringing in MARBL's
src/
directory asPOPDIR/source/marbl
, POP gets full MARBL repo asPOPDIR/externals/marbl
- Updates to scripts in
cime_config/
to use input file generation tool- User specifies MARBL parameter changes in
SourceMods/src.pop/marbl.parameters_input
- User specifies MARBL parameter changes in
- YAML vs XML
- PyYAML v3.11 is available on hobart with
module load lang/python/2.7.11
- PyYAML v3.12 is available on cheyenne with
module load python/2.7.13
- Also testing with PyYAML v3.12 on my laptop (appears to be latest version available)
- PyYAML v3.11 is available on hobart with
- Grazing as part of
zooplankton_type
?- I haven't looked at the code in detail, but I don't really like the
(max_grazing_prey_cnt, zooplankton_cnt)
indices. (Jessica is currently trying to run with 4 zooplankton classes, 1 of which grazes on 5 PFTs but the others graze on 2, 3, or 4)
- I haven't looked at the code in detail, but I don't really like the
- New marbl_dev tag: marbl_dev_n60. Moved
iron_frac_in_dust
andiron_frac_in_bc
from MARBL to POP. - Jessica uncovered a bug and something that we probably need to improve
- Issue 202: unexpected behavior if
marbl_in
has more than 512 lines - Should we add some namelist capability to setting tracer fallback values? She wants to easily test several different options for different PFT classes, and currently needs to rebuild just to make sure she is initializing tracers correctly
- Issue 202: unexpected behavior if
- Still working on generating the input file
- I think the python tool can now read an input file of its own
- Some support for elements of a derived type depending on other elements (e.g. size of
grazing%auto_ind(:)
isgrazing%auto_ind_cnt
) but I need to generalize it in a better manner - Working on design document -- this was a little backwards, as I wrote the design document after coming up with an initial design, but I hope it helps clean up the python code now that I have a better sense of what the YAML needs to contain
- New marbl_dev tag: marbl_dev_n59. This brought in the latest POP trunk tag (from Oct 8), compatible with
cesm2_0_beta07
; C compsets are bit-for-bit with tests frommarbl_dev_n58
andcesm2_0_alpha07b
but updates to CICE mean that G compsets change answers. - When should I make a new trunk tag? I'm thinking once we are happy with the input file generation tool. Two changes since last merge with the trunk (oldest at top):
Tag Summary: Merge latest marbl_dev_levy (n119) into marbl_dev This brings MARBL up to 0.20.0 (this version of MARBL improves the initialization process in MARBL) Tag Summary: Merge latest marbl_dev_levy (n121) into marbl_dev This adds more MARBL variables to marbl_in (only via user_nl_pop, there are no new default values for them)
- Getting closer to YAML / python combo for generating namelist
- Have included derived types for
autotrophs
andzooplankton
, but notgrazing
(yet) - Updated how arrays are initialized (using YAML's array notation rather than parsing a string with commas)
- Have included derived types for
- Problems that still need solving
- Need to know tracer count because that is dimension of
tracer_restore_vars(:)
-
autotrophs_type
andzooplankton_type
both contain members that should NOT be set by user (for example, user setsautotrophs%alphaPI_per_day
and then the Fortran code converts it tocgs
units and stores it inautotrophs%alphaPI
) -
grazing_type
contains two components that are allocatable arrays
- Need to know tracer count because that is dimension of
- Follow-up from discussion on how to use YAML to generate input file
- Support for arrays (sort of)
- Support for multiple defaults (resolution dependent, for example)
- Best approach for some refactoring Keith is doing
- abio and ciso can both read d14c forcing from file (or set to constant); we want to ensure consistency between the two modules with both are active
- abio and ecosys have same issue, but with atmospheric CO2. And do we want to force consistency or is it okay if, for example, ecosys gets value from coupler but abio reads from a file?
- September 26 (canceled due to travel)
This is a special meeting to discuss the possibility of MARBL providing a tool to generate its own namelist in CESM rather than having it handled by POP's build-namelist
script. The same tool would also provide ECOSYS_BASE_NT
and CISO_NT
to buildcpp
rather than relying on the user to update OCN_TRACER_MODULES_OPT
.
One possibility would be to use MARBL's stand-alone driver for this task. That would require building the Fortran tool before any other building is done in CESM [*]; one possible workflow would be:
- Will introduce a new python script in $POPROOT/cime_config/ that
- creates a place to build the MARBL driver (
$CASEROOT/Tools/marbl_drv/
or$EXEROOT/ocn/marbl_drv
?) if it doesn’t exist - copies
$POPROOT/source/marbl/{*.F90,makedep.py}
and marbl driver F90 files to new directory - replaces anything in new directory with version from
SourceMods/src.pop/
(if applicable) - Builds executable (will require yet another Makefile, should probably live in POP repo)
- creates a place to build the MARBL driver (
-
buildcpp
would run theget_tracer_cnt
test- requires
tracer_cnt.nml
in$POPROOT/marbl_tools
- uses
user_nl_marbl
as input file
- requires
-
buildnml
would run thegen_inputfile
test- requires
gen_inputfile.nml
in$POPROOT/marbl_tools
- uses
user_nl_marbl
asinputfile
- requires
[Minor] issues I've stumbled upon in first ten minutes of playing with the above proposal:
- CIME only copies
user_nl_pop
from$POPROOT/cime_config/
- Solution: will need new CIME tag to copy
user_nl_marbl
as well
- Solution: will need new CIME tag to copy
- POP needs
$MARBL/tests/driver_src/
as an external- Solution:
$POPROOT/marbl_tools/Makefile
is in POP repo and$POPROOT/marbl_tools/driver
comes from externals
- Solution:
I'm starting to play with this on hobart, and can maybe show some examples.
[*] |
Buildtimes: 15s locally with |
- Visit from UCI folks led to generic MARBL interface project
- Fortran interface module that links to full MARBL library, can be used from Matlab or Python
- Still need to figure out best way to distribute / test it
- Build namelist
-
MARBL 0.20.0: made MARBL more runtime-configurable, closed ~5 issue tickets
- no more namelist, MARBL uses "input files" that it parses line by line instead
- single init phase
- PFT counts are all set via input file instead of namelist
-
PFT_defaults = 'CESM2'
gives default PFTs,PFT_defaults = 'user-specified'
requires user to set all PFT-related parameters via input file - merged
autotroph_config_type
andautotroph_parms_type
back intoautotroph_type
(similar forzooplankton_type
andgrazing_type
)
- marbl_dev_n57: updates required to move from MARBL 0.19.2 to 0.20.0
-
marbl_dev_n58: added more
autotroph_type
objects tonamelist_definitions_pop.xml
to allow CESM users to usePFT_defaults = 'user-specified'
- September 12 (canceled due to travel)
- September 5 (canceled due to travel)
- Status update for #189
- Reworked initialization of PFTs -- default is still the same, but defaults when you change
autotroph_cnt
,zooplankton_cnt
, and / orgrazer_prey_cnt
are coded differently (for one thing, no longer dependent onsname
) - Created
gen_inputfile
test; Fortran-based way to produce input files
- Reworked initialization of PFTs -- default is still the same, but defaults when you change
- August 22 (canceled in favor of code review for #189)
- Catch up on CESM 2.0 timeline
- POP initial condition question: gx1v6 and gx1v7 use
ecosys_jan_IC_gx1v6_20150108.nc
butecosys_jan_IC_gx1v6_20161123.nc
is in the repo and contains diatP, diazP, spP, and Lig (plus Fe is different). Should we switch, or is this an intermediate file with plans to introduce a new version before the 2.0 release?
- Update on #189
- No more namelist!
- No more build-time configuration!
- Still need to clean up the code, but functionality is where we want it
- Show documentation updates (still on branch)
- Merge Keith science changes before 189?
- Initialization is very different, will be easier for me to handle conflicts than Keith
- My changes are bit-for-bit, which will make testing more manageable
- August 8 (travel)
- August 1 (travel)
- Retooling the initialization stage of MARBL... we need to separate init into distinct phases to allow us to define parameters and determine tracer count in a consistent manner, but instead of requiring GCMs to use
put()
calls between phases, could we pre-stage all theput
calls and then process them between phases? We could use a logical flag to track whichput
calls had been applied, and then error out withvariable not found
if any statements had NOT been applied. - The above point may render this moot, but how should we name the individual init phases? We are currently using a semantic naming convention (
config
,init
,complete_config_and_init
) but it's not entirely clear what each of those phases means. It would be nice to just useinit_phase1
,init_phase2
, etc... but that doesn't give users a clear indication of what is happening in each phase. Two points of confusion with the current scheme:- We tend to think of
init
as "setting parameters in the parameterization sense of the word" and config as "variables that may affect tracer count, and also other logical variables that aren't really part of a parameterization" but I am working on a "pre-config" phase that would determine PFT counts (and possibly be a good place for configuring sinking detritus types if we add that feature) and suddenly this semantic description doesn't seem useful - I'd like to come up with a good rule of thumb for answering "I am adding a new step to initialization, where does it go?" Two possible answers are "initialize it as soon as everything else it depends on is locked" or "initialize it in the phase prior to when it is needed by other features." I have put together a quick list of every process that occurs in initialization, including which phase it occurs in and any dependencies it has.
- We tend to think of
- Another possibility is to keep all the
init_phase#()
routines public, but also provide a blank namelist on the interface so that developers can start with a singleinit()
call and whatever default parameter values MARBL provides to get up and running, and then switch to the multi-phase calls withput()
statements in between if necessary. - CESM Tutorial -- anything MARBL-specific? Should we work something into the POP exercises?
- Working on #189, trying to rely less on build-time configuration options (which spawned the two discussion points above). This PR will let us remove
marbl_sizes.F90
altogether, which is a huge step forward.
- July 18 (canceled due to travel)
- New tracer module to add: abio gasses
- Already have issue to refactor marbl_mod.F90
- Talk about overhaul of tracer module system for MARBL 1.0 instead of MARBL 2.0?
- Related issues: tracer reset functionality and specifying which tracers are required for sflux / forcing
- Possible design:
marbl_tracer_type
class to track what tracers are enabled rather than relying on the tracer indexing type
- New tags
-
0.19.0: namelist size runtime configurable, consistent diagnostic names for autotroph Si Uptake,
molw_Fe
used inmarbl_mod.F90
-
0.19.1 & 0.19.2: lots of code clean-up (no interface changes, no answer changes). Two highlights: better error message when CO2 solver does not converge; consistent use of
CaCO3_form
throughout the code
-
0.19.0: namelist size runtime configurable, consistent diagnostic names for autotroph Si Uptake,
- Currently working on making PFT counts runtime configurable, need smart way to ensure consistency between POP build-time settings and MARBL run-time settings
- Use
put()
statements to override whatever is in MARBL namelist? - Have namelist defaults depend on POP build-time settings? For example
MARBL_OPT = CMIP6_SETUP
and several variables can depend onMARBL_OPT
(will then get ported to MARBL when it builds its own namelist)
- Use
- Merged
marbl_dev
(n52) onto trunk, so MARBL 0.19.2 will be available incesm2_0_alpha07a
.
- July 4 (National holiday)
- June 27 (canceled due to travel)
- June 20 (CESM workshop)
- Gloss over topics that we didn't really get to last week
- Testing question: should we switch from
T62_g16
toT62_g17
? Or test a combination of the two resolutions? See #179. - Unsorted tickets in the MARBL 1.0.0 project
- Lots of small updates (without interface changes) coming in marbl0.18.1
- Is now the right time to have MARBL produce list of diagnostics that POP converts into
tavg_contents
?
- New
marbl_dev
tag because I removedgcm_num_interior_forcing_elements
from the interface (hardcoded to 1 until we update the code)
- Try using github projects for MARBL 1.0.0 milestone?
- Also added project page for Post-MARBL 1.0.0 milestone to better visualize (maybe we want to change some tickets to MARBL 1.0.0?)
-
Three tickets without a milestone: should we try to get any (or all) of these into MARBL 1.0.0?
- I think 22 needs to be done before CESM 2.0 or not at all -- another major release with an inconsistent tavg variable name will make it harder to correct afterwards
-
168 -- apparently PGI 17.4 is somewhat buggy, because for some reason a string being set wasn't always populated with the right value the first time it was accessed... but the workaround for it actually ties in to #167
- This is already merged, tagged as marbl0.17.1
-
169 -- easy way to use PGI compiler on local machines (
--mach local-pgi
); not urgent to accept, but useful for testing since PGI 17.4 is free and fixed previous PGI bugs that prevented MARBL from building / running (though, as noted above, it's still not perfect) -
170 -- drop
gcm_num_elements_interior_forcing
from interface (has a related POP update) - 171 -- found some bad use whitespace when examining log files
-
172 -- split
thermodynamic_coefficients_type
to separate species concentrations from actual coefficients.
- MARBL 0.17.1 is on the trunk (as part of
marbl_dev_n46
, which containsmarbl_dev_klindsay_n73
andmarbl_dev_levy_n91
) - Alper made a new trunk tag for Jim (cleaning up some XML info, no expected conflicts), so
marbl_dev
is a little behind
- Best tool for linking meeting agenda to github?
- Using the github projects doesn't seem sustainable long-term
- How about a wiki?
- Some discussion on MARBL 1.0.0 milestone
- @mnlevy1981 will talk to @klindsay28 over coming weeks to prioritize open tickets
- Will see where we are when CESM 2.0 is frozen
There are currently four open pull requests, that will be merged in the following order:
- 165 -- bit-for-bit, no interface changes
-
166 -- bit-for-bit unless
ladjust_bury_coeff = .true.
, no interface changes - 164 -- round-off level answer changes, no interface changes
- 157 -- big answer changes (default parameter values changed); changes namelist as well so there is corresponding GCM tag to make
First three should be merged and tagged by the end of May (as MARBL version 0.16.1); last one will be merged later in the week
Two recent marbl_dev
tags:
-
marbl_dev_n43
-- changed default PE layouts for better out-of-the-box throughput (with and without ecosystem) on NCAR machines. See this google spreadsheet for timing details -
marbl_dev_n44
-- branch is up-to-date with the current trunk (May 16th, 2017 tag)
Upcoming marbl_dev
tags:
- All changes between
marbl_dev_klindsay_n58
andmarbl_dev_klindsay_n71
have been reviewed / are ready to merge -
marbl_dev_klindsay_n72
needs to be reviewed -
marbl_dev_klindsay_n73
and beyond are in progress, should be merged when ready / reviewed