-
Notifications
You must be signed in to change notification settings - Fork 60
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
(+) Refactor of MOM_file_parser #74
(+) Refactor of MOM_file_parser #74
Conversation
Codecov Report
@@ Coverage Diff @@
## dev/gfdl #74 +/- ##
=========================================
Coverage 29.03% 29.04%
=========================================
Files 244 244
Lines 71865 71849 -16
=========================================
Hits 20869 20869
+ Misses 50996 50980 -16
Continue to review full report at Codecov.
|
field_flag = 0 | ||
if (field) field_flag = 1 | ||
call min_across_PEs(field_flag, pelist) | ||
all_across_PEs = (field_flag < 1) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As written, the logic on lines 468 and 469 seems to be implementing the parallel version of .not.any()
, and not the parallel version of all()
. To correctly implement all_across_PEs()
as the analog of all()
, I think that line should be all_across_PEs = (field_flag > 0)
.
Exactly the same issue pertains to config_src/infra/FMS2/MOM_coms_infral.F90.
The fact that there are two identical routines in the two infra/FMS[12] directories, neither of which calls an any FMS infrastructure interfaces directly, suggests that these new routines should perhaps be implemented in src/framework/MOM_coms.F90.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like you are right. As you might have guessed, all_across_PEs
was not used anywhere. Perhaps adding it was a mistake.
I believe this is better placed in the infra
layer, because the conversions to integers and use of {min,max}_across_PEs
reflects the absence of logical MPI collectives in FMS and is thus a uniquely FMS operation. A different framework would not have this restriction.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
all_across_PEs
test has been updated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The changes to MOM_file_parser.F90 seem well conceived.
However, as noted in detail in a separate comment attached to one of the relevant lines, the logic in the new all_across_PEs()
routine looks to me like it is incorrect. Although any_across_PEs()
is being used, I don't see where all_across_PEs()
is being used, which could be how faulty logic could have slipped through. Moreover, I don't understand why the new any_across_PEs()
and all_across_PEs()
routines should be in the multiple config_src/infra/*/MOM_coms_infra.F90 files, rather than being implemented just once directly in src/framework/MOM_coms.F90.
f4d5d4b
to
01d03a2
Compare
01d03a2
to
55db1a2
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This PR is not compiling in static memory mode. The static_value=
argument at https://github.com/NOAA-GFDL/MOM6/blob/dev/gfdl/src/core/MOM_verticalGrid.F90#:~:text=call%20get_param(param_file%2C%20mdl%2C%20%22NK,static_value%3DNK_) (i.e. at about line 145 of core/MOM_verticalGrid.F90) needs to be changed to `default='.
This patch includes several minor changes to the MOM_file_parser and supporting modules in order to accommodate stronger unit testing. It includes the following API changes: - Removal of `static_value` from `get_param` - Redefined `link_parameter` and `parameter_block` as private - New functions: `all_across_PEs()`, `any_across_PEs()` `static_value` was not used in any known experiments (outside of internal GFDL testing), and the two derived types describe internal operations within `MOM_file_parser`, so we do not expect any disruptions from these changes. A detailed summary of the changes are listed below. - `assert()` is now used to detect same files with different IO units. Detection of reopenend files of the same name but different IO unit has been changed from `MOM_error(FATAL, ...)` to `assert()`, to reflect that this should be a logically impossible result. - Bugfix: Reopened files are now reported to all PEs. If an open file is re-opened, then only the root PE will detect this and will `return` immediately. However, the others will proceed into `populate_param_data` and will get stuck in a broadcast waiting for root. We fix this by communicating the reopened state to all PEs and allow all ranks to return before re-processing the data. Note that this could also be resolved by allowing all ranks to track IO unit numbers, but for now we do not attempt to change this behavior. - `newunit=` used to generate parameter file IO unit The parameter IO unit is now generated by `newunit=` rather than an explicit search for an unused IO unit. Note that this is a Fortran 2008 feature. Testing around available IO units has also been removed. - Removal of generic IO error handling Generic "IO error" tests, and corresponding `err=` arguments, have been removed in most cases. We now rely on the Fortran runtime to provide diagnostics on these errors, which should typically exceed any information that MOM6 could provide. - Removal of purported `namelist` support There were several blocks of code provided to support namelist syntax, but did not appear to be working, nor was there any known instance of it being used by anyone, so it has been removed. - `#define/undef/=` syntax testing across ranks Previously, only the root PE would test for consistency of the #define-like syntax, even though all ranks have this information. This required a second, awkwardly placed syntax test later in the subroutine. This test is redefined to run over all ranks, and the subsequent test has been removed. - `define/override` test reordering The `found_override` test when coupled to a `#define`-like declaration was unreachable due to the presence of an even stronger test related to valid syntax. This test has been moved to provide more detailed information about the nature of the error. - `link_parameter`, `parameter_block` defined as private Internal derived types of `MOM_file_parser` are redefined as private. This preserves the integrity of instances of these types, and also prevents creation of implicit object code required to access them externally. - Removal of `static_value` from `get_param` interface The `static_value` argument of `get_param` has been removed, since it is functionally equivalent to `default`. While this is an API change, there is no known case of anyone using this argument. - The `param_type%doc` fields are now properly deallocated after closed. - Quotes have been added around some filename error warnings, to help detect issues related to whitespace. - `any_across_PEs` and `all_across_PEs` New functions for calling `any()` and `all()` across PE ranks have been added. Behavior is in line with other functions, such as `min_across_PEs`.
ae156c2
to
9caa701
Compare
Thank you for detecting this error, I've updated the PR. |
This PR has passed pipeline testing at https://gitlab.gfdl.noaa.gov/ogrp/MOM6/-/pipelines/14861 . |
* additions for stochastic physics and ePBL perts * cleanup of code and enhancement of ePBL perts * Update MOM_diabatic_driver.F90 remove conflict with dev/emc * Update MOM_diabatic_driver.F90 further resolve conflict * Update MOM_diabatic_driver.F90 put id_sppt_wts, etc back. * add stochy_restart writing to mom_cap * additions for stochy restarts * clean up debug statements * clean up code * fix non stochastic ePBL calculation * re-write of stochastic code to remove CPP directives * remove blank link in MOM_diagnostics * clean up MOM_domains * make stochastics optional * correct coupled_driver/ocean_model_MOM.F90 and other cleanup * clean up of code for MOM6 coding standards * remove stochastics container * revert MOM_domains.F90 * clean up of mom_ocean_model_nuopc.F90 * remove PE_here from mom_ocean_model_nuopc.F90 * remove debug statements * stochastic physics re-write * move stochastics to external directory * doxygen cleanup * add write_stoch_restart_ocn to MOM_stochastics * add logic to remove incrments from restart if outside IAU window * revert logic wrt increments * add comments * update to gfdl 20210806 (NOAA-GFDL#74) * remove white space and fix comment * Update MOM_oda_incupd.F90 remove unused index bounds, and fix sum_h2 loop. Co-authored-by: pjpegion <[email protected]> Co-authored-by: Marshall Ward <[email protected]> * Fussing with zotero.bib. Getting a warning about a repeated bibliography entry for adcroft2004. Rob thinks this is a hash failure. * Still fussing with zotero.bib - it was complaining about the (unused) Kasahara reference. * Several little things, one is making sponge less verbose. - Pointing to OBC wiki file from the lateral parameterizations doc. - Using the MOM6 verbosity to control the time_interp verbosity. - Making the check for negative water depths more informative. * return a more accurate error message in MOM_stochasics * Working on boundary layer docs. * Done with EPBL docs? * Undoing some patches from others * Cleaning up too-new commits * Adding in that SAL commit again. * correction on type in directory name * Added some to vertical viscisity doc. * Cleaned up whitespace leftover from porous topomerge. - Spacing within expressions was uneven and made multiplation look like POW functions. Leftover from merging NOAA-GFDL#3. - No answer changes. * Fix out-of-bounds k index in PPM flux - An errant use of the porous face area led to an out-of-bounds k-index reported in NOAA-GFDL#19. - Closes NOAA-GFDL#19 * Adding Channel drag figure * Take cite out of figure caption. * Copyright year 2022
* remove white space and fix comment * Update MOM_oda_incupd.F90 remove unused index bounds, and fix sum_h2 loop. Co-authored-by: pjpegion <[email protected]> Co-authored-by: Marshall Ward <[email protected]>
* remove white space and fix comment * Update MOM_oda_incupd.F90 remove unused index bounds, and fix sum_h2 loop. Co-authored-by: pjpegion <[email protected]> Co-authored-by: Marshall Ward <[email protected]>
* remove white space and fix comment * Update MOM_oda_incupd.F90 remove unused index bounds, and fix sum_h2 loop. Co-authored-by: pjpegion <[email protected]> Co-authored-by: Marshall Ward <[email protected]>
Note that most of these commits are from previously squashed pull requests, and this PR is restoring them. - 6360dbb Merge branch 'main' into main_to_dev - bac8031 Merge pull request mom-ocean#1566 from jiandewang/EMC-FMS-mixed-mode-20220411 - e532d86 Merge pull request #88 from marshallward/missing_attrib_with_class_bugfix - d380f1d An alternate fix to class(*) issues with FMS 2022-01 - 8ecf333 Merge pull request #87 from jiandewang/feature/update-to-main-20220317 - ba37f94 Merge remote-tracking branch 'FSU/main' into feature/update-to-main-20220317 this is corresponding to MOM6 main 20220317 commit (hash # 399a7db) - 44313d9 Merge pull request #85 from jiandewang/feature/update-to-main-20220217 - 966707f Merge remote-tracking branch 'GFDL/main' into feature/update-to-main-20220217 this is corresponding to MOM6 main branch 20220217 commit (hash # 6f6d4d6), which originally based on GFDL-candidate-20220129 - 32c0e1e Merge pull request #81 from jiandewang/feature/update-to-main-20211220 - 9642b1d delete external/OCEAN_stochastic_phyiscs directory as Phil re-coded in external/stochastic_physics directory - e7c9ada solve minor conflict in mom_cap.F90 mom_ocean_model_nuopc.F90 and MOM_energetic_PBL.F90, add two new files: src/parameterizations/stochastic/MOM_stochastics.F90 and config_src/external/stochastic_physics/stochastic_physics.F90 - 90d5961 Merge pull request #78 from jiandewang/feature/update-to-GFDL-20211019 - fd02017 Merge remote-tracking branch 'GFDL/main' into feature/update-to-GFDL-20211019 - 36f17eb Merge pull request #72 from pjpegion/ocn_stoch_july2021 - a9a957e return a more accurate error message in MOM_stochasics - 56bb41e Merge branch 'ocn_stoch_july2021' of https://github.com/pjpegion/MOM6 into ocn_stoch_july2021 - ca2ae1c update to dev/emc - 14ca4a1 Merge pull request #76 from jiandewang/feature/update-to-GFDL-20210914 - 29016c2 Merge remote-tracking branch 'GFDL/main' into feature/update-to-GFDL-20210914 merge GFDL main 20210914 commit (hash # c09e199) - a8577df Merge branch 'NOAA-EMC:dev/emc' into ocn_stoch_july2021 - f8a8e4c update to gfdl 20210806 (#74) - 16e6af0 update to dev/emc - 237a510 add comments - 1b4273d revert logic wrt increments - 5b2040e add logic to remove incrments from restart if outside IAU window - c5f2b72 add write_stoch_restart_ocn to MOM_stochastics - bdf2dc7 doxygen cleanup - 8bc4acc move stochastics to external directory - a3fa3a1 Merge remote-tracking branch 'upstream/dev/emc' into ocn_stoch_july2021 - e4bc007 stochastic physics re-write - 202cbd4 update to dev/emc - 61717ee Merge remote-tracking branch 'origin/dev/emc' into ocn_stoch - 565e0bb remove debug statements - a4c0411 Merge remote-tracking branch 'upstream/dev/emc' into ocn_stoch - 689a73f remove PE_here from mom_ocean_model_nuopc.F90 - 8afe969 clean up of mom_ocean_model_nuopc.F90 - 25ed4fc revert MOM_domains.F90 - b8d9888 place stochastic array in fluxes container and make SPPT specific arrays allocatable - d984a7e remove stochastics container - eb88219 clean up of code for MOM6 coding standards - 6e3ea1b correct coupled_driver/ocean_model_MOM.F90 and other cleanup - 0b99c1f make stochastics optional - 85023f8 Merge remote-tracking branch 'upstream/dev/emc' into ocn_stoch - 80f9f44 clean up MOM_domains - 5443f8e remove blank link in MOM_diagnostics - 1727d9a re-write of stochastic code to remove CPP directives - 600ebf9 Merge remote-tracking branch 'upstream/dev/emc' into ocn_stoch - 6bb9d0b fix non stochastic ePBL calculation - 1d7ffa3 clean up code - 040e1f1 Merge pull request #13 from NOAA-EMC/dev/emc - 2cba995 Merge branch 'dev/emc' into ocn_stoch - 1dc0f4f Merge remote-tracking branch 'upstream/dev/emc' into dev/emc - 4bd9b9e clean up debug statements - 25ed5ef additions for stochy restarts - a2a374b add stochy_restart writing to mom_cap - 0c15f4c Update MOM_diabatic_driver.F90 - 167a62e Merge pull request #12 from pjpegion/dev/emc - bd477a9 Update MOM_diabatic_driver.F90 - 7212400 Update MOM_diabatic_driver.F90 - 7de295c cleanup of code and enhancement of ePBL perts - cd06356 Merge pull request #11 from NOAA-EMC/dev/emc - 9896d61 Merge pull request #9 from pjpegion/dev/emc_merge - 0a62737 Merge branch 'ocn_stoch' into dev/emc_merge - 3cad1ba Merge pull request #8 from NOAA-EMC/dev/emc - c2aa2a8 updates from dev/emc - 182ef34 additions for stochastic physics and ePBL perts - 671c714 Merge pull request #1 from NOAA-EMC/dev/emc
This patch includes several minor changes to the MOM_file_parser and
supporting modules in order to accommodate stronger unit testing.
It includes the following API changes:
Removal of
static_value
fromget_param
Redefined
link_parameter
andparameter_block
as privateNew functions:
all_across_PEs()
,any_across_PEs()
static_value
was not used in any known experiments (outside ofinternal GFDL testing), and the two derived types describe internal
operations within
MOM_file_parser
, so we do not expect any disruptionsfrom these changes.
A detailed summary of the changes are listed below.
assert()
is now used to detect same files with different IO units.Detection of reopenend files of the same name but different IO unit
has been changed from
MOM_error(FATAL, ...)
toassert()
, toreflect that this should be a logically impossible result.
Bugfix: Reopened files are now reported to all PEs.
If an open file is re-opened, then only the root PE will detect this
and will
return
immediately. However, the others will proceed intopopulate_param_data
and will get stuck in a broadcast waiting forroot.
We fix this by communicating the reopened state to all PEs and allow
all ranks to return before re-processing the data.
Note that this could also be resolved by allowing all ranks to track
IO unit numbers, but for now we do not attempt to change this
behavior.
newunit=
used to generate parameter file IO unitThe parameter IO unit is now generated by
newunit=
rather than anexplicit search for an unused IO unit. Note that this is a Fortran
2008 feature.
Testing around available IO units has also been removed.
Removal of generic IO error handling
Generic "IO error" tests, and corresponding
err=
arguments, havebeen removed in most cases. We now rely on the Fortran runtime to
provide diagnostics on these errors, which should typically exceed any
information that MOM6 could provide.
Removal of purported
namelist
supportThere were several blocks of code provided to support namelist syntax,
but did not appear to be working, nor was there any known instance of
it being used by anyone, so it has been removed.
#define/undef/=
syntax testing across ranksPreviously, only the root PE would test for consistency of the
#define-like syntax, even though all ranks have this information.
This required a second, awkwardly placed syntax test later in the
subroutine.
This test is redefined to run over all ranks, and the subsequent test
has been removed.
define/override
test reorderingThe
found_override
test when coupled to a#define
-like declarationwas unreachable due to the presence of an even stronger test related
to valid syntax.
This test has been moved to provide more detailed information about
the nature of the error.
link_parameter
,parameter_block
defined as privateInternal derived types of
MOM_file_parser
are redefined as private.This preserves the integrity of instances of these types, and also
prevents creation of implicit object code required to access them
externally.
Removal of
static_value
fromget_param
interfaceThe
static_value
argument ofget_param
has been removed, since itis functionally equivalent to
default
. While this is an API change,there is no known case of anyone using this argument.
The
param_type%doc
fields are now properly deallocated after closed.Quotes have been added around some filename error warnings, to help
detect issues related to whitespace.
any_across_PEs
andall_across_PEs
New functions for calling
any()
andall()
across PE ranks havebeen added. Behavior is in line with other functions, such as
min_across_PEs
.