Skip to content

Commit

Permalink
ESCOMP tag: cam6_3_134
Browse files Browse the repository at this point in the history
Merge pull request ESCOMP#891 from nusbaume/atmos_phys_update

cam6_3_134: Update atmospheric_physics external

ESCOMP commit: 1e09295
  • Loading branch information
nusbaume authored and Steve Goldhaber committed Apr 22, 2024
1 parent d1e76b8 commit ea17ee2
Show file tree
Hide file tree
Showing 22 changed files with 614 additions and 256 deletions.
128 changes: 128 additions & 0 deletions doc/ChangeLog
Original file line number Diff line number Diff line change
@@ -1,5 +1,133 @@
===============================================================

Tag name: cam6_3_134
Originator(s): nusbaume, jimmielin
Date: 31 Oct 2023
One-line Summary: Update atmospheric_physics external
Github PR URL: https://github.com/ESCOMP/CAM/pull/891

Purpose of changes (include the issue number and title text for each relevant GitHub issue):

This PR updates the atmospheric_physics external in CAM,
which requires some source modifications on the CAM-side.
There has also been some replacement of CAM code with CCPP
physics routines in order to reduce duplication.

This PR also fixes a bug found in Kessler where it was using
the wrong pressure when converting to/from potential temperature
via the exner function.

Finally, this change also brings in a rename of the SE
dycore's 'time_mod' module to 'se_dyn_time_mod' in order
to avoid name collision with GEOS-Chem code.

Fixes #752 -> Update atmospheric_physics external
Fixes #802 -> Kessler physics using inconsistent reference pressures

Closes #904 -> Rename SE dycore time_mod to dyn_time_mod (PR)

Describe any changes made to build system:

Added the "src/utils/cam_ccpp" and "src/atmos_phys/utilities" directory
to the list of directories used during compilation.

Describe any changes made to the namelist: N/A

List any changes to the defaults for the boundary datasets: N/A

Describe any substantial timing or memory changes: N/A

Code reviewed by: cacraigucar, PeterHjortLauritzen, jtruesdal, peverwhee

List all files eliminated:

R src/dynamics/se/dycore/time_mod.F90
- Renamed to 'se_dyn_time_mod'
R src/utils/ccpp_kinds.F90
- Moved to 'src/utils/cam_ccpp/ccpp_kinds.F90'

List all files added and what they do:

A src/utils/cam_ccpp/ccpp_constituent_prop_mod.F90
- Creates a "CCPP Constituent Properties" object for CAM.

List all existing files that have been modified, and describe the changes:

M Externals_CAM.cfg
- Update 'atmospheric_physics' external

M bld/configure
- Add 'src/utils/cam_ccpp' and 'src/atmos_phys/utilities' to build file paths

M src/dynamics/se/dp_coupling.F90
- Use CCPP-ized 'update_dry_static_energy_run' subroutine

M src/physics/cam/geopotential.F90
- Use CCPP-ized 'geopotential_temp_run' subroutine

M src/physics/cam/ref_pres.F90
- Update comment in order to avoid future confusion

M src/physics/cam/physpkg.F90
M src/physics/cam_dev/physpkg.F90
M src/physics/simple/held_suarez_cam.F90
M src/physics/simple/physpkg.F90
- Update physics calls and add new const. prop. DDT to work with new
atmospheric_physics external

M src/physics/simple/kessler_cam.F90
- Update physics calls and add new const. prop. DDT to work with new
atmospheric_physics external. Also replace use of "standard pressure"
with "reference pressure" in order to avoid unphysical mismatches.

M src/dynamics/se/dycore/fvm_consistent_se_cslam.F90
M src/dynamics/se/dycore/prim_advance_mod.F90
M src/dynamics/se/dycore/prim_advection_mod.F90
M src/dynamics/se/dycore/prim_driver_mod.F90
M src/dynamics/se/dycore/prim_init.F90
M src/dynamics/se/dycore/prim_state_mod.F90
M src/dynamics/se/stepon.F90
- Replace 'time_mod' with 'se_dyn_time_mod'

If there were any failures reported from running test_driver.sh on any test
platform, and checkin with these failures has been OK'd by the gatekeeper,
then copy the lines from the td.*.status files for the failed tests to the
appropriate machine below. All failed tests must be justified.

cheyenne/intel/aux_cam:

All tests have NLCOMP failures due to changes in input data paths on Cheyenne.

ERP_Ln9_Vnuopc.C96_C96_mg17.F2000climo.cheyenne_intel.cam-outfrq9s_mg3 (Overall: FAIL) details:
FAIL ERP_Ln9_Vnuopc.C96_C96_mg17.F2000climo.cheyenne_intel.cam-outfrq9s_mg3 NLCOMP
FAIL ERP_Ln9_Vnuopc.C96_C96_mg17.F2000climo.cheyenne_intel.cam-outfrq9s_mg3 MODEL_BUILD time=3
ERP_Ln9_Vnuopc.f09_f09_mg17.FCSD_HCO.cheyenne_intel.cam-outfrq9s (Overall: FAIL) details:
FAIL ERP_Ln9_Vnuopc.f09_f09_mg17.FCSD_HCO.cheyenne_intel.cam-outfrq9s NLCOMP
FAIL ERP_Ln9_Vnuopc.f09_f09_mg17.FCSD_HCO.cheyenne_intel.cam-outfrq9s COMPARE_base_rest
SMS_Lh12_Vnuopc.f09_f09_mg17.FCSD_HCO.cheyenne_intel.cam-outfrq3h (Overall: DIFF) details:
FAIL SMS_Lh12_Vnuopc.f09_f09_mg17.FCSD_HCO.cheyenne_intel.cam-outfrq3h NLCOMP
FAIL SMS_Lh12_Vnuopc.f09_f09_mg17.FCSD_HCO.cheyenne_intel.cam-outfrq3h BASELINE /glade/p/cesm/amwg/cesm_baselines/cam6_3_133: DIFF
- pre-existing failure

izumi/nag/aux_cam:

DAE_Vnuopc.f45_f45_mg37.FHS94.izumi_nag.cam-dae (Overall: FAIL) details:
FAIL DAE_Vnuopc.f45_f45_mg37.FHS94.izumi_nag.cam-dae RUN time=9
- pre-existing failure

ERS_Ln27_Vnuopc.ne5pg3_ne5pg3_mg37.FKESSLER.izumi_nag.cam-outfrq9s (Overall: DIFF) details:
FAIL ERS_Ln27_Vnuopc.ne5pg3_ne5pg3_mg37.FKESSLER.izumi_nag.cam-outfrq9s BASELINE /fs/cgd/csm/models/atm/cam/pretag_bl/cam6_3_133_nag: DIFF
- expected failure

izumi/gnu/aux_cam: All PASS

Summarize any changes to answers:
All compsets which use Kessler idealized physics
will have larger-than-roundoff answer changes due
to reference pressure bug fix.

===============================================================

Tag name: cam6_3_133
Originator(s): fvitt
Date: 19 Oct 2023
Expand Down
29 changes: 17 additions & 12 deletions src/dynamics/se/dp_coupling.F90
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ subroutine d_p_coupling(phys_state, phys_tend, pbuf2d, dyn_out)
use phys_control, only: use_gw_front, use_gw_front_igw
use hycoef, only: hyai, ps0
use fvm_mapping, only: dyn2phys_vector, dyn2phys_all_vars
use time_mod, only: timelevel_qdp
use se_dyn_time_mod, only: timelevel_qdp
use control_mod, only: qsplit
use test_fvm_mapping, only: test_mapping_overwrite_dyn_state, test_mapping_output_phys_state
use prim_advance_mod, only: tot_energy_dyn
Expand Down Expand Up @@ -546,15 +546,17 @@ subroutine derived_phys_dry(phys_state, phys_tend, pbuf2d)
use shr_const_mod, only: shr_const_rwv
use phys_control, only: waccmx_is
use geopotential, only: geopotential_t
use static_energy, only: update_dry_static_energy_run
use check_energy, only: check_energy_timestep_init
use hycoef, only: hyai, ps0
use shr_vmath_mod, only: shr_vmath_log
use qneg_module, only: qneg3
use dyn_tests_utils, only: vc_dry_pressure
use shr_kind_mod, only: shr_kind_cx
! arguments
type(physics_state), intent(inout), dimension(begchunk:endchunk) :: phys_state
type(physics_tend ), intent(inout), dimension(begchunk:endchunk) :: phys_tend
type(physics_buffer_desc), pointer :: pbuf2d(:,:)
type(physics_buffer_desc), pointer :: pbuf2d(:,:)

! local variables
integer :: lchnk
Expand All @@ -564,6 +566,10 @@ subroutine derived_phys_dry(phys_state, phys_tend, pbuf2d)

integer :: m, i, k, ncol, m_cnst
type(physics_buffer_desc), pointer :: pbuf_chnk(:)

!Needed for "update_dry_static_energy" CCPP scheme
integer :: errflg
character(len=shr_kind_cx) :: errmsg
!----------------------------------------------------------------------------

! Evaluate derived quantities
Expand Down Expand Up @@ -686,18 +692,17 @@ subroutine derived_phys_dry(phys_state, phys_tend, pbuf2d)
end do

! Compute initial geopotential heights - based on full pressure
call geopotential_t (phys_state(lchnk)%lnpint, phys_state(lchnk)%lnpmid , phys_state(lchnk)%pint , &
phys_state(lchnk)%pmid , phys_state(lchnk)%pdel , phys_state(lchnk)%rpdel , &
phys_state(lchnk)%t , phys_state(lchnk)%q(:,:,:), rairv(:,:,lchnk), gravit, zvirv , &
phys_state(lchnk)%zi , phys_state(lchnk)%zm , ncol )
call geopotential_t(phys_state(lchnk)%lnpint, phys_state(lchnk)%lnpmid , phys_state(lchnk)%pint, &
phys_state(lchnk)%pmid , phys_state(lchnk)%pdel , phys_state(lchnk)%rpdel , &
phys_state(lchnk)%t , phys_state(lchnk)%q(:,:,:), rairv(:,:,lchnk), gravit, zvirv , &
phys_state(lchnk)%zi , phys_state(lchnk)%zm , ncol)

! Compute initial dry static energy, include surface geopotential
do k = 1, pver
do i = 1, ncol
phys_state(lchnk)%s(i,k) = cpairv(i,k,lchnk)*phys_state(lchnk)%t(i,k) &
+ gravit*phys_state(lchnk)%zm(i,k) + phys_state(lchnk)%phis(i)
end do
end do
call update_dry_static_energy_run(pver, gravit, phys_state(lchnk)%t(1:ncol,:), &
phys_state(lchnk)%zm(1:ncol,:), &
phys_state(lchnk)%phis(1:ncol), &
phys_state(lchnk)%s(1:ncol,:), &
cpairv(1:ncol,:,lchnk), errflg, errmsg)

! Ensure tracers are all positive
call qneg3('D_P_COUPLING',lchnk ,ncol ,pcols ,pver , &
Expand Down
2 changes: 1 addition & 1 deletion src/dynamics/se/dycore/fvm_consistent_se_cslam.F90
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ module fvm_consistent_se_cslam
use cam_abortutils, only: endrun
use cam_logfile, only: iulog

use time_mod, only: timelevel_t
use se_dyn_time_mod, only: timelevel_t
use element_mod, only: element_t
use fvm_control_volume_mod, only: fvm_struct
use hybrid_mod, only: hybrid_t, config_thread_region, get_loop_ranges, threadOwnsVertLevel
Expand Down
2 changes: 1 addition & 1 deletion src/dynamics/se/dycore/prim_advance_mod.F90
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ subroutine prim_advance_exp(elem, fvm, deriv, hvcoord, hybrid,dt, tl, nets, net
use element_mod, only: element_t
use hybvcoord_mod, only: hvcoord_t
use hybrid_mod, only: hybrid_t
use time_mod, only: TimeLevel_t, timelevel_qdp, tevolve
use se_dyn_time_mod, only: TimeLevel_t, timelevel_qdp, tevolve
use fvm_control_volume_mod, only: fvm_struct
use cam_thermo, only: get_kappa_dry
use air_composition, only: thermodynamic_active_species_num
Expand Down
2 changes: 1 addition & 1 deletion src/dynamics/se/dycore/prim_advection_mod.F90
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ module prim_advection_mod
use element_mod, only: element_t
use fvm_control_volume_mod, only: fvm_struct
use hybvcoord_mod, only: hvcoord_t
use time_mod, only: TimeLevel_t, TimeLevel_Qdp
use se_dyn_time_mod, only: TimeLevel_t, TimeLevel_Qdp
use control_mod, only: nu_q, nu_p, limiter_option, hypervis_subcycle_q, rsplit
use edge_mod, only: edgevpack, edgevunpack, initedgebuffer, initedgesbuffer

Expand Down
8 changes: 4 additions & 4 deletions src/dynamics/se/dycore/prim_driver_mod.F90
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,8 @@ subroutine prim_init2(elem, fvm, hybrid, nets, nete, tl, hvcoord)
use dimensions_mod, only: irecons_tracer, fvm_supercycling
use dimensions_mod, only: fv_nphys, nc
use parallel_mod, only: syncmp
use time_mod, only: timelevel_t, tstep, phys_tscale, nsplit, TimeLevel_Qdp
use time_mod, only: nsplit_baseline,rsplit_baseline
use se_dyn_time_mod, only: timelevel_t, tstep, phys_tscale, nsplit, TimeLevel_Qdp
use se_dyn_time_mod, only: nsplit_baseline,rsplit_baseline
use prim_state_mod, only: prim_printstate
use control_mod, only: runtype, topology, rsplit, qsplit, rk_stage_user, &
nu, nu_q, nu_div, hypervis_subcycle, hypervis_subcycle_q, &
Expand Down Expand Up @@ -218,7 +218,7 @@ subroutine prim_run_subcycle(elem, fvm, hybrid,nets,nete, dt, tl, hvcoord,nsubst
!
!
use hybvcoord_mod, only : hvcoord_t
use time_mod, only: TimeLevel_t, timelevel_update, timelevel_qdp, nsplit
use se_dyn_time_mod, only: TimeLevel_t, timelevel_update, timelevel_qdp, nsplit
use control_mod, only: statefreq,qsplit, rsplit, variable_nsplit
use prim_advance_mod, only: applycamforcing
use prim_advance_mod, only: tot_energy_dyn,compute_omega
Expand Down Expand Up @@ -407,7 +407,7 @@ subroutine prim_step(elem, fvm, hybrid,nets,nete, dt, tl, hvcoord, rstep)
! tl%n0 time t + dt_q
!
use hybvcoord_mod, only: hvcoord_t
use time_mod, only: TimeLevel_t, timelevel_update
use se_dyn_time_mod, only: TimeLevel_t, timelevel_update
use control_mod, only: statefreq, qsplit, nu_p
use thread_mod, only: omp_get_thread_num
use prim_advance_mod, only: prim_advance_exp
Expand Down
2 changes: 1 addition & 1 deletion src/dynamics/se/dycore/prim_init.F90
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ subroutine prim_init1(elem, fvm, par, Tl)
use element_mod, only: element_t, allocate_element_desc
use fvm_mod, only: fvm_init1
use mesh_mod, only: MeshUseMeshFile
use time_mod, only: timelevel_init, timelevel_t
use se_dyn_time_mod, only: timelevel_init, timelevel_t
use mass_matrix_mod, only: mass_matrix
use derivative_mod, only: allocate_subcell_integration_matrix_cslam
use derivative_mod, only: allocate_subcell_integration_matrix_physgrid
Expand Down
8 changes: 4 additions & 4 deletions src/dynamics/se/dycore/prim_state_mod.F90
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ module prim_state_mod
use dimensions_mod, only: nlev, np, nc, qsize_d, ntrac_d
use parallel_mod, only: ordered
use hybrid_mod, only: hybrid_t
use time_mod, only: timelevel_t, TimeLevel_Qdp, time_at
use se_dyn_time_mod, only: timelevel_t, TimeLevel_Qdp, time_at
use control_mod, only: qsplit, statediag_numtrac
use global_norms_mod, only: global_integrals_general
use element_mod, only: element_t
Expand All @@ -24,7 +24,7 @@ subroutine prim_printstate(elem, tl,hybrid,nets,nete, fvm, omega_cn)
use air_composition, only: thermodynamic_active_species_idx_dycore, dry_air_species_num
use air_composition, only: thermodynamic_active_species_num,thermodynamic_active_species_idx
use cam_control_mod, only: initial_run
use time_mod, only: tstep
use se_dyn_time_mod, only: tstep
use control_mod, only: rsplit, qsplit
use perf_mod, only: t_startf, t_stopf
type (element_t), intent(inout) :: elem(:)
Expand Down Expand Up @@ -345,10 +345,10 @@ end subroutine prim_printstate_cslam_gamma
subroutine adjust_nsplit(elem, tl,hybrid,nets,nete, fvm, omega_cn)
use dimensions_mod, only: ksponge_end
use dimensions_mod, only: fvm_supercycling, fvm_supercycling_jet
use time_mod, only: tstep
use se_dyn_time_mod, only: tstep
use control_mod, only: rsplit, qsplit
use perf_mod, only: t_startf, t_stopf
use time_mod, only: nsplit, nsplit_baseline,rsplit_baseline
use se_dyn_time_mod, only: nsplit, nsplit_baseline,rsplit_baseline
use control_mod, only: qsplit, rsplit
use time_manager, only: get_step_size
use cam_abortutils, only: endrun
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
module time_mod
module se_dyn_time_mod
!------------------
use shr_kind_mod, only: r8=>shr_kind_r8
!------------------
Expand Down Expand Up @@ -132,4 +132,4 @@ subroutine TimeLevel_update(tl,uptype)
!$OMP BARRIER
end subroutine TimeLevel_update

end module time_mod
end module se_dyn_time_mod
6 changes: 3 additions & 3 deletions src/dynamics/se/dyn_comp.F90
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ module dyn_comp
use dimensions_mod, only: qsize, use_cslam
use element_mod, only: element_t, elem_state_t
use fvm_control_volume_mod, only: fvm_struct
use time_mod, only: nsplit
use se_dyn_time_mod, only: nsplit
use edge_mod, only: initEdgeBuffer, edgeVpack, edgeVunpack, FreeEdgeBuffer
use edgetype_mod, only: EdgeBuffer_t
use bndry_mod, only: bndry_exchange
Expand Down Expand Up @@ -963,11 +963,11 @@ subroutine dyn_run(dyn_state)
use air_composition, only: thermodynamic_active_species_idx_dycore
use prim_driver_mod, only: prim_run_subcycle
use dimensions_mod, only: cnst_name_gll
use time_mod, only: tstep, nsplit, timelevel_qdp
use se_dyn_time_mod, only: tstep, nsplit, timelevel_qdp
use hybrid_mod, only: config_thread_region, get_loop_ranges
use control_mod, only: qsplit, rsplit, ftype_conserve
use thread_mod, only: horz_num_threads
use time_mod, only: tevolve
use se_dyn_time_mod, only: tevolve

type(dyn_export_t), intent(inout) :: dyn_state

Expand Down
4 changes: 2 additions & 2 deletions src/dynamics/se/dyn_grid.F90
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ module dyn_grid
use prim_init, only: prim_init1
use edge_mod, only: initEdgeBuffer
use edgetype_mod, only: EdgeBuffer_t
use time_mod, only: TimeLevel_t
use se_dyn_time_mod, only: TimeLevel_t
use dof_mod, only: UniqueCoords, UniquePoints

implicit none
Expand Down Expand Up @@ -133,7 +133,7 @@ subroutine dyn_grid_init()
use hybrid_mod, only: hybrid_t, init_loop_ranges, &
get_loop_ranges, config_thread_region
use control_mod, only: qsplit, rsplit
use time_mod, only: tstep, nsplit
use se_dyn_time_mod, only: tstep, nsplit
use fvm_mod, only: fvm_init2, fvm_init3, fvm_pg_init
use dimensions_mod, only: irecons_tracer
use comp_gll_ctr_vol, only: gll_grid_write
Expand Down
2 changes: 1 addition & 1 deletion src/dynamics/se/restart_dynamics.F90
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ module restart_dynamics
use dimensions_mod, only: np, npsq, ne, nlev, qsize, nelemd, nc, ntrac, use_cslam
use dof_mod, only: UniquePoints
use element_mod, only: element_t
use time_mod, only: tstep, TimeLevel_Qdp
use se_dyn_time_mod, only: tstep, TimeLevel_Qdp

use edge_mod, only: initEdgeBuffer, edgeVpack, edgeVunpack, FreeEdgeBuffer
use edgetype_mod, only: EdgeBuffer_t
Expand Down
8 changes: 4 additions & 4 deletions src/dynamics/se/stepon.F90
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ subroutine stepon_run1( dtime_out, phys_state, phys_tend, &
use dp_coupling, only: d_p_coupling
use physics_buffer, only: physics_buffer_desc

use time_mod, only: tstep ! dynamics timestep
use se_dyn_time_mod,only: tstep ! dynamics timestep

real(r8), intent(out) :: dtime_out ! Time-step
type(physics_state), intent(inout) :: phys_state(begchunk:endchunk)
Expand Down Expand Up @@ -152,7 +152,7 @@ subroutine stepon_run2(phys_state, phys_tend, dyn_in, dyn_out)
use dp_coupling, only: p_d_coupling
use dyn_grid, only: TimeLevel

use time_mod, only: TimeLevel_Qdp
use se_dyn_time_mod, only: TimeLevel_Qdp
use control_mod, only: qsplit
use prim_advance_mod, only: tot_energy_dyn

Expand Down Expand Up @@ -207,7 +207,7 @@ subroutine stepon_run3(dtime, cam_out, phys_state, dyn_in, dyn_out)
use dyn_comp, only: dyn_run
use advect_tend, only: compute_adv_tends_xyz
use dyn_grid, only: TimeLevel
use time_mod, only: TimeLevel_Qdp
use se_dyn_time_mod,only: TimeLevel_Qdp
use control_mod, only: qsplit
! arguments
real(r8), intent(in) :: dtime ! Time-step
Expand Down Expand Up @@ -254,7 +254,7 @@ subroutine diag_dynvar_ic(elem, fvm)
use cam_history, only: write_inithist, outfld, hist_fld_active, fieldname_len
use dyn_grid, only: TimeLevel

use time_mod, only: TimeLevel_Qdp ! dynamics typestep
use se_dyn_time_mod, only: TimeLevel_Qdp ! dynamics typestep
use control_mod, only: qsplit
use hybrid_mod, only: config_thread_region, get_loop_ranges
use hybrid_mod, only: hybrid_t
Expand Down
Loading

0 comments on commit ea17ee2

Please sign in to comment.