Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Condense seed dispersal update in elm_drv with a fates procedure #9

Closed
6 tasks done
glemieux opened this issue Oct 11, 2022 · 4 comments
Closed
6 tasks done

Comments

@glemieux
Copy link
Owner

glemieux commented Oct 11, 2022

The regular seed dispersal update code in elm_drv should be neatly wrapped up into fates code.

  • move mpi call into a new elmfates_interface procedure
  • refactor bc_in and bc_out assignment wrap_seed_... calls into elmfates_interfaceMod, instead of elm_driver
  • create "dispersal" type to replace the elm_driver seed_ local variable
  • Update seed dispersal code to accommodate multiple pfts fates#23
  • move allocation of seed distribution arrays out of elm_driver into an initialization routine
    - [ ] Add global variable to set the distribution frequency (check against is_beg_curr_...)
  • Add fates-side mode selection variable/logic to turn dispersal on or off
@glemieux
Copy link
Owner Author

Order of operations:

Init:

  1. Allocate bc_in/out via allocate_bcin/bcout (called from elmfates init <- elm_inst_biogeochem)
  2. Zero out bc_in(s)%seed_in and bc_out(s)%seed_out via zero_bcs (called from elmfates init <- elm_inst_biogeochem)

Dynamics:

  1. Calculate and set bc_out%seed_out(pft) via call to SeedIn (called from ed_integrate_state_variables <- ed_ecosystem_dynamics <- dynamics_driv)
  2. Send seed from bc_out%seed_out into an local output buffer for the current gridcell
  3. Distribute seed information from current gridcell to all gridcells (wrapped mpi call in elm_drv out OMP thread area)
  4. Accumulate incoming seed from global gridcell-sized array and pass back into bc_in%seed_in values via wrap_seed_dispersal

@glemieux
Copy link
Owner Author

glemieux commented Oct 18, 2022

Unless we are planning on sharing the seed information with the host land model, we don't really need this to be formatted as part of the bc_in/out data structure. That said, this setup will provide future-proofing if so desired. The wrapped mpi call does need to take place within elm_drv outside of the OMP threading pragmas, but we could still use separate fates type structure.

@glemieux
Copy link
Owner Author

glemieux commented Oct 21, 2022

I need to refresh myself on how the probability density is used in conjunction with the seed mass. This PD value for each neighboring grid cell in held in the neighborhood type and not used currently. Since its held in the neighborhood_type and indexed by gridcell, the calculation needs to happen in WrapSeedGlobal outside of the fates threaded area.

  • Review application of PD functions

UPDATE: Yes, I need to update the incoming_global value based on the PD. See glemieux/fates#23 for further updates.

@glemieux
Copy link
Owner Author

glemieux commented Nov 1, 2022

Complete

@glemieux glemieux closed this as completed Nov 1, 2022
glemieux pushed a commit that referenced this issue Jun 13, 2023
cee/15.0.0 with GPU MPI buffers can crash in a system lib like this:

#4  0x00007fffe159e35b in (anonymous namespace)::do_free_with_callback(void*, void (*)(void*)) [clone .constprop.0] () from /opt/cray/pe/cce/15.0.0/cce/x86_64/lib/libtcmalloc_minimal.so.1
#5  0x00007fffe15a8f16 in tc_free () from /opt/cray/pe/cce/15.0.0/cce/x86_64/lib/libtcmalloc_minimal.so.1
#6  0x00007fffe99c2bcd in _dlerror_run () from /lib64/libdl.so.2
#7  0x00007fffe99c2481 in dlopen@@GLIBC_2.2.5 () from /lib64/libdl.so.2
#8  0x00007fffea7bce42 in _ad_cray_lock_init () from /opt/cray/pe/lib64/libmpi_cray.so.12
#9  0x00007fffed7eb37a in call_init.part () from /lib64/ld-linux-x86-64.so.2
#10 0x00007fffed7eb496 in _dl_init () from /lib64/ld-linux-x86-64.so.2
#11 0x00007fffed7dc58a in _dl_start_user () from /lib64/ld-linux-x86-64.so.2
#12 0x0000000000000001 in ?? ()
#13 0x00007fffffff42e7 in ?? ()
#14 0x0000000000000000 in ?? ()

Work around this by using cee/14.0.3.
glemieux pushed a commit that referenced this issue Nov 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant