Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update OMP #680

Merged
merged 15 commits into from
Feb 18, 2022
Merged

Update OMP #680

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
3003c67
Add narwhal intel, gnu, cray, aocc
apcraig Dec 30, 2021
3525b14
update narwhal_cray and perf_suite
apcraig Dec 31, 2021
80e8caa
Review and update OMP implementation
apcraig Jan 7, 2022
deda037
Merge branch 'ompB' of https://github.com/apcraig/cice into ompC
apcraig Jan 7, 2022
9c442bc
- Review diagnostics OMP implementation
apcraig Jan 10, 2022
3446cf6
Update EVP OMP implementation
apcraig Jan 11, 2022
f91f459
- Refactor puny/pi scalars in eap dynamics to improve performance
apcraig Jan 11, 2022
7905bf6
Clean up
apcraig Jan 12, 2022
667e998
Comment out temporary timers
apcraig Jan 12, 2022
6bb8e22
Update OMP env variables on Narwhal
apcraig Jan 13, 2022
0d4c2e3
Update gaffney OMP_STACKSIZE
apcraig Jan 14, 2022
d56737c
update OMP_STACKSIZE on cori
apcraig Jan 14, 2022
021e0f3
Update Onyx OMP_STACKSIZE
apcraig Jan 14, 2022
3a00b69
Update OMP_STACKSIZE on mustang
apcraig Jan 17, 2022
c8b9f17
- Update Tsfc values on land in various places in the code, was affec…
apcraig Jan 19, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 5 additions & 7 deletions cicecore/cicedynB/dynamics/ice_dyn_evp.F90
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -239,9 +239,7 @@ subroutine evp (dt)
call t2ugrid_vector(strairy)
endif

! tcraig, tcx, threading here leads to some non-reproducbile results and failures in icepack_ice_strength
! need to do more debugging
!$TCXOMP PARALLEL DO PRIVATE(iblk,ilo,ihi,jlo,jhi,this_block,ij,i,j)
!$OMP PARALLEL DO PRIVATE(iblk,ilo,ihi,jlo,jhi,this_block,ij,i,j)
do iblk = 1, nblocks

!-----------------------------------------------------------------
Expand Down Expand Up @@ -301,7 +299,7 @@ subroutine evp (dt)
enddo ! ij

enddo ! iblk
!$TCXOMP END PARALLEL DO
!$OMP END PARALLEL DO

call icepack_warnings_flush(nu_diag)
if (icepack_warnings_aborted()) call abort_ice(error_message=subname, &
Expand Down Expand Up @@ -354,7 +352,7 @@ subroutine evp (dt)
endif

enddo
!$OMP END PARALLEL DO
!$OMP END PARALLEL DO
endif

call ice_timer_start(timer_evp_2d)
Expand Down Expand Up @@ -399,7 +397,7 @@ subroutine evp (dt)
! stress tensor equation, total surface stress
!-----------------------------------------------------------------

!$TCXOMP PARALLEL DO PRIVATE(iblk,strtmp)
!$OMP PARALLEL DO PRIVATE(iblk,strtmp)
do iblk = 1, nblocks

! if (trim(yield_curve) == 'ellipse') then
Expand Down Expand Up @@ -445,7 +443,7 @@ subroutine evp (dt)
Tbu (:,:,iblk))

enddo
!$TCXOMP END PARALLEL DO
!$OMP END PARALLEL DO

call stack_velocity_field(uvel, vvel, fld2)
call ice_timer_start(timer_bound)
Expand Down
1 change: 0 additions & 1 deletion configuration/scripts/cice.batch.csh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,6 @@ EOFB
else if (${ICE_MACHINE} =~ narwhal*) then
if (${runlength} <= 0) then
set batchtime = "00:29:59"
set queue = "debug"
else
set queue = "standard"
endif
Expand Down
1 change: 1 addition & 0 deletions configuration/scripts/options/set_nml.timerstats
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
timer_stats = .true.
24 changes: 13 additions & 11 deletions doc/source/user_guide/ug_implementation.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1063,9 +1063,11 @@ Currently, the timers are set up as in :ref:`timers`.
Section :ref:`addtimer` contains instructions for adding timers.

The timings provided by these timers are not mutually exclusive. For
example, the column timer (5) includes the timings from 6–10, and
subroutine *bound* (timer 15) is called from many different places in
the code, including the dynamics and advection routines.
example, the Column timer includes the timings from several other
timers, while timer Bound is called from many different places in
the code, including the dynamics and advection routines. The
Dynamics, Advection, and Column timers do not overlap and represent
most of the overall model work.

The timers use *MPI\_WTIME* for parallel runs and the F90 intrinsic
*system\_clock* for single-processor runs.
Expand All @@ -1083,19 +1085,19 @@ The timers use *MPI\_WTIME* for parallel runs and the F90 intrinsic
+--------------+-------------+----------------------------------------------------+
| 2 | Timeloop | total minus initialization and exit |
+--------------+-------------+----------------------------------------------------+
| 3 | Dynamics | EVP |
| 3 | Dynamics | dynamics |
+--------------+-------------+----------------------------------------------------+
| 4 | Advection | horizontal transport |
+--------------+-------------+----------------------------------------------------+
| 5 | Column | all vertical (column) processes |
+--------------+-------------+----------------------------------------------------+
| 6 | Thermo | vertical thermodynamics |
| 6 | Thermo | vertical thermodynamics, part of Column timer |
+--------------+-------------+----------------------------------------------------+
| 7 | Shortwave | SW radiation and albedo |
| 7 | Shortwave | SW radiation and albedo, part of Thermo timer |
+--------------+-------------+----------------------------------------------------+
| 8 | Ridging | mechanical redistribution |
| 8 | Ridging | mechanical redistribution, part of Column timer |
+--------------+-------------+----------------------------------------------------+
| 9 | FloeSize | flow size |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice additions!

| 9 | FloeSize | flow size, part of Column timer |
+--------------+-------------+----------------------------------------------------+
| 10 | Coupling | sending/receiving coupler messages |
+--------------+-------------+----------------------------------------------------+
Expand All @@ -1107,13 +1109,13 @@ The timers use *MPI\_WTIME* for parallel runs and the F90 intrinsic
+--------------+-------------+----------------------------------------------------+
| 14 | Bound | boundary conditions and subdomain communications |
+--------------+-------------+----------------------------------------------------+
| 15 | BGC | biogeochemistry |
| 15 | BGC | biogeochemistry, part of Thermo timer |
+--------------+-------------+----------------------------------------------------+
| 16 | Forcing | forcing |
+--------------+-------------+----------------------------------------------------+
| 17 | 1d-evp | 1d evp |
| 17 | 1d-evp | 1d evp, part of Dynamics timer |
+--------------+-------------+----------------------------------------------------+
| 18 | 2d-evp | 2d evp |
| 18 | 2d-evp | 2d evp, part of Dynamics timer |
+--------------+-------------+----------------------------------------------------+
| 19 | UpdState | update state |
+--------------+-------------+----------------------------------------------------+
Expand Down