Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update to 1.2.0-alpha.6 #641

Merged
merged 20 commits into from
Aug 4, 2023
Merged

Update to 1.2.0-alpha.6 #641

merged 20 commits into from
Aug 4, 2023

Conversation

xylar
Copy link
Collaborator

@xylar xylar commented Jun 5, 2023 •
edited
Loading

This merge updates compass to v. 1.2.0-alpha.6. This update:

  • Brings in a new mache with support for Intel on Perlmutter and which allows for spack environment builds on non-E3SM machines
  • Updates to a new pyremap that supports specifying the NetCDF format and (where appropriate) the xarray NetCDF engine
  • Names spack clones the same as the spack environment that will be created in them (to avoid multiple developers trying to use the same clone of spack)
  • Adds support for deploying on supported machines that aren't part of mache (non-E3SM machines), including documentation and some examples (2 of my laptops)
  • Update versions of HDF5, NetCDF-C, NetCDF-Fortran, Parallel-NetCDF, ESMF and PETSc in spack environments
  • Drops the --with_netlib_lapack flag, since we currently require that library always wehen PETSc is installed.

Checklist

  • Developer's Guide has been updated
  • Documentation has been built locally and changes look as expected
  • The E3SM-Project submodule has been updated with relevant E3SM changes
  • Document (in a comment titled Testing in this PR) any testing that was used to verify the changes

closes #607

Testing

MPAS-Ocean with pr:

MALI with full_integration - as discussed with @matthewhoffman, validation failures in landice/circular_shelf/decomposition_test indicate that thresholds need to be adjusted. Check marks indicate that all other tests are passing:

  • Chicoma
    • gnu and mpich
  • Chrysalis
    • gnu and openmpi
  • Perlmutter
    • gnu and mpich

MPAS-Ocean with nonhydro:

  • Anvil
    • intel and impi
  • Chicoma
    • gnu and mpich
  • Chrysalis
    • intel and openmpi
  • Compy
    • intel and impi
  • Perlmutter
    • gnu and mpich

Deployed

MPAS-Ocean with pr:

MALI with full_integration (passing excep #654):

  • Chicoma
    • gnu and mpich
  • Chrysalis
    • gnu and openmpi
  • Perlmutter
    • gnu and mpich

MPAS-Ocean with nonhydro:

  • Anvil
    • intel and impi
    • gnu and openmpi
  • Chicoma
    • gnu and mpich
  • Chrysalis
    • intel and openmpi
    • gnu and openmpi
  • Compy
    • intel and impi
  • Perlmutter
    • gnu and mpich

@xylar xylar requested a review from matthewhoffman June 5, 2023 23:11
@xylar xylar self-assigned this Jun 5, 2023
@xylar xylar added the dependencies and deployment Changes relate to creating conda and Spack environments, and creating a load script label Jun 5, 2023
@xylar
Copy link
Collaborator Author

xylar commented Jun 5, 2023

This needs E3SM-Project/E3SM#5732 and an equivalent update to MALI-Dev.

@xylar xylar force-pushed the update-to-1.2.0-alpha.6 branch 3 times, most recently from 9f7e23f to f888406 Compare June 7, 2023 06:00
@xylar xylar added the in progress This PR is not ready for review or merging label Jun 7, 2023
@xylar
Copy link
Collaborator Author

xylar commented Jun 10, 2023

mache will need to be updated because of E3SM-Project/E3SM#5755

@xylar xylar mentioned this pull request Jun 14, 2023
Merged
@xylar xylar force-pushed the update-to-1.2.0-alpha.6 branch from c8792ce to 9f67ac7 Compare June 18, 2023 08:33
@xylar
Copy link
Collaborator Author

xylar commented Jun 18, 2023

ESMF_RegridWeightGen is giving me trouble with intel on Perlmutter, so it seems some more debugging is needed there.

A few other tests are also failing both for MPAS-Ocean and MALI so that will also need some debugging...

@xylar xylar force-pushed the update-to-1.2.0-alpha.6 branch 3 times, most recently from f16cd3b to b2ae644 Compare June 29, 2023 16:14
@xylar xylar force-pushed the update-to-1.2.0-alpha.6 branch 7 times, most recently from 552842d to 969fcef Compare July 22, 2023 12:43
@xylar xylar removed the in progress This PR is not ready for review or merging label Jul 22, 2023
@xylar
Copy link
Collaborator Author

xylar commented Jul 22, 2023

It doesn't seem possible to support Intel on Perlmutter at this time. I was not able to get ESMF working reliably with it. We will try again in the future, perhaps after we move to mbtempest instead of ESMF_RegridWeightGen.

@xylar xylar added the in progress This PR is not ready for review or merging label Jul 22, 2023
xylar added 8 commits July 31, 2023 10:42
@xylar
Drop pio from esmf spec
84ab6bf
@xylar
Fix quotes in spack env. yaml
a892626
@xylar
Switch from SCORPIO to ParallelIO
7d8db08 
Update to latest ESMF

Set `HAVE_ADIOS=false` in the load script
@xylar
For known machines, use config file in spack builds
9a5ec5f 
This will allow support for machines that are unknown to mache.
@xylar
Add support for new machine morpheus
7bce0ff 
This is my laptop as an example of a machine not supported by
mache.
@xylar
Skip exception when getting config from mache
b75efb3 
To support machines that aren't known to mache, we need to suppress
exceptions when trying to read mache config options for
a machine.
@xylar
Support config option for identifying known machines
445f5d1 
This merge adds a config option `hostname_contains` to a new
config section `discovery` that can be used to identify the
machine by hostname if `mache` is unable to identify it.

This capability has been added both when calling the configure
script and when setting up test cases.
@xylar
Add support for new machine eligos
ad9c19e
@xylar xylar force-pushed the update-to-1.2.0-alpha.6 branch from f138884 to f92cdc0 Compare July 31, 2023 08:42
xylar added 10 commits July 31, 2023 10:57
@xylar
Add documentation on adding a supported machine
84dac6d
@xylar
Update spack package versions
c6de1b9
@xylar
Update pyremap version
fc8c4f9 
This brings in pyremap 1.0.0, which supports specifying the
NetCDF format and xarray output engine.
@xylar
Add ncview to the development environment
ebac565
@xylar
Update developer's guide
45c968b
@xylar
Switch back to SCORPIO
2f4289c 
Build ESMF without PNetCDF and with internal ParallelIO
@xylar
Add hack to make sure scorpio, not esmf, provides libpio*.a
3e166e6
@xylar
Update to mache 1.16.0
83c80e5
@xylar
Remove --with_netlib_lapack flag
71e1532 
This is tedious to type and the suffix causes environment names
that are too long (spack can't create a valid shebang).
@xylar
Remove Intel-MPI from supported configs on Chrysalis
625485c 
Also remove Gnu from Compy in the docs (already marked as
unsuppored).
@xylar xylar force-pushed the update-to-1.2.0-alpha.6 branch from f92cdc0 to 625485c Compare July 31, 2023 08:57
@xylar
Copy link
Collaborator Author

xylar commented Jul 31, 2023

@matthewhoffman, maybe this is something you'll be able to test and approve once LCRC is back up?

@matthewhoffman matthewhoffman mentioned this pull request Aug 4, 2023
Merged
3 tasks
matthewhoffman
matthewhoffman approved these changes Aug 4, 2023
View reviewed changes
Copy link
Member

@matthewhoffman matthewhoffman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@xylar , thanks for the substantial amount of work in putting this together! My MALI full_integration tests look good, and I blessed the small changes expected from the update of the Albany version. I also tested with #634 and the depth-integrated solver works as expected with the new version of Albany in this PR.

One thing that surprised me, though, is I forgot to test merge E3SM-Project/E3SM#5732 into MALI-Dev/develop when I did my testing, but I did not run into any problems compiling. Does that surprise you? Assuming that's not an indication of anything amiss, I'm approving this.

matthewhoffman added a commit to matthewhoffman/compass that referenced this pull request Aug 4, 2023
@matthewhoffman
Update tolerance on circ shelf decomp test
cf74639 
This keeps tests passing after updating version of Albany in MPAS-Dev#641
@matthewhoffman matthewhoffman mentioned this pull request Aug 4, 2023
Merged
1 task
@xylar
Copy link
Collaborator Author

xylar commented Aug 4, 2023

As we discussed on slack,

One thing that surprised me, though, is I forgot to test merge E3SM-Project/E3SM#5732 into MALI-Dev/develop when I did my testing, but I did not run into any problems compiling. Does that surprise you? Assuming that's not an indication of anything amiss, I'm approving this.

The answer is that MALI is so far out of date that ADIOS support was never added so #5732 isn't even needed 😉

@xylar xylar merged commit 9b82e96 into MPAS-Dev:main Aug 4, 2023
@xylar xylar deleted the update-to-1.2.0-alpha.6 branch August 4, 2023 19:12
@xylar
Copy link
Collaborator Author

xylar commented Aug 4, 2023

Thank you @matthewhoffman!!!

matthewhoffman added a commit that referenced this pull request Aug 4, 2023
@matthewhoffman
Merge pull request #660 from matthewhoffman/landice/update_circshelf_tol
e9a8b11 
Update tolerance on circ shelf decomp test

This is required to allow circular shelf decomposition test to pass after updating version of Albany in #641. The differences are expected. Closes #654 .
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@matthewhoffman matthewhoffman matthewhoffman approved these changes

Assignees

@xylar xylar

Labels
dependencies and deployment Changes relate to creating conda and Spack environments, and creating a load script E3SM PR finished
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Need spack clone for each spack environment
2 participants
@xylar @matthewhoffman