Enable easier mache testing #523
Conversation
xylar
added
clean-up
dependencies and deployment
TestingSo far, I have only tested this on my Ubuntu laptop. I used: and could see in the logs that the expected fork and branch were cloned, and that mache 1.12.0rc1 was installed as expected. We will need to verify that this works on all 6 supported machines.
|
|
Might be a good activity for which to enlist the newly formed compass support group. |
|
Yep, I think the easiest way to do that is add these changes to #522 for testing. That will be handy anyway. I'll modify the notes accordingly. |
|
@jonbob, @mark-petersen, @matthewhoffman, @trhille and @darincomeau, I somewhat arbitrarily assigned each of you to a machine to test this on tomorrow as part of our follow-up spack testing. Please trade with someone if you don't have access to the machine I assigned you to (or have some other reason for preferring a different one). |
|
I'm getting the following error on perlmutter from the the printed out traceback: |
|
@darincomeau, I think this might relate to you trying to use a shared conda base as your starting point? You ideally shouldn't have a |
|
Ok removing that directory got me further, my environment must have been messed up from that previous error. Now executes cleanly, ends with I tried running that script, but get an error So it's still looking in system software for the spack environment. I do see I don't mean to clog this PR with my issues. |
|
@darincomeau, I see, I think you are perhaps following the instructions based on what I ran in my own test? That was on my laptop, where spack isn't needed. On Perlmutter, you need to follow the instructions in the Google Doc, which means you need the Your error results because you have a new |
|
Ah thanks @xylar , sorry I missed that part in the Google Doc. explains why I wasnt seeing CONDA_BASE do anything... Thanks for accommodating the lowest common denominator! |
xylar
force-pushed
the
simplify_local_mache
branch
from
d257d7d to
c34896d
Compare
|
For the record, this is what I did on chicoma. I started in a compass directory, on the head of xylar:simplify_local_mache. Then it installs gnu mpich with: |
xylar
force-pushed
the
simplify_local_mache
branch
from
c34896d to
f0f1d34
Compare
|
With the last commits, my previous command works to completion on chicoma. Unfortunately, I still can't compile E3SM/master: @xylar this is probably unrelated to this compass PR, but how did you get MPAS-Ocean to compile on chicoma? |
|
@mark-petersen, as always, you're going to have to comment out the You could try building But debugging the build command and the actual error message is really the starting point. |
There was a problem hiding this comment.
Thanks for doing this, @xylar. I'm approving based on my testing on Cori, in which I built spack with Albany and successfully ran the MALI full_integration test suite. I also had a quick look through the code diffs and nothing jumped out at me, although I can't pretend that I understand all of it.
|
To see the correct PIO error messages on chicoma I did this to the stand-alone make file: We then figured out that the netcdf and pnetcdf paths have a gnu version number in the path before the lib directory. We needed to use environmental variables that specify the correct path on chicoma: |
|
Thanks @mark-petersen. I will fix that in a separate PR. |
|
Also, is there any chance you didn't source the load script since you created it again? Safest to start with a fresh terminal. |
I did, but in the same terminal window. I think it's best if I pick this up again once Perlmutter comes back up. |
There was a problem hiding this comment.
I was able to successfully create the conda load script on chicoma with
export CONDA_BASE=/usr/projects/climate/mpeterse/miconda3
./conda/configure_compass_env.py \
--conda ${CONDA_BASE} \
--mache_fork xylar/mache \
--mache_branch update_cime_machines \
--update_spack \
--spack /lustre/scratch5/mpeterse/spack_test \
--tmpdir /lustre/scratch5/mpeterse/spack_tmp \
--compiler gnu \
--recreate
source load_dev_compass_1.2.0-alpha.4_chicoma-cpu_gnu_mpich.sh
then compile MPAS-Ocean stand-alone, and run the nightly test suite. Thanks for the help @xylar!
|
On Anvil I did this: and the compass env creation worked successfully. However, when I tried to build MALI after loading the env script, I got this error: I'll follow up with @xylar about this offline when I see him this morning. |
|
@matthewhoffman, it looks like you're missing the |
|
@xylar , yes, you are right, I forgot to include the albany flag. I tried again with adding it, and spack is clearly now building trilinos and albany. After a few hours it died with an albany build error: I can try again with a different compiler or mpi. I had selected a combination from the compass docs that is supported for anvil, but it wasn't clear to me if that meant we should expect albany to build. What do you recommend? (From my previous attempt without albany, it seems this PR works correctly. Do you agree that is sufficient for approving the PR?) |
|
@matthewhoffman, yes, I'm seeing the same error on Anvil. That's new since I last built albany (maybe something to check in with Irina and Mauro about -- maybe related to Anvil having old compilers). As long as you see that compass is trying to use a spack branch called |
|
@jonbob, how is testing on Compy going for you? Anything I can do to help? |
|
@matthewhoffman, I'm going to try again with |
@xylar now that Perlmutter is back up I tried again from sourcing the load compass script again in an (obviously) new terminal window, and then building with
EDIT: ok that time it completed in 22 minutes with |
xylar
force-pushed
the
simplify_local_mache
branch
3 times, most recently
from
7401d14 to
0b6f924
Compare
|
@xylar -- I am trying to push on testing this on compy today, but running into issues. I had checked out compass to get mambaforge, following the Compass Quick Start. Is it OK to use this same compass clone for the spack testing? |
|
@jonbob, yes, you can use the same clone for this testing. You just need to add my fork as a remote and check out this branch from it. |
xylar
force-pushed
the
simplify_local_mache
branch
2 times, most recently
from
2822028 to
f9e2b6e
Compare
The `configure_compass_env.py` script now takes two flags, `--mache_fork` and `--mache_branch` that are used to clone the fork and branch locally, then install mache from it. This saves the trouble of developers cloning it themselves and building the mache conda package.
I believe all systems have python3 available. This allows us to remove some awkward imports and to use f-strings instead of format commands.
xylar
force-pushed
the
simplify_local_mache
branch
from
f9e2b6e to
0f0ab0f
Compare
|
Thanks everyone!!! |
The
configure_compass_env.pyscript now takes two flags,--mache_forkand--mache_branchthat are used to clone the fork and branch locally, then install mache from it. This saves the trouble of developers cloning it themselves and building the mache conda package.This merge also simplifies the deployment scripts by requiring python3, which I believe all systems have available without loading any modules. This allows us to remove some awkward imports and to use f-strings instead of format commands.
This merge also removes the
cxx17spack variant of Albany because this variant is no longer supported (Albany always uses the c++17 standard).Checklist
changes look as expected
Testingin this PR) any testing that was used to verify the changescloses #520