wreck: need -o mpi=spectrum -o spectrum_root=blah or similar

[garlick]

Do we need to add some way to alter the MPI_ROOT, OPAL_LIBDIR, LD_LIBRARY_PATH, and LD_PRELOAD environment variables set in the spectrum MPI plugin so that they point to an installation other than system packaged one? It seems we are instructing our users to use side installed versions of MPI:

From [email protected] on behalf of Gyllenhaal, John C. [email protected]

The default spectrum-mpi module loaded also was changed to spectrum-mpi/rolling-release. This new MPI module allows us to roll out new versions of spectrum-mpi without breaking existing executables or requiring rebuilds on update (e.g. do rolling releases of MPI)! It also allows migration of executables between CORAL EA and CORAL systems even though they use different default MPIs. Please use spectrum-mpi/rolling-release if at all possible and let me know if you have difficulties. This will be discussed more at the Sierra Systems Getting Started Workshop tomorrow afternoon and in the slides that will be released a few days after that. I will send more details and update the quickstart guide later this week.

The underlying MPI used by the ‘rolling-release’ module is now controlled by the symbolic link /usr/tce/packages/spectrum-mpi/ibm/spectrum-mpi-rollingrel. That link on CORAL EA systems (RZMANTA, RAY, SHARK) will remain at spectrum-mpi-2018.04.27 for the time being (perhaps until we switch to jsrun on CORAL EA), so there should be no user visible change on CORAL EA.

On the CORAL systems (SIERRA, RZANSEL, BUTTE), that rollingrel link advanced to spectrum-mpi/2018.07.12 because we need that version for our acceptance testing in August. Only relinking is needed to get the new MPI libraries (existing application libraries do not need to be rebuilt) and existing executables may still be able to run with lrun/jsrun (but not guaranteed). If this causes problems for your application, please let me know. The previous default MPI version is still available via ‘ml spectrum-mpi/2018.06.01’.

[grondo]

closed by #1988