Installation: Detect and use MPI_LIB during config

[HenrikBengtsson]

Issue

Hi, it looks like configure detects environment variable MPI_INCLUDE but not MPI_LIB resulting in the following installation failure on R 4.1.0 on CentOS 7 (built from source with gcc 8.3.1).

$ module load mpi/openmpi-x86_64 ## Available via the official CentOS/EPEL repo

$ R --quiet
> Sys.getenv(c("MPI_INCLUDE", "MPI_LIB"))
                  MPI_INCLUDE                       MPI_LIB 
"/usr/include/openmpi-x86_64"      "/usr/lib64/openmpi/lib" 

> install.packages("pbdMPI")
Installing package into ‘/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8-sandbox’
(as ‘lib’ is unspecified)
trying URL 'https://cloud.r-project.org/src/contrib/pbdMPI_0.4-3.tar.gz'
Content type 'application/x-gzip' length 519462 bytes (507 KB)
==================================================
downloaded 507 KB

* installing *source* package ‘pbdMPI’ ...
** package ‘pbdMPI’ successfully unpacked and MD5 sums checked
** using staged installation
setting mpi include path from MPI_INCLUDE
configure: error: Must specify both the include and lib paths
ERROR: configuration failed for package ‘pbdMPI’
* removing ‘/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8-sandbox/pbdMPI’

The downloaded source packages are in
        ‘/scratch/henrik/RtmpdYT6OW/downloaded_packages’
Warning message:
In install.packages("pbdMPI") :
  installation of package ‘pbdMPI’ had non-zero exit status
> 

Workaround

The fix is to install as:

> install.packages("pbdMPI", configure.args="--with-mpi-type=OPENMPI --with-mpi-libpath=$MPI_LIB")
Installing package into ‘/c4/home/henrik/R/x86_64-pc-linux-gnu-library/4.1-CBI-gcc8-sandbox’
(as ‘lib’ is unspecified)
trying URL 'https://cloud.r-project.org/src/contrib/pbdMPI_0.4-3.tar.gz'
Content type 'application/x-gzip' length 519462 bytes (507 KB)
==================================================
downloaded 507 KB

* installing *source* package ‘pbdMPI’ ...
** package ‘pbdMPI’ successfully unpacked and MD5 sums checked
** using staged installation
setting mpi include path from MPI_INCLUDE
checking for sed... /usr/bin/sed
checking for mpicc... mpicc
checking for ompi_info... ompi_info
checking for pkg-config... /usr/bin/pkg-config
>> TMP_FOUND = Nothing found from mpicc --show & sed nor pkg-config ...
checking for openpty in -lutil... yes
checking for main in -lpthread... yes

******************* Results of pbdMPI package configure *****************
 
>> MPIRUN = /usr/lib64/openmpi/bin/mpirun
>> MPIEXEC = /usr/lib64/openmpi/bin/mpiexec
>> ORTERUN = /usr/lib64/openmpi/bin/orterun
>> TMP_INC = 
>> TMP_LIB = 
>> TMP_LIBNAME = 
>> TMP_FOUND = Nothing found from mpicc --show & sed nor pkg-config ...
>> MPI_ROOT = 
>> MPITYPE = OPENMPI
>> MPI_INCLUDE_PATH = /usr/include/openmpi-x86_64
>> MPI_LIBPATH = /usr/lib64/openmpi/lib
>> MPI_LIBNAME = 
>> MPI_LIBS =  -lutil -lpthread
>> MPI_DEFS = -DMPI2
>> MPI_INCL2 = 
>> MPI_LDFLAGS = 
>> PKG_CPPFLAGS = -I/usr/include/openmpi-x86_64  -DMPI2 -DOPENMPI
>> PKG_LIBS = -L/usr/lib64/openmpi/lib -lmpi  -lutil -lpthread
>> PROF_LDFLAGS = 
>> ENABLE_LD_LIBRARY_PATH = no
 
*************************************************************************
configure: creating ./config.status
config.status: creating src/Makevars
configure: creating ./config.status
config.status: creating src/Makevars
config.status: creating R/zzz.r
** libs
echo "MPIRUN = /usr/lib64/openmpi/bin/mpirun" > Makeconf
echo "MPIEXEC = /usr/lib64/openmpi/bin/mpiexec" >> Makeconf
echo "ORTERUN = /usr/lib64/openmpi/bin/orterun" >> Makeconf
echo "TMP_INC = " >> Makeconf
echo "TMP_LIB = " >> Makeconf
echo "TMP_LIBNAME = " >> Makeconf
echo "TMP_FOUND = Nothing found from mpicc --show & sed nor pkg-config ..." >> Makeconf
echo "MPI_ROOT = " >> Makeconf
echo "MPITYPE = OPENMPI" >> Makeconf
echo "MPI_INCLUDE_PATH = /usr/include/openmpi-x86_64" >> Makeconf
echo "MPI_LIBPATH = /usr/lib64/openmpi/lib" >> Makeconf
echo "MPI_LIBNAME = " >> Makeconf
echo "MPI_LIBS =  -lutil -lpthread" >> Makeconf
echo "MPI_DEFS = -DMPI2" >> Makeconf
echo "MPI_INCL2 = " >> Makeconf
echo "MPI_LDFLAGS = " >> Makeconf
echo "PKG_CPPFLAGS = -I/usr/include/openmpi-x86_64  -DMPI2 -DOPENMPI" >> Makeconf
echo "PKG_LIBS = -L/usr/lib64/openmpi/lib -lmpi  -lutil -lpthread" >> Makeconf
echo "PROF_LDFLAGS = " >> Makeconf
echo "ENABLE_LD_LIBRARY_PATH = no" >> Makeconf
...
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (pbdMPI)

The downloaded source packages are in
        ‘/scratch/henrik/Rtmp429dOm/downloaded_packages’

> library(pbdMPI)
> packageVersion("pbdMPI")
[1] ‘0.4.3’
> 

Wish/Suggestion

Make the pbdMPI configure script detect also MPI_LIB, which I'm quite sure is an official environment variable so that:

> install.packages("pbdMPI", configure.args="--with-mpi-type=OPENMPI")

be enough. Maybe it would result in:

> install.packages("pdbMPI")

working out of the box, which would be lovely.

FWIW, here's what the above environment module sets and provides:

$ module show mpi/openmpi-x86_64
------------------------------------------------------------------------------
   /etc/modulefiles/mpi/openmpi-x86_64:
------------------------------------------------------------------------------
conflict("mpi")
prepend_path("PATH","/usr/lib64/openmpi/bin")
prepend_path("LD_LIBRARY_PATH","/usr/lib64/openmpi/lib")
prepend_path("PYTHONPATH","/usr/lib64/python2.7/site-packages/openmpi")
prepend_path("MANPATH","/usr/share/man/openmpi-x86_64")
setenv("MPI_BIN","/usr/lib64/openmpi/bin")
setenv("MPI_SYSCONFIG","/etc/openmpi-x86_64")
setenv("MPI_FORTRAN_MOD_DIR","/usr/lib64/gfortran/modules/openmpi-x86_64")
setenv("MPI_INCLUDE","/usr/include/openmpi-x86_64")
setenv("MPI_LIB","/usr/lib64/openmpi/lib")
setenv("MPI_MAN","/usr/share/man/openmpi-x86_64")
setenv("MPI_PYTHON_SITEARCH","/usr/lib64/python2.7/site-packages/openmpi")
setenv("MPI_COMPILER","openmpi-x86_64")
setenv("MPI_SUFFIX","_openmpi")
setenv("MPI_HOME","/usr/lib64/openmpi")

$ mpirun --version
mpirun (Open MPI) 1.10.7

Report bugs to http://www.open-mpi.org/community/help/