Intel MPI module version is broken for "2018 Beta" and "2018 Beta Update 1"

[chuckatkins]

Running transaction
  Installing : intel-mpi-devel-ohpc-2017-37.1.x86_64                        1/1 
 
Scanning top-level dir = /opt/intel
--> Installing OpenHPC-style modulefile for MPI version=5.1.3
--> Installing OpenHPC-style modulefile for MPI version=2017
--> Installing OpenHPC-style modulefile for MPI version=5.1.3
--> Installing OpenHPC-style modulefile for MPI version=2017.3
--> Installing OpenHPC-style modulefile for MPI version=2017.2
--> Installing OpenHPC-style modulefile for MPI version=2017.1
--> Installing OpenHPC-style modulefile for MPI version=2017.2
--> Installing OpenHPC-style modulefile for MPI version=2018 Beta
/usr/bin/cp: target ‘Beta’ is not a directory
/usr/bin/cp: target ‘Beta’ is not a directory
Can't open /opt/ohpc/pub/moduledeps/gnu7/impi/2018: No such file or directory.
Can't open Beta: No such file or directory.
--> Installing OpenHPC-style modulefile for MPI version=2018 Beta.1
/usr/bin/cp: target ‘Beta.1’ is not a directory
/usr/bin/cp: target ‘Beta.1’ is not a directory
Can't open /opt/ohpc/pub/moduledeps/gnu7/impi/2018: No such file or directory.
Can't open Beta.1: No such file or directory.
  Verifying  : intel-mpi-devel-ohpc-2017-37.1.x86_64                        1/1 

Installed:
  intel-mpi-devel-ohpc.x86_64 0:2017-37.1                                       

Complete!

[koomie]

@chuckatkins, based on some other discussions regarding the pxse compatibility packages that came up on the user list, I have updated the approach taken in the 1.3.2 branch and was wondering if you could try them out in your environment? In theory, they should also handle the "beta" business you reported (it worked ok for me with Beta1).

The main difference in the latest builds is to remove the Requires from the .spec file as user's have gotten burned by inconsistencies from installations of different flavors of PXSE (e.g. composer vs. cluster install). Instead, they query for an RPM that owns icc and mpicc and use that rpm to determine the version and installation path. To still have the pre-requisite effect that you need to install pxse first, the initial tests are moved to a %pre installation section, and it punts if it can't find the local install.

The actual modulefile generation remains the same, but versioning is determined directly from the RPM metadata instead of things like icc -V.

If you want to try them out, you can grab the current 1.3.2 builds for CentOS at:

• http://build.openhpc.community/OpenHPC:/1.3:/Update2:/Factory/CentOS_7/x86_64/intel-compilers-devel-ohpc-2017-10.1.x86_64.rpm
• http://build.openhpc.community/OpenHPC:/1.3:/Update2:/Factory/CentOS_7/x86_64/intel-mpi-devel-ohpc-2017-30.1.x86_64.rpm

Thanks.

[chuckatkins]

That helps a bunch! I didn't install the RPMs but I tested the version code by pulling it out into seperate shell scripts and it seems to work well. On my test machine, I have 9 different versions installed from 16 u3 through 18 beta 1. I get the following version info:

$ ./version-icc.sh | sort
Checking for local PXSE compiler installation(s).
--> Version 16.0.3.210 detected
--> Version 16.0.4.258 detected
--> Version 17.0.0.098 detected
--> Version 17.0.1.132 detected
--> Version 17.0.2.174 detected
--> Version 17.0.3.191 detected
--> Version 17.0.4.196 detected
--> Version 18.0.0.061 detected
--> Version 18.0.0.082 detected

and

$ ./version-impi.sh | sort
Checking for local PXSE MPI installation(s).
--> Version 2017.0.098 detected
--> Version 2017.1.132 detected
--> Version 2017.2.174 detected
--> Version 2017.2.191 detected
--> Version 2017.3.196 detected
--> Version 2018.0.061 detected
--> Version 2018.0.082 detected
--> Version 5.1.3.210 detected
--> Version 5.1.3.258 detected

So the "proper" versioning seems to work well. The only problem currently is that there's still no way to tie the impi version to the compiler version. Should those remain entirely independent? While I understand that technically any impi version can be used with any compiler but should it? Is there way to construct the modules such that at least the default impi version used is the version that corresponds to the compiler it's deployed with? This would still allow you to explicitly mix and match compiler and mpi versions but just keep the default impi version aqligned with the compiler.

[chuckatkins]

I was thinking something like this:

mpiicc_subpath="linux/mpi/intel64/bin/mpiicc"
for file in ${versions}; do
    prefix=`echo ${file} | sed "s|${icc_subpath}||"`
    version=`rpm -q --qf '%{VERSION}.%{RELEASE}\n' -f ${file}`
    impi_version=`rpm -q --qf '%{VERSION}.%{RELEASE}\n' -f ${prefix}${mpiicc_subpath}`
    if [ -z "${impi_version}" ];then
        echo "--> Version ${version} detected"
    else
        echo "--> Version ${version} detected, Default IMPI Version ${impi_version}"
    fi
done

Where impi_version can then be used to set a variable in the module file, something like LMOD_FAMILY_MPI_DEFAULT_IMPI_VERSION and then the intel-mpi-devel package can create a .version file in the impi modules that looked something like

#%Module
if { $::env(LMOD_FAMILY_MPI_DEFAULT_IMPI_VERSION) ne "" } {
    set ModulesVersion $::env(LMOD_FAMILY_MPI_DEFAULT_IMPI_VERSION)
}

That do you think?

[chuckatkins]

Either way, the version matching issue is outside the scope of this one. I'll close this and open a new one to discuss and address referencing the PR.