Added Intel compiler v2016 support #691

DirkdeDraak · 2015-09-18T11:03:13Z

No description provided.

hpcugentbot · 2015-09-18T11:03:16Z

Automatic reply from Jenkins: Can I test this?

boegel · 2015-09-18T11:48:40Z

Jenkins: ok to test

hpcugentbot · 2015-09-18T11:50:53Z

Easyblocks unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1199/console for more details).

This pull request is now ready for review/testing.

Please try and find someone who can tackle this; contact @boegel if you're not sure what to do.

boegel · 2015-09-18T12:08:41Z

easybuild/easyblocks/i/icc.py

+            cfg_extras_map = {
+                'COMPONENTS': 'ALL',
+            }
+        super(EB_icc, self).install_step(silent_cfg_names_map=silent_cfg_names_map, silent_cfg_extras=cfg_extras_map)


this will crash for non-2016 versions, since cfg_extras_map is not defined

please change it to something like:

silent_cfg_extras = {} if LooseVersion(self.version) >= LooseVersion('2016'): silent_cfg_extras.update{'COMPONENTS': 'ALL'} super(EB_icc, self).install_step(silent_cfg_names_map=silent_cfg_names_map, silent_cfg_extras=silent_cfg_extras)

Yes, makes sense.

If we pass COMPONENTS=ALL, we get mkl, ipp, tbb, gdb and a whole other bunch of stuff installed. Do we want this? It actually makes the easyconfigs for the MKL, TBB, etc redundant. We can specify a subset of ALL:

intel-comp-l-all-vars__noarch intel-comp-l-all-common__noarch intel-comp-l-ps-common__noarch intel-comp-l-all-devel__i486 intel-comp-l-ps-ss-devel__i486 intel-comp-l-all__i486 intel-comp-l-ps-ss-wrapper__i486 intel-comp-l-all-devel__x86_64 intel-comp-l-ps-devel__x86_64 intel-comp-l-ps-ss-devel__x86_64 intel-openmp-l-all__i486 intel-openmp-l-ps__i486 intel-openmp-l-all__x86_64 intel-openmp-l-ps-mic__x86_64 intel-openmp-l-ps__x86_64 intel-openmp-l-ps-ss__x86_64 intel-openmp-l-all-devel__i486 intel-openmp-l-all-devel__x86_64 intel-openmp-l-ps-mic-devel__x86_64 intel-openmp-l-ps-devel__x86_64 intel-openmp-l-ps-ss-devel__x86_64 intel-tbb-libs__noarch intel-comp-all-doc__noarch intel-comp-ps-ss-doc__noarch intel-icc-doc__noarch intel-icc-ps-doc__noarch intel-icc-ps-ss-doc__noarch intel-ifort-ps-doc__noarch intel-icc-l-all__x86_64 intel-icc-l-ps-ss__x86_64 intel-icc-l-all-vars__noarch intel-icc-l-all-common__noarch intel-icc-l-ps-common__noarch intel-icc-l-all__i486 intel-icc-l-ps-ss-wrapper__i486 intel-icc-l-all-devel__i486 intel-icc-l-all-devel__x86_64 intel-icc-l-ps-devel__x86_64 intel-icc-l-ps-ss-devel__x86_64 intel-ifort-l-ps__i486 intel-ifort-l-ps__x86_64 intel-ifort-l-ps-vars__noarch intel-ifort-l-ps-common__noarch intel-ifort-l-ps-devel__i486 intel-ifort-l-ps-devel__x86_64 intel-mpirt-l-ps__i486 intel-mkl__i486 intel-mkl-ps__i486 intel-mkl__x86_64 intel-mkl-ps__x86_64 intel-mkl-common__noarch intel-mkl-ps-common__noarch intel-mkl-devel__x86_64 intel-mkl-ps-mic-devel__x86_64 intel-mkl-ps-f95-devel__x86_64 intel-mkl-gnu-devel__x86_64 intel-mkl-ps-gnu-devel__x86_64 intel-mkl-ps-pgi-devel__x86_64 intel-mkl-sp2dp-devel__x86_64 intel-mkl-ps-cluster-devel__x86_64 intel-mkl-devel__i486 intel-mkl-ps-f95-devel__i486 intel-mkl-gnu-devel__i486 intel-mkl-ps-gnu-devel__i486 intel-mkl-ps-pgi-devel__i486 intel-mkl-ps-cluster-common__noarch intel-mkl-ps-f95-common__noarch intel-mkl-ps-cluster__x86_64 intel-mkl-gnu__x86_64 intel-mkl-ps-gnu__x86_64 intel-mkl-ps-pgi__x86_64 intel-mkl-sp2dp__x86_64 intel-mkl-ps-mic__x86_64 intel-mkl-gnu__i486 intel-mkl-ps-gnu__i486 intel-mkl-ps-pgi__i486 intel-mkl-doc__noarch intel-mkl-ps-doc__noarch intel-mkl-ps-tbb__i486 intel-mkl-ps-tbb-devel__i486 intel-mkl-ps-tbb__x86_64 intel-mkl-ps-tbb-devel__x86_64 intel-mkl-ps-tbb-mic__x86_64 intel-mkl-ps-tbb-mic-devel__x86_64 intel-ipp-l-common__noarch intel-ipp-l-ps-common__noarch intel-ipp-l-st-devel__i486 intel-ipp-l-st__i486 intel-ipp-l-st__x86_64 intel-ipp-l-mt__i486 intel-ipp-l-mt__x86_64 intel-ipp-l-st-devel__x86_64 intel-ipp-l-ps-st-devel__x86_64 intel-ipp-l-mt-devel__i486 intel-ipp-l-mt-devel__x86_64 intel-ipp-l-doc__noarch intel-ipp-l-ps-doc__noarch intel-crypto-ipp-st-devel__i486 intel-crypto-ipp-st-devel__x86_64 intel-crypto-ipp-ps-st-devel__x86_64 intel-crypto-ipp-st__x86_64 intel-crypto-ipp-st__i486 intel-crypto-ipp-mt-devel__i486 intel-crypto-ipp-mt-devel__x86_64 intel-crypto-ipp-mt__x86_64 intel-crypto-ipp-mt__i486 intel-crypto-ipp-ss-st-devel__i486 intel-crypto-ipp-ss-st-devel__x86_64 intel-crypto-ipp-common__noarch intel-crypto-ipp-doc__noarch intel-tbb-source__noarch intel-tbb-devel__noarch intel-tbb-common__noarch intel-tbb-ps-common__noarch intel-tbb-doc__noarch intel-daal__i486 intel-daal__x86_64 intel-daal-common__noarch intel-daal-doc__noarch intel-gdb-gt__x86_64 intel-gdb-gt-src__noarch intel-gdb-gt-libelfdwarf__x86_64 intel-gdb-gt-devel__x86_64 intel-gdb-gt-common__noarch intel-gdb-gt-doc__noarch intel-gdb-ps-cdt__x86_64 intel-gdb-ps-cdt-source__x86_64 intel-gdb-ps-mic__x86_64 intel-gdb-ps-mpm__x86_64 intel-gdb-ps-doc__noarch intel-gdb-ps-doc-jp__noarch intel-gdb__i486 intel-gdb__x86_64 intel-gdb-source__noarch intel-gdb-python-source__noarch intel-gdb-common__noarch intel-gdb-doc__noarch intel-icsxe__noarch intel-ips__noarch intel-ipsc__noarch intel-ipsf__noarch intel-compxe__noarch intel-ccompxe__noarch intel-fcompxe__noarch intel-ipp-psxe__noarch intel-mkl-psxe__noarch intel-tbb-psxe__noarch intel-daal-psxe__noarch intel-mpi-psxe__x86_64 intel-mpi-rt-psxe__x86_64 intel-psf-ia__i486 intel-psf-intel__x86_64 intel-gdb-ps-common__noarch intel-psxe-common__noarch intel-psxe-doc__noarch intel-ccomp-doc__noarch intel-fcomp-doc__noarch intel-icsxe-doc__noarch intel-ips-doc__noarch intel-ipsc-doc__noarch intel-ipsf-doc__noarch intel-compxe-doc__noarch intel-ccompxe-doc__noarch intel-fcompxe-doc__noarch intel-icsxe-pset

omfg... how the hell da we figure out what we need from this mess? O_o

trail and error. But most are clear what they mean.

Seriously? There's no proper documentation for this?

None that I have found?
See https://software.intel.com/en-us/articles/intel-composer-xe-2015-silent-installation-guide under COMPONENTS

boegel · 2015-09-18T12:25:24Z

@DirkdeDraak: Thanks a lot for sharing your work! There are some things that need to be looked into to avoid breaking the installation of older versions.
Please see the remarks as constructive feedback, not criticism.

Let me know if you need help, or if you're not comfortable with tackling the remarks.

boegel · 2015-09-18T13:50:42Z

Jenkins: ok to test

boegel · 2015-09-18T13:50:52Z

Jenkins: test this please

hpcugentbot · 2015-09-18T14:03:29Z

Easyblocks unit test suite FAILed.

See https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1202/console for more details.

Please fix the reported issues by pushing additional commits to the branch corresponding with this pull request; contact @boegel if you're not sure what to do.

hpcugentbot · 2015-09-18T14:04:35Z

Easyblocks unit test suite FAILed.

See https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1203/console for more details.

Please fix the reported issues by pushing additional commits to the branch corresponding with this pull request; contact @boegel if you're not sure what to do.

hpcugentbot · 2015-09-18T14:08:58Z

Easyblocks unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1205/console for more details).

This pull request is now ready for review/testing.

Please try and find someone who can tackle this; contact @boegel if you're not sure what to do.

DirkdeDraak · 2015-09-18T14:14:31Z

I pushed my latest updated. This time only the icc.py again.

hpcugentbot · 2015-09-22T08:52:35Z

Easyblocks unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1217/console for more details).

This pull request is now ready for review/testing.

Please try and find someone who can tackle this; contact @boegel if you're not sure what to do.

… more specific path is not found. This is more in line with how Intel sets up the environment by sourcing the compilervars.sh

hpcugentbot · 2015-10-15T10:11:40Z

Easyblocks unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1289/console for more details).

This pull request is now ready for review/testing.

Please try and find someone who can tackle this; contact @boegel if you're not sure what to do.

wpoely86 · 2015-10-15T10:21:25Z

easybuild/easyblocks/i/icc.py

+            # load dependencies. 
+            gccversion = get_software_version('GCC')
+            # manual approach to at least have the system version of gcc
+            if not gccversion:


add a bit fat warn here. We don't really support this anymore. icc should always be installed alongside an EB GCC.

Hi Ward, not sure what you mean.
My icc.eb file does have a dependency on GCC
name = 'icc'
version = '2016.0.109'
toolchain = {'name': 'dummy', 'version': 'dummy'}gcc = 'GCC'
gccver = '5.2.0'
versionsuffix = '-%s-%s' % (gcc, gccver)

though during the install phase it is not available. Likely because the toolchains is dummy, dummy and it doesn't load the GCC version listed.
One thing I was thinking about, (based on earlier discussion with dependencies for Intel-toolchains), was that if the FFTW libraries are built, they will currently be built with the system-GCC, while the dependency is not active during install.

Yes, icc always needs a GCC. I was referring to your fallback to system GCC. I think you can do the following:

gccversion = get_software_version('GCC') if not gccversion: raise EasyBuildError("GCC module is not loaded?")

@wpoely86: no, please don't, you'll break the installation of icc in the ictce toolchain, where we don't include our own GCC underneath.

We should encourage using an EB-built GCC when installing/using icc, but not enforce it.

I wouldn't go further than adding a self.log.warning is the system GCC is being relied upon.

wpoely86 · 2015-10-15T11:17:23Z

@DirkdeDraak Thanks for the effort. We're currently looking into what the best options are: provide one easyconfig for everything (icc, ifort, mkl, impi, daal, etc) or split if off in serveral easyconfigs. The latter is more convenient for people who do not have a license for everything.

hpcugentbot · 2015-10-15T12:11:39Z

Easyblocks unit test suite PASSed (see https://jenkins1.ugent.be/job/easybuild-easyblocks-pr-builder/1291/console for more details).

This pull request is now ready for review/testing.

Please try and find someone who can tackle this; contact @boegel if you're not sure what to do.

geimer · 2015-10-29T08:09:03Z

+1 for installing components separately.

While installing C/C++ and Fortran compilers together would be a step forward, not everyone needs MKL and TBB. Also, you may want to use Intel MPI with GNU compilers or the Intel compilers with another MPI -- and having two MPIs in $PATH at the same time usually ends up in a huge mess. (Although I'm not sure whether Intel MPI is really included, as the referenced easyconfigs are based on the Composer edition.)

wpoely86 · 2015-10-29T08:16:44Z

@geimer did you just volunteer to test my mess? 😁

boegel · 2015-11-24T13:13:03Z

easybuild/easyblocks/i/icc.py

+        cfg_extras_map = {}
+        if LooseVersion(self.version) >= LooseVersion('2016'):
+            cfg_extras_map = {
+                'COMPONENTS': 'ALL',


I don't think this is a very good idea... This is icc, not "Intel world".

Is there documentation on what are valid values for COMPONENTS?

* 691: removed double modextravars and explicit gccversion = None fixed paths for TBB, debugger. dirmap will now hold original items if more specific path is not found. This is more in line with how Intel sets up the environment by sourcing the compilervars.sh removed make_module_extra Modified according to pull request easybuilders#691 Added support for installing Intel compilers v2016 added support for Intel 2016 compilers

* origin/pr/691: removed double modextravars and explicit gccversion = None fixed paths for TBB, debugger. dirmap will now hold original items if more specific path is not found. This is more in line with how Intel sets up the environment by sourcing the compilervars.sh removed make_module_extra Modified according to pull request easybuilders#691 Added support for installing Intel compilers v2016 added support for Intel 2016 compilers

boegel · 2015-12-11T23:35:30Z

will get merged via #756

fmasselink added 2 commits September 18, 2015 12:07

added support for Intel 2016 compilers

15b48de

Added support for installing Intel compilers v2016

782d164

boegel mentioned this pull request Sep 18, 2015

Several additions/changes to easyblocks #693

Closed

DirkdeDraak mentioned this pull request Sep 18, 2015

Added icc, ifort, iccifort 2016.0.109 files easybuilders/easybuild-easyconfigs#1973

Closed

boegel reviewed Sep 18, 2015
View reviewed changes

Modified according to pull request #691

12c6707

removed make_module_extra

01254a5

fixed paths for TBB, debugger. dirmap will now hold original items if…

441d1b6

… more specific path is not found. This is more in line with how Intel sets up the environment by sourcing the compilervars.sh

wpoely86 reviewed Oct 15, 2015
View reviewed changes

removed double modextravars and explicit gccversion = None

5e95c15

boegel added this to the v2.5.0 milestone Nov 24, 2015

boegel reviewed Nov 24, 2015
View reviewed changes

boegel mentioned this pull request Nov 25, 2015

support for 2016.* versions of Intel tools #751

Closed

wpoely86 mentioned this pull request Nov 26, 2015

Add Intel Parallel Studio 2016, update 1 easybuilders/easybuild-easyconfigs#2148

Merged

boegel mentioned this pull request Nov 26, 2015

EasyBuild 2.5.0 (roadmap) easybuilders/easybuild#170

Closed

16 tasks

wpoely86 mentioned this pull request Dec 2, 2015

update easyblocks for Intel tools to support 2016 versions #756

Merged

boegel merged commit 5e95c15 into easybuilders:develop Dec 14, 2015

boegel mentioned this pull request Nov 11, 2016

ifort module includes intel mpirun etc. #1032

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added Intel compiler v2016 support #691

Added Intel compiler v2016 support #691

DirkdeDraak commented Sep 18, 2015

hpcugentbot commented Sep 18, 2015

boegel commented Sep 18, 2015

hpcugentbot commented Sep 18, 2015

boegel Sep 18, 2015

DirkdeDraak Sep 18, 2015

wpoely86 Oct 14, 2015

boegel Nov 24, 2015

wpoely86 Nov 24, 2015

boegel Nov 24, 2015

wpoely86 Nov 24, 2015

boegel commented Sep 18, 2015

boegel commented Sep 18, 2015

boegel commented Sep 18, 2015

hpcugentbot commented Sep 18, 2015

hpcugentbot commented Sep 18, 2015

hpcugentbot commented Sep 18, 2015

DirkdeDraak commented Sep 18, 2015

hpcugentbot commented Sep 22, 2015

hpcugentbot commented Oct 15, 2015

wpoely86 Oct 15, 2015

DirkdeDraak Oct 15, 2015

wpoely86 Oct 15, 2015

boegel Oct 15, 2015

wpoely86 commented Oct 15, 2015

hpcugentbot commented Oct 15, 2015

geimer commented Oct 29, 2015

wpoely86 commented Oct 29, 2015

boegel Nov 24, 2015

boegel commented Dec 11, 2015

Added Intel compiler v2016 support #691

Added Intel compiler v2016 support #691

Conversation

DirkdeDraak commented Sep 18, 2015

hpcugentbot commented Sep 18, 2015

boegel commented Sep 18, 2015

hpcugentbot commented Sep 18, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

boegel commented Sep 18, 2015

boegel commented Sep 18, 2015

boegel commented Sep 18, 2015

hpcugentbot commented Sep 18, 2015

hpcugentbot commented Sep 18, 2015

hpcugentbot commented Sep 18, 2015

DirkdeDraak commented Sep 18, 2015

hpcugentbot commented Sep 22, 2015

hpcugentbot commented Oct 15, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wpoely86 commented Oct 15, 2015

hpcugentbot commented Oct 15, 2015

geimer commented Oct 29, 2015

wpoely86 commented Oct 29, 2015

Choose a reason for hiding this comment

boegel commented Dec 11, 2015