Build failure with --with-hwloc=external #2616

opoplawski · 2016-12-20T23:13:39Z

If I try to build the Fedora openmpi package with --with-hwloc=external I end up with:

In file included from ../../opal/mca/hwloc/hwloc.h:131:0,
                 from proc.h:21,
                 from proc.c:19:
../../opal/mca/hwloc/external/external.h:20:35: fatal error: /include/hwloc.h: No such file or directory
 #include MCA_hwloc_external_header
                                   ^
compilation terminated.

Full logs - https://kojipkgs.fedoraproject.org//work/tasks/5738/17005738/build.log
It does work with --with-hwloc=/usr, but I noticed a post on the mailing list suggesting that that was bad practice.

The text was updated successfully, but these errors were encountered:

ggouaillardet · 2016-12-21T01:19:56Z

@opoplawski thanks for the report, i will be pushing a fix shortly

- simply #include "hwloc.h" to use the external hwloc header - do use the external hwloc header instead of opal/mca/hwloc/hwloc.h Thanks Orion Poplawski for the report Fixes open-mpi#2616 Signed-off-by: Gilles Gouaillardet <[email protected]> (cherry picked from commit open-mpi/ompi@9649c44)

opoplawski · 2016-12-23T02:35:30Z

Looks good here. Thanks.

opoplawski · 2016-12-27T21:07:11Z

So, it worked okay with the #2618 patch, but now with 2.0.2rc2 I get lots of errors:

libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I../../../opal/include -I../../../ompi/include -I../../../oshmem/include -I../../../opal/mca/hwloc/hwloc1112/hwloc/include/private/autogen -I../../../opal/mca/hwloc/hwloc1112/hwloc/include/hwloc/autogen -I../../../ompi/mpiext/cuda/c -I../../../opal/include -I../../.. -I../../../orte/include -I/builddir/build/BUILD/openmpi-2.0.2rc2/opal/mca/event/libevent2022/libevent -I/builddir/build/BUILD/openmpi-2.0.2rc2/opal/mca/event/libevent2022/libevent/include -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -m64 -mtune=generic -finline-functions -fno-strict-aliasing -mcx16 -pthread -MT base/hwloc_base_dt.lo -MD -MP -MF base/.deps/hwloc_base_dt.Tpo -c base/hwloc_base_dt.c  -fPIC -DPIC -o base/.libs/hwloc_base_dt.o
In file included from ../../../opal/mca/hwloc/base/base.h:18:0,
                 from base/hwloc_base_dt.c:15:
../../../opal/mca/hwloc/hwloc.h:142:5: error: unknown type name 'hwloc_cpuset_t'
     hwloc_cpuset_t available;
     ^~~~~~~~~~~~~~
../../../opal/mca/hwloc/hwloc.h:152:5: error: unknown type name 'hwloc_obj_type_t'
     hwloc_obj_type_t type;
     ^~~~~~~~~~~~~~~~
../../../opal/mca/hwloc/hwloc.h:162:5: error: unknown type name 'hwloc_cpuset_t'
     hwloc_cpuset_t available;
     ^~~~~~~~~~~~~~
../../../opal/mca/hwloc/hwloc.h:214:22: error: unknown type name 'hwloc_topology_t'
 OPAL_DECLSPEC extern hwloc_topology_t opal_hwloc_topology;
                      ^~~~~~~~~~~~~~~~

and so on. Full logs here: https://kojipkgs.fedoraproject.org//work/tasks/5380/17085380/build.log

jsquyres · 2016-12-27T21:42:33Z

Do you have a version of hwloc that is too old?

opoplawski · 2016-12-27T22:59:06Z

No, just updated it to 1.11.5 to be sure of that.

ggouaillardet · 2016-12-27T23:47:34Z

there are way too many include directories in the command line, and the point of the patch was mainly to get rid of -I.
i will have a look from now

ggouaillardet · 2016-12-28T00:13:49Z

@jsquyres this is from the generated opal/mca/hwloc/Makefile

COMPILE = $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) \
        $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS)

so what we really need here is to override $(DEFAULT_INCLUDES) and not add some more stuff to $(AM_CPPFLAGS)

makes sense ?

the inline patch restore part of my initial patch

diff --git a/opal/mca/hwloc/Makefile.am b/opal/mca/hwloc/Makefile.am
index c5340da..2aa843b 100644
--- a/opal/mca/hwloc/Makefile.am
+++ b/opal/mca/hwloc/Makefile.am
@@ -10,9 +10,9 @@
 #
 
 # We do not want -I$(srcdir) in AM_CPPFLAGS, or there can be a
-# conflict between system hwloc.h and opal/mca/hwloc/hwloc.h.  So just
-# set only what we need to AM_CPPFLAGS.
-AM_CPPFLAGS = -I$(top_builddir)/opal/include
+# conflict between system hwloc.h and opal/mca/hwloc/hwloc.h.
+# So override DEFAULT_INCLUDES and only what is required
+DEFAULT_INCLUDES = -I$(top_builddir)/opal/include
 
 # main library setup
 noinst_LTLIBRARIES = libmca_hwloc.la

jsquyres · 2017-01-03T18:48:03Z

Odd. I see that INCLUDES is deprecated in favor of AM_CPPFLAGS, but I do not see mention of DEFAULT_INCLUDES in the latest Automake docs at all. I do see a single offhand reference to DEFAULT_INCLUDES in older Automake docs, though.

Is DEFAULT_INCLUDES really what we should be updating?

jsquyres · 2017-01-03T20:44:45Z

@opoplawski I'm looking into this issue. One thing I note in the full logs is right up near the beginning of configure:

./configure: line 5711: hostname: command not found

Might want to add an RPM dependency for the RPM containing hostname...?

@ggouaillardet I see the problem with --with-hwloc=external. You won't hit this situation unless you have an hwloc installed in a default compiler/linker location (e.g., /usr), and you use --with-hwloc=external. I.e., it's a different case than --with-hwloc=DIR.

The problem is:

opal/mca/hwloc/hwloc.h includes MCA_hwloc_IMPLEMENTATION_HEADER, which is "opal/mca/hwloc/external/external.h"
external.h includes MCA_hwloc_external_header, which is "hwloc.h"
Unfortunately, when we are building in the opal/mca/hwloc dir, there is a hwloc.h in the pwd.
We also have a -I. in the compile line.
The preprocessor therefore picks up ./hwloc.h instead of the external/system hwloc.h.
Hence, all the hwloc symbols and types are undefined.

This -I. comes from DEFAULT_INCLUDES, which may well be what @ggouaillardet was trying to say in the first place. 😄

Note: I'm 99.99% sure that we didn't put -I. in DEFAULT_INCLUDES -- it was put there by Automake.

I'm not sure what the Right approach is to fix this issue. I see a few options:

Rename opal/mca/hwloc/hwloc.h to something else.
- We'll need to s/hwloc.h/hwloc-whatever.h/ throughout the code base, but that's not a big deal.
- There may be other side effects, though -- I'm pretty sure the MCA infrastructure requires a header file with the same name as the framework. That might be difficult to work around.
Edit DEFAULT_INCLUDES so that it does not contain -I., either directly or indirectly. This might be what we have to do, but it sure is ugly -- it's undocumented, which, to me, makes it sound like an internal variable in Automake.
Add some kind of special case for handling building in the opal/mca/hwloc directory in the .h files.

Suggestions?

jsquyres · 2017-01-03T20:47:55Z

@opoplawski I should note that using --with-hwloc=/usr is likely a Good Enough workaround for now. We definitely need to fix this issue, but that should get you building for now (and possibly for the v2.0.2 release).

opoplawski · 2017-01-04T16:37:38Z

Yeah, I've used --with-hwloc=/usr for ages now, so that's no problem. The hostname thing can be ignored.

jsquyres · 2017-02-10T02:09:40Z

@ggouaillardet @artpol84 Still open for suggestions on how to solve this one (per #2616 (comment)). 😄

artpol84 · 2017-02-10T02:14:42Z

I'm not sure what the Right approach is to fix this issue. I see a few options:

1. Rename opal/mca/hwloc/hwloc.h to something else.
We'll need to s/hwloc.h/hwloc-whatever.h/ throughout the code base, but that's not a big deal.
There may be other side effects, though -- I'm pretty sure the MCA infrastructure requires a header file with the same name as the framework. That might be difficult to work around.
2. Edit DEFAULT_INCLUDES so that it does not contain -I., either directly or indirectly. This might be what we have to do, but it sure is ugly -- it's undocumented, which, to me, makes it sound like an internal variable in Automake.
3. Add some kind of special case for handling building in the opal/mca/hwloc directory in the .h files.

+1 for # 1

ggouaillardet · 2017-02-10T02:45:30Z

#1 looks like the best approach to me (though maybe a bit painful ...)
i do not mind taking a crack at it though, but before that, we should agree on how to rename the hwloc framework. i previously suggested hwtopo, but remain open to new ideas

ggouaillardet · 2017-02-10T02:47:08Z

hw is an other (and simpler) option

Refs. open-mpi#2616 Signed-off-by: Gilles Gouaillardet <[email protected]>

Refs. open-mpi#2616 the leia framework will be renamed into something else later R.I.P. Carrie Fisher Signed-off-by: Gilles Gouaillardet <[email protected]>

Frameworks are usually required to have a framework/framework.h file. However, this is sometimes problematic (see the hwloc use case/problem description, below). This commit allows frameworks to have an "autogen.options" file (i.e., project/mca/framework/autogen.options) that specifies things that autogen needs to know about the framework. Currently, the only option recognized in autogen.options is "framework_header", which allows a framework to specify that its header file is named something other than "framework.h" (the framework header file must still be in the project/mca/framework directory; it simply may be named something other than framework.h). More options may be introduced over time. The use case that motivated this is the hwloc framework (open-mpi#2616). Per MCA framework rules, the hwloc framework is required to have an opal/mca/hwloc/hwloc.h file. However, the hwloc library itself *also* has an hwloc.h file. This causes a problem when configuring Open MPI with --with-hwloc=external (meaning: do not use the hwloc embedded within the Open MPI source code tree -- instead, use an hwloc installation from outside the Open MPI source code tree). Specifically, when in the opal/mca/hwloc directory, the presence of "-I." in DEFAULT_INCLUDES (put there by Automake) causes a confusion between the hwloc.h in opal/mca/hwloc/hwloc.h and the system-installed hwloc.h. Chaos ensues (see the GitHub issue for more detail). The solution is to rename the opal/mca/hwloc/hwloc.h to something else (e.g., hwloc-internal.h), and extend autogen.pl to allow frameworks to have an alternate name for their framework header file. This commit introduces the autogen.pl mechanism to allow the alternate header file name. A follow-on commit will effect this change in the hwloc framework (and update all the places in the code base to use the new filename). Signed-off-by: Jeff Squyres <[email protected]>

jsquyres · 2017-02-10T13:15:55Z

@ggouaillardet and I came up with competing solutions overnight. 😄

I think I figured out overnight how to rename opal/mca/hwloc/hwloc.h without too much trouble -- see #2955.

jsquyres · 2017-02-22T13:37:17Z

@rhc54 Did you mean to close this issue?

rhc54 · 2017-02-22T14:12:49Z

No, it was an update into a related repo - another "git-ism" ☹️

Frameworks are usually required to have a framework/framework.h file. However, this is sometimes problematic (see the hwloc use case/problem description, below). This commit allows frameworks to have an "autogen.options" file (i.e., project/mca/framework/autogen.options) that specifies things that autogen needs to know about the framework. Currently, the only option recognized in autogen.options is "framework_header", which allows a framework to specify that its header file is named something other than "framework.h" (the framework header file must still be in the project/mca/framework directory; it simply may be named something other than framework.h). More options may be introduced over time. The use case that motivated this is the hwloc framework (open-mpi#2616). Per MCA framework rules, the hwloc framework is required to have an opal/mca/hwloc/hwloc.h file. However, the hwloc library itself *also* has an hwloc.h file. This causes a problem when configuring Open MPI with --with-hwloc=external (meaning: do not use the hwloc embedded within the Open MPI source code tree -- instead, use an hwloc installation from outside the Open MPI source code tree). Specifically, when in the opal/mca/hwloc directory, the presence of "-I." in DEFAULT_INCLUDES (put there by Automake) causes a confusion between the hwloc.h in opal/mca/hwloc/hwloc.h and the system-installed hwloc.h. Chaos ensues (see the GitHub issue for more detail). The solution is to rename the opal/mca/hwloc/hwloc.h to something else (e.g., hwloc-internal.h), and extend autogen.pl to allow frameworks to have an alternate name for their framework header file. This commit introduces the autogen.pl mechanism to allow the alternate header file name. A follow-on commit will effect this change in the hwloc framework (and update all the places in the code base to use the new filename). Signed-off-by: Jeff Squyres <[email protected]>

Frameworks are usually required to have a framework/framework.h file. However, this is sometimes problematic (see the hwloc use case/problem description, below). This commit allows frameworks to have an "autogen.options" file (i.e., project/mca/framework/autogen.options) that specifies things that autogen needs to know about the framework. Currently, the only option recognized in autogen.options is "framework_header", which allows a framework to specify that its header file is named something other than "framework.h" (the framework header file must still be in the project/mca/framework directory; it simply may be named something other than framework.h). More options may be introduced over time. The use case that motivated this is the hwloc framework (open-mpi#2616). Per MCA framework rules, the hwloc framework is required to have an opal/mca/hwloc/hwloc.h file. However, the hwloc library itself *also* has an hwloc.h file. This causes a problem when configuring Open MPI with --with-hwloc=external (meaning: do not use the hwloc embedded within the Open MPI source code tree -- instead, use an hwloc installation from outside the Open MPI source code tree). Specifically, when in the opal/mca/hwloc directory, the presence of "-I." in DEFAULT_INCLUDES (put there by Automake) causes a confusion between the hwloc.h in opal/mca/hwloc/hwloc.h and the system-installed hwloc.h. Chaos ensues (see the GitHub issue for more detail). The solution is to rename the opal/mca/hwloc/hwloc.h to something else (e.g., hwloc-internal.h), and extend autogen.pl to allow frameworks to have an alternate name for their framework header file. This commit introduces the autogen.pl mechanism to allow the alternate header file name. A follow-on commit will effect this change in the hwloc framework (and update all the places in the code base to use the new filename). Signed-off-by: Jeff Squyres <[email protected]> (cherry picked from commit a065b9b)

Frameworks are usually required to have a framework/framework.h file. However, this is sometimes problematic (see the hwloc use case/problem description, below). This commit allows frameworks to have an "autogen.options" file (i.e., project/mca/framework/autogen.options) that specifies things that autogen needs to know about the framework. Currently, the only option recognized in autogen.options is "framework_header", which allows a framework to specify that its header file is named something other than "framework.h" (the framework header file must still be in the project/mca/framework directory; it simply may be named something other than framework.h). More options may be introduced over time. The use case that motivated this is the hwloc framework (open-mpi#2616). Per MCA framework rules, the hwloc framework is required to have an opal/mca/hwloc/hwloc.h file. However, the hwloc library itself *also* has an hwloc.h file. This causes a problem when configuring Open MPI with --with-hwloc=external (meaning: do not use the hwloc embedded within the Open MPI source code tree -- instead, use an hwloc installation from outside the Open MPI source code tree). Specifically, when in the opal/mca/hwloc directory, the presence of "-I." in DEFAULT_INCLUDES (put there by Automake) causes a confusion between the hwloc.h in opal/mca/hwloc/hwloc.h and the system-installed hwloc.h. Chaos ensues (see the GitHub issue for more detail). The solution is to rename the opal/mca/hwloc/hwloc.h to something else (e.g., hwloc-internal.h), and extend autogen.pl to allow frameworks to have an alternate name for their framework header file. This commit introduces the autogen.pl mechanism to allow the alternate header file name. A follow-on commit will effect this change in the hwloc framework (and update all the places in the code base to use the new filename). Signed-off-by: Jeff Squyres <[email protected]>

Frameworks are usually required to have a framework/framework.h file. However, this is sometimes problematic (see the hwloc use case/problem description, below). This commit allows frameworks to have an "autogen.options" file (i.e., project/mca/framework/autogen.options) that specifies things that autogen needs to know about the framework. Currently, the only option recognized in autogen.options is "framework_header", which allows a framework to specify that its header file is named something other than "framework.h" (the framework header file must still be in the project/mca/framework directory; it simply may be named something other than framework.h). More options may be introduced over time. The use case that motivated this is the hwloc framework (open-mpi#2616). Per MCA framework rules, the hwloc framework is required to have an opal/mca/hwloc/hwloc.h file. However, the hwloc library itself *also* has an hwloc.h file. This causes a problem when configuring Open MPI with --with-hwloc=external (meaning: do not use the hwloc embedded within the Open MPI source code tree -- instead, use an hwloc installation from outside the Open MPI source code tree). Specifically, when in the opal/mca/hwloc directory, the presence of "-I." in DEFAULT_INCLUDES (put there by Automake) causes a confusion between the hwloc.h in opal/mca/hwloc/hwloc.h and the system-installed hwloc.h. Chaos ensues (see the GitHub issue for more detail). The solution is to rename the opal/mca/hwloc/hwloc.h to something else (e.g., hwloc-internal.h), and extend autogen.pl to allow frameworks to have an alternate name for their framework header file. This commit introduces the autogen.pl mechanism to allow the alternate header file name. A follow-on commit will effect this change in the hwloc framework (and update all the places in the code base to use the new filename). Signed-off-by: Jeff Squyres <[email protected]> (cherry picked from commit a065b9b)

Frameworks are usually required to have a framework/framework.h file. However, this is sometimes problematic (see the hwloc use case/problem description, below). This commit allows frameworks to have an "autogen.options" file (i.e., project/mca/framework/autogen.options) that specifies things that autogen needs to know about the framework. Currently, the only option recognized in autogen.options is "framework_header", which allows a framework to specify that its header file is named something other than "framework.h" (the framework header file must still be in the project/mca/framework directory; it simply may be named something other than framework.h). More options may be introduced over time. The use case that motivated this is the hwloc framework (open-mpi#2616). Per MCA framework rules, the hwloc framework is required to have an opal/mca/hwloc/hwloc.h file. However, the hwloc library itself *also* has an hwloc.h file. This causes a problem when configuring Open MPI with --with-hwloc=external (meaning: do not use the hwloc embedded within the Open MPI source code tree -- instead, use an hwloc installation from outside the Open MPI source code tree). Specifically, when in the opal/mca/hwloc directory, the presence of "-I." in DEFAULT_INCLUDES (put there by Automake) causes a confusion between the hwloc.h in opal/mca/hwloc/hwloc.h and the system-installed hwloc.h. Chaos ensues (see the GitHub issue for more detail). The solution is to rename the opal/mca/hwloc/hwloc.h to something else (e.g., hwloc-internal.h), and extend autogen.pl to allow frameworks to have an alternate name for their framework header file. This commit introduces the autogen.pl mechanism to allow the alternate header file name. A follow-on commit will effect this change in the hwloc framework (and update all the places in the code base to use the new filename). Signed-off-by: Jeff Squyres <[email protected]>

ggouaillardet added Severity: blocker Target: v2.0.x Target: v2.x labels Dec 21, 2016

ggouaillardet self-assigned this Dec 21, 2016

ggouaillardet mentioned this issue Dec 21, 2016

hwloc: correctly handle --with-hwloc=external #2617

Merged

ggouaillardet closed this as completed in #2617 Dec 21, 2016

This was referenced Dec 21, 2016

v2.0.x: hwloc: correctly handle --with-hwloc=external #2618

Closed

v2.x: hwloc: correctly handle --with-hwloc=external #2619

Merged

ggouaillardet assigned jsquyres Dec 28, 2016

ggouaillardet reopened this Dec 28, 2016

jsquyres added this to the v2.0.2 milestone Jan 3, 2017

jsquyres added Severity: critical and removed Severity: blocker labels Jan 3, 2017

jsquyres modified the milestones: v2.0.3, v2.0.2 Jan 3, 2017

jsquyres mentioned this issue Feb 9, 2017

configuration: OMPI v2.x with external hwloc from the distro #2946

Closed

ggouaillardet added a commit to ggouaillardet/ompi that referenced this issue Feb 10, 2017

hwloc: rename the hwloc framework into hwlocxxx

944980f

Refs. open-mpi#2616 Signed-off-by: Gilles Gouaillardet <[email protected]>

ggouaillardet mentioned this issue Feb 10, 2017

hwloc: rename the hwloc framework into something else #2954

Closed

jsquyres mentioned this issue Feb 10, 2017

Fix --with-hwloc=external #2955

Merged

rhc54 closed this as completed in 9649c44 Feb 22, 2017

rhc54 reopened this Feb 22, 2017

jsquyres added the Target: v3.0.x label Feb 22, 2017

jsquyres closed this as completed in #2955 Feb 28, 2017

This was referenced Feb 28, 2017

v2.0.x: --with-hwloc=external fix #3066

Merged

v2.1.x: --with-hwloc=external fix #3067

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build failure with --with-hwloc=external #2616

Build failure with --with-hwloc=external #2616

opoplawski commented Dec 20, 2016

ggouaillardet commented Dec 21, 2016

opoplawski commented Dec 23, 2016

opoplawski commented Dec 27, 2016

jsquyres commented Dec 27, 2016

opoplawski commented Dec 27, 2016

ggouaillardet commented Dec 27, 2016 •

edited

Loading

ggouaillardet commented Dec 28, 2016

jsquyres commented Jan 3, 2017

jsquyres commented Jan 3, 2017 •

edited

Loading

jsquyres commented Jan 3, 2017

opoplawski commented Jan 4, 2017

jsquyres commented Feb 10, 2017

artpol84 commented Feb 10, 2017

ggouaillardet commented Feb 10, 2017

ggouaillardet commented Feb 10, 2017

jsquyres commented Feb 10, 2017

jsquyres commented Feb 22, 2017

rhc54 commented Feb 22, 2017

Build failure with --with-hwloc=external #2616

Build failure with --with-hwloc=external #2616

Comments

opoplawski commented Dec 20, 2016

ggouaillardet commented Dec 21, 2016

opoplawski commented Dec 23, 2016

opoplawski commented Dec 27, 2016

jsquyres commented Dec 27, 2016

opoplawski commented Dec 27, 2016

ggouaillardet commented Dec 27, 2016 • edited Loading

ggouaillardet commented Dec 28, 2016

jsquyres commented Jan 3, 2017

jsquyres commented Jan 3, 2017 • edited Loading

jsquyres commented Jan 3, 2017

opoplawski commented Jan 4, 2017

jsquyres commented Feb 10, 2017

artpol84 commented Feb 10, 2017

ggouaillardet commented Feb 10, 2017

ggouaillardet commented Feb 10, 2017

jsquyres commented Feb 10, 2017

jsquyres commented Feb 22, 2017

rhc54 commented Feb 22, 2017

ggouaillardet commented Dec 27, 2016 •

edited

Loading

jsquyres commented Jan 3, 2017 •

edited

Loading