USD fails to build with gcc-11

[mont29]

https://github.com/PixarAnimationStudios/USD/blob/9b4cea6217bc49c166fd3686b786a2f18d3e6516/pxr/base/work/singularTask.h#L111

Current code builds fine on gcc-10, but this seems to require explicit std namespace with newer gcc: std::size_t old = count;

[mont29]

Note that this issue also exists in recent releases, at least in 21.02 one.

(in 21.02 release there is another similar one in demangle.cpp, where an #include <cxxabi.h> happens inside of the PXR_NAMESPACE_OPEN_SCOPE namespace scope. But this seems to have been fixed in current code already.)

[jilliene]

Filed as internal issue #USD-7093

[mont29]

Hi, any update on this issue? Would submitting a pull request help getting it fixed faster?

This is a one-line fix, but not sure which of both suggested solutions (in the report and in #1723 one) would be the desired one here...

[spiffmon]

Hi @mont29 and @sobotka - we do not build or validate against GCC's higher than our documented, but we would absolutely consider a PR against this repo!

[mont29]

While generating the PR in own fork I found a few more similar issues (for Blender we typically build a minimal USD).

The root of these issues is that GCC 11 is no longer doing some implicit includes for some std files: https://www.gnu.org/software/gcc/gcc-11/porting_to.html#header-dep-changes

[sybrenstuvel]

What is the status of this task? #1776 has been merged, so shouldn't this issue be resolved now?

[spiffmon]

Hi @sybrenstuvel , I believe the main reason we have not closed this Issue or #1723 is that we do not build or test with gcc11, and as @mont29 mentioned in #1776 , there could be other, similar issues yet lurking in the codebase. If someone confirms they can build a complete USD+Hydra+plugins build on gcc11, I think we'd be more comfortable closing this.

[TheJackiMonster]

I'm still getting a lot of warnings because of memset(this, 0, sizeof(*this)); for example. There are also still some errors (it reports 2 errors when I'm trying to build on Archlinux with GCC version 11.2.0). I didn't find the exact errors in the output though because using grep with the output didn't work well and all the output of the build script gets written to the standard output stream.

So I don't think this is solved and I personally don't like the output from the build script. But maybe I just missed a simple option to get rid of all those warnings hiding the actual errors.

[spiffmon]

Thanks for confirming there's more work to do, @TheJackiMonster

[pberto]

Guys what is the status here? GCC 11 support is kind of needed... RHEL9 does not even support any previous GCC version.

macOS supports building with clang 13.1, which supports GCC 11 as its primary compatibility target... so it does not seem incredible to be asking this...

I also think clang should be used as the primary compiler also on Linux, since we are at it.

[marktucker]

FWIW Houdini has been doing gcc 11 builds since Houdini 20/USD 23.08. Our shipped builds are done with 11.2, but internally, some devs work with 11.3. We don't use the standard build script, so there may be issues with that approach,. But no compile errors and no reported failures have been blamed on the compiler.

[sunyab]

I just tried building the dev branch on gcc 11.4.1 and had no problems as well.

@pberto What errors did you run into?

[pberto]

So, building dev works , but of course I was building release, where it does not. It seems boost-python related (unsurprisingly). @sunyab Can you try on your end building release (24.08)? You should hit the same issue. I replicated it even with regular build_usd.py and I am building with the default setting with the exception of materialx which I disabled as it's on by default.

I am using gcc 11.4.1

[sunyab]

@pberto My 24.08 build using build_usd.py with gcc11 completed without errors. What Python version are you using? My build was done with Python 3.9.18.

USD will be using its own embedded version of boost::python as of 24.11, so if the error is specific to the boost build system then hopefully whatever you're running into won't be relevant going forward.

[pberto]

3.9.18 too. I am now trying on the other twin RedHat9.4 system. I also tried other python inside venv but its painful because python-devel is not found (I am not even sure why). Will edit this soon.

[pberto]

So it seems related to.... numpy? Note I dont have errors, this is why I could not find anything in the log, but a search for "failed" reveled it. If you have any ideas...

./boost/bind.hpp: At global scope:
./boost/config/pragma_message.hpp:24:34: note: ‘#pragma message: The practice of declaring the Bind placeholders (_1, _2, ...) in the global namespace is deprecated. Please use <boost/bind/bind.hpp> + using namespace boost::placeholders, or define BOOST_BIND_GLOBAL_PLACEHOLDERS to retain the current behavior.’
   24 | # define BOOST_PRAGMA_MESSAGE(x) _Pragma(BOOST_STRINGIZE(message(x)))
      |                                  ^~~~~~~
./boost/bind.hpp:36:1: note: in expansion of macro ‘BOOST_PRAGMA_MESSAGE’
   36 | BOOST_PRAGMA_MESSAGE(
      | ^~~~~~~~~~~~~~~~~~~~
libs/python/src/numpy/dtype.cpp: In member function ‘int boost::python::numpy::dtype::get_itemsize() const’:
libs/python/src/numpy/dtype.cpp:101:83: error: ‘PyArray_Descr’ {aka ‘struct _PyArray_Descr’} has no member named ‘elsize’
  101 | int dtype::get_itemsize() const { return reinterpret_cast<PyArray_Descr*>(ptr())->elsize;}
      |                                                                                   ^~~~~~

    "g++"   -fvisibility-inlines-hidden -fPIC -m64 -pthread -O3 -finline-functions -Wno-inline -Wall -fvisibility=hidden  -DBOOST_ALL_NO_LIB=1 -DBOOST_NUMPY_SOURCE -DNDEBUG  -I"." -I"/home/jenkins/.local/lib/python3.9/site-packages/numpy/_core/include" -I"/usr/include/python3.9"  -c -o "/home/jenkins/Dev/openusd-grp/build/boost/bin.v2/libs/python/build/gcc-11/release/python-3.9/threading-multi/visibility-hidden/numpy/dtype.o" "libs/python/src/numpy/dtype.cpp"

...failed gcc.compile.c++ /home/jenkins/Dev/openusd-grp/build/boost/bin.v2/libs/python/build/gcc-11/release/python-3.9/threading-multi/visibility-hidden/numpy/dtype.o...

[mattyjams]

That sounds like you might be using NumPy 2.x, in which case you may be running afoul of this:
boostorg/python#431

NumPy 2.0.0 introduced an API change that required a modification in boost::python, which was made in boostorg/python#432 as commit boostorg/python@0474de0.

I raised this recently in #vfx_reference_platform on the ASWF Slack, but there isn't really much we might do about it at the moment, since the fix somehow missed Boost 1.86.0 and still isn't yet in an official release:
https://academysoftwarefdn.slack.com/archives/C017TA11X2S/p1724365162466289

I have a branch I've been maintaining that pushes most of OpenUSD's dependencies forward, and in there is a change to build_usd.py to apply the fix to the Boost source before building it:
mattyjams@93f42f3

Maybe try that and see if it helps?

[sunyab]

Agreed with @mattyjams that this is what you're running into. OpenUSD doesn't actually use boost::python's numpy support, but AFAICT there's no easy way to disable it when building boost. Aside from the patch, downgrading your NumPy version would also work.

This issue won't show up in 24.11, we've disabled the numpy extensions in our embedded version of boost::python.

[pberto]

So I did not try what @mattyjams suggested however it make sense. Anyway I managed to build it by simply pip uninstall numpy. Honestly I dont know if this make sense, but hey, it worked.