JDK8 OSX JIT - Unable to load j9jit29

[babsingh]

When running OpenJ9 JDK8 on OSX where gcc49 is unavailable, the following issue is seen:

JVMJ9VM011W Unable to load j9jit29: dlopen(/Users/[email protected]/Desktop/j2sdk-image/jre/lib/compressedrefs/libj9jit29.dylib, 2): Symbol not found: __ZNSt8__detail15_List_node_base7_M_hookEPS0_ Referenced from: /Users/[email protected]/Desktop/j2sdk-image/jre/lib/compressedrefs/libj9jit29.dylib Expected in: /usr/lib/libstdc++.6.0.9.dylib in /Users/[email protected]/Desktop/j2sdk-image/jre/lib/compressedrefs/libj9jit29.dylib

The JIT library can't be loaded.

The above error is seen because OpenJ9 JDK8 is built with gcc49 and is dependent on the libstdc++.6.0.9 library in gcc49. OSX doesn't have gcc49 by default; this causes the above error. To resolve the above error, gcc49 needs to be installed on OSX: brew install gcc49.

How to remove the dependency on the libstdc++.6.0.9 library in gcc49 when building OpenJ9 JDK8?

Possible approaches:

1. Try removing the dependency on libstdc++.
2. Build OpenJ9 using clang Xcode9 (similar to JDK11) but use gcc Xcode 4 for OpenJDK8 components.
3. Can we compile with gcc49 so that the code works with gcc42 (default gcc compiler on OSX)? this is infeasible.

[babsingh]

fyi - @nbhuiyan @0xdaryl @pshipton @DanHeidinga @sxa555

[pshipton]

@babsingh libstdc++ should be statically linked with the jit, this is how it works on other platforms.

[pshipton]

I think its a setting somewhere that needs to be enabled for the Mac builds.

[pshipton]

found #221

[babsingh]

@pshipton

Apple does not support statically linked binaries on Mac OS X. A statically linked binary assumes binary compatibility at the kernel system call interface, and we do not make any guarantees on that front. Rather, we strive to ensure binary compatibility in each dynamically linked system library and framework.

Reference: https://developer.apple.com/library/archive/qa/qa1118/_index.html

[pshipton]

Considering the other options then, I'm not sure what 3 means, and I don't think 1 is feasible in the short term (i.e. a lot of work to rewrite the jit code). I believe this was considered earlier and rejected in favor of statically linking. I'd pursue option 2.

[nbhuiyan]

@pshipton

I think its a setting somewhere that needs to be enabled for the Mac builds.

That setting already applies to mac builds here. The linker does not complain that it is an invalid option, but simply says that it was unused.

[babsingh]

Can we compile with gcc49 so that the code works with gcc42 (default gcc compiler on OSX)?

more details on point 3. things work after installing gcc49. thought that gcc49 may have an option that allows compatibility with gcc42 or older gcc versions. so, things built with gcc49 could work with the dependencies in gcc42. rethinking, users with no gcc will still experience issues. so, point 3 won't work. also, i don't think it's possible to enforce compatibility between gcc49 and gcc42.

[lmaisons]

@pshipton

Apple does not support statically linked binaries on Mac OS X. A statically linked binary assumes binary compatibility at the kernel system call interface, and we do not make any guarantees on that front. Rather, we strive to ensure binary compatibility in each dynamically linked system library and framework.

Reference: https://developer.apple.com/library/archive/qa/qa1118/_index.html

Also from the referenced page:

A: Before discussing this issue, it's important to be clear about terminology:

• A static library is a library of code that can be linked into a binary that will, eventually, be dynamically linked to the system libraries and frameworks.

• A statically linked binary is one that does not import system libraries and frameworks dynamically, but instead makes direct system calls into the kernel.

Apple fully supports static libraries; if you want to create one, just start with the appropriate Xcode project or target template.

Does statically linking libstdc++ on macOS actually result in what apple considers a statically linked binary? I would have expected it to call dynamically into a libc equivalent rather than making kernel calls directly.

EDIT:

Are -static-libgcc -static-libstdc++ supported on the version of GCC we use on macOS?

[babsingh]

Are -static-libgcc -static-libstdc++ supported on the version of GCC we use on macOS?

Refer to Nazim's comment above. The linker does not complain that it is an invalid option, but simply says that it was unused.

[DanHeidinga]

Build OpenJ9 using clang Xcode9 (similar to JDK11) but use gcc Xcode 4 for OpenJDK8 components.

@babsingh Any update on the progress using this approach?

[nbhuiyan]

@lmaisons

Are -static-libgcc -static-libstdc++ supported on the version of GCC we use on macOS?

-static-libstdc++ is a recognized option for gcc-4.9, but appears to do nothing. Also, gcc-4.9 does not report that the option was unused.

[babsingh]

Build OpenJ9 using clang Xcode9 (similar to JDK11) but use gcc Xcode 4 for OpenJDK8 components. Any update on the progress using this approach?

Downloaded XCode 9 from https://developer.apple.com. It took more than an hour. Xcode9 is a xip file, which requires 10.11.5+ for unzipping. Currently on 10.10.5. Found a 10.11.6 machine. Moving forward to building it. Will post results by the end of today.

[babsingh]

Update:

Build OpenJ9 using clang Xcode9 (similar to JDK11) but use gcc Xcode 4 for OpenJDK8 components.

• Xcode9 needs minimum OSX 10.13 and Xcode4 doesn't work on OSX >= 10.12. Xcode4 needs OSX <= 10.11. So, Xcode9 and Xcode4 can't be used together.
• Xcode4 and Xcode7 can be used together. Tried building OpenJ9 using Xcode7 + clang. OpenJ9 build fine with the JIT disabled. JIT compilation instantly failed. Not sure if something was missing in the setup for the JIT. For the JIT, I changed export PLATFORM=amd64-osx-gcc to export PLATFORM=amd64-osx-clang in runtime/compiler/makefile.ftl. @nbhuiyan Can you check if the JIT can be built using Xcode7 + clang? the same way the JIT is built for JDK11 using Xcode9 + clang.

Changes to the build process: details. ./configure needs to be run differently.

bash ./configure \
    --with-freemarker-jar=/Users/jenkins/openj9_workspace/openj9-openjdk-jdk8/freemarker-2.3.8/lib/freemarker.jar \
    --with-xcode-path=/Applications/Xcode4/Xcode.app \
    --with-openj9-cc=/Applications/Xcode7/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang \
    --with-openj9-cxx=/Applications/Xcode7/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ \
    --with-openj9-developer-dir=/Applications/Xcode7/Xcode.app/Contents/Developer

Reference: ibmruntimes/openj9-openjdk-jdk8#147.

The following environment variables won't work anymore:

export OPENJ9_USE_CUSTOM_COMPILER=1
export OPENJ9_DEVELOPER_DIR="/Applications/Xcode7/Xcode.app/Contents/Developer"
export OPENJ9_CC="/usr/local/bin/gcc-4.9"
export OPENJ9_CXX="/usr/local/bin/g++-4.9"

[nbhuiyan]

Some updates regarding this issue...

In order to succeed running the configure script and building OpenJDK8, we need to set the environment variable MACOSX_DEPLOYMENT_TARGET=10.8. I discovered that this forces Clang in Xcode7 to fall back to always using stdlibc++ (a very outdated version of it), which is why @babsingh faced issues with building the JIT. With some messy manual work, I was able to get a build using Xcode4 + Xcode7:

1. Disable building the JIT by commenting out the build and clean commands in runtime/compiler/module.xml

                <commands>
<!--
			<command line="$(MAKE) -C compiler -f makefile clean" type="clean"/>
			<command line="$(MAKE) -C compiler -f makefile" type="all"/>
-->
		</commands>

2. export MACOSX_DEPLOYMENT_TARGET=10.8 needed by the configure script. Also, set these necessary env variables:

export SED=gsed
export TAR=gtar
export UMA_SUPPRESS_WARNINGS_AS_ERRORS=1
export OMR_WARNINGS_AS_ERRORS=0
export SDKPATH=/Applications/Xcode4/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk

3. Run configure script

bash ./configure \
    --with-freemarker-jar=/Users/jenkins/openj9_workspace/openj9-openjdk-jdk8/freemarker-2.3.8/lib/freemarker.jar \
    --with-xcode-path=/Applications/Xcode4/Xcode.app \
    --with-openj9-cc=/Applications/Xcode7/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang \
    --with-openj9-cxx=/Applications/Xcode7/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ \
    --with-openj9-developer-dir=/Applications/Xcode7/Xcode.app/Contents/Developer \
    --with-boot-jdk=/path/to/bootjdk/

4. make images or make all until JIT phase is about to begin.
5. In a different terminal (or unset MACOSX_DEPLOYMENT_TARGET and set it back later):
   1. cd build/macosx-x86_64-normal-server-release/vm/compiler/
   2. export VERSION_MAJOR=8
   3. export OPENJDK_VERSION_NUMBER_FOUR_POSITIONS=8.0.0.0
   4. make
6. Once that completes, continue the build we interrupted before.

JIT will no longer depend on libstdc++, and instead depend on libc++.

[babsingh]

The above manual work can be avoided if MACOSX_VERSION_MIN is changed from 10.7.0 to 10.9.0 in JDK8 extensions repo. With ibmruntimes/openj9-openjdk-jdk8#206 (fix), OpenJ9 JDK8 can be built in one step; the JIT doesn't need to be built separately.

export MACOSX_DEPLOYMENT_TARGET=10.9.0

cd <PATH_TO>/openj9-openjdk-jdk8

bash ./get_source.sh

bash ./configure 
    --with-freemarker-jar=/Users/jenkins/openj9_workspace/openj9-openjdk-jdk8/freemarker-2.3.8/lib/freemarker.jar  
    --with-xcode-path=/Applications/Xcode4/Xcode.app 
    --with-openj9-cc=clang
    --with-openj9-cxx=clang++
    --with-openj9-developer-dir=/Applications/Xcode7/Xcode.app/Contents/Developer

make LOG=trace images

[babsingh]

@nbhuiyan opened #3629 to change runtime/compiler/makefile.ftl from:

  ifeq ($(VERSION_MAJOR),8)
    export PLATFORM=amd64-osx-gcc
  else
    export PLATFORM=amd64-osx-clang
  endif

to:

  export PLATFORM=amd64-osx-clang

With #3629 and ibmruntimes/openj9-openjdk-jdk8#206, entire OpenJ9 can be built with Xcode7 and the default clang/clang++. This will remove the dependency on libstdc++.6.0.9 and allow us to close this issue.