Setup OSX Machines for JDK8 #3323

babsingh · 2018-10-17T14:59:21Z

Setting up the machine

Install Mac OS X 10.10 (Yosemite)
The instructions below must be executed with sudo permissions.
Install brew

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Download Xcode 4.6.3 from https://developer.apple.com/downloads.

sudo mkdir /Applications/Xcode4

Drag Xcode.app for Xcode4 into /Applications/Xcode4. Alternate command line variant to mount and copy the Xcode dmg file. This will help in automating the setup process on OSX machines:

# Mount the Xcode dmg file
hdiutil attach /path/to/XCODE.dmg

# Copy Xcode.app to the desired location
cp -a /Volumes/Xcode/Xcode.app <PATH_TO_DESIRED_LOCATION>

# Unmout the Xcode dmg file
hdiutil detach /dev/disk...

Download Xcode 7.2.1 from https://developer.apple.com/downloads.

sudo mkdir /Applications/Xcode7

Drag Xcode.app for Xcode7 into /Applications/Xcode7. The above command line variant can be used to mount and copy Xcode7 to a desired location. brew won't allow gcc-4.9 to be installed with Xcode4. So, switch to Xcode7.

sudo xcode-select -switch /Applications/Xcode7/Xcode.app

Accept Xcode license if needed.

sudo xcodebuild -license accept

Install Bash version 4. Instructions: https://gist.github.com/samnang/1759336. ./get_source.sh asks for Bash version 4. Bash version 4 will be located here: /usr/local/bin/bash. Extension repo scripts will try to use /bin/bash. /bin/bash will point to Bash version 3 after installing Bash version 4. Associate /bin/bash symlink to Bash version 4:

rm -f /bin/bash
ln -s /usr/local/bin/bash /bin/bash

Install gnu-sed. Default sed on OSX won't process carriage returns (\r) and new-lines (\n). gsed will doesn't have this issue.

brew install gnu-sed

Install gnu-tar. Default tar on OSX doesn't support --exclude-vcs option where gtar supports --exclude-vcs.

brew install gnu-tar

Install other required tools.

brew install nasm autoconf git freetype

Only required on OSX 10.10 (Yosemite). Edit line 143 (typedef void (^dispatch_block_t)(void);) in /usr/include/dispatch/object.h:

#ifdef __clang__
typedef void (^dispatch_block_t)(void);
#else
typedef void* dispatch_block_t;
#endif

Building OpenJ9 JDK8

Download the boot JDK from AdoptOpenJDK. Technically, JDK7 should be used as the boot JDK but JDK7 isn't available at AdoptOpenJDK. So, I used OpenJDK8 with Hotspot as the boot JDK, and no issues were seen: https://adoptopenjdk.net/releases.html#x64_mac.
Clone openj9-openjdk-jdk8 extensions repo:

git clone https://github.com/ibmruntimes/openj9-openjdk-jdk8.git
cd ./openj9-openjdk-jdk8

Download and unzip freemarker.jar:

wget https://sourceforge.net/projects/freemarker/files/freemarker/2.3.8/freemarker-2.3.8.tar.gz/download -O freemarker.tgz
gtar -xzf freemarker.tgz freemarker-2.3.8/lib/freemarker.jar --strip-components=2
rm -f freemarker.tgz

The default BSD tar on Mac won't strip component. gtar should be used to --strip-components.

Set the following environment variables:

export SED=gsed
export TAR=gtar

export MACOSX_DEPLOYMENT_TARGET=10.9.0
export SDKPATH=/Applications/Xcode4/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk

export JAVA_HOME=<PATH_TO_BOOT_JDK>

^^^ Make sure that the above file and executable paths are valid on the machine. ^^^

Info about the above environment variables:

# SED=gsed overrides the default sed OSX since it doesn't process new lines \n and carriage returns \r.
# TAR=gtar overrides the default tar on OSX since it doesn't support --exclude-vcs option.

# Setting UMA_SUPPRESS_WARNINGS_AS_ERRORS and OMR_WARNINGS_AS_ERRORS: warnings won't be treated as errors.

# If MACOSX_DEPLOYMENT_TARGET is not set to 10.9.0, ./configure will fail.
# SDKPATH points to the Xcode4 SDK directory

# JAVA_HOME should point to the boot JDK.

Run ./configure:

# Compressedrefs
bash ./configure \
    --with-freemarker-jar=<PATH_TO_FREEMARKER_JAR> \
    --with-xcode-path=<PATH_TO_XCODE4_APP> \
    --with-openj9-cc=<PATH_TO_OPENJ9_CC> \
    --with-openj9-cxx=<PATH_TO_OPENJ9_CXX> \
    --with-openj9-developer-dir=<PATH_TO_OPENJ9_DEVELOPER_DIR>

# Non-compressedrefs
bash ./configure \
    --with-freemarker-jar=<PATH_TO_FREEMARKER_JAR> \
    --with-xcode-path=<PATH_TO_XCODE4_APP> \
    --with-openj9-cc=<PATH_TO_OPENJ9_CC> \
    --with-openj9-cxx=<PATH_TO_OPENJ9_CXX> \
    --with-openj9-developer-dir=<PATH_TO_OPENJ9_DEVELOPER_DIR> \
    --with-noncompressedrefs

PATH_TO_XCODE4_APP=/Applications/Xcode4/Xcode.app
PATH_TO_OPENJ9_CC=<POINT_TO_XCODE7_CLANG>
PATH_TO_OPENJ9_CXX=<POINT_TO_XCODE7_CLANG++>
PATH_TO_OPENJ9_DEVELOPER_DIR=/Applications/Xcode7/Xcode.app/Contents/Developer

# OPENJ9_DEVELOPER_DIR, OPENJ9_CC and OPENJ9_CXX are used to specify custom
# tools to build OpenJ9. The default tools Xcode4 + gcc-4.2 will fail to build OpenJ9.

Run make:

# Variant 1
make all

# In JDK8, build.log isn't generated. Also, make's output may have insufficient info for debugging.
# For debugging, make can be run as follows:
make LOG=trace all 2>&1 | tee make_images.log

# Shorter and quicker variant if one doesn't want to run "make all".
make images

If make is successful, a j2sdk-images and a j2re-image should be generated in the build directory.

The text was updated successfully, but these errors were encountered:

babsingh · 2018-10-17T15:00:48Z

fyi - @DanHeidinga @groeges @AdamBrousseau @jdekonin

babsingh · 2018-10-17T15:01:04Z

related to #36

jdekonin · 2018-10-17T15:39:37Z

@cwillhelm can you please setup the remaining 3 macs using these instructions. Please provide any questions or feedback here. Thanks.

cwillhelm · 2018-10-17T15:43:19Z

For these statements Drag Xcode.app for Xcode4 into /Applications/Xcode4

is it as simple as doing cp Xcode.app into /Applications/Xcode4 Or are there other files / steps that need to be done? I ask because this configuration is being done completely from the CLI.

babsingh · 2018-10-17T15:51:55Z

@cwillhelm Xcode comes as a *.dmg file. To access Xcode.app, mounting and unzipping is done by the OS when you double click the *.dmg file. If you can convert *.dmg -> Xcode.app via cmdline, then cp should probably work. You will have to figure out the cmds.

babsingh · 2018-10-17T17:37:31Z

fyi @sxa555

cwillhelm · 2018-10-17T17:52:45Z

Do you specifically need MacOS 10.10, or can we use a newer version? (10.13)

babsingh · 2018-10-17T18:35:03Z

not sure. it will depend on whether Xcode4 and Xcode7 work with a newer OSX version. i followed the instructions from here: https://github.com/manasthakur/jdk-tips/wiki/Building-OpenJDK-8-on-Mac-OS-X-Yosemite.

babsingh · 2018-10-17T21:37:44Z

Added instructions to Build OpenJ9 JDK8 in the first post.

babsingh · 2018-10-18T03:16:25Z

A good source for build platform requirements used for OpenJDK + Hotspot by other providers: https://wiki.openjdk.java.net/display/Build/Supported+Build+Platforms. Mac OSX version seems to be flexible.

JDK 8 build platforms supported by Oracle

OS and CPU	OS Type/Vendor	OS Version	Compiler/Toolchain Type/Vendor	Compiler/Toolchain version	Notes
Mac OSX x86_64	Mac OS X	10.7 (Lion)	Xcode / gcc	4.6.2

Other JDK 8 build platforms

Platform	Status	OS Type/Vendor and Version	Compiler/Toolchain Type/Vendor and Version	Notes
Mac OSX x86_64	Works correctly	Mac OS X 10.8 (Mountain Lion)	XCode / gcc 4.2.1	Maintained by SAP

cwillhelm · 2018-10-19T15:09:43Z

Some additional steps to do prior to starting install (because brew can't be run as root):

mkdir /Users/bruin

dscl . -create /Users/bruin
dscl . -create /Users/bruin UserShell /bin/bash
dscl . -create /Users/bruin RealName “new” 
dscl . -create /Users/bruin UniqueID "510"     # make sure no other user has this id 
dscl . -create /Users/bruin PrimaryGroupID 20
dscl . -create /Users/bruin NFSHomeDirectory /Users/bruin
dscl . -passwd /Users/bruin brewtest      # change me 

visudo        # add bruin to sudoers
sudo chown -R bruin:admin /Users/bruin

su - bruin
sudo xcode-select --reset
brew install wget        # if not already installed

babsingh · 2018-11-06T03:07:37Z

Changes to the OpenJ9 JDK8 build process on OSX. The following environment variables won't work anymore:

export OPENJ9_USE_CUSTOM_COMPILER=1
export OPENJ9_DEVELOPER_DIR="/Applications/Xcode7/Xcode.app/Contents/Developer"
export OPENJ9_CC="/usr/local/bin/gcc-4.9"
export OPENJ9_CXX="/usr/local/bin/g++-4.9"

Reference: ibmruntimes/openj9-openjdk-jdk8#147. Instead of using the above environment variables, ./configure needs to be run differently:

bash ./configure \
    --with-freemarker-jar=<PATH_TO_FREEMARKER_JAR> \
    --with-xcode-path=<PATH_TO_XCODE4_APP> \
    --with-openj9-cc=<PATH_TO_OPENJ9_CC> \
    --with-openj9-cxx=<PATH_TO_OPENJ9_CXX> \
    --with-openj9-developer-dir=<PATH_TO_OPENJ9_DEVELOPER_DIR>

Examples:

PATH_TO_XCODE4_APP=/Applications/Xcode4/Xcode.app
PATH_TO_OPENJ9_CC=/Applications/Xcode7/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang
PATH_TO_OPENJ9_CXX=/Applications/Xcode7/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++
PATH_TO_OPENJ9_DEVELOPER_DIR=/Applications/Xcode7/Xcode.app/Contents/Developer

The instructions in the main description above have also been updated.

babsingh · 2018-11-06T03:17:37Z

^^^ fyi: @sxa555 @groeges @AdamBrousseau @jdekonin

babsingh · 2018-11-06T03:22:01Z

Mounting and copying Xcode dmg file

This will help in automating the setup process on OSX machines:

# Mount the Xcode dmg file
hdiutil attach /path/to/XCODE.dmg

# Copy Xcode.app to the desired location
cp -a /Volumes/Xcode/Xcode.app <PATH_TO_DESIRED_LOCATION>

# Unmout the Xcode dmg file
hdiutil detach /dev/disk...

Reference: http://osxdaily.com/2011/12/17/mount-a-dmg-from-the-command-line-in-mac-os-x/

jdekonin · 2018-11-06T12:57:48Z

I also needed to install pkg-config and I didn't need to modify /usr/include/dispatch/object.h or /bin/bash. The system I used was 10.11.6 (El Capitan), an update to 10.12 (Sierra) causes failures with xcode4.

sxa · 2018-11-08T11:23:38Z

@jdekonin So I can now undo the change to object.h on the machines that we needed previously?

babsingh · 2018-11-08T21:21:20Z

undo the change to object.h on the machines that we needed previously?

The object.h change is needed to workaround an OSX 10.10 specific bug. @jdekonin setup the machines with OSX 10.11 where the object.h workaround is not needed.

Note: One won't be able to edit object.h on OSX >= 10.11 without disabling System Integrity Protection (SIP), which is a tedious step.

fyi - @sxa555

babsingh · 2018-11-09T23:23:59Z

Update:

Referring to comment. Once #3629 and ibmruntimes/openj9-openjdk-jdk8#206 are merged, entire OpenJ9 can be built with Xcode7 and the default clang/clang++. This will remove the dependency on gcc49/libstdc++.6.0.9, and resolve Unable to load j9jit29 issue on Mac's with no gcc49.

I have updated the build instructions above. Three important changes needed once #3629 and ibmruntimes/openj9-openjdk-jdk8#206 are merged:

Remove gcc49 from machines: brew uninstall gcc49. gcc49 is no longer needed.
Set export MACOSX_DEPLOYMENT_TARGET=10.9.0 before running ./configure.
During ./configure, point --with-openj9-cc and --with-openj9-cxx to Xcode7's clang and clang++ respectively. How to find Xcode7's clang? <PATH_TO_XCODE7's>/xcodebuild -find clang.

fyi - @jdekonin @groeges @sxa555

Based on content in eclipse-openj9#3323, added a section for macOS. [ci skip] Signed-off-by: Sue Chaplain <[email protected]>

- Both compressed and non Related eclipse-openj9#3323 [skip ci] Signed-off-by: Adam Brousseau <[email protected]>

babsingh · 2019-04-03T16:03:53Z

The knowledge for setting up OSX machines for OpenJ9 JDK8 development has been transferred to the infra team. Machines have been setup, and the infra team has developed the required scripts for setting up OSX machines for OpenJ9 JDK8 development. Thus, closing this issue.

fyi - @jdekonin @sxa555.

pshipton added the comp:build label Oct 17, 2018

groeges mentioned this issue Oct 19, 2018

Set up macos machines to be able to build OpenJ9 adoptium/infrastructure#524

Closed

babsingh mentioned this issue Nov 6, 2018

JDK8 OSX JIT - Unable to load j9jit29 #3496

Closed

sxa mentioned this issue Nov 8, 2018

Update macos.sh with new OpenJ9 parameters adoptium/temurin-build#720

Merged

This was referenced Nov 13, 2018

Changes to OSX JDK8/JDK11 Setup adoptium/temurin-build#735

Closed

Changes to OSX JDK8/JDK11 Setup adoptium/infrastructure#561

Closed

SueChaplain mentioned this issue Nov 21, 2018

Add build instructions for OpenJDK8 for macOS #3772

Merged

SueChaplain added a commit to SueChaplain/openj9 that referenced this issue Nov 27, 2018

[WIP] Add build instructions for OpenJDK8 for macOS

4a76a3a

Based on content in eclipse-openj9#3323, added a section for macOS. [ci skip] Signed-off-by: Sue Chaplain <[email protected]>

AdamBrousseau added a commit to AdamBrousseau/openj9 that referenced this issue Dec 14, 2018

Enable option to build OSX for JDK8

4b7b88e

- Both compressed and non Related eclipse-openj9#3323 [skip ci] Signed-off-by: Adam Brousseau <[email protected]>

AdamBrousseau mentioned this issue Dec 14, 2018

Enable option to build OSX for JDK8 #4051

Merged

jdekonin mentioned this issue Jan 31, 2019

System test fails to compile on osx1011-x86-1 but compiles on osx1011-x86-2 #4078

Closed

babsingh closed this as completed Apr 3, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Setup OSX Machines for JDK8 #3323

Setup OSX Machines for JDK8 #3323

babsingh commented Oct 17, 2018 •

edited

Loading

babsingh commented Oct 17, 2018

babsingh commented Oct 17, 2018

jdekonin commented Oct 17, 2018

cwillhelm commented Oct 17, 2018

babsingh commented Oct 17, 2018 •

edited

Loading

babsingh commented Oct 17, 2018

cwillhelm commented Oct 17, 2018

babsingh commented Oct 17, 2018 •

edited

Loading

babsingh commented Oct 17, 2018

babsingh commented Oct 18, 2018 •

edited

Loading

cwillhelm commented Oct 19, 2018 •

edited

Loading

babsingh commented Nov 6, 2018 •

edited

Loading

babsingh commented Nov 6, 2018

babsingh commented Nov 6, 2018 •

edited

Loading

jdekonin commented Nov 6, 2018

sxa commented Nov 8, 2018

babsingh commented Nov 8, 2018

babsingh commented Nov 9, 2018

babsingh commented Apr 3, 2019

Setup OSX Machines for JDK8 #3323

Setup OSX Machines for JDK8 #3323

Comments

babsingh commented Oct 17, 2018 • edited Loading

Setting up the machine

Building OpenJ9 JDK8

babsingh commented Oct 17, 2018

babsingh commented Oct 17, 2018

jdekonin commented Oct 17, 2018

cwillhelm commented Oct 17, 2018

babsingh commented Oct 17, 2018 • edited Loading

babsingh commented Oct 17, 2018

cwillhelm commented Oct 17, 2018

babsingh commented Oct 17, 2018 • edited Loading

babsingh commented Oct 17, 2018

babsingh commented Oct 18, 2018 • edited Loading

JDK 8 build platforms supported by Oracle

Other JDK 8 build platforms

cwillhelm commented Oct 19, 2018 • edited Loading

babsingh commented Nov 6, 2018 • edited Loading

babsingh commented Nov 6, 2018

babsingh commented Nov 6, 2018 • edited Loading

Mounting and copying Xcode dmg file

jdekonin commented Nov 6, 2018

sxa commented Nov 8, 2018

babsingh commented Nov 8, 2018

babsingh commented Nov 9, 2018

babsingh commented Apr 3, 2019

babsingh commented Oct 17, 2018 •

edited

Loading

babsingh commented Oct 17, 2018 •

edited

Loading

babsingh commented Oct 17, 2018 •

edited

Loading

babsingh commented Oct 18, 2018 •

edited

Loading

cwillhelm commented Oct 19, 2018 •

edited

Loading

babsingh commented Nov 6, 2018 •

edited

Loading

babsingh commented Nov 6, 2018 •

edited

Loading