From 92111a277b87d867efbe27a061f651efa9293b2f Mon Sep 17 00:00:00 2001 From: CHENGJin Date: Tue, 10 Jul 2018 17:04:31 -0400 Subject: [PATCH] Fix various script issues with Windows JDK11 The changes are to resolve all script issues detected when compiling JDK11 on Windows. Signed-off-by: CHENGJin --- buildenv/Build_Instructions_V11.md | 457 ++++++++++++++++++ .../compiler/build/toolcfg/win-msvc/common.mk | 4 + runtime/ddr/blacklist | 2 + runtime/ddr/module.xml | 3 + 4 files changed, 466 insertions(+) create mode 100644 buildenv/Build_Instructions_V11.md diff --git a/buildenv/Build_Instructions_V11.md b/buildenv/Build_Instructions_V11.md new file mode 100644 index 00000000000..99618924d3f --- /dev/null +++ b/buildenv/Build_Instructions_V11.md @@ -0,0 +1,457 @@ + + +Building OpenJDK Version 11 with OpenJ9 +====================================== + +Our website describes a simple [build process](http://www.eclipse.org/openj9/oj9_build.html) +that uses Docker and Dockerfiles to create a build environment that contains everything +you need to easily build a Linux binary of **OpenJDK V10** with the Eclipse OpenJ9 virtual machine. +A more complete set of build instructions are included here for multiple platforms: + +- [Linux :penguin:](#linux) +- [AIX :blue_book:](#aix) +- [Windows :ledger:](#windows) +- [MacOS :apple:](#macos) +- [ARM :iphone:](#arm) + +---------------------------------- + +## Linux +:penguin: +This build process provides detailed instructions for building a Linux x86-64 binary of **OpenJDK V11** with OpenJ9 on Ubuntu 16.04. The binary can be built directly on your system, in a virtual +machine, or in a Docker container :whale:. + +If you are using a different Linux distribution, you might have to review the list of libraries that are bundled with your distribution and/or modify the instructions to use equivalent commands to the Advanced Packaging Tool (APT). For example, for Centos, substitute the `apt-get` command with `yum`. + +If you want to build a binary for Linux on a different architecture, such as Power Systems™ or z Systems™, the process is very similar and any additional information for those architectures are included as Notes :pencil: as we go along. + +### 1. Prepare your system +:penguin: +Instructions are provided for preparing your system with and without the use of Docker technology. + +Skip to [Setting up your build environment without Docker](#setting-up-your-build-environment-without-docker). + +#### Setting up your build environment with Docker :whale: +If you want to build a binary by using a Docker container, follow these steps to prepare your system: + +1. The first thing you need to do is install Docker. You can download the free Community edition from [here](https://docs.docker.com/engine/installation/), which also contains instructions for installing Docker on your system. You should also read the [Getting started](https://docs.docker.com/get-started/) guide to familiarise yourself with the basic Docker concepts and terminology. + +2. Obtain the [Linux on 64-bit x86 systems Dockerfile](https://github.com/eclipse/openj9/blob/master/buildenv/docker/jdk10/x86_64/ubuntu16/Dockerfile) to build and run a container that has all the correct software pre-requisites. + + :pencil: Dockerfiles are also available for the following Linux architectures: [Linux on 64-bit Power systems™](https://github.com/eclipse/openj9/blob/master/buildenv/docker/jdk10/ppc64le/ubuntu16/Dockerfile) and [Linux on 64-bit z Systems™](https://github.com/eclipse/openj9/blob/master/buildenv/docker/jdk10/s390x/ubuntu16/Dockerfile) + + Either download one of these Dockerfiles to your local system or copy and paste one of the following commands: + + - For Linux on 64-bit x86 systems, run: +``` +wget https://raw.githubusercontent.com/eclipse/openj9/master/buildenv/docker/jdk10/x86_64/ubuntu16/Dockerfile +``` + + - For Linux on 64-bit Power systems, run: +``` +wget https://raw.githubusercontent.com/eclipse/openj9/master/buildenv/docker/jdk10/ppc64le/ubuntu16/Dockerfile +``` + + - For Linux on 64-bit z Systems, run: +``` +wget https://raw.githubusercontent.com/eclipse/openj9/master/buildenv/docker/jdk10/s390x/ubuntu16/Dockerfile +``` + +3. Next, run the following command to build a Docker image, called **openj9**: +``` +docker build -t openj9 -f Dockerfile . +``` + +4. Start a Docker container from the **openj9** image with the following command, where `-v` maps any directory, ``, +on your local system to the containers `/root/hostdir` directory so that you can store the binaries, once they are built: +``` +docker run -v :/root/hostdir -it openj9 +``` + +:pencil: Depending on your [Docker system configuration](https://docs.docker.com/engine/reference/commandline/cli/#description), you might need to prefix the `docker` commands with `sudo`. + +Now that you have the Docker image running, you are ready to move to the next step, [Get the source](#2-get-the-source). + +#### Setting up your build environment without Docker + +If you don't want to user Docker, you can still build an **OpenJDK V10** with OpenJ9 directly on your Ubuntu system or in a Ubuntu virtual machine. Use the +[Linux on x86 Dockerfile](https://github.com/eclipse/openj9/blob/master/buildenv/docker/jdk10/x86_64/ubuntu16/Dockerfile) like a recipe card to determine the software dependencies +that must be installed on the system, plus a few configuration steps. + +:pencil: +Not on x86? We also have Dockerfiles for the following Linux architectures: [Linux on Power systems](https://github.com/eclipse/openj9/blob/master/buildenv/docker/jdk10/ppc64le/ubuntu16/Dockerfile) and [Linux on z Systems](https://github.com/eclipse/openj9/blob/master/buildenv/docker/jdk10/s390x/ubuntu16/Dockerfile). + +1. Install the list of dependencies that can be obtained with the `apt-get` command from the following section of the Dockerfile: +``` +apt-get update \ + && apt-get install -qq -y --no-install-recommends \ + autoconf \ + ca-certificates \ + ... +``` + +2. This build uses the same gcc and g++ compiler levels as OpenJDK, which might be +backlevel compared with the versions you use on your system. Create links for +the compilers with the following commands: +``` +ln -s g++ /usr/bin/c++ +ln -s g++-4.8 /usr/bin/g++ +ln -s gcc /usr/bin/cc +ln -s gcc-4.8 /usr/bin/gcc +``` + +3. Download and setup **freemarker.jar** into a directory. +``` +cd / +wget https://sourceforge.net/projects/freemarker/files/freemarker/2.3.8/freemarker-2.3.8.tar.gz/download -O freemarker.tgz +tar -xzf freemarker.tgz freemarker-2.3.8/lib/freemarker.jar --strip=2 +rm -f freemarker.tgz +``` + +4. Download and setup the boot JDK using the latest AdoptOpenJDK v9 build. +``` +cd / +wget -O bootjdk9.tar.gz https://api.adoptopenjdk.net/openjdk9-openj9/releases/x64_linux/latest/binary +tar -xzf bootjdk9.tar.gz +rm -f bootjdk9.tar.gz +ls | grep -i jdk | xargs -I % sh -c 'mv % bootjdk9' + +export JAVA_HOME="//bootjdk9" +export PATH="${JAVA_HOME}/bin:${PATH}" +``` + +### 2. Get the source +:penguin: +First you need to clone the Extensions for OpenJDK for OpenJ9 project. This repository is a git mirror of OpenJDK without the HotSpot JVM, but with an **openj9** branch that contains a few necessary patches. Run the following command: +``` +git clone https://github.com/ibmruntimes/openj9-openjdk-jdk11.git +``` +Cloning this repository can take a while because OpenJDK is a large project! When the process is complete, change directory into the cloned repository: +``` +cd openj9-openjdk-jdk11 +``` +Now fetch additional sources from the Eclipse OpenJ9 project and its clone of Eclipse OMR: +``` +bash get_source.sh +``` + +### 3. Configure +:penguin: +When you have all the source files that you need, run the configure script, which detects how to build in the current build environment. +``` +bash configure --with-freemarker-jar=//freemarker.jar +``` +:warning: You must give an absolute path to freemarker.jar + +:pencil: If you require a heap size greater than 57GB, enable a noncompressedrefs build with the `--with-noncompressedrefs` option during this step. + +### 4. Build +:penguin: +Now you're ready to build **OpenJDK V11** with OpenJ9: +``` +make all +``` +:warning: If you just type `make`, rather than `make all` your build will fail, because the default `make` target is `exploded-image`. If you want to specify `make` instead of `make all`, you must add `--default-make-target=images` when you run the configure script. For more information, read this [issue](https://github.com/ibmruntimes/openj9-openjdk-jdk9/issues/34). + +Two Java builds are produced: a full developer kit (jdk) and a runtime environment (jre) +- **build/linux-x86_64-normal-server-release/images/jdk** +- **build/linux-x86_64-normal-server-release/images/jre** + + :whale: If you built your binaries in a Docker container, copy the binaries to the containers **/root/hostdir** directory so that you can access them on your local system. You'll find them in the directory you set for `` when you started your Docker container. See [Setting up your build environment with Docker](#setting-up-your-build-environment-with-docker). + + :pencil: On other architectures the **/jdk** and **/jre** directories are in **build/linux-ppc64le-normal-server-release/images** (Linux on 64-bit Power systems) and **build/linux-s390x-normal-server-release/images** (Linux on 64-bit z Systems) + +### 5. Test +:penguin: +For a simple test, try running the `java -version` command. +Change to the /jre directory: +``` +cd build/linux-x86_64-normal-server-release/images/jre +``` +Run: +``` +./bin/java -version +``` + +Here is some sample output: + +``` +openjdk version "10-internal" 2018-03-20 +OpenJDK Runtime Environment (build 10-internal+0-adhoc..openj9-openjdk-jdk10) +Eclipse OpenJ9 VM (build master-4ecf784, JRE 10 Linux amd64-64 Compressed References 20180411_000000 (JIT enabled, AOT enabled) +OpenJ9 - 4ecf784 +OMR - cb9e24d +JCL - eaa06eb based on jdk-10+46) +``` +:penguin: *Congratulations!* :tada: + +---------------------------------- + +## AIX +:blue_book: + +:construction: +This section is still under construction. Further contributions expected. + +The following instructions guide you through the process of building an **OpenJDK V11** binary that contains Eclipse OpenJ9 on AIX 7.2. + +### 1. Prepare your system +:blue_book: +You must install the following AIX Licensed Program Products (LPPs): + +- [Java9_64](https://adoptopenjdk.net/releases.html?variant=openjdk9-openj9#ppc64_aix) +- [xlc/C++ 13.1.3](https://www.ibm.com/developerworks/downloads/r/xlcplusaix/) +- x11.adt.ext + +A number of RPM packages are also required. The easiest method for installing these packages is to use `yum`, because `yum` takes care of any additional dependent packages for you. + +Download the following file: [yum_install_aix-ppc64.txt](aix/jdk10/yum_install_aix-ppc64.txt) + +This file contains a list of required RPM packages that you can install by specifying the following command: +``` +yum shell yum_install_aix-ppc64.txt +``` + +It is important to take the list of package dependencies from this file because it is kept right up to date by our developers. + +Download and setup freemarker.jar into your home directory by running the following commands: + +``` +cd / +wget https://sourceforge.net/projects/freemarker/files/freemarker/2.3.8/freemarker-2.3.8.tar.gz/download -O freemarker.tgz +tar -xzf freemarker.tgz freemarker-2.3.8/lib/freemarker.jar --strip=2 +rm -f freemarker.tgz +``` + +### 2. Get the source +:blue_book: +First you need to clone the Extensions for OpenJDK for OpenJ9 project. This repository is a git mirror of OpenJDK without the HotSpot JVM, but with an **openj9** branch that contains a few necessary patches. Run the following command: +``` +git clone https://github.com/ibmruntimes/openj9-openjdk-jdk11.git +``` +Cloning this repository can take a while because OpenJDK is a large project! When the process is complete, change directory into the cloned repository: +``` +cd openj9-openjdk-jdk11 +``` +Now fetch additional sources from the Eclipse OpenJ9 project and its clone of Eclipse OMR: + +``` +bash get_source.sh +``` +### 3. Configure +:blue_book: +When you have all the source files that you need, run the configure script, which detects how to build in the current build environment. +``` +bash configure --with-freemarker-jar=//freemarker.jar \ + --with-cups-include= \ + --disable-warnings-as-errors +``` +where `` is the location where you stored **freemarker.jar** and `` is the absolute path to CUPS. For example `/opt/freeware/include`. + +:pencil: If you require a heap size greater than 57GB, enable a noncompressedrefs build with the `--with-noncompressedrefs` option during this step. + +### 4. build +:blue_book: +Now you're ready to build OpenJDK with OpenJ9: +``` +make all +``` +:warning: If you just type `make`, rather than `make all` your build will fail, because the default `make` target is `exploded-image`. If you want to specify `make` instead of `make all`, you must add `--default-make-target=images` when you run the configure script. For more information, read this [issue](https://github.com/ibmruntimes/openj9-openjdk-jdk9/issues/34). + +Two Java builds are produced: a full developer kit (jdk) and a runtime environment (jre) +- **build/aix-ppc64-normal-server-release/images/jdk** +- **build/aix-ppc64-normal-server-release/images/jre** + +### 5. Test +:blue_book: +For a simple test, try running the `java -version` command. +Change to the /jdk directory: +``` +cd build/aix-ppc64-normal-server-release/images/jdk +``` +Run: +``` +./bin/java -version +``` + +Here is some sample output: + +``` +openjdk version "10-internal" 2018-03-20 +OpenJDK Runtime Environment (build 10-internal+0-adhoc..openj9-openjdk-jdk10) +Eclipse OpenJ9 VM (build master-4ecf784, JRE 10 AIX ppc64-64 Compressed References 20180411_000000 (JIT enabled, AOT enabled) +OpenJ9 - 4ecf784 +OMR - cb9e24d +JCL - eaa06eb based on jdk-10+46) +``` +:blue_book: *Congratulations!* :tada: + +---------------------------------- + +## Windows +:ledger: + +The following instructions guide you through the process of building a Windows **OpenJDK V11** binary that contains Eclipse OpenJ9. This process can be used to build binaries for Windows 7, 8, and 10. + +### 1. Prepare your system +:ledger: +You must install a number of software dependencies to create a suitable build environment on your system: + +- [Cygwin](https://cygwin.com/install.html), which provides a Unix-style command line interface. Install all packages in the `Devel` category. Included in this package is mingw, a minimalist subset of GNU tools for Microsoft Windows. OpenJ9 uses the mingw/GCC compiler and is tested with version 6.4.0 of same. Other versions are expected to work also. In the `Archive` category, install the packages `zip` and `unzip`. In the `Utils` category, install the `cpio` package. Install any further package dependencies that are identified by the installer. More information about using Cygwin can be found [here](https://cygwin.com/docs.html). +- [Windows JDK 10](https://adoptopenjdk.net/releases.html?variant=openjdk10#x64_win), which is used as the boot JDK. +- [Microsoft Visual Studio 2017]( https://visualstudio.microsoft.com/thank-you-downloading-visual-studio/?sku=Community&rel=15), which is the default compiler level used by OpenJDK11; or [Microsoft Visual Studio 2013]( https://go.microsoft.com/fwlink/?LinkId=532495), which is chosen to compile if VS2017 isn't installed. +- [Freemarker V2.3.8](https://sourceforge.net/projects/freemarker/files/freemarker/2.3.8/freemarker-2.3.8.tar.gz/download) +- [mingw-w64](https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win32/Personal%20Builds/mingw-builds/installer/mingw-w64-install.exe) + +Note: +mingw-w64 must be installed if compiling with VS2017 in that the current version of cygwin has not yet integrated the latest changes of mingw-w64 for the moment. Meanwhile, add the binary path of mingw-w64 to the `PATH` environment variable. e.g. +``` +set PATH=%PATH%;C:\mingw-w64\x86_64-8.1.0-win32-seh-rt_v6-rev0\mingw64\bin +or +export PATH="$PATH;C:\mingw-w64\x86_64-8.1.0-win32-seh-rt_v6-rev0\mingw64\bin" (in Cygwin) +``` + +Update your `LIB` and `INCLUDE` environment variables to provide a path to the Windows debugging tools with the following commands: + +``` +set INCLUDE=C:\Program Files\Debugging Tools for Windows (x64)\sdk\inc;%INCLUDE% +set LIB=C:\Program Files\Debugging Tools for Windows (x64)\sdk\lib;%LIB% +``` + + You can download Visual Studio, Freemarker manually or obtain them using the [wget](http://www.gnu.org/software/wget/faq.html#download) utility. If you choose to use `wget`, follow these steps: + +- Open a cygwin terminal and change to the `/temp` directory: +``` +cd /cygdrive/c/temp +``` + +- Run the following commands: +``` +wget https://go.microsoft.com/fwlink/?LinkId=532495 -O vs2013.exe +wget https://sourceforge.net/projects/freemarker/files/freemarker/2.3.8/freemarker-2.3.8.tar.gz/download -O freemarker.tgz +``` +- Before installing Visual Studio, change the permissions on the installation file by running `chmod u+x vs2013.exe`. +- Install Visual Studio by running the file `vs2013.exe` (There is no special step required for downloading/installing VS2017. Please follow the guide of the downloaded installer to install all required components, especially for VC compiler). + +Not all of the shared libraries that are included with Visual Studio are registered during installation. +In particular, the `msdia120.dll` libraries must be registered manually. +To do so, execute the following from a command prompt: + +**VS2013** +``` +regsvr32 "C:\Program Files (x86)\Microsoft Visual Studio 12.0\DIA SDK\bin\msdia120.dll" +regsvr32 "C:\Program Files (x86)\Microsoft Visual Studio 12.0\DIA SDK\bin\amd64\msdia120.dll" +``` +**VS2017** +``` +regsvr32 "C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\DIA SDK\bin\msdia140.dll" +regsvr32 "C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\DIA SDK\bin\amd64\msdia140.dll" +``` + +- To unpack the Freemarker and Freetype compressed files, run: +``` +tar -xzf freemarker.tgz freemarker-2.3.8/lib/freemarker.jar --strip=2 +``` + +### 2. Get the source +:ledger: +First you need to clone the Extensions for OpenJDK for OpenJ9 project. This repository is a git mirror of OpenJDK without the HotSpot JVM, but with an **openj9** branch that contains a few necessary patches. + +Run the following command in the Cygwin terminal: +``` +git clone https://github.com/ibmruntimes/openj9-openjdk-jdk11.git +``` +Cloning this repository can take a while because OpenJDK is a large project! When the process is complete, change directory into the cloned repository: +``` +cd openj9-openjdk-jdk11 +``` +Now fetch additional sources from the Eclipse OpenJ9 project and its clone of Eclipse OMR: + +``` +bash get_source.sh +``` +### 3. Configure +:ledger: +When you have all the source files that you need, run the configure script, which detects how to build in the current build environment. +``` +bash configure --disable-warnings-as-errors \ + --with-toolchain-version=2013 or 2017 \ + --with-freemarker-jar=/cygdrive/c/temp/freemarker.jar +``` +Note: there is no need to specify --with-toolchain-version for 2017 as it will be located automatically + +:pencil: Modify the paths for freemarker if you manually downloaded and unpacked these dependencies into different directories. If Java 10 is not available on the path, add the `--with-boot-jdk=` configuration option. + +:pencil: If you require a heap size greater than 57GB, enable a noncompressedrefs build with the `--with-noncompressedrefs` option during this step. + +### 4. build +:ledger: +Now you're ready to build OpenJDK with OpenJ9: +``` +make all +``` + +Two Java builds are produced: a full developer kit (jdk) and a runtime environment (jre) +- **build/windows-x86_64-normal-server-release/images/jdk** +- **build/windows-x86_64-normal-server-release/images/jre** + +### 5. Test +:ledger: +For a simple test, try running the `java -version` command. +Change to the /jdk directory: +``` +cd build/windows-x86_64-normal-server-release/images/jdk +``` +Run: +``` +./bin/java -version +``` + +Here is some sample output: + +``` +openjdk version "11-internal" 2018-09-25 +OpenJDK Runtime Environment (build 11-internal+0-adhoc.Administrator.openj9-openjdk-jdk11) +Eclipse OpenJ9 VM (build master-11410ac2, JRE 11 Windows 7 amd64-64-Bit Compressed References 20180724_000000 (JIT enabled, AOT enabled) +OpenJ9 - 11410ac2 +OMR - e2e4b67c +JCL - a786f96b13 based on jdk-11+21) +``` + +:ledger: *Congratulations!* :tada: + +---------------------------------- + +## MacOS +:apple: + +:construction: +We haven't created a full build process for macOS yet? Watch this space! + +---------------------------------- + +## ARM +:iphone: + +:construction: +We haven't created a full build process for ARM yet? Watch this space! diff --git a/runtime/compiler/build/toolcfg/win-msvc/common.mk b/runtime/compiler/build/toolcfg/win-msvc/common.mk index cb9d28aeb24..d84abfe4646 100644 --- a/runtime/compiler/build/toolcfg/win-msvc/common.mk +++ b/runtime/compiler/build/toolcfg/win-msvc/common.mk @@ -237,6 +237,10 @@ SOLINK_FLAGS+=-nologo -nodefaultlib -incremental:no -debug SOLINK_LIBPATH+=$(PRODUCT_LIBPATH) SOLINK_SLINK+=$(PRODUCT_SLINK) j9thr j9hookable kernel32 oldnames msvcrt msvcprt ws2_32 +ifneq (,$(filter 2015 2017, $(MSVC_VERSION))) + SOLINK_SLINK+=ucrt vcruntime +endif + SOLINK_DEF?=$(JIT_SCRIPT_DIR)/j9jit.def SOLINK_ORG?=0x12900000 diff --git a/runtime/ddr/blacklist b/runtime/ddr/blacklist index b7148b271fe..2b98691ed91 100644 --- a/runtime/ddr/blacklist +++ b/runtime/ddr/blacklist @@ -59,6 +59,8 @@ type:_UNWIND_CODE type:UNWIND_INFO type:PUNWIND_INFO type:_UNWIND_INFO +type:PSLIST_HEADER +type:_SLIST_HEADER # Ignore one (or all) of sets of types whose names differ only in case # (so we don't try to create Java source files whose names differ only diff --git a/runtime/ddr/module.xml b/runtime/ddr/module.xml index 7a5709da126..6b344cf8890 100644 --- a/runtime/ddr/module.xml +++ b/runtime/ddr/module.xml @@ -419,6 +419,9 @@ SPDX-License-Identifier: EPL-2.0 OR Apache-2.0 OR GPL-2.0 WITH Classpath-excepti + + +