Use NASM assembler on x86-64 Windows and update NASM set-up process on OSX and xLinux makefiles

[nbhuiyan]

Depends on: #3293 , #3461 , #3493

This PR builds on top of the changeset in #3293 , and contains commits that are specific to x86_64 Linux, and will be rebased once #3293 is merged.

Issue: #3148

Signed-off-by: Nazim Uddin Bhuiyan [email protected]

[nbhuiyan]

@0xdaryl Ready for review

[0xdaryl]

Jenkins test sanity all jdk11

[0xdaryl]

Jenkins test sanity all jdk11

[nbhuiyan]

The CI build failures on win64 so far has been due to how -I include path search directives are handled in NASM 2.13, and the fact that we are using cygwin for the jenkins builds. Prior to NASM version 2.14, -I include paths provided in the command line would not undergo any further processing, and would simply be concatenated with the filename specified in the %include preprocessor directive, requiring us to have a trailing \ or / in the include file search paths.

When running NASM through Cygwin, it appears as though the full paths enclosed in quotes for the -I command line arguments get processed into a version without the trailing slash, which results in being unable to open the file to include due to the missing path separator between the file name and the containing directory when NASM naively concatenates them. Having absolute paths enclosed in quotes is a good idea in the situation that a parent directory contains a space in its name.

This will not be a problem in NASM version 2.14 and newer, as the -I option semantics have been vastly improved and we are no longer required to ensure a trailing slash in the -I paths. If want to use the full path in -I, which is not a requirement, even a slightly older version of NASM will be unable to build the assembly files. Given that NASM 2.14 was released very recently, perhaps it is too soon to make that a requirement when there is a workaround to it.

The workaround (currently in this PR) that I have been able to come up with supports NASM version 2.14 as well as the previous releases. This involves not enclosing the -I paths in quotes, and using relative paths with a trailing slash from compiler directory to the include file search directories instead of absolute paths. That way, we do not have to worry about the possibility of spaces in a parent directory resulting in issues. The target file and the source files are still going to be enclosed in quotes without any issues. This will allow us to build the JIT using both Cygwin and native Windows shells.

I would still like the transition to NASM v2.14 to happen soon, as it would clean up some hacks that were needed in CMake, but with this approach, we can make the transition smoother as it would work across the 2 versions.

Currently, the %include directives for Windows have been ifdef'd out as I experimented with different command formats. If we can agree on my current implementation, I will apply the same approach to macOS and xLinux as well and clean this up.

[0xdaryl]

Jenkins test sanity all jdk11

[0dvictor]

I don't think requiring NASM 2.14 is unreasonable. The workaround being applies has serve limitations: 1. it is quite common seeing spaces in the path, for example the Desktop folder in Windows 7 is under "Documents and Settings"; 2. search paths of .inc files are different from the old PASM ones and other C/C++ files. In addition, none of existing code requires any early version of NASM and hence I don't think requiring the latest version is too soon. As a reference, we introduced the requirement of Clang 7 only one month after it was released.

However, if upgrading/installing the required NASM 2.14 cannot make the deadline of a targeted release, I accept this workaround with two one extra notes in the building instructions and/or release notes:

1. Mark NASM 2.14 as a requirement;
2. Document the limitation that no space can exist in build path.

[nbhuiyan]

@0dvictor

Spaces can exist in the build path, since the paths being provided with the -I option are relative, and are the following:

NASM_INCLUDES=\
    ../oti \
    ../compiler \
    ../compiler/x/runtime \
    ../compiler/x/amd64/runtime

Relative -I paths are used when building other components of OpenJ9 as well, so we are not adding any new constraints.

[0dvictor]

@nbhuiyan Thank you for clarification. Now I see spaces in the paths to build should be okay.

[0xdaryl]

Jenkins test sanity all jdk11

[0xdaryl]

Jenkins test sanity all jdk11

[0xdaryl]

Jenkins test sanity plinux,aix jdk11

[0xdaryl]

Sanity tests have passed, as well as additional stress tests performed within IBM. This PR is also needed to enable #4249. Merging.