Simplify setup and use of Graal, and JDK 9 Support #232
Conversation
smarr
added
enhancement
|
In my opinion, language implementors using SOMns should be able to work with the Truffle DSL without needing to understanding Graal in much detail. As such, my preference would be the latter option of using the embedded Graal. Structuring the repo in this way suggests that Graal is more of a black box than a component the implementor needs to be worried about. The JVMCI-enabled JDK is still a problem (from my recent experience with @potanin, it seems that this is really the hardest part). If we were to take this path, when could we shift to >Java9? |
|
@Richard-Roberts when ever someone puts in the work to make sure we are Java 9 compatible... |
|
@smarr did i get this correctly: embedded graal basically does what graal-basic does and creates a graal vm in a subdirectory of SOMns? |
|
@daumayr there seems to be confusion about the terminology here. GraalBasic is HotSpot (incl. JVMCI) + Graal repo. The embedded Graal is just the Graal repo, so no HotSpot. For running SOMns without JIT compilation, we need Truffle + any JVM. I am not trying to create a "GraalVM" in a subdirectory of SOMns. |
|
With GraalVM i ment the result of compiling GraalBasic, i.e. the contents of ~/.local/graal-basic |
|
~/.local/graal-basic is the combination of HotSpot (incl. JVMCI) + Graal, which would not be necessary anymore. |
|
because like with JVMTI you can put JVMCI compilers in the JVM arguments? |
|
Yes, indeed. So far, the |
|
So, outside of the CI, are there any advantages of compiling Graal ourselves? |
|
@Richard-Roberts not sure you are referring to what you think you are referring to. In either scenario, Graal is compiled "ourselves" to make sure that it matches our Truffle version. We just don't need to compile HotSpot anymore. |
|
Sorry, I meant to say "are there any significant advantages of compiling Hotpot ourselves?" |
|
Not much, except that it is easier in the CI, because it doesn't require a download. |
We never really used them consistently anyway, removal seems to be unproblematic. Signed-off-by: Stefan Marr <[email protected]>
|
I did spent some time on trying to get things running on Java 9. Trying to use the Not sure what we can do without having to do major changes, for instance to avoid any unsafe use. |
Signed-off-by: Stefan Marr <[email protected]>
|
Thanks to the help of @eregon, I managed to get things working on Java 9. With my current set of changes, it should be possible to build SOMns having only Java 9 on the system. We are still build as Java 8 however. Furthermore, the default behavior will use the embedded Graal compiler, which can be disabled by passing the |
This adds the `-EG` flag to disable the use of the embedded Graal, which we might want to use on the CI system to avoid having to compile Graal, and to have more control over when we update the compiler to get reliable performance numbers. - adapt CI, and add Java 9 runs on Travis using embedded Graal - skip Graal builing on CI in most cases with $SG (skip graal) Signed-off-by: Stefan Marr <[email protected]>
|
Ok, to summarize the latest status (updated the initial issue description, too): The most recent patch in this PR introduces support to build SOMns on JDK 9. Think this is the best way forward. We use the embedded Graal by default and add support for fallbacks. Note that all testing is still done on Java 8, and only basic functionality is tested on Java 9. |
With this change, we recommend using Java 9 exclusively and do not refer to GraalBasic anymore. While I intent to keep using it, it seems just a too big hurdle for users. Requiring Java 9 should be simpler for them. Signed-off-by: Stefan Marr <[email protected]>
Signed-off-by: Stefan Marr <[email protected]>
|
@smarr Is Graal bundled with JDK 9 on both Linux and macOS? I seems to recall that was only the case on Linux but it's been a long time. The bundled Graal could also be very old and not compatible with recent Truffle. |
|
@eregon don't know. But I am referring to the Graal that's embedded into SOMns. With respect to flags and stuff, it seems to work equally well on Ubuntu and macOS. |
|
Ah, that should be fine then. I thought you meant the internal Graal in JDK 9 used for jaotc. |
smarr
changed the title
Currently, all documentation and the setup is advising people to use GraalBasic, which requires to compile a JVMCI-enabled HotSpot. (JVMCI is the Java Virtual Machine Compiler Interface, which enables the use of Graal)
The main reason for this setup is that it simplifies CI, and allows us to change Graal somewhat independently from the Truffle used by SOMns. This is important to be able to attribute performance changes correctly to the Graal compiler or something else.
Another reason is historical: Graal and Truffle used to be part of separate repositories.
However, now with Graal and Truffle being part of the same repo, we have the situation that SOMns always comes with its own Graal, which we do not actually use.
It seems also too complicated to build HotSpot for many users.
Thus it would be desirable to avoid this extra hurdle and allow them to use prebuilt binaries.
Options for how to use Graal
There are a two different options we could take here, and I'd like to hear your opinion: @daumayr @Richard-Roberts.
1. keep using Graal external to SOMns as default
pro:
con: it does not make it easier for people to use Graal
2. use the embedded Graal per default
pro: avoids having the Graal source externally, avoids mismatch issues
con: means we either need an option to use external Graal, or bind ourselves to the Graal version of Truffle
Even with the support of using the embedded Graal, one hurdle for users is still the need for a JVMCI-enabled Java 8/HotSpot. In the long term, this is going to be fixed by supporting >Java 9.In the mean time, it means people need to either compile HotSpot themselves, they need to download it from Oracle Labs, and accept their license, or we need to start building and hosting binaries ourselves. The later is not too complicated, and we could do an automatic download of the right binary depending on the operating system to shield the user somewhat from it.@Richard-Roberts was also thinking of improving distribution issues more generally.
But, there remains work to be done.
Java 9 Support
The most recent patch in this PR introduces support to build SOMns on JDK 9.
We do not have full Java 9 support, which would require more work, which currently is not very useful because of a performance bug in Java.
However, with the ability to build on JDK 9, we can advise all users to simply use Java 9, which avoids any separate compilation of HotSpot.
Think this is the best way forward. We use the embedded Graal by default and add support for fallbacks. Note that all testing is still done on Java 8, and only basic functionality is tested on Java 9.