Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for --jvm option in Enso runner #10374

Merged
merged 29 commits into from
Jul 6, 2024
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
43929db
Support for --jvm option in Enso launcher
JaroslavTulach Jun 26, 2024
7d31eab
Iterate installed GraalVM runtimes
JaroslavTulach Jun 26, 2024
67f1632
feat: engineDistributionRoot parameter
4e6 Jun 26, 2024
e67cc87
Generate NI executable as ./built-distribution/enso-engine-0.0.0-dev-…
JaroslavTulach Jun 26, 2024
8f18b2f
Ignore --jvm when already in the right JVM
JaroslavTulach Jun 28, 2024
54bb7d8
Recognize JAVA_OPTS environment variable and pass it to the JVM
JaroslavTulach Jun 28, 2024
ec51515
Let EditionManager work with any EditionProvider, not just updating one
JaroslavTulach Jun 28, 2024
f1ad21b
Don't use UpdatingEditionManager in regular execution
JaroslavTulach Jun 28, 2024
cfedcb6
Derive languageHome from code location
JaroslavTulach Jun 29, 2024
2f3e832
Search for the parser in component subdirectory
JaroslavTulach Jun 29, 2024
406291e
Setup logging before processing --jvm option
JaroslavTulach Jun 29, 2024
cc48ec3
Providing missing argument to makeEditionProvider
JaroslavTulach Jun 29, 2024
9c81fe5
Adding update flag to FakeEditionProvider
JaroslavTulach Jun 29, 2024
eafa192
Change log entry
JaroslavTulach Jun 29, 2024
5ea0814
Benchmarks need to be executed in --jvm mode
JaroslavTulach Jun 29, 2024
8c393bf
Merge remote-tracking branch 'origin/develop' into wip/jtulach/JvmOpt…
JaroslavTulach Jul 1, 2024
fa7a62d
Merge remote-tracking branch 'origin/develop' into wip/jtulach/JvmOpt…
JaroslavTulach Jul 3, 2024
13cb3d4
native-image needs location without .exe on Windows
JaroslavTulach Jul 3, 2024
3fb6a1b
There is no runner anymore
JaroslavTulach Jul 4, 2024
8e4834e
Use .exe in native runner tests
JaroslavTulach Jul 5, 2024
59f3087
run_benchmarks without bash script wrapper
JaroslavTulach Jul 5, 2024
83f31be
Supressing question mark operator is useless here warning
JaroslavTulach Jul 5, 2024
1ec943a
Updating documentation to new location of runner
JaroslavTulach Jul 6, 2024
13c9518
Just return benchmarks
JaroslavTulach Jul 6, 2024
ffb2fb8
No need to disable linter
JaroslavTulach Jul 6, 2024
60595ba
Use with_executable_extension
JaroslavTulach Jul 6, 2024
49700cd
Use with_executable_extension II
JaroslavTulach Jul 6, 2024
2bd08b1
cargo fmt
JaroslavTulach Jul 6, 2024
bbbbf25
Using longer local variable name
JaroslavTulach Jul 6, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
103 changes: 56 additions & 47 deletions build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -2614,55 +2614,60 @@ lazy val `engine-runner` = project
assembly := assembly
.dependsOn(`runtime-fat-jar` / assembly)
.value,
rebuildNativeImage :=
NativeImage
.buildNativeImage(
"runner",
staticOnLinux = false,
additionalOptions = Seq(
"-Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.NoOpLog",
"-H:IncludeResources=.*Main.enso$",
"-H:+AddAllCharsets",
"-H:+IncludeAllLocales",
"-ea",
// useful perf & debug switches:
// "-g",
// "-H:+SourceLevelDebug",
// "-H:-DeleteLocalSymbols",
// you may need to set smallJdk := None to use following flags:
// "--trace-class-initialization=org.enso.syntax2.Parser",
"-Dnic=nic"
),
mainClass = Some("org.enso.runner.Main"),
initializeAtRuntime = Seq(
"org.jline.nativ.JLineLibrary",
"org.jline.terminal.impl.jna",
"io.methvin.watchservice.jna.CarbonAPI",
"zio.internal.ZScheduler$$anon$4",
"org.enso.runner.Main$",
"sun.awt",
"sun.java2d",
"sun.font",
"java.awt",
"com.sun.imageio",
"com.sun.jna.internal.Cleaner",
"com.sun.jna.Structure$FFIType",
"akka.http"
rebuildNativeImage := Def
.taskDyn {
NativeImage
.buildNativeImage(
"enso",
JaroslavTulach marked this conversation as resolved.
Show resolved Hide resolved
targetDir = engineDistributionRoot.value / "bin",
staticOnLinux = false,
additionalOptions = Seq(
"-Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.NoOpLog",
"-H:IncludeResources=.*Main.enso$",
"-H:+AddAllCharsets",
"-H:+IncludeAllLocales",
"-ea",
// useful perf & debug switches:
// "-g",
// "-H:+SourceLevelDebug",
// "-H:-DeleteLocalSymbols",
// you may need to set smallJdk := None to use following flags:
// "--trace-class-initialization=org.enso.syntax2.Parser",
"-Dnic=nic"
),
mainClass = Some("org.enso.runner.Main"),
initializeAtRuntime = Seq(
"org.jline.nativ.JLineLibrary",
"org.jline.terminal.impl.jna",
"io.methvin.watchservice.jna.CarbonAPI",
"zio.internal.ZScheduler$$anon$4",
"org.enso.runner.Main$",
"sun.awt",
"sun.java2d",
"sun.font",
"java.awt",
"com.sun.imageio",
"com.sun.jna.internal.Cleaner",
"com.sun.jna.Structure$FFIType",
"akka.http"
)
)
)
.dependsOn(NativeImage.additionalCp)
.dependsOn(NativeImage.smallJdk)
.dependsOn(assembly)
.dependsOn(
buildEngineDistribution
)
.value,
buildNativeImage := NativeImage
.incrementalNativeImageBuild(
rebuildNativeImage,
"runner"
}
.dependsOn(NativeImage.additionalCp)
.dependsOn(NativeImage.smallJdk)
.dependsOn(assembly)
.dependsOn(
buildEngineDistribution
)
.value
.value,
buildNativeImage := Def.taskDyn {
NativeImage
.incrementalNativeImageBuild(
rebuildNativeImage,
"enso",
targetDir = engineDistributionRoot.value / "bin"
)
}.value
)
.dependsOn(`version-output`)
.dependsOn(yaml)
Expand Down Expand Up @@ -3559,6 +3564,10 @@ ThisBuild / buildEngineDistribution := {
buildEngineDistribution.result.value
}

ThisBuild / engineDistributionRoot := {
engineDistributionRoot.value
}

lazy val buildEngineDistributionNoIndex =
taskKey[Unit]("Builds the engine distribution without generating indexes")
buildEngineDistributionNoIndex := {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,11 @@ class MainModule(serverConfig: LanguageServerConfig, logLevel: Level) {
val distributionManager = new DistributionManager(environment)

val editionProvider =
EditionManager.makeEditionProvider(distributionManager, Some(languageHome))
EditionManager.makeEditionProvider(
distributionManager,
Some(languageHome),
false
)
val editionResolver = EditionResolver(editionProvider)
val editionReferenceResolver = new EditionReferenceResolver(
contentRoot.file,
Expand Down
98 changes: 96 additions & 2 deletions engine/runner/src/main/java/org/enso/runner/Main.java
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
import java.nio.file.InvalidPathException;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.HashMap;
Expand Down Expand Up @@ -53,6 +54,7 @@

/** The main CLI entry point class. */
public final class Main {
private static final String JVM_OPTION = "jvm";
private static final String RUN_OPTION = "run";
private static final String INSPECT_OPTION = "inspect";
private static final String DUMP_GRAPHS_OPTION = "dump-graphs";
Expand Down Expand Up @@ -127,6 +129,15 @@ private static Options buildOptions() {
.longOpt(RUN_OPTION)
.desc("Runs a specified Enso file.")
.build();
var jvm =
cliOptionBuilder()
.hasArg(true)
.numberOfArgs(1)
.optionalArg(true)
.argName("jvm")
.longOpt(JVM_OPTION)
.desc("Specifies whether to run JVM mode and optionally selects a JVM to run with.")
.build();
var inspect =
cliOptionBuilder()
.longOpt(INSPECT_OPTION)
Expand Down Expand Up @@ -451,6 +462,7 @@ private static Options buildOptions() {
options
.addOption(help)
.addOption(repl)
.addOption(jvm)
.addOption(run)
.addOption(inspect)
.addOption(dumpGraphs)
Expand Down Expand Up @@ -966,7 +978,7 @@ private URI parseUri(String string) {
*
* @param args the command line arguments
*/
public static void main(String[] args) throws IOException {
public static void main(String[] args) throws Exception {
new Main().launch(args);
}

Expand Down Expand Up @@ -1294,9 +1306,91 @@ private void println(String msg) {
System.out.println(msg);
}

private void launch(String[] args) {
private void launch(String[] args) throws IOException, InterruptedException, URISyntaxException {
var options = buildOptions();
var line = preprocessArguments(options, args);

if (line.hasOption(JVM_OPTION)) {
var jvm = line.getOptionValue(JVM_OPTION);
var current = System.getProperty("java.home");
if (jvm == null) {
jvm = current;
}
if (current == null || !current.equals(jvm)) {
var loc = Main.class.getProtectionDomain().getCodeSource().getLocation();
var commandAndArgs = new ArrayList<String>();
JVM_FOUND:
if (jvm == null) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The JVM_OPTION takes zero or one argument. If the argument is present, it should be a path to the GraalVM to use to execute Enso process. The code below uses ProcessBuilder to construct such process - more or less the code is a copy of what is in bin/enso and bin/enso.bat files.

When the --jvm option is used without additional argument, then we should somehow locate the GraalVM in the installation. CCing @4e6 and @radeusgd (as author of enso launcher which deals with similar issues) to guide this part of the implementation.

var env = new Environment() {};
var dm = new DistributionManager(env);
var paths = dm.paths();
var files = paths.runtimes().toFile().listFiles();
JaroslavTulach marked this conversation as resolved.
Show resolved Hide resolved
if (files != null) {
for (var d : files) {
var java = new File(new File(d, "bin"), "java").getAbsoluteFile();
if (java.exists()) {
commandAndArgs.add(java.getPath());
break JVM_FOUND;
}
}
}
commandAndArgs.add("java");
} else {
commandAndArgs.add(new File(new File(new File(jvm), "bin"), "java").getAbsolutePath());
}
var jvmOptions = System.getenv("JAVA_OPTS");
if (jvmOptions != null) {
for (var op : jvmOptions.split(" ")) {
if (op.isEmpty()) {
continue;
}
commandAndArgs.add(op);
}
}

commandAndArgs.add("--add-opens=java.base/java.nio=ALL-UNNAMED");
commandAndArgs.add("--module-path");
var component = new File(loc.toURI().resolve("..")).getAbsoluteFile();
if (!component.getName().equals("component")) {
component = new File(component, "component");
}
if (!component.isDirectory()) {
throw new IOException("Cannot find " + component + " directory");
}
commandAndArgs.add(component.getPath());
commandAndArgs.add("-m");
commandAndArgs.add("org.enso.runtime/org.enso.EngineRunnerBootLoader");
var it = line.iterator();
while (it.hasNext()) {
var op = it.next();
if (JVM_OPTION.equals(op.getLongOpt())) {
continue;
}
var longName = op.getLongOpt();
if (longName != null) {
commandAndArgs.add("--" + longName);
} else {
commandAndArgs.add("-" + op.getOpt());
}
var values = op.getValuesList();
if (values != null) {
commandAndArgs.addAll(values);
}
}
commandAndArgs.addAll(line.getArgList());
var pb = new ProcessBuilder();
pb.inheritIO();
pb.command(commandAndArgs);
var p = pb.start();
var exitCode = p.waitFor();
if (exitCode == 0) {
throw exitSuccess();
} else {
throw doExit(exitCode);
}
}
}

launch(options, line);
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,8 @@ object DependencyPreinstaller {

val editionProvider = EditionManager.makeEditionProvider(
distributionManager,
Some(languageHome)
Some(languageHome),
true
)
val editionResolver = EditionResolver(editionProvider)
val edition = editionResolver
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,9 @@ import java.nio.file.Path
import scala.util.Try

/** A helper class for resolving editions. */
class EditionManager private (editionProvider: UpdatingEditionProvider) {
class EditionManager private (
editionProvider: editions.provider.EditionProvider
) {
private val editionResolver = EditionResolver(editionProvider)
private val engineVersionResolver =
editions.EngineVersionResolver(editionProvider)
Expand Down Expand Up @@ -46,18 +48,26 @@ class EditionManager private (editionProvider: UpdatingEditionProvider) {
object EditionManager {

/** Create an [[EditionProvider]] that can locate editions from the
* distribution and the language home.
* distribution (if updating) and the language home.
*/
def makeEditionProvider(
final def makeEditionProvider(
distributionManager: DistributionManager,
languageHome: Option[LanguageHome]
): UpdatingEditionProvider = {
val config = new GlobalConfigurationManager(distributionManager).getConfig
new UpdatingEditionProvider(
getSearchPaths(distributionManager, languageHome),
distributionManager.paths.cachedEditions,
config.editionProviders
)
languageHome: Option[LanguageHome],
updating: Boolean
): editions.provider.EditionProvider = {
val config = new GlobalConfigurationManager(distributionManager).getConfig
val searchPaths = getSearchPaths(distributionManager, languageHome)
val cachePath = distributionManager.paths.cachedEditions
if (updating) {
new UpdatingEditionProvider(
searchPaths,
cachePath,
config.editionProviders
)
} else {
val actualSearchPaths = (searchPaths ++ List(cachePath)).distinct
new editions.provider.FileSystemEditionProvider(actualSearchPaths)
}
}

/** Get search paths associated with the distribution and language home. */
Expand All @@ -77,6 +87,6 @@ object EditionManager {
distributionManager: DistributionManager,
languageHome: Option[LanguageHome] = None
): EditionManager = new EditionManager(
makeEditionProvider(distributionManager, languageHome)
makeEditionProvider(distributionManager, languageHome, false)
JaroslavTulach marked this conversation as resolved.
Show resolved Hide resolved
)
}
Original file line number Diff line number Diff line change
Expand Up @@ -38,12 +38,8 @@ class UpdatingEditionProvider(
case Right(value) => Right(value)
}

/** Finds all editions available on the [[searchPaths]]. */
override def findAvailableEditions(): Seq[String] =
provider.findAvailableEditions()

/** Finds all available editions, performing an update if asked to. */
def findAvailableEditions(update: Boolean): Seq[String] = {
override def findAvailableEditions(update: Boolean): Seq[String] = {
if (update) {
updater.updateEditions()
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,5 @@ trait EditionProvider {
): Either[EditionLoadingError, Editions.Raw.Edition]

/** Finds all editions that are currently available. */
def findAvailableEditions(): Seq[String]
def findAvailableEditions(update: Boolean = false): Seq[String]
}
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,9 @@ class FileSystemEditionProvider(searchPaths: List[Path])
}

/** Finds all editions available on the [[searchPaths]]. */
override def findAvailableEditions(): Seq[String] =
override def findAvailableEditions(
ignoreUpdateRequest: Boolean
): Seq[String] =
searchPaths.flatMap(findEditionsAt).distinct

private def findEditionName(path: Path): Option[String] =
Expand Down
Loading
Loading