Generate native image for engine-runner (#3638)

This PR adds a possibility to generate native-image for engine-runner.
Note that due to on-demand loading of stdlib, programs that make use of it are currently not yet supported
(that will be resolved at a later point).
The purpose of this PR is only to make sure that we can generate a bare minimum runner because due to lack TruffleBoundaries or misconfiguration in reflection config, this can get broken very easily.
To generate a native image simply execute:
```
sbt> engine-runner-native/buildNativeImage
... (wait a few minutes)
```
The executable is called `runner` and can be tested via a simple test that is in the resources. To illustrate the benefits
see the timings difference between the non-native and native one:
```
>time built-distribution/enso-engine-0.0.0-dev-linux-amd64/enso-0.0.0-dev/bin/enso --no-ir-caches --in-project test/Tests/ --run engine/runner-native/src/test/resources/Factorial.enso 6
720

real	0m4.503s
user	0m9.248s
sys	0m1.494s
> time ./runner --run engine/runner-native/src/test/resources/Factorial.enso 6
720

real	0m0.176s
user	0m0.042s
sys	0m0.038s
```

# Important Notes
Notice that due to a [bug in GraalVM](oracle/graal#4200), which is already fixed in 22.x, and us still being on 21.x for the time being, I had to add a workaround to our sbt build to build a different fat jar for native image. To workaround it I had to exclude sqlite jar. Hence native image task is on `engine-runner-native` and not on `engine-runner`.

Will need to add the above command to CI.

.gitignore

@@ -118,6 +118,7 @@ build-cache/
 ##################
 
 *.build_artifacts.txt
+/runner
 
 ######################
 ## Enso-Development ##

CHANGELOG.md

@@ -355,6 +355,7 @@
 - [Update Scala compiler and libraries][3631]
 - [Support importing module methods][3633]
 - [Support Autosave for open buffers][3637]
+- [Generate native-image for engine-runner][3638]
 - [Support pattern matching on constants][3641]
 - [Builtin Date_Time, Time_Of_Day and Zone types for better polyglot
   support][3658]
@@ -404,6 +405,7 @@
 [3631]: https://github.com/enso-org/enso/pull/3631
 [3633]: https://github.com/enso-org/enso/pull/3633
 [3637]: https://github.com/enso-org/enso/pull/3637
+[3637]: https://github.com/enso-org/enso/pull/3638
 [3641]: https://github.com/enso-org/enso/pull/3641
 [3658]: https://github.com/enso-org/enso/pull/3658
 [3671]: https://github.com/enso-org/enso/pull/3671

build.sbt

@@ -1534,6 +1534,59 @@ lazy val `engine-runner` = project
   .dependsOn(`polyglot-api`)
   .dependsOn(`logging-service`)
 
+// A workaround for https://github.com/oracle/graal/issues/4200 until we upgrade to GraalVM 22.x.
+// sqlite-jdbc jar is problematic and had to be exluded for the purposes of building a native
+// image of the runner.
+lazy val `engine-runner-native` = project
+  .in(file("engine/runner-native"))
+  .settings(
+    assembly/assemblyExcludedJars := {
+      val cp = (assembly / fullClasspath).value
+      (assembly/assemblyExcludedJars).value ++
+          cp.filter(_.data.getName.startsWith("sqlite-jdbc"))
+    },
+    assembly / mainClass := (`engine-runner` / assembly / mainClass).value,
+    assembly / assemblyMergeStrategy := (`engine-runner` / assembly / assemblyMergeStrategy).value,
+    assembly / assemblyJarName := "runner-native.jar",
+    assembly / assemblyOutputPath := file("runner-native.jar"),
+    assembly := assembly
+        .dependsOn(`engine-runner` / assembly)
+        .value,
+    rebuildNativeImage := NativeImage
+      .buildNativeImage(
+        "runner",
+        staticOnLinux = true,
+        additionalOptions = Seq(
+          "-Dorg.apache.commons.logging.Log=org.apache.commons.logging.impl.NoOpLog",
+          "-H:IncludeResources=.*Main.enso$",
+          "--allow-incomplete-classpath",
+          "--macro:truffle",
+          "--language:js",
+          //          "-g",
+          //          "-H:+DashboardAll",
+          //          "-H:DashboardDump=runner.bgv"
+          "-Dnic=nic"
+        ),
+        mainClass = Option("org.enso.runner.Main"),
+        cp = Option("runtime.jar"),
+        initializeAtRuntime = Seq(
+          // Note [WSLoggerManager Shutdown Hook]
+          "org.enso.loggingservice.WSLoggerManager$",
+          "io.methvin.watchservice.jna.CarbonAPI"
+        )
+      )
+      .dependsOn(assembly)
+      .dependsOn(VerifyReflectionSetup.run)
+      .value,
+    buildNativeImage := NativeImage
+      .incrementalNativeImageBuild(
+        rebuildNativeImage,
+        "runner"
+      )
+      .value
+  )
+  .dependsOn(`engine-runner`)
+
 lazy val launcher = project
   .in(file("engine/launcher"))
   .configs(Test)

docs/infrastructure/native-image.md

@@ -190,3 +190,24 @@ Manager seems to find recursive accesses of some ephemeral-like classes named
 Image and they lead to warnings. For now no clues have been found that ignoring
 these classes would impact the native build, it seems that they can be ignored
 safely.
+
+### Engine runner Configuration
+
+The Native Image generation for the Engine Runner is currently in a preview
+state. Limitations are currently mostly due to
+[Java interop](https://www.pivotaltracker.com/story/show/183260380) and loading
+of stdlib components. To generate the Native Image for runner simply execute
+
+```
+sbt> engine-runner-native/buildNativeImage
+```
+
+and execute the binary on a sample factorial test program
+
+```
+> runner --run engine/runner-native/src/test/resources/Factorial.enso 6
+```
+
+The task that generates the Native Image, along with all the necessary
+configuration, reside in a separate project due to a bug in the currently used
+GraalVM version.

engine/runner-native/src/main/resources/META-INF/native-image/org/enso/runner/jni-config.json

@@ -0,0 +1,15 @@
+[
+  {
+    "name": "java.lang.ClassLoader",
+    "methods": [
+      { "name": "getPlatformClassLoader", "parameterTypes": [] },
+      { "name": "loadClass", "parameterTypes": ["java.lang.String"] }
+    ]
+  },
+  {
+    "name": "java.lang.ClassNotFoundException"
+  },
+  {
+    "name": "java.lang.NoSuchMethodError"
+  }
+]

engine/runner-native/src/main/resources/META-INF/native-image/org/enso/runner/predefined-classes-config.json

@@ -0,0 +1,8 @@
+[
+  {
+    "type":"agent-extracted",
+    "classes":[
+    ]
+  }
+]
+

engine/runner-native/src/main/resources/META-INF/native-image/org/enso/runner/proxy-config.json

@@ -0,0 +1 @@
+[]