default cli-app project doesn't work with GraalVM because of PicoCLI

[picaron]

I tried building a native image for a blank cli-app project generated with the mn command and it doesn't work at runtime because of the way PicoCLI works

I was using micronaut 1.0.0.RC2 and graalvm 1.0.0-rc7-graal (both installed with SDKMan)

I had to switch to the Clikt library (for Kotlin) and then I was able to generate a native-image for my cli-app

[graemerocher]

Picoli includes its own Graal integration, a tutorial is here https://picocli.info/picocli-on-graalvm.html

@remkop anything we can do to make Micronaut + Graal work more out of the box? Like modifications to the profile etc.

[remkop]

@graemerocher Some early thoughts:

• the picocli.codegen.aot.graalvm.ReflectionConfigGenerator tool in the picocli-codegen module works, but I still need to release it to Maven Central. I was planning to do this later as part of picocli 3.7 but that's still some time away so perhaps I should release just the codegen module for the existing picocli-3.6.1. (This is not a problem, just mentioning it for completeness.)

• the picocli.codegen.aot.graalvm.ReflectionConfigGenerator tool itself uses reflection to find the annotated fields and methods for which to generate entries in the GraalVM configuration file. So the application's @picocli.CommandLine.Command-annotated class needs to be compiled and in the classpath when you run the ReflectionConfigGenerator tool.

• You mention modifications to the profile, so I'm guessing you are looking for a new command, perhaps something like generate-graalvm-config, that would be available for applications generated in Micronaut CLI with the cli profile. This command would generate a reflect.json file for a specified @picocli.CommandLine.Command-annotated class.

Question: will the @picocli.CommandLine.Command-annotated class be compiled and on the classpath when running Micronaut CLI?

I guess there should be a separate picocli-codegen feature to bring in the picocli-codegen dependency; also, if the cli profile supports a generate-graalvm-config command, then the cli profile should automatically add both the picocli and the picocli-codegen features, or this command won't work...

[remkop]

FYI I'm working on an annotation processor for picocli that would allow things like generating the reflect.json file (or completion scripts etc) at compile time but that is still early stages.

[graemerocher]

@remkop I think it may make sense to use Micronaut's visitor API to achieve this without needing reflection and the classes themselves on the class path (just the source code needed).

Also it would work for both Groovy and Java (instead of limiting yourself to just Java and the annotation processing API). See for example https://github.com/micronaut-projects/micronaut-core/blob/master/configurations/openapi/src/main/java/io/micronaut/openapi/visitor/OpenApiApplicationVisitor.java

[remkop]

Ok, that is certainly interesting!

Either way, both the Micronaut visitor API and standard Java annotation processors will need parts of the picocli internal model to become part of the picocli public API. Figuring out these API changes is part of the work I'm doing now in the picocli annotation processor. It's important to get this right so I want to take my time with it.

As an interim solution, I could release picocli-codegen-3.6.1 to Maven Central, and the GraalVM section of the Micronaut documentation could be updated to explain how to generate a reflect-cli.json config file for picocli apps.

[remkop]

My previous comment is wrong/incomplete: It is also possible to re-implement the functionality that picocli.codegen.aot.graalvm.ReflectionConfigGenerator provides with just the Micronaut visitor API.

I don't have time to do this now but I'd be happy to answer questions if anyone wants to contribute such a tool to Micronaut.

[remkop]

Update: picocli-codegen 3.7.0 is now available from Maven Central.

The picocli-codegen README now has usage instructions with code snippets for Gradle and Maven to generate a cli-reflect.json file automatically as part of the build.

@graemerocher I believe this goes a long way towards what you are looking for in terms of tooling to make Micronaut + Graal work more out of the box. What is left is documentation:

• point to the picocli-codegen README. See PR Update graalFAQ.adoc #760
• note that the native-image -H:ReflectionConfigurationFiles option takes multiple comma-separated files, so users can specify both the config file generated manually with GraalClassLoadingAnalyzer and the one generated during the build by picocli-codegen. See PR Update graalServices.adoc #761

[remkop]

@picaron If you are using this and if you have any feedback on how to improve this further, please give a shout.

[graemerocher]

Thanks @remkop , looks good

[remkop]

@graemerocher Please let me know if #760 or #761 need tweaking before they can be merged.

[graemerocher]

@remkop looks good, thanks

[graemerocher]

@remkop actually if you can rebsumit the PRs to the 1.0.x branch that would help

[remkop]

I did it from the User Guide web page "Improve this doc", not from an IDE. Not sure what branch that targets. Is it possible to cherry-pick the commit onto other branches after accepting the PR? I don't have my environment set up...

[graemerocher]

@remkop ok no worries, will do manual merge

[remkop]

@graemerocher I saw the PRs got merged, thanks!

However, I don't see the changes yet in the online user manual (latest version nor the snapshot version), is that expected?

[graemerocher]

I believe this was resolved, if any changes let us know

[t-buss]

Hi,
not sure if this is really resolved, as I am running in some problems that seem to be related:

>>> $ sdk current java

Using java version 19.0.0-grl
>>> $ sdk current micronaut

Using micronaut version 1.1.1
>>> $ mn create-cli-app example.git-star --features=graal-native-image,http-client
| Generating Java project...
| Application created at /home/tbuss/Repos/transformer/git-star
>>> $ cd git-star/
>>> git-star/ $ ls
build.gradle      Dockerfile  gradlew*      micronaut-cli.yml  src/
docker-build.sh*  gradle/     gradlew.bat*  settings.gradle
>>> git-star/ $ vim Dockerfile
>>> git-star/ $ ./gradlew assemble

BUILD SUCCESSFUL in 8s
10 actionable tasks: 10 executed
>>> git-star/ $ ./docker-build.sh
Sending build context to Docker daemon  54.97MB
Step 1/8 : FROM oracle/graalvm-ce:1.0.0-rc15 as graalvm
 ---> e319c61ef084
Step 2/8 : COPY . /home/app/git-star
 ---> 59549105f0d3
Step 3/8 : WORKDIR /home/app/git-star
 ---> Running in c93a13623cb6
Removing intermediate container c93a13623cb6
 ---> bbe702a2b1d8
Step 4/8 : RUN native-image --no-server -cp build/libs/git-star-*.jar example.GitStarCommand
 ---> Running in d57190f045f8
[git-star:8]    classlist:   5,227.33 ms
[git-star:8]        (cap):   1,276.30 ms
[git-star:8]        setup:   2,867.72 ms
[git-star:8]   (typeflow):  38,611.84 ms
[git-star:8]    (objects):  56,966.59 ms
[git-star:8]   (features):   2,149.31 ms
[git-star:8]     analysis: 100,434.50 ms
[git-star:8]     universe:   1,967.74 ms
[git-star:8]      (parse):   6,614.95 ms
[git-star:8]     (inline):  18,425.55 ms
[git-star:8]    (compile):  57,505.04 ms
[git-star:8]      compile:  86,515.65 ms
[git-star:8]        image:   8,141.26 ms
[git-star:8]        write:   1,379.17 ms
[git-star:8]      [total]: 207,047.26 ms
Removing intermediate container d57190f045f8
 ---> 8f6ab5e72833
Step 5/8 : FROM frolvlad/alpine-glibc
 ---> dbb18830337d
Step 6/8 : EXPOSE 8080
 ---> Using cache
 ---> 1047f8548cd8
Step 7/8 : COPY --from=graalvm /home/app/git-star .
 ---> ae59fec935c0
Step 8/8 : ENTRYPOINT ["./git-star", "-v"]
 ---> Running in 99d89b148fb8
Removing intermediate container 99d89b148fb8
 ---> ee2ff9fb1b73
Successfully built ee2ff9fb1b73
Successfully tagged git-star:latest


To run the docker container execute:
    $ docker run -p 8080:8080 git-star
>>> git-star/ $ docker run git-star
Exception in thread "main" picocli.CommandLine$InitializationException: Could not instantiate class example.GitStarCommand: io.micronaut.context.exceptions.NoSuchBeanException: No bean of type [example.GitStarCommand] exists. Ensure the class is declared a bean and if you are using Java or Kotlin make sure you have enabled annotation processing.
	at picocli.CommandLine$DefaultFactory.create(CommandLine.java:2848)
	at picocli.CommandLine$Model$CommandReflection.extractCommandSpec(CommandLine.java:4835)
	at picocli.CommandLine$Model$CommandSpec.forAnnotatedObject(CommandLine.java:3142)
	at picocli.CommandLine.<init>(CommandLine.java:147)
	at picocli.CommandLine.run(CommandLine.java:1792)
	at picocli.CommandLine.run(CommandLine.java:1716)
	at io.micronaut.configuration.picocli.PicocliRunner.run(PicocliRunner.java:139)
	at io.micronaut.configuration.picocli.PicocliRunner.run(PicocliRunner.java:117)
	at example.GitStarCommand.main(GitStarCommand.java:19)
Caused by: io.micronaut.context.exceptions.NoSuchBeanException: No bean of type [example.GitStarCommand] exists. Ensure the class is declared a bean and if you are using Java or Kotlin make sure you have enabled annotation processing.
	at io.micronaut.configuration.picocli.MicronautFactory.lambda$create$0(MicronautFactory.java:71)
	at java.util.Optional.orElseThrow(Optional.java:290)
	at io.micronaut.configuration.picocli.MicronautFactory.create(MicronautFactory.java:71)
	at picocli.CommandLine$DefaultFactory.create(CommandLine.java:2847)
	... 8 more

This seems to be an error with the reflective access of PicoCLI.
I also tried what the PicoCLI README suggested and added dependencies and Gradle task:

dependencies {
    compile 'info.picocli:picocli:4.0.0-alpha-3'
    generateConfig 'info.picocli:picocli-codegen:4.0.0-alpha-3'
}

task(generateGraalReflectionConfig, dependsOn: 'classes', type: JavaExec) {
    main = 'picocli.codegen.aot.graalvm.ReflectionConfigGenerator'
    classpath = configurations.generateConfig + sourceSets.main.runtimeClasspath
    def outputFile = "${buildDir}/resources/main/META-INF/native-image/${project.group}/${project.name}/reflect-config.json"
    args = ["--output=$outputFile", 'example.GitStarCommand']
}
assemble.dependsOn generateGraalReflectionConfig

which resulted in this error message:

> Task :generateGraalReflectionConfig FAILED
Exception in thread "main" java.lang.NoSuchMethodError: picocli.CommandLine.execute([Ljava/lang/String;)I
        at picocli.codegen.aot.graalvm.ReflectionConfigGenerator.main(ReflectionConfigGenerator.java:92)

Help would be appreciated, and I would also volunteer to update the docs accordingly.
Thanks in advance!

[remkop]

@t-buss The CommandLine.execute method was added to picocli in 4.0.0-alpha-3. The fact that you're seeing a java.lang.NoSuchMethodError suggests that there is also an older version of picocli in the classpath preceding the picocli-4.0.0-alpha-3.jar. Can you verify if this is the case, perhaps by printing the classpath?

(Something like this:)

task(generateGraalReflectionConfig, dependsOn: 'classes', type: JavaExec) {
    main = 'picocli.codegen.aot.graalvm.ReflectionConfigGenerator'
    classpath = configurations.generateConfig + sourceSets.main.runtimeClasspath
    logger.warn("classpath = " + classpath)
    ...

[remkop]

@t-buss One way to fix the problem is to add the 'info.picocli:picocli:4.0.0-alpha-3'
dependency to the generateConfig classpath.

Alternatively use the same version of picocli and picocli-codegen that is on the runtime classpath to generate the reflect-config.

[remkop]

@t-buss Did this solve the problem?

[t-buss]

@remkop We managed to resolve it and adding the dependency to the generateConfig classpath was the first step. Thank you!
However, the mechanism to automatically load reflection config files seems to be broken, we had to manually add the file when calling native-image in the Dockerfile:

RUN native-image --no-server -cp build/libs/git-star-*.jar -H:ReflectionConfigurationFiles=build/resources/main/META-INF/native-image/reflect-config.json example.GitStarCommand

[remkop]

@t-buss Thanks for the update. I will take a look.

[remkop]

@t-buss Picocli 4.0.0-beta-1b has an annotation processor that includes the refect.config in your jar, so -H:ReflectionConfigurationFiles is no longer needed. Hopefully this resolves the issue you mentioned.