Annotation processors are not re-run in dev mode

[gunnarmorling]

Testing Quarkus with an annotation processor that generates source code, the MapStruct team noticed that the AP isn't re-run after code changes in dev mode.

[dmlloyd]

I wonder if there is some way to just capture the arguments that were passed to javac originally.

[geoand]

Is there a way perhaps for Maven to write those arguments to a file from which we could read at recompile time?

[gsmet]

You don't have the capability to get the whole Maven configuration in a particular mojo?

[geoand]

I am not a Maven internals expert, but that does sounds like it could work.

But would looking at the configuration be enough to know figure out the compiler arguments?

[chris922]

I got the dev-mode (partially) working with our MapStruct annotation processor (see mapstruct/mapstruct-examples#59 / mapstruct/mapstruct-examples@f05e97d):

Previously I defined the annotation processor using the maven-compile-plugin:

      <plugin>
        <artifactId>maven-compiler-plugin</artifactId>
        <version>3.8.0</version>
        <configuration>
          <source>1.8</source>
          <target>1.8</target>
          <annotationProcessorPaths>
            <path>
              <groupId>org.mapstruct</groupId>
              <artifactId>mapstruct-processor</artifactId>
              <version>${org.mapstruct.version}</version>
            </path>
          </annotationProcessorPaths>
        </configuration>
      </plugin>

This will not work together with the dev-mode. I assume the annotation processor path defined in the plugin will not be picked up automatically.

The solution was quite simple, just add our processor as a regular provided scoped dependency (thanks @filiphr for the idea). The processor itself contains a META-INF/services/javax.annotation.processing.Processor file so that the Java compiler will pick it up automatically.

    <dependency>
      <groupId>org.mapstruct</groupId>
      <artifactId>mapstruct-processor</artifactId>
      <version>${org.mapstruct.version}</version>
      <scope>provided</scope>
    </dependency>

One thing is still not working, and I guess it's quite hard to get this working:

The annotation processor will pick up files annotated with @Mapper, in case you make a change to this file everything is file. Nevertheless this class depends on other classes (in this example PersonDto and Person). When I make a change to one of these files the @Mapper annotated class will obviously not be regenerated and thus our annotation processor will not run.
Work-around: Make a temporary change to the @Mapper class so that it will be picked up again.

As already mentioned.. I think it will be quite hard to find a solution that will recompile the @Mapper file when a dependent class will be changed. Doing this in general could lead to recompiling a lot of files as soon as you changed a file.. maybe it's possible to just recompile the linked files that have an annotation processor!?

[gsmet]

On Sat, Mar 16, 2019 at 1:44 PM Christian Bandowski < ***@***.***> wrote: As already mentioned.. I think it will be quite hard to find a solution that will recompile the @Mapper file when a dependent class will be changed. Doing this in general could lead to recompiling a lot of files as soon as you changed a file.. maybe it's possible to just recompile the linked files that have an annotation processor!?

It's Saturday so crazy ideas are allowed. I wonder if we could have something like a HotReloadClassGroup and you would trigger the compilation of the whole group every time you change a class of the group. It wouldn't be automatic but it would be done on a declarative basis. For MapStruct, I think you could build the group from the annotations + checking the attributes/getters/... of the annotated class (it would be very similar to what we do in ReflectiveHierarchyStep) at augmentation time (so in the processor of an extension). Then you would produce a BuildItem and work from there. Obviously working on a group would be very naive, having a proper dependency tree would be more accurate but I think it would be good enough for a first step. I think it would be worth a try if you want to play with it. I could see how it could be helpful for some extensions.

…

-- Guillaume

[gunnarmorling]

So I have no clear understanding of how the Quarkus' dev mode environment and the compiler interact, but it might be worth checking out what Gradle is doing in regards to "incremental annotation processing". In particular, they are using the originatingElement information passed to the Filer methods such as createSourceFile() to deduct which annotation processors need to be executed again after given classes have changed.

[geoand]

Very interesting @gunnarmorling, definitely something worth checking.

For the time being, the devmode / compiler interaction is quite simple, see:

quarkus/core/devmode/src/main/java/io/quarkus/dev/JavaCompilationProvider.java

Line 31 in 6cf20b0

public void compile(Set<File> filesToCompile, Context context) {

[famod]

I just stumbled upon this. In my case it is hibernate-jpamodelgen.
Adding a provided dependency is less than ideal because we have many submodules (yes, we could define it to the root parent...) and hibernate-jpamodelgen has JaxB dependencies that I don't want to have on the compile or test (runtime) classpath.

PS: Noticed the problem because a message is printed on reload:

The following options were not recognized by any processor: '[fullyAnnotationConfigured]', line -1 in [unknown source]

cc @harthorst @tkalmar

[famod]

I kind of lost track of what was changed for dev mode in the last months but I suppose this is still a problem? /cc @stuartwdouglas

In virtually all of the recent projects I'm seeing lately in my company we have the following annotation processors:

• mapstruct
• jpa-modelgen
• immutables
• and possibly lombok

Not having them (fully) supported in dev mode is a real issue because it is very likely you will touch something in dev mode that is subject to annotation processing.

Leaving that "class group" challenge aside for a moment, wouldn't it be a good first improvement to "just" read the processors from maven config and run them for every change?
Not sure I'm making sense at all, but I'd love to see some progress on this.

[famod]

I almost submitted a "yay all is well" comment but I had second thoughts:

In a Quarkus 1.12.2 project we are setting up currently, all of the processors mentioned above do work, but only because Eclipse is triggering the processors!
This might also explain why we are seeing so many full reloads/restarts instead of smaller (agent based?) code swappings... (/cc @stuartwdouglas, does that make sense?)

Strangely, the provided dependency workaround doesn't work at all for us. 🤔 We might have a second look later.

[ghost]

My setup/situation is similar to what @famod mentions above.

Annotation processors:

• MapStruct
• jpa-modelgen
• Lombok

Instead of Eclipse I use IntelliJ IDEA. And in such a project with these annotation processors it seems to me that Quarkus has to do a full restart every time.

In another project, I use Kotlin without Lombok, MapStruct and jpa-modelgen. And there a lot of the changes can be applied with a hot replacement/code swapping (without full restart).

[FroMage]

I have a fix for that pending in #36168 for Maven, if anybody wants to help with Gradle, help will be welcome ;)

[geoand]

@snazy would you be interested in ^?

[snazy]

Still have the plan to use Gradle's continuous build with Quarkus' dev-mode, but no so much progress yet (beside some very early experiments).