Devmode, Maven: Broken classpath when a local module depends via its parent on another local module

[famod]

Describe the bug
Imagine the following Maven module structure:

• level0
• level1 depends on level0
• level2 depends on level1
  • level2.submodule (depends on level1 via parent level2)
• runner depends on level2 and level0

Each level* module contains a simple @ApplicationScoped service that calls the service from the "next-lower" level (e.g. level1 -> level0).
runner contains a REST resource with two methods, one calling the level2 service and one directly calling the level0 service.

Both methods are working as expected with runner-jar, but in devmode the call to level2 fails in ArC with a NoClassDefFoundError for the level0 service which is very confusing because the direct call to level0 does work!
This is with the current master that includes #10421. With 1.6.0.Final (or any earlier version) which does not yet contain that change, you'll see a ClassCastException instead.

Expected behavior
No error, should work in devmode the way it works with runner-jar.

Actual behavior

• master: java.lang.NoClassDefFoundError: org/acme/level0/Level0Service
• 1.6.0.Final or earlier:

  java.lang.ClassCastException: class org.acme.level0.Level0Service_ClientProxy cannot be cast to class org.acme.level0.Level0Service (org.acme.level0.Level0Service_ClientProxy is in unnamed module of loader io.quarkus.bootstrap.classloading.QuarkusClassLoader @439b15f2; org.acme.level0.Level0Service is in unnamed module of loader 'app')
  

To Reproduce
Steps to reproduce the behavior:

1. See Reproducer-PR: DevMojoIT test for #10484 (Devmode broken classpath with parent deps) #10485

Configuration
n/a

Screenshots
n/a

Environment (please complete the following information):

• Output of uname -a or ver:

  MINGW64_NT-10.0-18363 BIGBLUE 3.0.7-338.x86_64 2019-11-21 23:07 UTC x86_64 Msys
  

• Output of java -version:

  openjdk version "11.0.7" 2020-04-14
  OpenJDK Runtime Environment AdoptOpenJDK (build 11.0.7+10)
  OpenJDK 64-Bit Server VM AdoptOpenJDK (build 11.0.7+10, mixed mode)
  

• GraalVM version (if different from Java): n/a
• Quarkus version or git rev: master (NCDFE) or 1.6.0.Final or earlier (CCE)
• Build tool (ie. output of mvnw --version or gradlew --version): Apache Maven 3.6.3

Additional context
I ran into this issue with Apache CXF: https://quarkusio.zulipchat.com/#narrow/stream/187030-users/topic/ClassCastException.20in.20dev.20mode.20but.20not.20with.20runner-jar
But in the end this has nothing to do with CXF specifically.

I'm 99% sure that I already know the root cause of this bug:

In DevMojo, level1 is not added to the DevModeContext and ends up in the classPathManifest which in turn does (of course) not contain any other local projects/modules (which are part of the DevModeContext).
Because of this, level1 cannot see level0 in its classloader (or sees it in a diferent classloader before #10421).

level1 is not added to the DevModeContext because LocalProject.collectSelfWithLocalDeps(LocalProject, Set<AppArtifactKey>, List<LocalProject>) is working with getRawModel().getDependencies(), but that will only include the direct dependencies of a module, not the ones it inherits from its parent(s)! (it is just a "raw" model after all!)

So it seems that LocalProject will have to check the parent models as well, but that would mean custom merging and leaves the question whether this should really be done "by hand" or whether this should be offloaded to some OOTB Maven interpolation instead.
There is also a direct usage of getRawModel().getDependencies() in DevMojo.filterExtensionDependencies().

There are also some more methods in LocalProject that are not considering any parent(s), see usages of rawModel.getBuild() .

PS: From a design perspective, IMHO, rawModel should not be exposed like this by LocalProject.
I'd also strongly suggest to introduce a model cache to avoid costly re-parsing of already parsed parents.

[quarkusbot]

/cc @quarkusio/devtools

[famod]

cc @aloubyansky & @geoand

I would have taken a stab this already but it seems some things have to be decided first and I didn't want to jump in the wrong direction. I left all my thoughts under "Additional context" and since I don't think I will have (much) time to work on this in the next few days, you are more than welcome to take the lead.

[geoand]

Thanks a lot for the analysis @famod!

@aloubyansky is the best person to comment on this

[geoand]

@aloubyansky if you want me to check out something, feel free to ask :)