Package relocation doesn't relocate references in project source. Results in NoClassDefFound.

[kzarzycki]

Hi,
I'm having issue, with running your shadowJar plugin, more precisely when using "relocate" feature of it.
I'm trying to relocate all google libraries that I use( especially guava classes), to embed them in my jar and be independent of guava that is used in the platform I want to launch my jar in (it's Apache Storm, my jar is a Storm topology).

To achieve this I specify shadowJar task as following:

shadowJar {
     zip64 = true
    exclude 'META-INF/*.RSA', 'META-INF/*.SF', 'META-INF/*.DSA'
    relocate 'com.google', 'mycompany.com.google'
}

I hope I'm right, that that should be it, i.e. all my classes in my jar, that used com.google.* classes, are now "rewritten" to use mycompany.com.google.* classes.
But I have a problem to run the shadow jar:

$ java -cp myjar-1.0-all.jar mycompany.MyMainClass
...
Exception in thread "main" java.lang.NoClassDefFoundError: com/google/inject/PrivateModule
...

When I unpacked the jar, I see that indeed all com.google classes landed in mycompany/com/google directory. There are empty directories in com/google, but I believe this is not the issue.
Do you know what is the issue? I'll be really grateful for your help as it is a major blocker to my project.

Environment:

• Shadow version : 0.9.0-M5

• JVM: 1.8.0_05 (Oracle Corporation 25.5-b02)

• Gradle: 1.12-20140531062749+0000
  to have Shadow working with JVM 1.8, I had to asm to 5.x, I did this by overriding asm dependencies in buildscript:

  classpath 'org.ow2.asm:asm:5.0.3'
  classpath 'org.ow2.asm:asm-commons:5.0.3'

[stephan-gh]

I can confirm this issue with Java 7 as well. The dependency (Guava in this case) is relocated correctly and all references in the dependency will be relocated to their new path.
But the plugin does not relocate the references from the source of the project, they're still pointing to the old package path.
By the way, relocating packages from the source of the project does also not work, so if I have a package notrelocated in my project with a single class in it and want to relocate it to relocated it is still in notrelocated in the JAR.
https://github.com/Minecrell/GradleTest

I have not tested it with Java 8, but maybe you can update ASM to version 5 in the plugin if the older versions do not support Java 8?

[kzarzycki]

Exactly, I was just writing my comment about this:) As I believe shadowJar relocates the references inside the dependencies jars. But it does not relocate the references inside the class files of the project it is defined in (as you named it "source of the project").

So now I need to understand how to relocate references in the source of the project. One workaround would be to generate jar out of it first and then make shadowJar out of all jars including the one from sources.
Does anyone suggest a gradle snippet for doing it inside one project?

Thanks

[stephan-gh]

If you need it now I think you could use the older version of the shadow plugin (0.8). It works a little bit different and the new version is easier and faster to use but the relocation is working properly with it (Java 7, you need to test if you can get it working with Java 8).
Here is the README, if you wonder how to use it: https://github.com/johnrengelman/shadow/tree/0.8

[kzarzycki]

Thanks Minecrell! OK, I'll try version 0.8 then, we will see whether I can get all the features I need to use from it as well (java 8, zip64, relocate). I will inform about the output

[johnrengelman]

Thanks for the report. It's definitely a bug, the relocation code is only kicking in when handling the classes in JAR dependencies.

I'll get this fixed for 1.0.0.

[johnrengelman]

@kzarzycki v0.8 won't support zip64, that was one of the driving factors for the re-write in 0.9.0.
I've already got a test in place and a fix for the relocation issue. There is one more bug I'm going to try and fix and then I'll be releasing a v1.0.0 (hopefullly this afternoon).

[kzarzycki]

Thanks John, Minecrell!

John, that's very good that you got a fix already, I'll wait for the new version of your awesome plugin:)

Unfortunately, it seems there will be another issue that is now related to JDK8, lambdas & asm 5.x. As I believe it's more an issue of asm than shadow, I have just submitted a bug on OW2 asm issue tracker:
http://forge.ow2.org/tracker/?func=detail&aid=317531&group_id=23&atid=100023

But it also might be an issue of how you use asm in shadow, so I wanted to submit an issue for you too.
Could you please look into that too? I've created a sample project for showing this issue here:
https://github.com/kzarzycki/asm-lambda-deserialization-error-test

It should help you jump right to the problem. I'll be gratefull if you could look into that.

[johnrengelman]

I don't personally use the relocation feature and I'm not using Java 8 on any projects yet, so this will likely be far down my list. Contributions are always welcome. Can you please open a new ticket with the above information?

[stephan-gh]

Working for me in version 1.0.0. Thanks!