issue with problematic zip headers in pekko-protobuf-v3 jar #685
Comments
|
We may need to go to sbt-assembly team to discuss this. pekko-protobuf-v3 jar is generated using sbt-assembly. The issue may be best addressed by getting a change in sbt-assembly to avoid zip64 format. |
|
sbt-assembly uses the JDK zip filesystem to create jars. So, it's the JDK that creates the broken zip archives and it seems the problem has already been fixed in more recent JDKs (though, of course, it might be a problem in older JDKs that might not be maintained to the same degree). |
|
Or, wait the fix was for flagging the problem not for fixing the generation... |
I'm surprised that the JDK changes are not causing more problems. Our CI builds that create the pekko snapshot jars should be using the latest JDKs - as we can see the pekko-http CI builds are using the latest JDKs too and hitting the issue with the new zip64 checks. We could probably temporarily fix the tests by forcing the pekko-http tests to use older versions of the JDKs (from a few months ago) but it seems better to see if we can somehow fix the jar generation. |
|
Since the issue is with the protobuf-v3 jar, I guess it might be related to sharding / jarjarabrams which seems to use ant internally... |
sbt-assembly uses the |
1 similar comment
sbt-assembly uses the |
|
Seems like a good idea to use commons-compress. I've worked on Apache POI and the modern Microsoft file formats are basically zip files. We've had to switch to commons-compress because its zip support is superior to the JDK equivalents. Zip64 issues being one of the main drivers as to why we had to switch. |
|
We have at least 3 components dealing with jar files: assembly, jarjarabrams for sharding and also sbt-reproducible-builds. Can someone try to reproduce the issue? I cannot reproduce it with the jars I build locally. You can use |
|
Ok, indeed, I can reproduce it and the broken artifact is the one in |
I think in this case it's the main jar generation in sbt-assembly but I have to confirm |
|
Btw. the issue in the latest jar is not related to zip64 but a generally broken zip extra field entry. Maybe @raboof has an idea what could be broken about the stripper? |
|
Ok, let's redeem all of the components above and claim it on the arch nemesis. I went down into the abyss and read the zip file handling code of the JDK, commons-compress, and more, one worse than the other and now that? |
Im confused? |
|
On another note, this may also be related to raboof/sbt-reproducible-builds#277 |
Sorry, that was obscure... The problem is caused by osgi. |
No worries, I thought I was the only obscure one |
Affects pekko-http nightly build. So far, only affects pekko-protobuf-v3 jars generated from the 'main' branch as opposed to the '1.0.x' branch. Doesn't happen with Java 8 but happens with Java 11 and 17.
https://github.com/apache/incubator-pekko-http/actions/runs/6243149241/job/16948118095
This is the openjdk issue: https://bugs.openjdk.org/browse/JDK-8313765
See also: https://bugs.openjdk.org/browse/JDK-8315828
The text was updated successfully, but these errors were encountered: