Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adding jar offset rewriting and capability to use custom preambles #29

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

mbjarland
Copy link

@mbjarland mbjarland commented Nov 21, 2017

First of all, thank you for this library. I have been looking for a way to do this for ages and this really solves the problem in an elegant and once-and-for-all way.

With that being said, the current state of this library still left me with two items missing from my wish list of perfection:

  • jar file meta data offsets. The current method of just appending text to the front of the jar file leaves the jar file integrity compromised as none of the jar/zip internal offsets are accurate after the prepend. We would like to rewrite the resulting jar file zip meta data offsets so that tools such as unzip would still consider the zip file valid (as in with valid offsets. Try adding a longer preamble script with the current implementation and then either executing it with java or just running unzip -l on the resulting jar).
  • the ability to use custom preamble scripts to be able to optionally support launch accelerators such as drip. Or whatever insanity users of this library come up with.

I'm somewhat new to clojure and solving this problem led me down quite a journey. In (not so) short:

  • look around and realize there is no library on the jvm to read the zip file meta data as specified by the zip file specification. I'm not talking about java's ZipFile and friends which only deal with entries and have no concept of things like central directories or local headers as defined in the zip file specification.
  • be dissappointed and meditate over the un-reality of the fact that the base distribution unit of all java appliactions is jar files and we have nothing that can actually read them properly.
  • pick yourself up and boldly charge into the frey. Zip files. I mean...how hard can it be?
  • figure out which binary format library to use in clojure. The top contenders seemed to be octet and buffy, but there are others. Try them all out and try to implement zip files in them.
  • realize that none of them solve the problem the way I needed to. As none of them solve the problem out of the box, settle for the simplicity of octet. It has a clean and terse specification dsl.
  • understand the zip file format and how the offset calculations work and why adding a prelude without fixing the offsets breaks the zip file.
  • realize that octet was missing support for the custom length field pattern (ref lengths, see the linked octet pull request for details) used in the zip file specification and it was thus not possible to write a spec in octet for the zip file format. Could not get buffy to work either. Count to ten.
  • write a somewhat extensive pull request to octet adding the required support for reference length fields.
  • wait for pull request to be accepted. Thank you @niwinz for accepting it.
  • with the proper tooling for writing a specification for zip files finally in place in octet, write the library missing from the jvm: clj-zip-meta. Yay, we can now read zip/jar file meta data and have it returned as clojure data structures.
  • iterate...publish to clojars.
  • make changes to this library now using the ability to rewrite zip offsets using the newly written lib. Also add custom preamble support.
  • fight with command line parsing issues for an eternity to get optional (as in, use if installed) drip support working.
  • create this pull request

apologies for the rambling. It was some journey and I would like to thank the writers of this library for writing something so good that it was worth the above trouble to improve on.

If you deem this change acceptable, great. If not, this still solves my local problem so again, many thanks for this library.

@mbjarland
Copy link
Author

Oh, and I fully expect that somebody will tell me this problem is already solved somehow. If so, great! I still learned quite a lot going through the process so time well spent. If not, perhaps I have done a small contribution that might save somebody else some time somewhere down the line.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant