New release needed to fix issues in other dependent projects

[ayappanec]

Projects like spark , kafka etc uses lz4-java artifact from central maven repository and the older releases don't have support for more platforms. So issues are seen in platforms which don't have support. Since the scenario has changed recently with lz4-java supporting more platforms , a new release will help a lot. Thanks.

[odaira]

Once I merge all the necessary PRs and fix important issues, I plan to release a new version, probably sometime in May.

[danielfree]

@odaira It's great to see the restart of development. I think the frame format 1.5 would be an important feature for the next release, similar to #61 did, also can be found at https://github.com/danielfree/lz4-java and a further development is done at https://github.com/nathants/java-lz4

[odaira]

@danielfree Do you have any recommendation for which implementation of the frame support I should take a look at?

[danielfree]

@odaira My fork (credit goes to @gyscos) and https://github.com/nathants/java-lz4 contain only JNI implementation, while #61 is Java implementation, so it depends on which one we need.

There are still some issues in the JNI implementation, e.g. the available() method is not what it should be for an arbitrary length file, as well as the skip() method - I think these methods should represent the whole file instead of one buffer/block.

[odaira]

Thanks, I have reviewed and merged #61, as the platform independence should be the first priority.

[ayappanec]

@odaira Any update on this ?

[odaira]

I think these are my TODOs:

• Merge and write tests for read concated stream #76. It was withdrawn, but I think it is worth merging.
• Run the performance tests myself.
• Check and see if there is any other fix we need to merge from Kafka's implementation of the LZ4 frame format. lz4 streaming format 1.4 #21 4cdeb71#commitcomment-22335353

[maropu]

Hi, I'm interested in #76. If nobody takes on this , could I take over this (e.g., apply review and add tests, ...)?

[maropu]

I checked this patch (master...maropu:ConcatByteStream) fixed the issue described in #76 and passed related tests in spark (e.g., compression and shuffling stuffs).

[odaira]

@maropu that would be great! Are you going to submit a new PR?

[maropu]

@odaira ok, I'll do, thanks!

[maropu]

Could u check #105? Thanks!

[odaira]

I think I am done with my TODOs. Code-wise, we are ready to release the next version.

I can use Java 8's javac to build a jar file for distribution, can't I? Java 7 is no longer publicly updated, and we don't need to consider Java 7 users.

[maropu]

Spark v2.2 already removed java7, so it's okay for that.

[ijuma]

Kafka still supports Java 7, so it would be great if the release was built with Java 7.

[odaira]

Released 1.4.0. Used javac from Java 7.

[ijuma]

Thanks! We are already using it in kafka trunk. :)