New release needed to fix issues in other dependent projects

Question

New release needed to fix issues in other dependent projects

Closed this issue 7 years ago · 17 comments

Projects like spark , kafka etc uses lz4-java artifact from central maven repository and the older releases don't have support for more platforms. So issues are seen in platforms which don't have support. Since the scenario has changed recently with lz4-java supporting more platforms , a new release will help a lot. Thanks.

Answer 1 · 2017-04-25T16:12:37.000Z

Once I merge all the necessary PRs and fix important issues, I plan to release a new version, probably sometime in May.

Answer 2 · 2017-04-26T01:52:03.000Z

@odaira It's great to see the restart of development. I think the frame format 1.5 would be an important feature for the next release, similar to #61 did, also can be found at https://github.com/danielfree/lz4-java and a further development is done at https://github.com/nathants/java-lz4

Answer 3 · 2017-05-04T21:21:36.000Z

@danielfree Do you have any recommendation for which implementation of the frame support I should take a look at?

Answer 4 · 2017-05-05T02:24:40.000Z

@odaira My fork (credit goes to @gyscos) and https://github.com/nathants/java-lz4 contain only JNI implementation, while #61 is Java implementation, so it depends on which one we need.

There are still some issues in the JNI implementation, e.g. the available() method is not what it should be for an arbitrary length file, as well as the skip() method - I think these methods should represent the whole file instead of one buffer/block.

Answer 5 · 2017-05-22T17:41:07.000Z

Thanks, I have reviewed and merged #61, as the platform independence should be the first priority.

Answer 6 · 2017-06-09T06:48:35.000Z

@odaira Any update on this ?

Answer 7 · 2017-06-12T18:41:28.000Z

I think these are my TODOs:

Merge and write tests for #76. It was withdrawn, but I think it is worth merging.
Run the performance tests myself.
Check and see if there is any other fix we need to merge from Kafka's implementation of the LZ4 frame format. #21 4cdeb71#commitcomment-22335353

Answer 8 · 2017-06-23T13:25:40.000Z

Hi, I'm interested in #76. If nobody takes on this , could I take over this (e.g., apply review and add tests, ...)?

Answer 9 · 2017-06-25T07:14:34.000Z

I checked this patch (master...maropu:ConcatByteStream) fixed the issue described in #76 and passed related tests in spark (e.g., compression and shuffling stuffs).

Answer 10 · 2017-06-26T20:49:37.000Z

@maropu that would be great! Are you going to submit a new PR?

Answer 11 · 2017-06-26T23:16:24.000Z

@odaira ok, I'll do, thanks!

Answer 12 · 2017-06-26T23:22:21.000Z

Could u check #105? Thanks!

Answer 13 · 2017-07-13T22:59:19.000Z

I think I am done with my TODOs. Code-wise, we are ready to release the next version.

I can use Java 8's javac to build a jar file for distribution, can't I? Java 7 is no longer publicly updated, and we don't need to consider Java 7 users.

Answer 14 · 2017-07-14T06:40:05.000Z

Spark v2.2 already removed java7, so it's okay for that.

Answer 15 · 2017-07-22T08:18:01.000Z

Kafka still supports Java 7, so it would be great if the release was built with Java 7.

Answer 16 · 2017-08-22T22:43:20.000Z

Released 1.4.0. Used javac from Java 7.

Answer 17 · 2017-08-22T23:16:35.000Z

Thanks! We are already using it in kafka trunk. :)