How to compress file larger than 2GB?

[darouwan]

I am trying to compress file over 2GB size. As we know, both byte array and inputstream have 2^31-1 length limit, which is 2GB. So compressing file over 2GB size in one time seems impossible technically.

I ever tried to compress file in multi parts, where read one big file into some 2G length buffer and compress them respectively. However it is a little difficult to manage them.

So any better idea to achieve my purpose?

[odaira]

Although I have never tried it myself, how about using https://github.com/xerial/larray? I think you can get a direct ByteBuffer from LArray (or LBuffer?) and can give it to lz4-java.

[odaira]

Would you still need help for this?

[darouwan]

No more need for help, thanks!