Stream ended prematurely when trying to readAllBytes
yash29896 opened this issue · 0 comments
yash29896 commented
So I'm trying to switch our compressor to lz4-java from snappy and I was trying something like this for a POC
When I try a readAllBytes() on the LZ4FrameInputStream it throw the below exception an breaks.
ERROR
Exception in thread "main" java.io.IOException: Stream ended prematurely
at net.jpountz.lz4.LZ4FrameInputStream.readInt(LZ4FrameInputStream.java:245)
at net.jpountz.lz4.LZ4FrameInputStream.readBlock(LZ4FrameInputStream.java:259)
at net.jpountz.lz4.LZ4FrameInputStream.read(LZ4FrameInputStream.java:353)
at java.base/java.io.InputStream.readNBytes(InputStream.java:396)
at java.base/java.io.InputStream.readAllBytes(InputStream.java:333)
at com.yash.lz4.App.main(App.java:22)
Calling Function
public static void main( String[] args ) throws IOException
{
ByteHandler handler = new LZ4Compressor();
byte[] b = {4, 34, 77, 24, 96, 112, 115, 6, 0, 0, -128, 3, 0, 1, 0, 0, 0}; // Test data from one of our unit tests
InputStream inputStream = handler.deserialize(new ByteArrayInputStream(b));
byte[] i = inputStream.readAllBytes();
}
I've done some debugging and it seems like its not marking the frame as finished.
private void readBlock() throws IOException {
// Reads Ok up to this point
// Reads the last 6 bytes 3, 0, 1, 0, 0, 0
int blockSize = readInt(in);
final boolean compressed = (blockSize & LZ4FrameOutputStream.LZ4_FRAME_INCOMPRESSIBLE_MASK) == 0;
blockSize &= ~LZ4FrameOutputStream.LZ4_FRAME_INCOMPRESSIBLE_MASK; // blocksize = 6
// Check for EndMark
if (blockSize == 0) { // Since this is 6 its doesnt enter the loop
......
frameInfo.finish();
return;
}
Hence the next time it tries to read the stream which has already been completed it leaks into readBlock(); and breaks.
public int read() throws IOException {
while (!firstFrameHeaderRead || buffer.remaining() == 0) {
if (!firstFrameHeaderRead || frameInfo.isFinished()) {
if (firstFrameHeaderRead && readSingleFrame) {
return -1;
}
if (!nextFrameInfo()) {
return -1;
}
}
readBlock();
}
This is our Compressor implementation
public final class LZ4Compressor implements ByteHandler {
public OutputStream serialize(final OutputStream toSerialize) {
try {
return new LZ4FrameOutputStream(toSerialize);
} catch (IOException e) {
throw new UncheckedIOException(e);
}
}
public InputStream deserialize(final InputStream toDeserialize) {
try {
return new LZ4FrameInputStream(toDeserialize);
} catch (IOException e) {
throw new UncheckedIOException(e);
}
}
public ByteHandler getInstance() {
return new LZ4Compressor();
}
}