Question: is seems block format does not allows progressive decoding?
xcrh opened this issue · 1 comments
I'm not realy sure if this is issue at all, but I've got idea one may sometimes want to do a progressive block decoding. Say, specs say block could be fairly large and for e.g. networking it implies considerable latency, well over of network packet size. So I could imagine someone may want to try progressive block decoding to reduce latency a bit.
However, as far as I understand, current stream format assumes receiver have to receive complete block before being able to decode it reasonably, because some data required to begin decoding could be quite close to end of block. I'm not really sure if it actually a problem if such uses were not primary goal and it is somewhat advanced/uncommon usage scenario. I'm also not sure how it could affect entropy coding, IIRC you've told it is easier the way you do it right now.
When you are using lz5.exe there is -B option that divides input data into blocks of a given size. Then the subsequent blocks can be decompressed independently or progressively (if -BD is used).
lz5.exe uses LZ5F encapsulation described in lz5frame.h and https://github.com/inikep/lz5/blob/lz5_v2.0/lz5_Frame_format.md.