conspack/cl-conspack

Decoding files chunk by chunk

Opened this issue · 0 comments

Hello,

Instead of being forced to decode the file fully into memory before gaining access to the objects, it
would be nice if I could decode and process complete chunks as they came off of the stream.

Would a patch like this be useful?

(defun decode-file-chunks (path chunk-func)
  (with-open-file (stream path :element-type '(unsigned-byte 8))
    (let (eof)
      (tracking-refs ()
        (loop as object = (handler-case
                              (decode-stream stream)
                            (end-of-file () (setf eof t) (values)))
       until eof
       collect (funcall chunk-func object))))))

Thank you!