msgpack/msgpack-python

Relax "bytes" requirement when reading

smurfix opened this issue · 1 comments

The unpacker currently requires the file to return bytes.

cdef read_from_file(self):
        next_bytes = self.file_like_read(
                min(self.read_size,
                    self.max_buffer_size - (self.buf_tail - self.buf_head)
                    ))
        if next_bytes:
            self.append_buffer(PyBytes_AsString(next_bytes), PyBytes_Size(next_bytes))
        else:
            self.file_like = None

Is it possible to relax this?

The returned object must of course support the buffer protocol, but beyond that it shouldn't be required to be a bytes type. It could be a memoryview into an in-memory "file". There should be no requirement to copy the data; the unpacker does that anyway.

I never see such file-like. By typing.IOBase, read() must return str or bytes.
Is that file-like existing widely-used library? Or is it just in your program?

You can use Unpacker.feed(). Isn't it simpler than custom made file-like returning memoryview?