umarbutler/orjsonl

No gzip support && open closed

getorca opened this issue · 1 comments

To me it would be essential to support compression.

As well, I believe,

return map(orjson.loads, open(path, 'rb'))
, could lead to a memory leak when the open file isn't closed.

To me it would be essential to support compression.

I've just updated the library to support gzip, bzip2, xz and Zstandard compression.

, could lead to a memory leak when the open file isn't closed.

Good pickup. The file didn't appear to close even when the map object had been exhuasted. I've fixed that by changing stream() into a generator function and enclosing for json in line: yield orjson.loads(json) in a with statement.