Blosc/bcolz

How to read carray dir sequentially from disk

jrzaurin opened this issue · 0 comments

I am aware is not an "issue" per se, but I do like this package and I don't seem to manage to read from disk line by line.

I want to save a large collections of numpy arrays to disk and then read them row by row or batch by batch

For example:

import numpy as np
import bcolz
rm = np.random.rand(1000,1000)
c = bcolz.carray(rm, rootdir="test.bc")
c.flush()
del(c)

How would I then read from test.bc row by row? or with a set of row indexes?

Again sorry, I am aware this is not an issue, but stackoverflow has not been very useful

Thanks