traversc/qs

Reading specific chunks from disk

gevro opened this issue · 1 comments

gevro commented

Hi,
I have a very large data object, made by analyzing a large dataset. The large data object is in the same format every time. However, it is too big to load all into memory.

Is there a way to read chunks of an RDS object from disk, without having to load the whole object into memory? Then I could process each chunk separately.

It would also be helpful to be able to split into chunks and merge chunks.

Thanks.

Sorry, it's not possible. I would suggest splitting up your results into multiple chunks/files.