Problem saving oracle large object.
Closed this issue · 1 comments
cartal commented
Hi,
I'm trying to save my oracle
object using the oracle.to_hdf5
function, but I run into the following error:
---------------------------------------------------------------------------
OverflowError Traceback (most recent call last)
<ipython-input-17-b7f5462aee49> in <module>
----> 1 oracle.to_hdf5("/home/large.celloracle.oracle")
~/my-conda-envs/celloracle/lib/python3.6/site-packages/celloracle/trajectory/oracle_core.py in to_hdf5(self, file_path)
91 dump_hdf5(obj=self, filename=file_path,
92 data_compression=compression_opts, chunks=(2048, 2048),
---> 93 noarray_compression=compression_opts, pickle_protocol=2)
94
95
~/my-conda-envs/celloracle/lib/python3.6/site-packages/celloracle/utility/hdf5_processing.py in dump_hdf5(obj, filename, data_compression, chunks, noarray_compression, pickle_protocol)
92 fletcher32=False, shuffle=False)
93 else:
---> 94 serialized = _obj2uint(attribute, compression=noarray_compression, protocol=pickle_protocol)
95 _file.create_dataset(f"&{k}", data=serialized,
96 chunks=tuple((min(1024, len(serialized)),)),
~/my-conda-envs/celloracle/lib/python3.6/site-packages/celloracle/utility/hdf5_processing.py in _obj2uint(obj, compression, protocol)
23 An array encoding in bytes (uint8) the object pickled
24 """
---> 25 zstr = zlib.compress(pickle.dumps(obj, protocol=protocol), compression)
26 return np.frombuffer(zstr, dtype=np.uint8)
27
OverflowError: cannot serialize a string larger than 4GiB
Which seems to be caused by this issue.
Do you have any idea how to fix this?
KenjiKamimoto-ac commented
Hi Carlos, Thank you for using CellOracle!
I updated pickle protocol in the CellOracle now. Could you please uninstall CellOracle and re-install the updated CellOracle, and then please try to save your file again?