HDF5 append_dataset error
Closed this issue · 1 comments
My first ever issue report, please bear with me and let me know if I'm not doing this right.
In my experiment code I am using nplab.datafile.append_dataset
to save some data, but it is now throwing an exception. Code to reproduce:
import nplab
import numpy as np
datafile = nplab.current_datafile()
datafile.create_dataset('myData1', data=np.array([1,2,3])) # this works ok
datafile.append_dataset('myData2', np.array([1,2,3])) # throws exception
datafile.append_dataset('myData1', np.array(4)) # throws exception
traceback
Traceback (most recent call last):
File "C:\Users\...\datafile_test.py", line 13, in <module>
datafile.append_dataset('myData2', np.array([1,2,3])) # throws exception
File "C:\Users\...\GitHub\nplab\nplab\datafile.py", line 290, in append_dataset
maxshape=maxshape, chunks=True)
File "C:\Users\...\GitHub\nplab\nplab\datafile.py", line 252, in require_dataset
*args, **kwargs)
File "C:\Users\...\GitHub\nplab\nplab\datafile.py", line 234, in create_dataset
dset = super(Group, self).create_dataset(name, shape, dtype, data, *args, **kwargs)
File "C:\Users\...\Anaconda3\lib\site-packages\h5py\_hl\group.py", line 116, in create_dataset
dsid = dataset.make_new_dset(self, shape, dtype, data, **kwds)
File "C:\Users\...\Anaconda3\lib\site-packages\h5py\_hl\dataset.py", line 120, in make_new_dset
shuffle, fletcher32, maxshape, scaleoffset)
File "C:\Users\...\Anaconda3\lib\site-packages\h5py\_hl\filters.py", line 83, in generate_dcpl
raise TypeError("Scalar datasets don't support chunk/filter options")
TypeError: Scalar datasets don't support chunk/filter options
Any ideas? It was working fine a couple of months ago. h5py version is 2.8, nplab repo is up to date.
I branched off and started rolling back. The problem appears at commit fc65cb1 and is caused by a change in argument order in function Group.create_dataset
in datafile.py, which is called by the function append_dataset
that I was using. Setting the order back to what it was before resolves the problem.
I think we should roll back and keep the argument order as it was originally before fc65cb1 since the current order breaks my experiment, and it would also be consistent with the h5py library we are using, which is probably why it was implemented with that order in the first place. Do you agree @eoinell @mjh250 ? I'm happy to make the commit for that.