NeuralEnsemble/python-neo

TDT read issue when 'StoreName' key is missing from header

Closed this issue · 2 comments

Describe the bug

I'm having issues with reading TDT files (not sure where to check the TDT version, but these files are from 2014), where
'StoreName' key is missing from one of the entries in info:

v = np.dtype(dt).type(info[k])

Here is a snippet how the infos look like for one the problematic files:

[
    OrderedDict({'NumPoints': b'32', 'DataFormat': b'0', 'SampleFreq': b'24414.062500'}), 
    ...
    OrderedDict({'StoreName': b'Conx', 'HeadName': b'Conx', 'Enabled': b'1', 'CircType': b'5', 'NumChan': b'20', 'StrobeMode': b'0', 'TankEvType': b'33025', 'NumPoints': b'2048', 'DataFormat': b'2', 'SampleFreq': b'24414.062500'}), 
    ...
    OrderedDict({'StoreName': b'LFPx', 'HeadName': b'LFPx', 'Enabled': b'1', 'CircType': b'5', 'NumChan': b'4', 'StrobeMode': b'0', 'TankEvType': b'33025', 'NumPoints': b'64', 'DataFormat': b'0', 'SampleFreq': b'1017.252625'})]
]

I'm not sure what signal the first entry should correspond to, the neural signals have the 'Conx' stream name.
Is there a way to fix this?

To Reproduce
I can share one of these files privately.

Additional context

Traceback (most recent call last):
  File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/pydevd.py", line 1534, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/Users/weian/catalystneuro/turner-lab-to-nwb/check_datalist.py", line 11, in <module>
    stream_names, stream_ids = TdtRecordingExtractor.get_streams(folder_path=problematic_files[0])
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/weian/anaconda3/envs/turner_lab_to_nwb_env/lib/python3.12/site-packages/spikeinterface/extractors/neoextractors/neobaseextractor.py", line 71, in get_streams
    neo_reader = cls.get_neo_io_reader(cls.NeoRawIOClass, **neo_kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/weian/anaconda3/envs/turner_lab_to_nwb_env/lib/python3.12/site-packages/spikeinterface/extractors/neoextractors/neobaseextractor.py", line 64, in get_neo_io_reader
    neo_reader.parse_header()
  File "/Users/weian/anaconda3/envs/turner_lab_to_nwb_env/lib/python3.12/site-packages/neo/rawio/baserawio.py", line 179, in parse_header
    self._parse_header()
  File "/Users/weian/anaconda3/envs/turner_lab_to_nwb_env/lib/python3.12/site-packages/neo/rawio/tdtrawio.py", line 101, in _parse_header
    _info_channel_groups = read_tbk(tbk_filename)
                           ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/weian/anaconda3/envs/turner_lab_to_nwb_env/lib/python3.12/site-packages/neo/rawio/tdtrawio.py", line 542, in read_tbk
    v = np.dtype(dt).type(info[k])
                          ~~~~^^^
KeyError: 'StoreName'

Hi @weiglszonja

Would you have time to try to make a PR with a fix? Basically running that line only if StoreName in info

@alejoe91 I opened #1447, I have seen another example where not only StoreName is missing but other field(s) as well. I think these files can be salvaged with this solution.