Backward compatibility: cannot get recording or sorting from old workspace

[khl02007]

With the current version in repo I can load workspace created with old version. but cannot load recording or sorting.

when trying to load recording:

Computing sha1 of /stelmo/nwb/recording/ginny20211103_anagh_.nwb_02_r1_15_franklab_tetrode_hippocampus/recording.h5
---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
Input In [20], in <cell line: 1>()
----> 1 recording = workspace.get_recording_extractor(recording_id=recording_id)

File ~/repos/sortingview/sortingview/workspace/Workspace.py:238, in Workspace.get_recording_extractor(self, recording_id)
    236     # old workspace, maybe we need to bring sha1 files over to new system
    237     recording_object = _migrate_files_from_old_kachery_recursive(recording_object)
--> 238 return load_recording_extractor(recording_object)

File ~/repos/sortingview/sortingview/load_extractors/load_recording_extractor.py:60, in load_recording_extractor(recording_object)
     58     recording = si.ConcatenateSegmentRecording(recording_list=recording_list)
     59 else:
---> 60     raise Exception(f'Unexpected recording format: {recording_format}')
     61 setattr(recording, 'sortingview_object', recording_object)
     62 return recording

Exception: Unexpected recording format: h5_v1

when trying to load sorting

---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
Input In [19], in <cell line: 1>()
----> 1 sorting = workspace.get_sorting_extractor(sorting_id=sorting_id)

File ~/repos/sortingview/sortingview/workspace/Workspace.py:246, in Workspace.get_sorting_extractor(self, sorting_id)
    244     # old workspace, maybe we need to bring sha1 files over to new system
    245     sorting_object = _migrate_files_from_old_kachery_recursive(sorting_object)
--> 246 return load_sorting_extractor(sorting_object)

File ~/repos/sortingview/sortingview/load_extractors/load_sorting_extractor.py:40, in load_sorting_extractor(sorting_object)
     38     sorting = se2.NpzSortingExtractor(npz_file_path)
     39 else:
---> 40     raise Exception(f'Unexpected sorting format: {sorting_format}')
     41 setattr(sorting, 'sortingview_object', sorting_object)
     42 return sorting

Exception: Unexpected sorting format: h5_v1

[khl02007]

also cannot use the following methods for old workspaces:
get_sorting_curation_authorized_users
create_curation_feed_for_sorting
set_sorting_curation_authorized_users

[khl02007]

and could not create a new spikesortingview with the old workspace (since it requires being able to load the recording)

---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
Input In [31], in <cell line: 5>()
      1 # workspace.create_curation_feed_for_sorting(sorting_id=sorting_id)
      2 # workspace.set_sorting_curation_authorized_users(sorting_id=sorting_id, user_ids=['kyuhyun9056@gmail.com'])
      3 
      4 # Prepare a visualization and print the figURL
----> 5 url = workspace.spikesortingview(recording_id=recording_id, sorting_id=sorting_id, label='Test workspace')
      6 print(url)

File ~/repos/sortingview/sortingview/workspace/_spikesortingview.py:6, in spikesortingview(self, recording_id, sorting_id, label)
      5 def spikesortingview(self, *, recording_id: str, sorting_id: str, label: str):
----> 6     R = self.get_recording_extractor(recording_id)
      7     S = self.get_sorting_extractor(sorting_id)
      9     curation_feed: Union[kcl.Feed, None] = self.get_curation_feed_for_sorting(sorting_id)

File ~/repos/sortingview/sortingview/workspace/Workspace.py:238, in Workspace.get_recording_extractor(self, recording_id)
    236     # old workspace, maybe we need to bring sha1 files over to new system
    237     recording_object = _migrate_files_from_old_kachery_recursive(recording_object)
--> 238 return load_recording_extractor(recording_object)

File ~/repos/sortingview/sortingview/load_extractors/load_recording_extractor.py:60, in load_recording_extractor(recording_object)
     58     recording = si.ConcatenateSegmentRecording(recording_list=recording_list)
     59 else:
---> 60     raise Exception(f'Unexpected recording format: {recording_format}')
     61 setattr(recording, 'sortingview_object', recording_object)
     62 return recording

Exception: Unexpected recording format: h5_v1

[khl02007]

but indeed you can read the recording_ids and sorting_ids. also get_sorting_curation works!

[magland]

Thanks for the report. I will work on implementing the h5_v1 formats for recording/sorting. Will probably have that ready in time to test prior to our meeting.

The other functionalities you mentioned are not supported - I would think this is okay because these are just readonly objects... More specifically, the following are not supported:

get_sorting_curation_authorized_users
create_curation_feed_for_sorting
set_sorting_curation_authorized_users

But note that curations CAN be created using the new system - requires creating new workspace.

[magland]

@khl02007 I have implemented the h5_v1 format, and that's now on the main branch, but not yet tested.

Would be great if you could try it out!

[khl02007]

I can confirm get_recording_extractor and get_sorting_extractor now works with old workspace