Only send a reasonable amount of data to the GUI

Question

Only send a reasonable amount of data to the GUI

Closed this issue 2 years ago · 2 comments

khl02007 commented 3 years ago

data for some views (e.g. raster) for long recordings can be very large and need not be viewed all at once

Answer 1 · 2022-02-14T06:27:03.000Z

Not sure if this is related:

In [1]: import sortingview as sv

In [2]: workspace_uri = "workspace://bb11da07535a50656b0ec7860af71a7443d30391c15
   ...: 151e480d13853298e363f?label=J1620210531_.nwb_raw data valid times no pre
   ...: maze no premaze_24"

In [3]: workspace_uri
Out[3]: 'workspace://bb11da07535a50656b0ec7860af71a7443d30391c15151e480d13853298e363f?label=J1620210531_.nwb_raw data valid times no premaze no premaze_24'

In [4]: workspace = sv.load_workspace(workspace_uri)

In [5]: workspace.spikesortingview?
Signature:
workspace.spikesortingview(
    *,
    recording_id: str,
    sorting_id: str,
    label: str,
    include_curation: bool,
)
Docstring: <no docstring>
File:      ~/repos/sortingview/sortingview/workspace/_spikesortingview.py
Type:      method

In [6]: recording_id = workspace.recording_ids[0]

In [7]: sorting_id = workspace.sorting_ids[0]

In [8]: workspace.label
Out[8]: 'J1620210531_.nwb_raw data valid times no premaze no premaze_24'

In [9]: workspace.spikesortingview(
   ...:     recording_id=recording_id,
   ...:     sorting_id=sorting_id,
   ...:     label=workspace.label,
   ...:     include_curation=True,
   ...: )
Preparing spikesortingview data
Initial pass: segment 0
Initial pass: segment 1
Initial pass: segment 2
Initial pass: segment 3
Initial pass: segment 4
Initial pass: segment 5
Initial pass: segment 6
Initial pass: segment 7
Initial pass: segment 8
Initial pass: segment 9
Initial pass: segment 10
Initial pass: segment 11
Initial pass: segment 12
Initial pass: segment 13
Initial pass: segment 14
Initial pass: segment 15
Initial pass: segment 16
Initial pass: segment 17
Initial pass: segment 18
Initial pass: segment 19
Initial pass: segment 20
Initial pass: segment 21
Initial pass: segment 22
Initial pass: segment 23
Initial pass: segment 24
Initial pass: segment 25
Initial pass: segment 26
Initial pass: segment 27
Initial pass: segment 28
Initial pass: segment 29
Initial pass: segment 30
Initial pass: segment 31
Initial pass: segment 32
Initial pass: segment 33
Initial pass: segment 34
Initial pass: segment 35
Initial pass: segment 36
Initial pass: segment 37
Initial pass: segment 38
Initial pass: segment 39
Initial pass: segment 40
Initial pass: segment 41
Initial pass: segment 42
Initial pass: segment 43
Initial pass: segment 44
Initial pass: segment 45
Initial pass: segment 46
Initial pass: segment 47
Initial pass: segment 48
Initial pass: segment 49
Initial pass: segment 50
Initial pass: segment 51
Initial pass: segment 52
Initial pass: segment 53
Initial pass: segment 54
Initial pass: segment 55
Initial pass: segment 56
Initial pass: segment 57
Initial pass: segment 58
Initial pass: segment 59
Initial pass: segment 60
Initial pass: segment 61
Initial pass: segment 62
Initial pass: segment 63
Initial pass: segment 64
Initial pass: segment 65
Initial pass: segment 66
Initial pass: segment 67
Initial pass: segment 68
Segment 0 of 69
Segment 1 of 69
Segment 2 of 69
Segment 3 of 69
Segment 4 of 69
Segment 5 of 69
Segment 6 of 69
Segment 7 of 69
Segment 8 of 69
Segment 9 of 69
Segment 10 of 69
Segment 11 of 69
Segment 12 of 69
Segment 13 of 69
Segment 14 of 69
Segment 15 of 69
Segment 16 of 69
Segment 17 of 69
Segment 18 of 69
Segment 19 of 69
Segment 20 of 69
Segment 21 of 69
Segment 22 of 69
Segment 23 of 69
Segment 24 of 69
Segment 25 of 69
Segment 26 of 69
Segment 27 of 69
Segment 28 of 69
Segment 29 of 69
Segment 30 of 69
Segment 31 of 69
Segment 32 of 69
Segment 33 of 69
Segment 34 of 69
Segment 35 of 69
Segment 36 of 69
Segment 37 of 69
Segment 38 of 69
Segment 39 of 69
Segment 40 of 69
Segment 41 of 69
Segment 42 of 69
Segment 43 of 69
Segment 44 of 69
Segment 45 of 69
Segment 46 of 69
Segment 47 of 69
Segment 48 of 69
Segment 49 of 69
Segment 50 of 69
Segment 51 of 69
Segment 52 of 69
Segment 53 of 69
Segment 54 of 69
Segment 55 of 69
Segment 56 of 69
Segment 57 of 69
Segment 58 of 69
Segment 59 of 69
Segment 60 of 69
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
Input In [9], in <module>
----> 1 workspace.spikesortingview(
      2     recording_id=recording_id,
      3     sorting_id=sorting_id,
      4     label=workspace.label,
      5     include_curation=True,
      6 )

File ~/repos/sortingview/sortingview/workspace/_spikesortingview.py:9, in spikesortingview(self, recording_id, sorting_id, label, include_curation)
      6 S = self.get_sorting_extractor(sorting_id)
      8 print('Preparing spikesortingview data')
----> 9 X = SpikeSortingView.create(
     10     recording=R,
     11     sorting=S,
     12     segment_duration_sec=60 * 20,
     13     snippet_len=(20, 20),
     14     max_num_snippets_per_segment=100,
     15     channel_neighborhood_size=7
     16 )
     18 if include_curation:
     19     sorting_curation_uri = self.get_curation_subfeed(sorting_id).uri

File ~/repos/sortingview/sortingview/SpikeSortingView/SpikeSortingView.py:38, in SpikeSortingView.create(recording, sorting, segment_duration_sec, snippet_len, max_num_snippets_per_segment, channel_neighborhood_size)
     29 @staticmethod
     30 def create(*,
     31     recording: LabboxEphysRecordingExtractor,
   (...)
     36     channel_neighborhood_size: int
     37 ):
---> 38     data_uri = prepare_spikesortingview_data(
     39         recording=recording, sorting=sorting,
     40         segment_duration_sec=segment_duration_sec,
     41         snippet_len=snippet_len,
     42         max_num_snippets_per_segment=max_num_snippets_per_segment,
     43         channel_neighborhood_size=channel_neighborhood_size
     44     )
     45     return SpikeSortingView(data_uri)

File ~/repos/sortingview/sortingview/SpikeSortingView/prepare_spikesortingview_data.py:120, in prepare_spikesortingview_data(recording, sorting, segment_duration_sec, snippet_len, max_num_snippets_per_segment, channel_neighborhood_size)
    118 peak_channel_ind = channel_ids.tolist().index(peak_channel_id)
    119 if len(spike_train) > 0:
--> 120     spike_amplitudes = traces_with_padding[spike_train - start_frame_with_padding, peak_channel_ind]
    121     f.create_dataset(f'segment/{iseg}/unit/{unit_id}/spike_amplitudes', data=spike_amplitudes)
    122 else:

IndexError: index -4294966208 is out of bounds for axis 0 with size 35951218

Answer 2 · 2022-02-14T06:27:20.000Z

this was a 32 channel, 24 hr long recording