Question: Preprocessing required for sorting?

[ckwalters]

Hello,
I am trying out mountainsort for the first time on neuropixels data. I am using the spikeglx data example script which converts the recording to float32 at the whitening step. Sorting runs fine on the whitened float recording, but hangs at the detect_spikes step if I try to sort the same recording if it isn't whitened or converted to float. Is converting to float necessary? (Or maybe this is related to the large number of contacts in neuropixels data)?
Thank you!

traceback when I stopped the frozen/hanging sorting:

Traceback (most recent call last):
  File "c:\Users\CKW\Code\npxanalysis\mountainsort5_SORTING.py", line 180, in <module>
    main()
  File "c:\Users\CKW\Code\npxanalysis\mountainsort5_SORTING.py", line 128, in main
    sorting = ms5.sorting_scheme2(
              ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\CKW\anaconda3\envs\mountainsort\Lib\site-packages\mountainsort5\schemes\sorting_scheme2.py", line 87, in sorting_scheme2
    sorting1 = sorting_scheme1(
               ^^^^^^^^^^^^^^^^
  File "C:\Users\CKW\anaconda3\envs\mountainsort\Lib\site-packages\mountainsort5\schemes\sorting_scheme1.py", line 71, in sorting_scheme1
    times, channel_indices = detect_spikes(
                             ^^^^^^^^^^^^^^
  File "C:\Users\CKW\anaconda3\envs\mountainsort\Lib\site-packages\mountainsort5\core\detect_spikes.py", line 46, in detect_spikes
    candidate_values[inds2[i]].append(traces[inds1[i], inds2[i]])
KeyboardInterrupt

[magland]

Hi @ckwalters . Yes mountainsort expects whitened data. At the very least the channels should be normalized to have variance 1 because the detect threshold is normalized based on that assumption. That's if I can remember correctly. Have a look at the source code for detection

https://github.com/flatironinstitute/mountainsort5/blob/main/mountainsort5/core/detect_spikes.py

[ckwalters]

Got it, thanks for the quick reply!