Template-Gui 3D Clusters, individual waveforms, and fixing abrupt cutoffs.
DJ-Hayden opened this issue · 3 comments
Hey Everyone!
Apologies if this is the wrong place for this; it's dual KiloSort and Phy.
I'm hoping to start working on a 3D plot of each waveform's PCAs (or other stats) to help with fine scale cluster cutting. Long story short, I'd like to make a mini Plexon Offline Sorter. However, to work on that I have a few questions.
More Phy specific:
- Is there an already appropriate/fixed way to GET a single waveform from a given cluster? I'm aware I can iterate through a cluster, but I'm not sure if there's built-in functionality to point at a dot in PC space and immediately have access to its' cluster parent.
More KiloSort specific:
-
Do you have any insight as to why the Amplitude Histogram or FeatureView's have abrupt cut offs on otherwise well defined units? Even on the eMouse data from KiloSort's tutorial I see some pretty severe cutting off of Amplitudes/Feature Space even when a stable unit is some 30-40 (uV?) from 0? I'm aware the algorithm is template matching, so I suspect there is a threshold step somewhere in the configuration file, but repeated runs with different (i.e. low) thresholds have shown a consistent cutting off.
-
Is there any way to fix those severe cut offs or prevent them from happening? Would KiloSort be able to keep the "cut off" data for later merging/3D cluster cutting, perhaps as a giant "Unsorted" cluster that we could from?
-
Should 1) and 2) be unfixable and there is no ability to recover waveforms just below the cutoff threshold, I was thinking of the following. I could use a combination of simple voltage threshold detection plus the KiloSort times to get a giant list of spike times. After getting all unique spike times, I could sort them into "Unsorted" (every spike time not picked up by KiloSort) or "Unit##" (based on KiloSort). Then I could just send all of that to Phy. A 3D view of all the "Unsorted" spikes overlayed with the "Unit##" would hopefully cover those abrupt cutoffs (since the "Unsorted" stuff would theoretically contain the things KiloSort rejected).
Regards,
Dustin
I share all of these needs/questions, but haven't found any reliable way to mitigate the amplitude cutoff (even in clearly defined units).
Working in Phy makes all my past frustrations with the Offline Sorter (OS) interface seem totally trivial. A 3D PC-space view with split/merge functionality would go a very long way in making Phy more usable. In OS, trimming & defining units in the 3D space of [PC1, timestamp, PC2] is my absolute go-to method for sorting and visually confirming unit isolation; especially over long recordings.
Likewise, the OS ability to hold alt while 'scribbling' the cursor over a cluster of points in the 2D/3D PC-space plots and seeing the waveform highlighted is crucial for getting a sense of just what a particular PC actually looks like in the waveform. For example, detecting when the spread in a cluster is due to a gradual (or not so gradual) shift in waveform amplitude due to sliding, or when two adjacent clusters are actually just a slight phase shift of the same unit within the waveform snippet/template.
Very curious if any progress has been made on this [& if so, thanks in advance!].
Hey czuba!
I'm just about to get started in earnest on this! However, I think there's a bit of a problem. As I understand it, all the principal component dimensions you see are computed independently for each channel, meaning you cannot compare between channels/clusters. I was going to just implement the PCA myself using a method that allows comparisons in PC space to have meaning, but then I realized that Phy wasn't loading all the unit waveforms anyway (for efficiency sake), so it'd be useless to try and cluster cut anyway (because not all putative spikes are available). I'll send another email to the team and see if they could clarify some of the conjectures I have.
I think I'll likely end up having to make my own version of OfflineSorter. I can use Kilosort/Phy for a quick pass, but then it'd be a separate program to ensure all clusters are well separated.
Regards,
Dustin
The latest version in the dev branch contains significant improvements and customization options in the views. Hopefully this will tackle your concerns, otherwise please feel free to open new issues directly on the phy repository.