JaneliaSciComp/JRCLUST

Multiple errors in displaying results of 'jrc manual'

Closed this issue · 12 comments

Output of ver

MATLAB Version: 9.5.0.944444 (R2018b)
Operating System: Microsoft Windows 10 Home Version 10.0 (Build 17134)
Java Version: Java 1.8.0_152-b16 with Oracle Corporation Java HotSpot(TM) 64-Bit Server VM mixed mode

The command you used

jrc bootstrap
jrc detect-sort myparam.prm (appears to finish normally)
jrc manual myparam.prm (first error message appears)

Click a cluster
Keypress s
Select a clustering method and click OK
Click Yes in response to Keep Splits prompt (second error message appears)

If manually sorting, what it is you tried to do

Split a cluster using s key, using any of the provided clustering algorithms.

Console output

First error message

At this point, the clusters are not displayed in order of site number and do not all have mean waveforms displayed on them - some clusters do for some channels, some clusters for all channels, some clusters for none. Additionally, not all figures are displayed properly. Clicking one of the clusters causes the blank figures to be re-plotted and to actually contain data. Screenshots of results can be found here.

Error using matlab.graphics.axis.Axes/set
Value must be a 1x2 vector of numeric type in which the second element is larger than the first and may be Inf

Error in jrclust.views.Figure/axApply (line 282)
hFun(hAx, varargin{:});

Error in jrclust.views.plotFigPos>plotPosUnit (line 86)
hFigPos.axApply('default', @set, 'YLim', [floor(min(YData(:))-1), ceil(max(YData(:))+1)]);

Error in jrclust.views.plotFigPos (line 14)
plotPosUnit(hFigPos, c1Data, hCfg, 0, maxAmp);

Error in jrclust.curate.CurateController/updateFigPos (line 8)
jrclust.views.plotFigPos(hFigPos, obj.hClust, obj.hCfg, obj.selected, obj.maxAmp);

Error in jrclust.curate.CurateController/updateSelect (line 37)
obj.updateFigPos();

Error in jrclust.curate.CurateController/plotAllFigures (line 52)
obj.updateSelect(1);

Error in jrclust.curate.CurateController/beginSession (line 15)
obj.plotAllFigures();

Error in JRC/curate (line 46)
obj.hCurate.beginSession();

Error in JRC/run (line 27)
obj.curate();

Error in jrc (line 20)
hJRC.run();

Second error

After confirming I am finished splitting, a dialogue box appears reading "Operation failed", with an 'okay' button (see screenshots linked above). In the console appears:
Warning: Failed to split: Index exceeds the number of array elements (2234).
> In jrclust.curate.CurateController/splitCluster (line 43)
In jrclust.curate.CurateController/autoSplit (line 183)
In jrclust.curate.CurateController/keyPressFigWav (line 137)
In jrclust.curate.CurateController/plotAllFigures>@(varargin)obj.keyPressFigWav(varargin{:}) (line 40)

This seems to not happen if I am splitting a cluster that was created immediately beforehand by merging two previously existing clusters. However, it also does not happen 100% of the time on clusters which have not previously been merged.

If I perform a merge, the clusters are then re-sorted by channel, but still do not all have mean waveforms overlaid.

I've seen the first error when clusters have spikes from all over the probe. Can you load your spikeSites and spikesByCluster fields from your res.mat file and do

cellfun(@(spikes) numel(unique(spikeSites(spikes))), spikesByCluster)

for me?

For the second error, can you tell me if this happens if you split one cluster into 3 or more clusters?

cellfun(@(spikes) numel(unique(spikeSites(spikes))), spikesByCluster):
Columns 1 through 11
237 3 1 1 2 2 2 5 8 2 2

Columns 12 through 22
4 4 9 8 4 2 5 2 2 2 10

Columns 23 through 33
5 3 5 4 2 2 8 2 4 4 5

Columns 34 through 44
3 1 5 3 4 7 5 1 4 4 4

Columns 45 through 55
3 4 8 5 5 4 4 3 5 3 2

Columns 56 through 66
2 5 4 3 3 2 5 2 2 1 1

Columns 67 through 77
1 1 2 2 2 2 1 4 1 2 1

Columns 78 through 88
3 3 1 2 1 2 2 3 1 2 4

Columns 89 through 99
1 2 1 3 5 2 2 3 1 1 3

Columns 100 through 110
2 1 4 3 3 2 2 2 3 3 3

Columns 111 through 121
6 5 3 1 2 3 1 1 2 4 4

Columns 122 through 132
3 5 3 6 3 7 3 5 3 3 2

Columns 133 through 143
3 2 7 5 2 1 4 4 2 3 2

Columns 144 through 154
2 6 1 2 4 5 5 3 4 7 4

Columns 155 through 165
4 5 1 2 3 1 4 3 4 3 2

Columns 166 through 176
1 4 4 1 2 3 3 3 4 7 4

Columns 177 through 187
4 3 5 3 2 1 7 3 4 5 2

Columns 188 through 192
2 2 3 1 1

3-Splitting appears to fail in (EDIT: MOST OF) the same cases as 2-splitting; if I try to split a cluster 3 ways, and get an operation failure, it will also fail to split 2 ways, and vice versa.

ETA: I did find one cluster where 2-splitting failed and 3-splitting succeeded.

cellfun(@(spikes) numel(unique(spikeSites(spikes))), spikesByCluster):
Columns 1 through 23
237 3 1 1 2 2 2 5 8 2 2 4 4 9 8 4 2 5 2 2 2 10 5

Columns 24 through 46
3 5 4 2 2 8 2 4 4 5 3 1 5 3 4 7 5 1 4 4 4 3 4

Columns 47 through 69
8 5 5 4 4 3 5 3 2 2 5 4 3 3 2 5 2 2 1 1 1 1 2

Columns 70 through 92
2 2 2 1 4 1 2 1 3 3 1 2 1 2 2 3 1 2 4 1 2 1 3

Columns 93 through 115
5 2 2 3 1 1 3 2 1 4 3 3 2 2 2 3 3 3 6 5 3 1 2

Columns 116 through 138
3 1 1 2 4 4 3 5 3 6 3 7 3 5 3 3 2 3 2 7 5 2 1

Columns 139 through 161
4 4 2 3 2 2 6 1 2 4 5 5 3 4 7 4 4 5 1 2 3 1 4

Columns 162 through 184
3 4 3 2 1 4 4 1 2 3 3 3 4 7 4 4 3 5 3 2 1 7 3

Columns 185 through 192
4 5 2 2 2 3 1 1

Yeah, that first cluster is enormous and should not have been merged. Any chance you can send me this data so I can see why that might have happened? Fixing the first problem might address the second.

Any chance you can send me this data so I can see why that might have happened?

Sure. It should be in your inbox shortly.

@oliriley I think I've addressed at least part of the problem you're seeing. Can you pull the latest and run

jrc sort /path/to/params.prm

and then try manually curating? You'll still run into an error when you select the 1st cluster (this is a bigger issue), but everything should be in order and splits should work okay.

Will give it a try!

@aliddell you were right that it gives the same issues with giant first cluster (237 sites still) but the initial clusters are arranged by site number, and splitting works fine. Can post screenshots/console logs if it would be helpful.

Something to note: the the giant cluster-one exists even though its waveform similarity is <0.75, way below the minimum that should be allowed (I believe default is 0.98?). That makes me suspect that whatever is causing all the merges may be related to a missing check on similarity score.

@oliriley These spikes are all assigned to the same cluster based on feature similarity, before any waveform-based merging occurs. It's the same issue as #87, actually. It's going to take some careful thought to come up with a good solution to it.

@oliriley with the latest commit I'm no longer seeing this issue. Can you verify it works for you as well?

I'm testing now. It seems to have cleared up the first-large-cluster issue nicely!

Excellent! Good to close?