JaneliaSciComp/JRCLUST

Error using reshape in jrclust.detect.DetectController/detect when I am trying to run the "detect" on multiple row data files.

Closed this issue · 10 comments

If you have a usage or support question, come join our Gitter and ask your question there.
If you found a bug, please provide all of the following:

Output of ver

Output of gpuDevice

The command you used

(e.g., jrc detect-sort myparam.prm)

If manually sorting, what it is you tried to do

Console output

output of ver: MATLAB Version: 9.5.0.944444 (R2018b)
output of gpuDevice:
Name: 'GeForce GTX 1080 Ti'
Index: 1
ComputeCapability: '6.1'
SupportsDouble: 1
DriverVersion: 10.1000
ToolkitVersion: 9.1000
MaxThreadsPerBlock: 1024
MaxShmemPerBlock: 49152
MaxThreadBlockSize: [1024 1024 64]
MaxGridSize: [2.1475e+09 65535 65535]
SIMDWidth: 32
TotalMemory: 1.1811e+10
AvailableMemory: 9.6515e+09
MultiprocessorCount: 28
ClockRateKHz: 1683000
ComputeMode: 'Default'
GPUOverlapsTransfers: 1
KernelExecutionTimeout: 1
CanMapHostMemory: 1
DeviceSupported: 1
DeviceSelected: 1

The command: jrc detect myparam.prm (It inlcudes multiple raw data files)
Error using reshape
To RESHAPE the number of elements must not change.
Error in jrclust.detect.DetectController/detect (line
104)
res.spikesRaw = reshape(fread(fid, inf, '*int16'),
res.rawShape);
Error in JRC/detect (line 16)
dRes = obj.hDetect.detect();
Error in JRC/run (line 13)
obj.detect();
Error in jrc (line 20)
hJRC.run();

There should be a file called myparam_raw.jrc in your output directory. What is its size in bytes, and what is the value of rawShape in your myparam_res.mat file?

Yes, the size of the file in byte:
myparam_raw.jrc - bytes: 442630260; the size of rawShape= [41 6 389679]
There is no myparam_res.mat file at this point, only _filt.jrc, and _raw.jrc

I can share the row data files and parameter file via dropbox. Would you like to get them? The previous version (3.2.5) of JRCLUS could handle (detect and sort) the files, and then I was able to migrate from v3 to v4 using jrc importv3 . BUT there was an issue when I used the manual sorting v4, the trace view did not work on multiple recording files.

Ah, I think I have an idea of what's happening, but yes, if I could see the data files (and the prm files, the v3.2.5 one too if you still have it) to confirm, that'd be fantastic. I'll be out for the next week, so no rush.

You have to change to path for files in mylist.txt for version v3.

https://www.dropbox.com/sh/m3ot6db52uzjsrh/AAAvC5GrdUeknmucMCPsNMuna?dl=0

764R_2018_11_08_14_27_44.dat -row data first.
764R_2018_11_08_15_39_48.dat -row data second

764R_2018_11_08_14_27_44-764R_2018_11_08_15_39_48_BF_v3.prm
764R_2018_11_08_14_27_44_for_v4.prm
764_BF.prb (64 channels selected from 148 only - linear probe 8*8, distance 100um between contact sites, 200um between shanks)
mylist.txt (change the path for row data for v3)

Other symptoms: when I debug the script (at this row jrclust.detect.DetectController/detect line 104) and I just read manually the binary file (* _raw.jrc):

fid = fopen(obj.hCfg.rawFile, 'r'); %detect.m line 103
fread(fid, inf, '*int16') %detect.m line 104 only reading script extracted

It gives results -> empty matrix []. BUT when I close the debug, and try to read the binary file again, there is a matrix as an output.

Do you have any proposes to fix this issue? The point is the JRC 4.0 is not supporting to merge multiple binary files as 3.x supported. I have tried on several other recording. It is simply not working.

2019-09-03 15:18:57 Finished load 93/93 (took 16.52 s) 2019-09-03 15:19:04 Finished processing file H:\828L\1963-1964\1964_828L_2019_08_07_12_02_38.dat (2/2) (took 999.87 s)
Error using reshape
To RESHAPE the number of elements must not change.

Error in jrclust.detect.DetectController/detect (line 105)
res.spikesRaw = reshape(fread(fid, inf, '*int16'), res.rawShape);

Error in JRC/detect (line 16)
dRes = obj.hDetect.detect();

Error in JRC/run (line 13)
obj.detect();

Error in jrc (line 20)
hJRC.run();

I removed files from Dropbox because it was full. If you need againg please contact me.

It is working with your updated detect.m within the JRCLUST-4.0.0+jrclust+detect@DetectController.

@gombpeti yeah I meant to get back to you about that. I'm glad it's working now.