Spindles and Slow waves summary error
sjg2203 opened this issue · 6 comments
I am having issues when trying to get spindles.summary() or slowwaves.summary(), it returns this error
27-Jun-23 15:53:49 | ERROR | Wrong data amplitude for CHAN000 (trimmed STD = 4045.123). Unit of data MUST be uV! Channel will be skipped.
27-Jun-23 15:53:49 | ERROR | Wrong data amplitude for CHAN001 (trimmed STD = 4026.372). Unit of data MUST be uV! Channel will be skipped.
27-Jun-23 15:53:49 | ERROR | Wrong data amplitude for CHAN002 (trimmed STD = 4033.459). Unit of data MUST be uV! Channel will be skipped.
27-Jun-23 15:53:49 | ERROR | Wrong data amplitude for CHAN003 (trimmed STD = 4037.286). Unit of data MUST be uV! Channel will be skipped.
27-Jun-23 15:53:49 | ERROR | Wrong data amplitude for CHAN004 (trimmed STD = 4041.202). Unit of data MUST be uV! Channel will be skipped.
27-Jun-23 15:53:49 | ERROR | Wrong data amplitude for CHAN005 (trimmed STD = 4035.873). Unit of data MUST be uV! Channel will be skipped.
27-Jun-23 15:53:49 | WARNING | All channels have bad amplitude. Returning None.
Traceback (most recent call last):
File "C:\Users\user01\AppData\Local\Programs\Python\Python310\lib\site-packages\IPython\core\interactiveshell.py", line 3508, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-3-b8a02c28cf04>", line 186, in <module>
dfsp=sp.summary(grp_chan=True,grp_stage=True)
AttributeError: 'NoneType' object has no attribute 'summary'It was working perfectly fine before, and now it crashes when passing the second recording. I am guessing it is somewhat similar to this issue #77, as trimmed SDs are >1,000. I am using the RawEDF from MNE package and didn't find a way to change the units.
I tried your solution edf_sp=edf_sp.get_data(edf_sp,units="uV") and passing sf in sp=yasa.spindles_detect(edf_sp,sf=sf,hypno=hypno_up,include=(2,3)), without any success...
Any idea on how to fix this unit issue?
@raphaelvallat I have also been facing this - I just then had to sufficiently preprocess the data to decrease the trimmed SDs, could we may be allow this as an argument?
@umair-hassan can you elaborate on how you managed to get your data to work? Tried resampling and filtering but still isn't enough to make it work
Can you run the lines below and report the results here?
edf_sp=edf_sp.get_data(edf_sp,units="uV")
pd.DataFrame(edf_sp).describe([0.01, 0.1, 0.25., 0.5, 0.75, 0.9, 0.99]) # maybe need to do edf_sp.TI think you might have to clip the data to a certain value to avoid getting an excessively high standard deviation
Indeed I had to transpose the dataframe, otherwise pd.DataFrame(edf_sw).describe([0.01,0.1,0.25,0.5,0.75,0.9,0.99]) would return a MemoryError.
This is what I got as a result:
edf_sw.T
pd.DataFrame(edf_sw).describe([0.01,0.1,0.25,0.5,0.75,0.9,0.99])
Out[11]:
0 1 ... 4 5
count 1.608192e+07 1.608192e+07 ... 1.608192e+07 1.608192e+07
mean 1.586486e+01 1.801441e+01 ... 3.812313e+01 3.169889e+01
std 1.089357e+02 1.053010e+02 ... 1.305417e+02 1.073779e+02
min -5.821799e+03 -5.852781e+03 ... -5.708089e+03 -5.781588e+03
1% -8.388190e+01 -8.454108e+01 ... -7.036851e+01 -7.300526e+01
10% -2.785077e+01 -2.389563e+01 ... -1.071183e+01 -1.532616e+01
25% -5.767910e+00 -1.153582e+00 ... 1.235981e+01 7.745480e+00
50% 9.723049e+00 1.466697e+01 ... 3.147631e+01 2.785077e+01
75% 2.982834e+01 3.246509e+01 ... 5.355917e+01 4.993362e+01
90% 5.685512e+01 5.916228e+01 ... 8.223392e+01 7.860838e+01
99% 1.732021e+02 1.498009e+02 ... 1.910002e+02 1.626551e+02
max 8.909444e+03 5.322463e+03 ... 1.080000e+04 8.750909e+03
[12 rows x 6 columns]Any idea on how I can clip those data?
Apologies for the slow reply. I think you can simply clip the an arbitrary value like 500, which should remove the <1% of outlier values (e.g. max value for channel 4 is 1.080000e+04 = 10800). You can use np.clip(edf_sw, a_min=-500, a_max=500).
Let me know if that worked,
Thanks
Raphael
Apologies for the delayed answer
np.clip(edf_sw,a_min=-500,a_max=500) works like a charm, thanks!