ValueError early in rapidtide2 call

[dowdlelt]

Testing out rapidtide2, just using some very basic default settings in python 3. using the command:
$rapidtide2 pb05.sub-8006sham_oc_b_FSanaticor.r01.combine.nii rapid_out -L

and I am running into an issue. Perhaps I am doing something incorrectly?

Bulk of error copied below:

filtering to lfo band
applying zero phase antialiasing filter
Traceback (most recent call last):
File "/home/dowdlelt/anaconda3/bin/rapidtide2", line 4, in
import('pkg_resources').run_script('rapidtide==1.5.0', 'rapidtide2')
File "/home/dowdlelt/anaconda3/lib/python3.6/site-packages/pkg_resources/init.py", line 658, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/home/dowdlelt/anaconda3/lib/python3.6/site-packages/pkg_resources/init.py", line 1438, in run_script
exec(code, namespace, namespace)
File "/home/dowdlelt/anaconda3/lib/python3.6/site-packages/rapidtide-1.5.0-py3.6.egg/EGG-INFO/scripts/rapidtide2", line 2430, in
main()
File "/home/dowdlelt/anaconda3/lib/python3.6/site-packages/rapidtide-1.5.0-py3.6.egg/EGG-INFO/scripts/rapidtide2", line 1876, in main
padlen=int(inputfreq * optiondict['padseconds']))
File "/home/dowdlelt/anaconda3/lib/python3.6/site-packages/rapidtide-1.5.0-py3.6.egg/rapidtide/tide_funcs.py", line 2131, in dolpfiltfilt
[b, a] = signal.butter(order, 2.0 * cutofffreq / samplefreq)
File "/home/dowdlelt/anaconda3/lib/python3.6/site-packages/scipy/signal/filter_design.py", line 2394, in butter
output=output, ftype='butter')
File "/home/dowdlelt/anaconda3/lib/python3.6/site-packages/scipy/signal/filter_design.py", line 1959, in iirfilter
raise ValueError("Digital filter critical frequencies "
ValueError: Digital filter critical frequencies must be 0 < Wn < 1

This is using the latest release from github. I've also tried without -L, as well as with directly specifying the frequencies.

The entire output from the command is copied below:

$ rapidtide2 pb05.sub-8006sham_oc_b_FSanaticor.r01.combine.nii rap
id_out -L
/home/dowdlelt/anaconda3/lib/python3.6/site-packages/h5py/init.py:34: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type.
from ._conv import register_converters as _register_converters
/home/dowdlelt/anaconda3/lib/python3.6/site-packages/statsmodels/compat/pandas.py:56: FutureWarning: The pandas.core.datetools module is deprecated and will be removed in a future version. Please use the pandas.tseries module instead.
from pandas.core import datetools
/home/dowdlelt/anaconda3/lib/python3.6/site-packages/rapidtide-1.5.0-py3.6.egg/rapidtide
rapidtide2 version: v1.5.0 13-g74ea537-dirty
using standard scipy.fftpack
numba exists
memprofiler does not exist
nibabel exists
no aggressive optimization
using numba if present
prefiltering to lfo band
setting internal precision to double
setting output precision to single
input file is NIFTI
545468 spatial locations, 445 timepoints
startpoint set to minimum, (0)
endppoint set to maximum, ( 444 )
no regressor file specified - will use the global mean regressor
used 290466 voxels to calculate global mean signal
regressor start time, end time, and step 0.0 600.7500106096268 1.350000023841858
first fMRI point is at 0.0 seconds relative to time origin
filtering to lfo band
applying zero phase antialiasing filter
Traceback (most recent call last):
File "/home/dowdlelt/anaconda3/bin/rapidtide2", line 4, in
import('pkg_resources').run_script('rapidtide==1.5.0', 'rapidtide2')
File "/home/dowdlelt/anaconda3/lib/python3.6/site-packages/pkg_resources/init.py", line 658, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/home/dowdlelt/anaconda3/lib/python3.6/site-packages/pkg_resources/init.py", line 1438, in run_script
exec(code, namespace, namespace)
File "/home/dowdlelt/anaconda3/lib/python3.6/site-packages/rapidtide-1.5.0-py3.6.egg/EGG-INFO/scripts/rapidtide2", line 2430, in
main()
File "/home/dowdlelt/anaconda3/lib/python3.6/site-packages/rapidtide-1.5.0-py3.6.egg/EGG-INFO/scripts/rapidtide2", line 1876, in main
padlen=int(inputfreq * optiondict['padseconds']))
File "/home/dowdlelt/anaconda3/lib/python3.6/site-packages/rapidtide-1.5.0-py3.6.egg/rapidtide/tide_funcs.py", line 2131, in dolpfiltfilt
[b, a] = signal.butter(order, 2.0 * cutofffreq / samplefreq)
File "/home/dowdlelt/anaconda3/lib/python3.6/site-packages/scipy/signal/filter_design.py", line 2394, in butter
output=output, ftype='butter')
File "/home/dowdlelt/anaconda3/lib/python3.6/site-packages/scipy/signal/filter_design.py", line 1959, in iirfilter
raise ValueError("Digital filter critical frequencies "
ValueError: Digital filter critical frequencies must be 0 < Wn < 1

[bbfrederick]

Hmm. That’s puzzling, since I don’t think it’s supposed to use the butterworth filter by default (thats what’s crashing it). Try replacing -L with -F 0.0,0.15. That will explicitly disable the high pass filter, which I think is the problem. I’m currently away from my computer (playing hookie for the afternoon to sightsee at ismrm) but I’ll check this evening. If you have some way of transferring the file I could take a look at that too. Btw what is are you using? Blaise.

…

Sent from my iPhone

On Jun 18, 2018, at 4:11 PM, Logan Dowdle ***@***.***> wrote: Testing out rapidtide2, just using some very basic default settings in python 3. using the command: $rapidtide2 pb05.sub-8006sham_oc_b_FSanaticor.r01.combine.nii rapid_out -L and I am running into an issue. Perhaps I am doing something incorrectly? Bulk of error copied below: filtering to lfo band applying zero phase antialiasing filter Traceback (most recent call last): File "/home/dowdlelt/anaconda3/bin/rapidtide2", line 4, in import('pkg_resources').run_script('rapidtide==1.5.0', 'rapidtide2') File "/home/dowdlelt/anaconda3/lib/python3.6/site-packages/pkg_resources/init.py", line 658, in run_script self.require(requires)[0].run_script(script_name, ns) File "/home/dowdlelt/anaconda3/lib/python3.6/site-packages/pkg_resources/init.py", line 1438, in run_script exec(code, namespace, namespace) File "/home/dowdlelt/anaconda3/lib/python3.6/site-packages/rapidtide-1.5.0-py3.6.egg/EGG-INFO/scripts/rapidtide2", line 2430, in main() File "/home/dowdlelt/anaconda3/lib/python3.6/site-packages/rapidtide-1.5.0-py3.6.egg/EGG-INFO/scripts/rapidtide2", line 1876, in main padlen=int(inputfreq * optiondict['padseconds'])) File "/home/dowdlelt/anaconda3/lib/python3.6/site-packages/rapidtide-1.5.0-py3.6.egg/rapidtide/tide_funcs.py", line 2131, in dolpfiltfilt [b, a] = signal.butter(order, 2.0 * cutofffreq / samplefreq) File "/home/dowdlelt/anaconda3/lib/python3.6/site-packages/scipy/signal/filter_design.py", line 2394, in butter output=output, ftype='butter') File "/home/dowdlelt/anaconda3/lib/python3.6/site-packages/scipy/signal/filter_design.py", line 1959, in iirfilter raise ValueError("Digital filter critical frequencies " ValueError: Digital filter critical frequencies must be 0 < Wn < 1 This is using the latest release from github. I've also tried without -L, as well as with directly specifying the frequencies. The entire output from the command is copied below: $ rapidtide2 pb05.sub-8006sham_oc_b_FSanaticor.r01.combine.nii rap id_out -L /home/dowdlelt/anaconda3/lib/python3.6/site-packages/h5py/init.py:34: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type. from ._conv import register_converters as _register_converters /home/dowdlelt/anaconda3/lib/python3.6/site-packages/statsmodels/compat/pandas.py:56: FutureWarning: The pandas.core.datetools module is deprecated and will be removed in a future version. Please use the pandas.tseries module instead. from pandas.core import datetools /home/dowdlelt/anaconda3/lib/python3.6/site-packages/rapidtide-1.5.0-py3.6.egg/rapidtide rapidtide2 version: v1.5.0 13-g74ea537-dirty using standard scipy.fftpack numba exists memprofiler does not exist nibabel exists no aggressive optimization using numba if present prefiltering to lfo band setting internal precision to double setting output precision to single input file is NIFTI 545468 spatial locations, 445 timepoints startpoint set to minimum, (0) endppoint set to maximum, ( 444 ) no regressor file specified - will use the global mean regressor used 290466 voxels to calculate global mean signal regressor start time, end time, and step 0.0 600.7500106096268 1.350000023841858 first fMRI point is at 0.0 seconds relative to time origin filtering to lfo band applying zero phase antialiasing filter Traceback (most recent call last): File "/home/dowdlelt/anaconda3/bin/rapidtide2", line 4, in import('pkg_resources').run_script('rapidtide==1.5.0', 'rapidtide2') File "/home/dowdlelt/anaconda3/lib/python3.6/site-packages/pkg_resources/init.py", line 658, in run_script self.require(requires)[0].run_script(script_name, ns) File "/home/dowdlelt/anaconda3/lib/python3.6/site-packages/pkg_resources/init.py", line 1438, in run_script exec(code, namespace, namespace) File "/home/dowdlelt/anaconda3/lib/python3.6/site-packages/rapidtide-1.5.0-py3.6.egg/EGG-INFO/scripts/rapidtide2", line 2430, in main() File "/home/dowdlelt/anaconda3/lib/python3.6/site-packages/rapidtide-1.5.0-py3.6.egg/EGG-INFO/scripts/rapidtide2", line 1876, in main padlen=int(inputfreq * optiondict['padseconds'])) File "/home/dowdlelt/anaconda3/lib/python3.6/site-packages/rapidtide-1.5.0-py3.6.egg/rapidtide/tide_funcs.py", line 2131, in dolpfiltfilt [b, a] = signal.butter(order, 2.0 * cutofffreq / samplefreq) File "/home/dowdlelt/anaconda3/lib/python3.6/site-packages/scipy/signal/filter_design.py", line 2394, in butter output=output, ftype='butter') File "/home/dowdlelt/anaconda3/lib/python3.6/site-packages/scipy/signal/filter_design.py", line 1959, in iirfilter raise ValueError("Digital filter critical frequencies " ValueError: Digital filter critical frequencies must be 0 < Wn < 1 — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[dowdlelt]

Thanks for the response - there is no huge rush on this, just jumping back into it after the practicalfmri post. I've used it in the past (I do not know what version, but considerably earlier) without error.

Trying with -F 0.0,01.5 led to an identical error, at a identical location. Notable, if I disable the anti aliasing filter ('-a') it works, but I'm assuming that is a default for a reason.

Currently using Ubuntu 16.04, and pulled and installed the latest rapidtide release this am. Python 3.6.3, installed via anaconda.

Thanks for the help, I'll keep digging through it as well, though I am at the limits of my knowledge already.

Edit:
It completed using the -a flag, but I would like to bring up a small point, the following error:

[...]rapidtide-1.5.0-py3.6.egg/rapidtide/tide_funcs.py:497: FutureWarning: rcond parameter will change to the default of machine precision times max(M, N) where M and N are the input matrix dimensions.
To use the future default and silence this warning we advise to pass rcond=None, to keep using the old, explicitly pass rcond=-1.
solution = np.linalg.lstsq(np.mat(xc).T, np.mat(y).T)

was printed to the terminal an extraordinary amount of times.

[bbfrederick]

The antialiasing filter is needed, but that’s one of the oldest unmodified bits of the code, and the last butterworth filter. I’ll switch it over to a trapezoidal fft filter (like every single other filter) tonight. Should have done it long ago, was just lazy. Blaise

…

Sent from my iPhone

On Jun 18, 2018, at 5:14 PM, Logan Dowdle ***@***.***> wrote: Thanks for the response - there is no huge rush on this, just jumping back into it after the practicalfmri post. I've used it in the past (I do not know what version, but considerably earlier) without error. Trying with -F 0.0,01.5 led to an identical error, at a identical location. Notable, if I disable the anti aliasing filter ('-a') it works, but I'm assuming that is a default for a reason. Thanks for the help, I'll keep digging through it as well, though I am at the limits of my knowledge already. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[bbfrederick]

Ok, that's just weird. The antialiasing filter is already a trapezoidal FFT. In fact, you can't get to the line just before the crash - it depends on a value being True that is defined False, and never changed. Can you try going into the rapidtide directory, then changing to rapidtide/data/examples/src, and running the script "testfmri"? What happens when you do that might give me a clue. Also, what OS are you using? Blaise On Mon, Jun 18, 2018 at 6:53 PM, Blaise Frederick < blaise.frederick@gmail.com> wrote:

…

The antialiasing filter is needed, but that’s one of the oldest unmodified bits of the code, and the last butterworth filter. I’ll switch it over to a trapezoidal fft filter (like every single other filter) tonight. Should have done it long ago, was just lazy. Blaise Sent from my iPhone On Jun 18, 2018, at 5:14 PM, Logan Dowdle ***@***.***> wrote: Thanks for the response - there is no huge rush on this, just jumping back into it after the practicalfmri post. I've used it in the past (I do not know what version, but considerably earlier) without error. Trying with -F 0.0,01.5 led to an identical error, at a identical location. Notable, if I disable the anti aliasing filter ('-a') it works, but I'm assuming that is a default for a reason. Thanks for the help, I'll keep digging through it as well, though I am at the limits of my knowledge already. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA_R0zaW38vcOIlJY-57hMcH7cdPLd9Aks5t98PKgaJpZM4Ur1Aw> .

-- Blaise Frederick

[dowdlelt]

OS is Ubuntu 16.04,

I get a new error if I do that, after the standard deprecation warnings I see this:

/home/dowdlelt/anaconda3/lib/python3.6/site-packages/rapidtide-1.5.0-py3.6.egg/rapidtide
rapidtide2 version: v1.5.0 13-g74ea537-dirty
using standard scipy.fftpack
numba exists
memprofiler does not exist
nibabel exists
no aggressive optimization
using numba if present
option --despecklepasses not recognized
usage: rapidtide2 fmrifilename outputname
[-r LAGMIN,LAGMAX] [-s SIGMALIMIT] [-a] [--nowin[...]

[dowdlelt]

In digging through the rapidtide2 code, on line 1868, I see:

if optiondict['zpfilter']:
print('applying zero phase antialiasing filter')
if optiondict['verbose']:
print(' input freq:', inputfreq)
print(' fmri freq:', fmrifreq)
print(' npoints:', np.shape(reference_y)[0])
print(' filtorder:', optiondict['filtorder'])
reference_y_filt = tide.dolpfiltfilt(inputfreq, 0.5 * fmrifreq, reference_y, optiondict['filtorder'],
padlen=int(inputfreq * optiondict['padseconds']))
else:
if optiondict['trapezoidalfftfilter']:
print('applying trapezoidal antialiasing filter')
reference_y_filt = tide.dolptrapfftfilt(inputfreq, 0.25 * fmrifreq, 0.5 * fmrifreq, reference_y,
padlen=int(inputfreq * optiondict['padseconds']))
else:
print('applying brickwall antialiasing filter')
reference_y_filt = tide.dolpfftfilt(inputfreq, 0.5 * fmrifreq, reference_y,
padlen=int(inputfreq * optiondict['padseconds']))
reference_y = rt_floatset(reference_y_filt.real)

So, it looks like it will use the tide.dolpfilt if zpfilter is set to true. tide.dolpfilt appears to be the butterworth filter, on line 2488 in tide_funcs.py.

zpfilter is set to True in rapidtide2, on line 1169. Could this be the issue?

Update - it was not. Changing it to false had no effect. No doubt I did not dig deep enough....

[bbfrederick]

Yes - that’s certainly it. But I looked last night and on my copy on my laptop it’s set to false (that’s why I said it was impossible)! I must have fixed it within the last few weeks and not pushed it. I’ll straighten that out tonight, but in the meantime by all means try fixing that. Blaise.

…

Sent from my iPhone

On Jun 20, 2018, at 3:42 PM, Logan Dowdle ***@***.***> wrote: In digging through the rapidtide2 code, on line 1868, I see: if optiondict['zpfilter']: print('applying zero phase antialiasing filter') if optiondict['verbose']: print(' input freq:', inputfreq) print(' fmri freq:', fmrifreq) print(' npoints:', np.shape(reference_y)[0]) print(' filtorder:', optiondict['filtorder']) reference_y_filt = tide.dolpfiltfilt(inputfreq, 0.5 * fmrifreq, reference_y, optiondict['filtorder'], padlen=int(inputfreq * optiondict['padseconds'])) else: if optiondict['trapezoidalfftfilter']: print('applying trapezoidal antialiasing filter') reference_y_filt = tide.dolptrapfftfilt(inputfreq, 0.25 * fmrifreq, 0.5 * fmrifreq, reference_y, padlen=int(inputfreq * optiondict['padseconds'])) else: print('applying brickwall antialiasing filter') reference_y_filt = tide.dolpfftfilt(inputfreq, 0.5 * fmrifreq, reference_y, padlen=int(inputfreq * optiondict['padseconds'])) reference_y = rt_floatset(reference_y_filt.real) So, it looks like it will use the tide.dolpfilt if zpfilter is set to true. tide.dolpfilt appears to be the butterworth filter, on line 2488 in tide_funcs.py. zpfilter is set to True in rapidtide2, on line 1169. Could this be the issue? — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[dowdlelt]

Alright, I think I have sorted it out, for some reason there is effectively a duplication of the rapidtide script in anaconda, though they are the same version. I'm not familiar enough with python/anaconda/setup.pys/etc to be certain what happen, but it looks like zpfilter was still set to true in this file:
/home/dowdlelt/anaconda3/lib/python3.6/site-packages/rapidtide-1.5.0-py3.6.egg/EGG-INFO/scripts/rapidtide2
but not in the other main file that was just in the rapidtide folder, here:
/home/dowdlelt/anaconda3/lib/python3.6/site-packages/rapidtide-1.5.0-py3.6.egg/rapidtide/

Altering that one remaining True to False dodged the error entirely.

Thank you for your help with this, but it appears I may have generated this issue myself. In either case, I believe this is resolved.

[bbfrederick]

Just to follow up - I did a deeper dive into the code this morning an found another place the butterworth filter was used I had completely forgotten about - one of my resampling routines used it for resampling input regressors. That now uses the trapezoidal filter like everything else. I also completely removed the zpfilter option. I will never want to use it, so the whole thing is stripped out. It should now be impossible to use a butterworth unless you select it with the -b option on the command line. Blaise

…

On Jun 20, 2018, at 5:52 PM, Logan Dowdle ***@***.***> wrote: Alright, I think I have sorted it out, for some reason there is effectively a duplication of the rapidtide script in anaconda, though they are the same version. I'm not familiar enough with python/anaconda/setup.pys/etc to be certain what happen, but it looks like zpfilter was still set to true in this file: /home/dowdlelt/anaconda3/lib/python3.6/site-packages/rapidtide-1.5.0-py3.6.egg/EGG-INFO/scripts/rapidtide2 but not in the other main file that was just in the rapidtide folder, here: /home/dowdlelt/anaconda3/lib/python3.6/site-packages/rapidtide-1.5.0-py3.6.egg/rapidtide/ Altering that one remaining True to False dodged the error entirely. Thank you for your help with this, but it appears I may have generated this issue myself. In either case, I believe this is resolved. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA_R02rdHQt96lCC3b1sWYfT6yNNfn-iks5t-m-1gaJpZM4Ur1Aw>.