threadpool.py init error
Laserdance100 opened this issue · 11 comments
Problem1:
soapy_power -r 2.56M -f 88M:98M -B 500k -F rtl_power -O output.txt --even -T 1 --debug
Detached kernel driver
Found Rafael Micro R820T tuner
Reattached kernel driver
DEBUG: pyfftw module found (using 2 threads by default)
DEBUG: scipy.fftpack module found
Detached kernel driver
Found Rafael Micro R820T tuner
DEBUG: Applying fixes for RTLSDR quirks...
INFO: Using device: RTLSDR
DEBUG: SoapySDR stream - args: {'buffers': '100'}
[INFO] Using format CF32.
DEBUG: SoapySDR stream - buffer size: 8192
DEBUG: SoapySDR stream - read timeout: 0.103200
INFO: repeats: 106667
INFO: samples: 640002 (time: 0.25000 s)
INFO: max_buffer_size (samples): 32768000 (repeats: 5461333.33, time: 12.80000 s)
INFO: buffer_size (samples): 647168 (repeats: 107861.33, time: 0.25280 s)
INFO: buffer_repeats: 1
Traceback (most recent call last):
File "/usr/bin/soapy_power", line 9, in
load_entry_point('soapy-power==1.4.0', 'console_scripts', 'soapy_power')()
File "/home/X/.local/lib/python3.5/site-packages/soapypower/main.py", line 310, in main
max_threads=args.max_threads, max_queue_size=args.max_queue_size
File "/home/X/.local/lib/python3.5/site-packages/soapypower/power.py", line 280, in sweep
reset_stream=reset_stream, max_threads=max_threads, max_queue_size=max_queue_size
File "/home/X/.local/lib/python3.5/site-packages/soapypower/power.py", line 192, in setup
lnb_lo=lnb_lo, max_threads=max_threads, max_queue_size=max_queue_size)
File "/home/X/.local/lib/python3.5/site-packages/soapypower/psd.py", line 31, in init
thread_name_prefix='PSD_thread'
File "/home/X/.local/lib/python3.5/site-packages/soapypower/threadpool.py", line 7, in init
super().init(max_workers or os.cpu_count() or 1, thread_name_prefix)
TypeError: init() takes from 1 to 2 positional arguments but 3 were given
code in threadpool.py:
class ThreadPoolExecutor(concurrent.futures.ThreadPoolExecutor):
"""ThreadPoolExecutor which allows setting max. work queue size"""
def init(self, max_workers=0, thread_name_prefix='', max_queue_size=0):
super().init(max_workers or os.cpu_count() or 1, thread_name_prefix)
fix to the problem1
super().init(max_workers or os.cpu_count() or 1)
Then it works. I have even done my first .csv export spectrum analysis with my airspy. I know two others softwares that do the .csv export for the RTLSDR, but none with the airspy.
Problem2:
soapy_power params:
{'bin_size': 100.0,
'crop': 90.0,
'device': 'rtlsdr',
'gain': 20,
'hops': 0,
'interval': 60.0,
'ppm': 0,
'sample_rate': 2160000,
'single_shot': True,
'start_freq': 24.0,
'stop_freq': 1765.0}
Found Rafael Micro R820T tuner
Found Rafael Micro R820T tuner
Exact sample rate is: 2160000.051498 Hz
INFO: Using device: RTLSDR
WARNING: number of overlapping FFT bins should be even, changing overlap/crop factor to 0.90909
[INFO] Using format CF32.
INFO: repeats: 665
INFO: samples: 14630 (time: 0.00677 s)
INFO: max_buffer_size (samples): 32768000 (repeats: 1489454.55, time: 15.17037 s)
INFO: buffer_size (samples): 16384 (repeats: 744.73, time: 0.00759 s)
INFO: buffer_repeats: 1
INFO: overlap: 0.90909
INFO: bin_size: 98181.82 Hz
INFO: bins: 22
INFO: bins (after crop): 2
INFO: sample_rate: 2.160 MHz
INFO: sample_rate (after crop): 0.196 MHz
INFO: freq_range: 1741.000 MHz
INFO: hopping: YES
INFO: hop_size: 0.196 MHz
INFO: hops: 8867
INFO: min_center_freq: 24.098 MHz
INFO: max_center_freq: 1765.058 MHz
INFO: min_freq (after crop): 24.000 MHz
INFO: max_freq (after crop): 1765.156 MHz
ERROR: len(x_axis) != len(y_axis)
It seems that len(y_axis) = hops * bins_after_crop, but that len(x_axis) is slightly bigger. Flooring required?
Problem3:
soapy_power params:
{'bin_size': 1000.0,
'crop': 90.0,
'device': 'rtlsdr',
'gain': 20,
'hops': 0,
'interval': 60.0,
'ppm': 0,
'sample_rate': 2160000,
'single_shot': True,
'start_freq': 24.0,
'stop_freq': 1765.0}
Found Rafael Micro R820T tuner
Found Rafael Micro R820T tuner
Exact sample rate is: 2160000.051498 Hz
INFO: Using device: RTLSDR
WARNING: number of FFT bins should be even, changing to 4
WARNING: number of overlapping FFT bins should be even, changing overlap/crop factor to 1.00000
Traceback (most recent call last):
File "/usr/bin/soapy_power", line 9, in
load_entry_point('soapy-power==1.5.0', 'console_scripts', 'soapy_power')()
File "/usr/lib/python3.5/site-packages/soapy_power-1.5.0-py3.5.egg/soapypower/main.py", line 330, in main
File "/usr/lib/python3.5/site-packages/soapy_power-1.5.0-py3.5.egg/soapypower/power.py", line 105, in freq_plan
ZeroDivisionError: float division by zero
It happen when cropping is set to 100%, which is not a great idea anyway.
code in power.py:
freq_range = max_freq - min_freq
hopping = True if freq_range >= sample_rate_crop else False
hop_size = self.nearest_freq(sample_rate_crop, bin_size)
hops = math.ceil(freq_range / hop_size) if hopping else 1
Problem4:
When I set a 2dB gain in the program, it set a 20dB gain in the device.
So a 0-49.6dB device can be used only with 0, 10, 20, 30, 40dB gains!
Best regards.
Problem 1 is now fixed in Git master (thread_name_prefix is only supported on Python >= 3.6).
I will look into problems 2 and 3 tomorrow. But why are you doing so much cropping? 90% crop is extremely high, you are discarding most FFT bins.
Problem 4 seems strange, will look into it.
Btw. thanks for your report.
I can confirm, I have this issue in soapy_power 1.5.0 but not in git master
zero@gato soapy_power % qspectrumanalyzer (git)-[master]
soapy_power params:
{'bandwidth': 20000000.0,
'bin_size': 1.0,
'crop': 0.0,
'device': '',
'gain': 0,
'hops': 0,
'interval': 1.0,
'ppm': 0,
'sample_rate': 20000000.0,
'single_shot': False,
'start_freq': 100.0,
'stop_freq': 300.0}
[INFO] Opening HackRF device instance {device=HackRF One, driver=hackrf, part_id=a000cb3c005f434b, serial=000000000000000015d463dc38437825, version=2017.02.1}...
INFO: Using device: HackRF One
INFO: repeats: 100
INFO: samples: 2000000 (time: 0.10000 s)
INFO: max_buffer_size (samples): 32768000 (repeats: 1638.40, time: 1.63840 s)
INFO: buffer_size (samples): 2097152 (repeats: 104.86, time: 0.10486 s)
INFO: buffer_repeats: 1
Traceback (most recent call last):
File "/usr/lib/python-exec/python3.4/soapy_power", line 11, in <module>
load_entry_point('soapy-power==1.5.0', 'console_scripts', 'soapy_power')()
File "/usr/lib64/python3.4/site-packages/soapypower/__main__.py", line 348, in main
max_threads=args.max_threads, max_queue_size=args.max_queue_size
File "/usr/lib64/python3.4/site-packages/soapypower/power.py", line 280, in sweep
reset_stream=reset_stream, max_threads=max_threads, max_queue_size=max_queue_size
File "/usr/lib64/python3.4/site-packages/soapypower/power.py", line 192, in setup
lnb_lo=lnb_lo, max_threads=max_threads, max_queue_size=max_queue_size)
File "/usr/lib64/python3.4/site-packages/soapypower/psd.py", line 31, in __init__
thread_name_prefix='PSD_thread'
File "/usr/lib64/python3.4/site-packages/soapypower/threadpool.py", line 7, in __init__
super().__init__(max_workers or os.cpu_count() or 1, thread_name_prefix)
TypeError: __init__() takes 2 positional arguments but 3 were given
I've just rebuilt simplesoapy, qspectrumanalyzer and soapy_power from git.
Seems that the problems1 and 4 are fixed. Perhaps you should transform the gain qt-spin-box to a qt-double-spin-box 00.0, so we can use gains like 49.6dB.
Is this precision in setting gain really useful? Aren't whole numbers enough?
Btw. if you need really precise control of gain, you can set gain in GUI to -1 and add --specific-gains option to additional backend parameters in Settings. Then you can specify gains of individual amplification elements (e.g. LNA, VGA and AMP on HackRF). Or you can add '-g' option and specify total gain with more precision than in GUI.
Ok. I see why you kept the gain setting integer. Since there are possibility to specify a more precise gain easily, I guess it's enough :)
@Laserdance100 Nonetheless I have implemented this change ;-)
@Laserdance100 I will leave that Problem 2 and Problem 3 for now, I can't reproduce it with sane settings.
If I will get more time, I may look at it again, but if you would like to look at it yourself, I am ready to merge pull request ;-)
I looked at problem3 a bit:
It seems that the code (on my computer) suffers a bit from floating arithmetic precision.
related Python packages used:
numpy (1.12.1)
QSpectrumAnalyzer (2.1.0)
SimpleSoapy (1.4.1)
SimpleSpectral (1.0.0)
soapy-power (1.6.0)
How to reproduce: with a RTLSDR, set the sampling rate to 2.56MHz, then do a singleshot from 100MHz to 110.24MHz with 640kHz bin size (1s setting, 0.0dB gain, 0 ppm, 0% crop)
It will perform 4 hops of 2.56MHz
(100M-102.56M; 102.56M-105.12M; 105.12M-107.68M; 107.68M-110.24M)
Each hop contain 4 bins of 640kHz
first hop:
(100M-100.64M; 100.64M-101.28M; 101.28M-101.92M; 101.92M-102.56M)
then for each hop the SoapyPowerBinFormat function of writer.py(soapypower) provide an y-axis to the parse_output function of soapy_power.py(qspectrumanalyser)
this y-axis has a size of 4 (of course there are 4 bins in each hop)
so the x-axis should have a size of 4 too.
it should be from my point of view (100.32M; 100.96M; 101.6M; 102.24M)
and absolutely not (100M; 100.64M; 101.28M; 101.92M; 102.56M)
and possibly (100M; 100.64M; 101.28M; 101.92M) depending on how you wrote the rest of your program.
you computed the x-axis like that:
x_axis = np.arange(start_freq, stop_freq, step)
note that you are using float numbers.
from numpy.arange docs:
"When using a non-integer step, such as 0.1, the results will often not be consistent. It is better to use linspace for these cases."
np.arange(100, 102.56, 0.64) gives
array([ 100. , 100.64, 101.28, 101.92, 102.56])
but np.arange(100e6, 102.56e6, 0.64e6) gives
array([ 1.00000000e+08, 1.00640000e+08, 1.01280000e+08,
1.01920000e+08])
That's the well known float precision problem. Here, it leads to the array being of size 5 or 4!
another example, you compute 8/2and 8%2 with floats.
you could have 4.00000000000000000000000000000026
and 0.00000000000000000000000000000037
(it's not a great example cause there is actually no problem with 8 and 2, but you should have understood what I was pointing, and what could possibly occur)
So I guess you should code something like this
def parse_output(self, data):
"""Parse data from soapy_power"""
header, y_axis = data
time_start = header.time_start
time_stop = header.time_stop
start_freq = int(header.start)
stop_freq = int(header.stop)
step = int(header.step)
samples = header.samples
note the integer casts!
linspace_step_count = (stop_freq - start_freq)/step
linspace_start = start_freq+step/2
linspace_stop = linspace_start+(linspace_step_count*step)
x_axis = np.linspace(linspace_start, linspace_stop, linspace_step_count)
As I don't really know if that change is breaking your code somewhere else, I think that you have to think about this problem.
About the problem4:
It's quite easy. It's a problem which occurs if you have a near 100% crop setting, which is then rounded as 100% crop by the program.
I think it would be great to check if the crop value of 100%, if that's the case, report and stop.
Best regards
Can you please test latest QSpectrumAnalyzer Git master branch? Problem 3 should be hopefully fixed with latest commit xmikos/qspectrumanalyzer@4571075.
I've just tested it with master, the problem3 is fixed :)
About the problem4 (near 100% crop setting, which is then rounded to 100%):
a 100% crop error would be more helpful than the current ZeroDivisionError: float division by zero
But who would have a near 100% crop anyway?
I'll close the issue. Thanks for the fix ;)