Issue about how to improve the efficiency of MU processing in spikeinterface?
Closed this issue · 2 comments
Our data is collected by Intan devices in RHD format, which is then converted into raw files and stored using spikeinterface. We then read the raw files again and obtain MUA (Multi-Unit Activity) through the detect_peaks( ) method. However, our data size is extremely large, and the processing speed is very slow and unstable. There are frequent instances where the processing gets stuck and doesn't complete. In these situations, sometimes the results are outputted, and sometimes they are not. Additionally, the computer does not show any error messages, and I have observed that there is no memory overflow. This situation is quite perplexing. Therefore, I am wondering if there are methods in spikeinterface to enhance the efficiency and stability of MUA processing. For example: (1) How to resolve the issue of MUA processing getting stuck and improve the overall stability of the program; (2) How to read raw files, process them into MUA data, and store them simultaneously, instead of reading all raw files first, processing, and then writing the output; (3) How to utilize global_job_kwargs to enhance processing speed; (4) How to improve the overall processing speed of the program.
Looking forward to everyone's response and answers.
@sunjie454, This is the package for probes, you might want to open your issues on the actual spikeinterface tracker instead since it doesn't seem like this is a probe-based issue. https://github.com/SpikeInterface/spikeinterface/issues
@sunjie454, This is the package for probes, you might want to open your issues on the actual spikeinterface tracker instead since it doesn't seem like this is a probe-based issue. https://github.com/SpikeInterface/spikeinterface/issues
@zm711 Thanks for your answer very mach. But, the probeInterface cannot resolve my problem. Anyway , thanks for your answer.