adjtomo/seisflows

Parallelize 'default' preprocessing module

bch0w opened this issue · 1 comments

bch0w commented

Currently, the internal "default" preprocessing steps (waveform filtering, misfit quantification, adjoint source creation) are run in serial for each event (but in parallel for all events). These tasks are embarrassingly parallel and could be sped up significantly through parallelization using something like concurrent.futures

That being said, using "pyatoa" preprocessing allows for parallel preprocessing, as well as expanded capabilities for misfit quantification including windowing and figure generation. See the Pyaflowa core module in Pyatoa for more information.

bch0w commented

See https://github.com/adjtomo/pyatoa/blob/abb9bd47e14d2c2d469aac73f1bf6b5a6b9d16c0/pyatoa/scripts/process_data_w_mpi.py for an example of using Mpi4Py to parallelize preprocessing. May be useful when writing in parallelization to SeisFlows