aramis-lab/clinica

Enable debugging for converters using multiprocessing

Closed this issue · 0 comments

Some of our converters like ADNI-to-BIDS use multiprocessing when converting images using a Pool of workers where the number of workers is set to n_procs:

with Pool(processes=n_procs) as pool:
create_file_ = partial(
create_file,
modality=modality,
bids_dir=bids_dir,
mod_to_update=mod_to_update,
)
output_file_treated = pool.map(create_file_, images_list)

When setting n_procs to one, this still creates a Pool with one worker which prevent using the classical Python debugger for example.

I believe we should only create the Pool of workers when n_jprocs >= 2, and run the conversion sequentially when n_procs==1:

create_file_ = partial(
    create_file,
    modality=modality,
    bids_dir=bids_dir,
    mod_to_update=mod_to_update,
)
output_file_treated = [create_file_(image) for image in images_list]

This wouldn't change the user experience, but would greatly improve the developer experience when debugging image conversions.