Evaluation Framework for Multimodal Biomedical Image Registration Methods

Code of the paper Is Image-to-Image Translation the Panacea for Multimodal Image Registration? A Comparative Study (arXiv)

Open-access data: Datasets for Evaluation of Multimodal Image Registration

Overview

This repository provides an open-source quantitative evaluation framework for multimodal biomedical registration, aiming to contribute to the openness and reproducibility of future research.

evaluate.py is the main script to call the registration methods and calculate their performance.
./Datasets/ contains detailed descriptions of the evaluation datasets, and instructions and scripts to customise them.
The *.sh scripts provide examples to set large-scale evaluations.
plot.py and show_samples.py can be used to plot the registration performance and visualise the modality-translation results (see paper for examples).
Each folder contains the modified implementation of a method, whose compatibility with this evaluation framework is tested (see paper for details).
Other files should be self-explanatory, otherwise, please open an issue.

Usage

Image-to-Image translation

pix2pix and CycleGAN: run commands_*.sh to train and predict_*.sh to translate

# train and test 
cd pytorch-CycleGAN-and-pix2pix/
./commands_{dataset}.sh {fold} {gpu_id} # no {fold} for Histological data

# modality mapping of evaluation data
# {Dataset}_patches -> {Dataset}_patches_fake
./predict_{dataset}.sh

# for RIRE dataset
# RIRE_temp -> RIRE_slices_fake
./predict_rire.sh

DRIT++: run commands_*.sh to train and predict_all.sh to translate

# train and test 
cd ../DRIT/src/
./commands_{dataset}.sh

# modality mapping of evaluation data
# {Dataset}_patches -> {Dataset}_patches_fake
./predict_{dataset}.sh

# for RIRE dataset
# ../../pytorch-CycleGAN-and-pix2pix/datasets/rire_cyc_train -> RIRE_slices_fake
./predict_rire.sh

StarGANv2: run commands_*.sh to train and predict_all.sh to translate

# train (for all datasets)
cd ../stargan-v2/
./commands_{dataset}.sh {fold} {gpu_id} # no {fold} for Histological data

# test
# modality mapping of evaluation data
# {Dataset}_patches -> {Dataset}_patches_fake
./predict_{dataset}.sh

# for RIRE dataset
# RIRE_temp -> RIRE_slices_fake
./predict_rire.sh

CoMIR: run commands_train.sh and predict_all.sh

# train and test (for all datasets)
cd ../CoMIR/
./commands_train.sh

# modality mapping of evaluation data
# {Dataset}_patches -> {Dataset}_patches_fake
./predict_all.sh {gpu_id}

Evaluate registration performance

Run python evaluate.py -h or python evaluate_3D.py -hto see the options.

Dependencies

environment.yml includes the full list of packages used to run most of the experiments. Some packages might be unnecessary. And here are some exceptions:

SimpleElastix is required to compute the Mutual Information baseline performance.
For CoMIR, to reduce GPU memory usage, the inference on GPU requires pytorch>=1.6 to use the Automatic Mixed Precision package, otherwise it uses half-precision.

Citation

Please consider citing our paper and dataset if you find the code useful for your research.

@article{luImagetoImageTranslationPanacea2021,
  title = {Is {{Image}}-to-{{Image Translation}} the {{Panacea}} for {{Multimodal Image Registration}}? {{A Comparative Study}}},
  shorttitle = {Is {{Image}}-to-{{Image Translation}} the {{Panacea}} for {{Multimodal Image Registration}}?},
  author = {Lu, Jiahao and {\"O}fverstedt, Johan and Lindblad, Joakim and Sladoje, Nata{\v s}a},
  year = {2022},
  month = nov,
  journal = {PLOS ONE},
  volume = {17},
  number = {11},
  pages = {e0276196},
  issn = {1932-6203},
  doi = {10.1371/journal.pone.0276196},
  langid = {english}
}

@datasettype{luDatasetsEvaluationMultimodal2021,
  title = {Datasets for {{Evaluation}} of {{Multimodal Image Registration}}},
  author = {Lu, Jiahao and {\"O}fverstedt, Johan and Lindblad, Joakim and Sladoje, Nata{\v s}a},
  year = {2021},
  month = apr,
  publisher = {{Zenodo}},
  doi = {10.5281/zenodo.5557568},
  language = {eng}
}

MIDA-group/MultiRegEval