/Fully_Unsupervised_CNN_Registration_Keras

Fully unsupervised 2D/3D image registration with ConvNet.

Primary LanguagePythonMIT LicenseMIT

2D/3D Medical Imaging Registration via Fully Unsupervised ConvNet

This is a Keras/Tensorflow implementation of my paper:

Chen, J., Li, Y., Du, Y. and Frey, E.C. (2020), Generating anthropomorphic phantoms using fully unsupervised deformable image registration with convolutional neural networks. Med. Phys., 47: 6366-6380. https://doi.org/10.1002/mp.14545

Chen, Junyu, et al. "A fully unsupervised approach to create patient-like phantoms via Convolutional neural networks." Journal of Nuclear Medicine 61.supplement 1 (2020): 522-522.

For most updated scripts (TensorFlow 2.X), see "CNN_MedPhy_TF2x/" folder.

"CNN_MedPhy_TF2x/img.zip" contains an example image pair of moving and fixed images. Extract "pat.pkl" to "CNN_MedPhy_TF2x/", then you should be able to run "CNN_MedPhy_TF2x/register3D_example.py" without changing code. It took about 16 GB of GPU memory for this 3D registration.

We treat CNN as an optimization tool that iteratively minimizes the loss function via reparametrization in each iteration. This means that the algorithm is fully unsupervised and thus no prior training is required. The registration loss function is defined as:

, where I_d and I_f are, respectively, the deformed and the fixed image, L_sim represents the loss function for image similarity, and R represents the regularization applied on the deformation field.

The effects of different loss functions:

The effects of different regularizations:

Sample results for XCAT phantom to and patient CT registration:

Some deformed phantom and SPECT simulations:

If you find this code is useful in your research, please consider to cite:

@article{https://doi.org/10.1002/mp.14545,
author = {Chen, Junyu and Li, Ye and Du, Yong and Frey, Eric C.},
title = {Generating anthropomorphic phantoms using fully unsupervised deformable image registration with convolutional neural networks},
journal = {Medical Physics},
volume = {47},
number = {12},
pages = {6366-6380},
keywords = {computerized Phantom, convolutional Neural Networks, deep Neural Networks, image Registration, medical Image Simulation},
doi = {https://doi.org/10.1002/mp.14545},
url = {https://aapm.onlinelibrary.wiley.com/doi/abs/10.1002/mp.14545},
eprint = {https://aapm.onlinelibrary.wiley.com/doi/pdf/10.1002/mp.14545},
year = {2020}
}