medigan: A Python repository from cdchushig

medigan: medical generative (adversarial) networks

❌ Problem 1: Data scarcity in medical imaging.
❌ Problem 2: Scarcity of readily reusable generative models in medical imaging.
✅ Solution: medigan
1. dataset sharing via generative models 🎁
2. data augmentation 🎁
3. domain adaptation 🎁
4. synthetic data evaluation method testing with multi-model datasets 🎁

medigan provides functions for sharing and re-use of pretrained generative models in medical imaging.

Features:

Instead of training your own, use one a generative models from medigan to generate synthetic data.
Search and find a model in medigan using search terms (e.g. "Mammography" or "Endoscopy").
Contribute your own generative model to medigan to increase its visibility, re-use, and impact.

Available models

Output type	Modality	Model type	Output size	Base dataset	`model_id`	Hosted on	Reference
_{Breast Calcification}	_mammography	_dcgan	_128x128	_Inbreast	_{00001_DCGAN_MMG_CALC_ROI}	_{Zenodo (5187714)}
_{Breast Mass}	_mammography	_dcgan	_128x128	_Optimam	_{00002_DCGAN_MMG_MASS_ROI}	_{Zenodo (5188557)}	_{Alyafi et al (2019)}
_{Breast Density Transfer}	_mammography	_cyclegan	_1332x800	_BCDR	_{00003_CYCLEGAN_MMG_DENSITY_FULL}	_{Zenodo (5547263)}	_{Garrucho et al (2022)}
_{Breast Mass with Mask}	_mammography	_pix2pix	_256x256	_BCDR	_{_{00004_PIX2PIX_MMG_MASSES_W_MASKS}}	_{Zenodo (7093759)}
_{Breast Mass}	_mammography	_dcgan	_128x128	_BCDR	_{00005_DCGAN_MMG_MASS_ROI}	_{Zenodo (6555188)}	_{Szafranowska et al (2022)}
_{Breast Mass}	_mammography	_wgan-gp	_128x128	_BCDR	_{00006_WGANGP_MMG_MASS_ROI}	_{Zenodo (6554713)}	_{Szafranowska et al (2022)}
_{Brain Tumors on Flair, T1, T1c, T2 with Masks}	_{brain MRI}	_{inpaint GAN}	_256x256	_{BRATS 2018}	_{00007_INPAINT_BRAIN_MRI}	_{Zenodo (7041737)}	_{Kim et al (2020)}
_{Breast Mass (Mal/Benign)}	_mammography	_c-dcgan	_128x128	_CBIS-DDSM	_{00008_C-DCGAN_MMG_MASSES}	_{Zenodo (6647349)}
_{Polyp with Mask}	_endoscopy	_pggan	_256x256	_HyperKvasir	_{00009_PGGAN_POLYP_PATCHES_W_MASKS}	_{Zenodo (6653743)}	_{Thambawita et al (2022)}
_{Polyp with Mask}	_endoscopy	_fastgan	_256x256	_HyperKvasir	_{00010_FASTGAN_POLYP_PATCHES_W_MASKS}	_{Zenodo (6660711)}	_{Thambawita et al (2022)}
_{Polyp with Mask}	_endoscopy	_singan	_≈250x250	_HyperKvasir	_{00011_SINGAN_POLYP_PATCHES_W_MASKS}	_{Zenodo (6667944)}	_{Thambawita et al (2022)}
_{Breast Mass (Mal/Benign)}	_mammography	_c-dcgan	_128x128	_BCDR	_{00012_C-DCGAN_MMG_MASSES}	_{Zenodo (6755693)}
_{Breast Density Transfer MLO}	_mammography	_cyclegan	_1332x800	_OPTIMAM	_{00013_CYCLEGAN_MMG_DENSITY_OPTIMAM_MLO}	_{Zenodo (6818095)}	_{Garrucho et al (2022)}
_{Breast Density Transfer CC}	_mammography	_cyclegan	_1332x800	_OPTIMAM	_{00014_CYCLEGAN_MMG_DENSITY_OPTIMAM_CC}	_{Zenodo (6818103)}	_{Garrucho et al (2022)}
_{Breast Density Transfer MLO}	_mammography	_cyclegan	_1332x800	_CSAW	_{00015_CYCLEGAN_MMG_DENSITY_CSAW_MLO}	_{Zenodo (6818105)}	_{Garrucho et al (2022)}
_{Breast Density Transfer CC}	_mammography	_cyclegan	_1332x800	_CSAW	_{00016_CYCLEGAN_MMG_DENSITY_CSAW_CC}	_{Zenodo (6818107)}	_{Garrucho et al (2022)}
_{Lung Nodules}	_{chest x-ray}	_dcgan	_128x128	_NODE21	_{00017_DCGAN_XRAY_LUNG_NODULES}	_{Zenodo (6943691)}
_{Lung Nodules}	_{chest x-ray}	_wgan-gp	_128x128	_NODE21	_{00018_WGANGP_XRAY_LUNG_NODULES}	_{Zenodo (6943761)}
_{Chest Xray Images}	_{chest x-ray}	_pggan	_1024x1024	_ChestX-ray14	_{00019_PGGAN_CHEST_XRAY}	_{Zenodo (6943803)}
_{Chest Xray Images}	_{chest x-ray}	_pggan	_1024x1024	_ChestX-ray14	_{00020_PGGAN_CHEST_XRAY}	_{Zenodo (7046280)}	_{Segal et al (2021)}
_{Brain T1-T2 MRI Modality Transfer}	_{brain MRI}	_cyclegan	_224x192	_{CrossMoDA 2021}	_{00021_CYCLEGAN_BRAIN_MRI_T1_T2}	_{Zenodo (7074555)}	_{Joshi et al (2022)}
_{Cardiac MRI Age Transfer}	_{cardiac MRI}	_wgan	_256x256	_{UK Biobank}	_{00022_WGAN_CARDIAC_AGING}	_{Zenodo (7446930)}	_{Campello et al (2022)}

Model information can be found in:

model documentation (e.g. the parameters of the models' generate functions)
global.json file (e.g. metadata for model description, selection, and execution)
medigan paper (e.g. analysis and comparisons of models and FID scores)

Installation

To install the current release, simply run:

pip install medigan

Getting Started

Examples and notebooks are located at examples folder

Documentation is available at medigan.readthedocs.io

Generation example

DCGAN

Create mammography calcification images using DCGAN model

# import medigan and initialize Generators
from medigan import Generators
generators = Generators()

# generate 6 samples with model 1 (00001_DCGAN_MMG_CALC_ROI). 
# Also, auto-install required model dependencies.
generators.generate(model_id=1, num_samples=6, install_dependencies=True)

CYCLEGAN

Create mammograms translated from Low-to-High Breast Density using CYCLEGAN model

from medigan import Generators
generators = Generators()
# model 3 is "00003_CYCLEGAN_MMG_DENSITY_FULL"
generators.generate(model_id=3, num_samples=1)

→

Search Example

Search for a model inside medigan using keywords

# import medigan and initialize Generators
from medigan import Generators
generators = Generators()

# list all models
print(generators.list_models())

# search for models that have specific keywords in their config
keywords = ['DCGAN', 'Mammography', 'BCDR']
results = generators.find_matching_models_by_values(keywords)

Get Model as Dataloader

We can directly receive a torch.utils.data.DataLoader object for any of medigan's generative models.

from medigan import Generators
generators = Generators()
# model 4 is "00004_PIX2PIX_MMG_MASSES_W_MASKS"
dataloader = generators.get_as_torch_dataloader(model_id=4, num_samples=3)

Visualize the contents of the dataloader.

from matplotlib import pyplot as plt
import numpy as np

plt.figure()
# subplot with 2 rows and len(dataloader) columns
f, img_array = plt.subplots(2, len(dataloader)) 

for batch_idx, data_dict in enumerate(dataloader):
    sample = np.squeeze(data_dict.get("sample"))
    mask = np.squeeze(data_dict.get("mask"))
    img_array[0][batch_idx].imshow(sample, interpolation='nearest', cmap='gray')
    img_array[1][batch_idx].imshow(mask, interpolation='nearest', cmap='gray')
plt.show()

Visualize A Model

With our interface, it is possible to generate sample by manually setting the conditional inputs or latent vector values. The sample is updated in realtime, so it's possible to observe how the images changes when the parameters are modified. The visualization is available only for models with accessible input latent vector. Depending on a model, a conditional input may be also available or synthetic segmentation mask.

from medigan import Generators

generators = Generators()
# model 10 is "00010_FASTGAN_POLYP_PATCHES_W_MASKS"
generators.visualize(10)

Contribute A Model

Create an init.py file in your model's root folder.

Next, run the following code to contribute your model to medigan.

Your model will be stored on Zenodo.
Also, a Github issue will be created to add your model's metadata to medigan's global.json.
To do so, please provide a github access token (get one here) and a zenodo access token (get one here), as shown below. After creation, the zenodo access token may take a few minutes before being recognized in zenodo API calls.

from medigan import Generators
generators = Generators()

# Contribute your model
generators.contribute(
    model_id = "00100_YOUR_MODEL", # assign an ID
    init_py_path ="path/ending/with/__init__.py",
    model_weights_name = "10000",
    model_weights_extension = ".pt",
    generate_method_name = "generate", # in __init__.py
    dependencies = ["numpy", "torch"], 
    creator_name = "YOUR_NAME",
    creator_affiliation = "YOUR_AFFILIATION",
    zenodo_access_token = 'ZENODO_ACCESS_TOKEN',
    github_access_token = 'GITHUB_ACCESS_TOKEN',

Thank you for your contribution!

You will soon receive a reply in the Github issue that you created for your model by running generators.contribute().

Contributions in General

We welcome contributions to medigan. Please send us an email or read the contributing guidelines regarding contributing to the medigan project.

Reference

If you use a medigan model in your work, please cite its respective publication (see references).

Please also consider citing the medigan paper:

R. Osuala, G. Skorupko, N. Lazrak, et al., “medigan: A Python Library of Pretrained Generative Models for Enriched Data Access in Medical Imaging,” arXiv preprint arXiv:2209.14472 (2022).

BibTeX entry:

@article{osuala2022medigan,
  title = {{medigan: A Python Library of Pretrained Generative Models for Enriched Data Access in Medical Imaging}},
  shorttitle = {{medigan}},
  doi = {10.48550/ARXIV.2209.14472},
  url = {https://arxiv.org/abs/2209.14472},
  journal={arXiv preprint arXiv:2209.14472},
  author = {Osuala, Richard and Skorupko, Grzegorz and Lazrak, Noussair and Garrucho, Lidia and García, Eloy and Joshi, Smriti and Jouide, Socayna and Rutherford, Michael and Prior, Fred and Kushibar, Kaisar and Diaz, Oliver and Lekadir, Karim},
  year = {2022}
}

cdchushig/medigan