M3d-CAM

M3d-CAM is an easy to use PyTorch library that allows the generation of 3D/ 2D attention maps for both classification and segmentation with multiple methods such as Guided Backpropagation, Grad-Cam, Guided Grad-Cam and Grad-Cam++.
All you need to add to your project is a single line of code:

model = medcam.inject(model, output_dir="attention_maps", save_maps=True)

Features

Works with classification and segmentation data / models
Works with 2D and 3D data
Supports Guided Backpropagation, Grad-Cam, Guided Grad-Cam and Grad-Cam++
Attention map evaluation with given ground truth masks
Option for automatic layer selection

Installation

Install PyTorch from https://pytorch.org/get-started/locally/
Install M3d-CAM via pip with: pip install medcam
Install M3d-CAM via conda with: conda install -c conda-forge medcam

Documentation

M3d-CAM is fully documented and you can view the documentation under:
https://meclabtuda.github.io/M3d-Cam/

Examples

	#1 Classification (2D)	#2 Segmentation (2D)	#3 Segmentation (3D)
Image
Guided backpropagation
Grad-Cam
Guided Grad-Cam
Grad-Cam++

Usage

# Import M3d-CAM
from medcam import medcam

# Init your model and dataloader
model = MyCNN()
data_loader = DataLoader(dataset, batch_size=1, shuffle=False)

# Inject model with M3d-CAM
model = medcam.inject(model, output_dir="attention_maps", save_maps=True)

# Continue to do what you're doing...
# In this case inference on some new data
model.eval()
for batch in data_loader:
    # Every time forward is called, attention maps will be generated and saved in the directory "attention_maps"
    output = model(batch)
    # more of your code...

Usage through GaNDLF

The Generally Nuanced Deep Learning Framework (GaNDLF) [ref] provides an end-to-end solution for various deep learning workloads (segmentation, regression, classification, and more) and M3d-CAM can be called from GaNDLF directly via its configuration [ref] in an extremely simple manner:

medcam: 
{
  backend: "gcam",
  layer: "auto"
}

Demos

Classification

You can find a Jupyter Notebook on how to use M3d-CAM for classification using a resnet152 at demos/Medcam_classification_demo.ipynb or open it directly in Google Colab:

2D Segmentation

TODO

3D Segmentation

You can find a Jupyter Notebook on how to use M3d-CAM with the nnUNet for handeling 3D data at demos/Medcam_nnUNet_demo.ipynb or open it directly in Google Colab:

Citations

If you are using M3D-CAM for your article, please consider citing this tech report and the following papers:

@misc{2007.00453,
Author = {Karol Gotkowski and Camila Gonzalez and Andreas Bucher and Anirban Mukhopadhyay},
Title = {M3d-CAM: A PyTorch library to generate 3D data attention maps for medical deep learning},
Year = {2020},
Eprint = {arXiv:2007.00453},
}

@inproceedings{selvaraju2017grad,
  title={Grad-cam: Visual explanations from deep networks via gradient-based localization},
  author={Selvaraju, Ramprasaath R and Cogswell, Michael and Das, Abhishek and Vedantam, Ramakrishna and Parikh, Devi and Batra, Dhruv},
  booktitle={Proceedings of the IEEE international conference on computer vision},
  pages={618--626},
  year={2017}
}

@inproceedings{chattopadhay2018grad,
  title={Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks},
  author={Chattopadhay, Aditya and Sarkar, Anirban and Howlader, Prantik and Balasubramanian, Vineeth N},
  booktitle={2018 IEEE Winter Conference on Applications of Computer Vision (WACV)},
  pages={839--847},
  year={2018},
  organization={IEEE}
}

MECLabTUDA/M3d-Cam