WhiteBox - Part1
The White Box Project is a project that introduces many ways to solve the part of the black box of machine learning. In this part, i've introduced and experimented with ways to interpret and evaluate models in the field of image.
I shared Korean versions for each reference to study methodology and English. Please refer to the reference.
참고자료별로 영어공부겸 한국어로 번역한 자료가 있습니다.
Keywords : Explainable AI, XAI, Interpretable AI, Attribution Method, Saliency Map
Requirements
pytorch >= 1.2.0
torchvision == 0.4.0
How to Run
Model Train
python main.py --train --target=['mnist','cifar10']
Model Selectivity Evaluation
python main.py --eval=selectivity --target=['mnist','cifar10'] --method=['VGB','IB','DeconvNet','IG','GB','GC','GBGC']
Model ROAR & KAR Evaluation
For ROAR and KAR, the saliency map of each attribution methods that you want to evaluate must be saved prior to the evaluation.
python main.py --eval=['ROAR','KAR'] --target=['mnist','cifar10'] --method=['VGB','IB','DeconvNet','IG','GB','GC','GBGC']
Dataset
- MNIST
- CIFAR-10
Saliency Maps
Attribution Methods
- Vanilla Backpropagation (VBP) [Notebook]
- Input x Backpropagation (IB) [Notebook]
- DeconvNet [1] [Notebook]
- Guided Backpropagation (GB) [2] [Notebook]
- Integrated Gradients (IG) [3] [Notebook]
- Grad-CAM (GC) [4] [Notebook]
- Guided Grad-CAM (GB-GC) [4] [Notebook]
Ensemble Methods
- SmoothGrad (SG) [5]
- SmoothGrad-Squared (SG-SQ) [6]
- SmoothGrad-VAR (SG-VAR) [6]
Evaluation Methods
- Coherence
- Selectivity
- Remove and Retrain (ROAR) [6]
- Keep and Retrain (KAR) [6]
Experiments
Model Architecture & Performance
[Notebook]
| Architecture | MNIST | CIFAR-10 |
|---|---|---|
 |
 |
 |
Evaluation Results
Coherence
[Notebook]
Coherence is a qualitative evaluation method that shows the importance of images. Attributions should fall on discriminative features (e.g. the object of interest).
MNIST
CIFAR-10
Selectivity
[Notebook]
Selecticity is a method for quantitative evaluation of the attribution methods. The evaluation method is largely divided into two courses. First, the feature map for the image is created and the most influential part is deleted from the image. The second is to create the feature map again with the modified image and repeat the first process.
As a result, IB, GB and GB-GC were the most likely attribution methods to degrade the performance of models for the two datasets.
MNIST
CIFAR-10
ROAR/KAR
ROAR/KAR is a method for quantitative evaluation of the attribution methods that how the performance of the classifier changes as features are removed based on the attribution method.
- ROAR : replace N% of pixels estimated to be most important [Notebook]
- KAR : replace N% of pixels estimated to be least important
- Retrain Model and measure change in test accuracy
Reference
-
[1] Zeiler, M. D., & Fergus, R. (2014, September). Visualizing and understanding convolutional networks. In European conference on computer vision (pp. 818-833). Springer, Cham. (Korean version)
-
[2] Springenberg, J. T., Dosovitskiy, A., Brox, T., & Riedmiller, M. (2014). Striving for simplicity: The all convolutional net. arXiv preprint arXiv:1412.6806.
-
[3] Sundararajan, M., Taly, A., & Yan, Q. (2017, August). Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference on Machine Learning-Volume 70 (pp. 3319-3328). JMLR. org.
-
[4] Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017). Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE International Conference on Computer Vision (pp. 618-626). (Korean version)
-
[5] Smilkov, D., Thorat, N., Kim, B., Viégas, F., & Wattenberg, M. (2017). Smoothgrad: removing noise by adding noise. arXiv preprint arXiv:1706.03825. (Korean version)
-
[6] Hooker, S., Erhan, D., Kindermans, P. J., & Kim, B. (2018). Evaluating feature importance estimates. arXiv preprint arXiv:1806.10758. (Korean version)