/Model-Guidance

Code for the paper: Studying How to Efficiently and Effectively Guide Models with Explanations. ICCV 2023.

Primary LanguagePythonMIT LicenseMIT

Studying How to Efficiently and Effectively Guide Models with Explanations

Sukrut Rao* · Moritz Böhle* · Amin Parchami-Araghi · Bernt Schiele

IEEE/CVF International Conference on Computer Vision (ICCV) 2023

Paper | Code | Video

Setup

Prerequisites

All the required packages can be installed using conda with the provided environment.yml file.

Data

Scripts to download and preprocess the VOC2007 and COCO2014 datasets have been provided in the datasets directory. Please refer to the README file provided there.

ImageNet Pre-trained Weights

A script to download the pre-trained ImageNet weights for B-cos and X-DNN backbones has been provided in the weights directory. Please refer to the README file provided there.

Training Models

To train a model, use:

python train.py [options]

The list of options and their descriptions can be found by using:

python train.py -h

Training without Model Guidance

For example, to train a B-cos model on VOC2007, use:

python train.py --model_backbone bcos --dataset VOC2007 --learning_rate 1e-4 --train_batch_size 64 --total_epochs 300

Fine-tuning with Model Guidance

For example, to optimize B-cos attributions using the Energy loss at the Input layer, use:

python train.py --model_backbone bcos --dataset VOC2007 --learning_rate 1e-4 --train_batch_size 64 --total_epochs 50 --optimize_explanations --model_path models/VOC2007/bcos_standard_attrNone_loclossNone_origNone_resnet50_lr1e-04_sll1.0_layerInput/model_checkpoint_f1_best.pt --localization_loss_lambda 1e-3 --layer Input --localization_loss_fn Energy --pareto

Code for training on the Waterbirds-100 dataset and scripts for visualizing explanations will be added soon.

Acknowledgements

This repository uses and builds upon code from the following repositories:

Citation

Please cite our paper as follows:

@InProceedings{Rao_2023_ICCV,
    author    = {Rao, Sukrut and B\"ohle, Moritz and Parchami-Araghi, Amin and Schiele, Bernt},
    title     = {Studying How to Efficiently and Effectively Guide Models with Explanations},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    month     = {October},
    year      = {2023},
    pages     = {1922-1933}
}