BNN-concepts

PyTorch implementation of the paper "Bayesian Neural Networks Avoid Encoding Complex and Perturbation-Sensitive Concepts" in ICML 2023 (pdf).

Requirements

Python 3.9
pytorch 1.7.1
CUDA 11.0
numpy 1.22.3
torchvision 0.8.2

Models were trained and tested on the NVIDIA GeForce RTX 3080 GPU.

Usage

Training

Use the following script to train the models.

./scripts/bash_train.sh

You can change the --gpu_id argument in the script to specify the index of GPU on your machine. Model parameters will be saved to the checkpoints folder by default.

Verifying the sensitivity of high-order interactions to perturbations

Use the following script to run the experiment and reproduce the results in Figure 3 in the paper.

./scripts/bash_verify_stability.sh

Results will be saved to the results/verify_stability folder by default.

Demonstrating BNNs are less likely to encode high-order interactions

Use the following script to run the experiment and reproduce the results in Figure 4 in the paper.

./scripts/bash_compare_BNN_with_std_DNN.sh

Results will be saved to the results/compare_BNN_with_std_DNN folder by default.

Citation

@inproceedings{ren2023bayesian,
  title={Bayesian Neural Networks Avoid Encoding Complex and Perturbation-Sensitive Concepts},
  author={Ren, Qihan and Deng, Huiqi and Chen, Yunuo and Lou, Siyu and Zhang, Quanshi},
  booktitle={International Conference on Machine Learning},
  pages={28889--28913},
  year={2023},
  volume={202},
  series={Proceedings of Machine Learning Research},
  month={23--29 Jul},
  publisher={PMLR},
}