Siyeong-Lee/Deconstructing-MINE

Official implementation of the paper "Combating the Instability of Mutual Information-based Losses via Regularization" (UAI 2022)

Combating the Instability of Mutual Information-based Losses via Regularization

• Accepted to UAI 2022
• https://openreview.net/forum?id=HFbf9PIjqgq
• This repository is the official implementation by the authors.
• Watch our poster or the 1 minute video presentation of this paper for more details.

Environment

• Python 3.7.4
• pip-20.1.1
• Additional packages can be located in requirements.txt and benchmark.requirements.txt (Only for the experiments_self_contrastive.py and experiments_classification.py.

How to reproduce experiments

One-hot experiments

• comparing_onehot.ipynb, observing_imine_different_target.ipynb

Gaussian benchmark

• Run with python3 experiments.py --batch_size 64 --loss {LOSS} --problem gaussian
• We modified code from https://colab.research.google.com/github/google-research/google-research/blob/master/vbmi/vbmi_demo.ipynb

Contrastive learning benchmark (CLB)

• CIFAR-100: Run with python3 experiments_self_contrastive.py --dataset cifar100 --loss {LOSS} --batch_size 100 --model resnet18 --epochs 150 --remove_fc --seed {SEED} --device 0
• CIFAR-10: Run with python3 experiments_self_contrastive.py --dataset cifar10 --loss {LOSS} --batch_size 10 --model resnet18 --epochs 100 --remove_fc --seed {SEED} --device 0

Supervised learning benchmark (CLB)

• CIFAR-100: Run with python3 experiments_classification.py --dataset cifar100 --loss {LOSS} --batch_size 100 --model resnet18 --epochs 100 --seed {SEED} --device 0
• CIFAR-10: Run with python3 experiments_classification.py --dataset cifar10 --loss {LOSS} --batch_size 10 --model resnet18 --epochs 40 --seed {SEED} --device 0