chingyaoc/RINCE

CVPR 2022, Robust Contrastive Learning against Noisy Views

Robust Contrastive Learning against Noisy Views

This repository provides a PyTorch implementation of the Robust InfoNCE loss proposed in paper Robust Contrastive Learning against Noisy Views.

Requirements:

• PyTorch >= 1.5.0

Pseudo Code

The implementation only requires a small modification to the InfoNCE code.

# bsz : batch size (number of positive pairs)
# pos : exponent for positive example, shape=[bsz]
# neg : sum of exponents for negative examples, shape=[bsz]
# q, lam : hyperparameters of RINCE

info_nce_loss = -log(pos / (pos + neg))
rince_loss = -pos**q / q + (lam * (pos + neg))**q / q

ImagNet Experiments

The code can be found in ImageNet/moco-v3 and ImageNet/simclr.

Citation

If you find this repo useful for your research, please consider citing the paper

@article{chuang2022robust,
  title={Robust Contrastive Learning against Noisy Views},
  author={Chuang, Ching-Yao and Hjelm, R Devon and Wang, Xin and Vineet, Vibhav and Joshi, Neel and Torralba, Antonio and Jegelka, Stefanie and Song, Yale},
  journal={arXiv preprint arXiv:2201.04309},
  year={2022}
}

For any questions, please contact Ching-Yao Chuang (cychuang@mit.edu).