Contrastive Multimodal Fusion with TupleInfoNCE
ICCV 2021
Yunze Liu, Qingnan Fan, Shanghang Zhang, Hao Dong, Thomas Funkhouser, Li Yi*
Official PyTorch code for TupleInfoNCE, a novel contrastive learning objective for multi-modal learning. It contrasts tuples based not only on positive and negative correspondences but also by composing new negative tuples using modalities describing different scenes.
If you find this code useful for your research, please consider citing:
@inproceedings{liu2021contrastive,
title={Contrastive multimodal fusion with tupleinfonce},
author={Liu, Yunze and Fan, Qingnan and Zhang, Shanghang and Dong, Hao and Funkhouser, Thomas and Yi, Li},
booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
pages={754--763},
year={2021}
}
Our project structure is borrowed from ESANet, so we thank the authors for their wonderful work. We also thank the anonymous reviewers for their constructive feedback.
MIT