RH-Lin/m3ixup

Official Pytorch Implementation of "Adapt and Explore: Multimodal Mixup for Representation Learning"

Multimodal Mixup for Representation Learning

Official Implementation of "Adapt and Explore: Multimodal Mixup for Representation Learning"

Table of Contents

• Setup
• Download Datasets
• Download Pre-trained Models
• Evaluation
• Train
• Acknowledgments

Setup

conda env create -f m3ixup.yml
conda activate m3ixup
poetry install

Download Datasets

cd download/
bash download_unsupervised_dataset.sh
bash download_supervised_dataset.sh
bash download_rl_dataset.sh

Download Pre-trained Models

To replicate the results, download the pretrained models:

cd download/
bash download_unsupervised_pretrain_m3ixup_models.sh
bash download_supervised_pretrain_m3ixup_models.sh
bash download_rl_pretrain_m3ixup_models.sh

Acknowledgments

We would like to express our gratitude to huggingface and gmc, which are of great help to our work.