Implementation of "Deeply Coupled Cross-Modal Prompt Learning" in ACL 2023 findings

Installation

This codebase is tested on Ubuntu 20.04.2 LTS with python 3.8. Follow the below steps to create environment and install dependencies.

Setup conda environment (recommended).

# Create a conda environment
conda create -y -n cmpa python=3.8

# Activate the environment
conda activate cmpa

# Install torch (requires version >= 1.8.1) and torchvision
# Please refer to https://pytorch.org/ if you need a different cuda version
pip install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html

Install dassl library.

# Instructions borrowed from https://github.com/KaiyangZhou/Dassl.pytorch#installation

# Clone this repo
git clone https://github.com/KaiyangZhou/Dassl.pytorch.git
cd Dassl.pytorch/

# Install dependencies
pip install -r requirements.txt

# Install this library (no need to re-build if the source code is modified)
python setup.py develop
cd ..

Clone CMPA code repository and install requirements

# Clone CMPA code base
git clone https://github.com/GingL/CMPA.git

cd cmpa/
# Install requirements

pip install -r requirements.txt

Data preparation

Please follow the instructions at DATASETS.md to prepare all datasets.

Training and Evaluation

Please refer to the RUN.md for detailed instructions on training, evaluating and reproducing the results using our pre-trained models.

Acknowledgements

Our code is based on Co-CoOp and CoOp, MaPLe repository. We thank the authors for releasing their code. If you use our model and code, please consider citing these works as well.