This demo is a part of the final project for the CS336 course - Multimedia Information Retrieval at UIT.
- Windows 11
- Python 3.11.7
To get a local copy up and running follow these simple steps.
- Clone the repo
git clone https://github.com/nganngants/CIR-demo
- Install Python dependencies
pip install -r requirements.txt
- Download CIRR datasets.
Here's a brief description of each and every file and folder in the repo:
utils.py
: Utils filemodel.py
: Combiner model definition filedata_utils.py
: Dataset loading and preprocessing utils fileextract_features.py
: Feature extraction filehubconf.py
: Torch Hub config fileapp.py
: Flask server filestatic
: Flask static files foldertemplates
: Flask templates folder
To properly work with the codebase CIRR datasets should have the following structure:
project_base_path
└─── cirr_dataset
└─── dev
| dev-0-0-img0.png
| dev-0-0-img1.png
| dev-0-1-img0.png
| ...
└─── test1
| test1-0-0-img0.png
| test1-0-0-img1.png
| test1-0-1-img0.png
| ...
└─── cirr
└─── captions
| cap.rc2.test1.json
| cap.rc2.train.json
| cap.rc2.val.json
└─── image_splits
| split.rc2.test1.json
| split.rc2.train.json
| split.rc2.val.json
Before launching the demo it is necessary to extract the features using the following command
python extract_features.py
Start the server and run the demo using the following command
python app.py
By default, the server run on port 5000 of localhost address: http://127.0.0.1:5000/
This project is based on the official repository of the paper Effective conditioned and composed image retrieval combining CLIP-based features
Our reproduction of the methods is based on the respective official repository, we thank the authors to release their code. If you use the related part, please cite the corresponding paper.
@inproceedings{baldrati2022effective,
title={Effective Conditioned and Composed Image Retrieval Combining CLIP-Based Features},
author={Baldrati, Alberto and Bertini, Marco and Uricchio, Tiberio and Del Bimbo, Alberto},
booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
pages={21466--21474},
year={2022}
}