CAIR: Multi-Scale Color Attention Network for Instagram Filter Removal (ECCV 2022 Workshop)

Woon-Ha Yeo, Wang-Taek Oh, Kyung-Su Kang, Young-Il Kim, Han-Cheol Ryu

Challenge

AIM 2022 Instagram Filter Removal Challenge

Filtered Image	Original Image	CAIR* (ours)	IFRNet, CVPRW2021	CIFR, CVPRW2022

Installation & Setting

This implementation is based on NAFNet.

python 3.9.5
pytorch 1.11.0
cuda 11.3

pip install -r requirements.txt
python setup.py develop --no_cuda_ext

CAIR

Data Prepration

You can download IFFI(Instagram Filter Fashion Image) dataset on challenge website

You can train on IFFI dataset by following these steps:

Datasets directory structure

  datasets
  └──IFFI
     └──IFFI-dataset-train
     |  └──0
     |  └──1
     |  └──2
     |  └──...
     └──IFFI-dataset-lr-train
     |  └──0
     |  └──1
     |  └──2
     |  └──...
     └──IFFI-dataset-lr-challenge-test-wo-gt
        └──0
        └──1
        └──2
        └──...

Your Instagram Filter Removal Challenge dataset to lmdb format. If you want to use ensemble learning, set --ensemble true.

python scripts/data/dataset_to_lmdb.py --basedir ./datasets/IFFI --ensemble false
python scripts/data/dataset_to_lmdb.py --basedir ./datasets/IFFI --ensemble true

Result data into submission format

python scripts/data/test_dataset.py --resultdir ./results/model_name

Train/Test

If you want to train/test other model, replace option file with others in options/ folder.

Train

python -m torch.distributed.launch --nproc_per_node=4 --master_port=4321 basicsr/train.py -opt options/train/CAIR/CAIR_M-width32.yml --launcher pytorch

Test

# General test
python -m torch.distributed.launch --nproc_per_node=4 --master_port=4321 basicsr/test.py -opt options/test/CAIR/CAIR_M-width32.yml --launcher pytorch
#  TTA
python -m torch.distributed.launch --nproc_per_node=4 --master_port=4321 basicsr/test.py -opt options/test/CAIR-TTA/CAIR_M-width32.yml --launcher pytorch

Ensemble learning

Data preparation

For ensemble learning, train and test set should be inferenced with three models, and then three images from three models should be contatenated.

# CAIR_M-width32
python -m torch.distributed.launch --nproc_per_node=4 --master_port=4321 basicsr/test.py -opt ./options/test/Ensemble/CAIR_M-width32.yml --launcher pytorch
# CAIR_S-width32
python -m torch.distributed.launch --nproc_per_node=4 --master_port=4319 basicsr/test.py -opt ./options/test/Ensemble/CAIR_S-width32.yml --launcher pytorch
python scripts/data/concat_ensemble_input.py

Train ensemble network

python -m torch.distributed.launch --nproc_per_node=4 --master_port=4321 basicsr/train.py -opt options/train/Ensemble/CAIR_Ensemble.yml --launcher pytorch

Test ensemble network

python -m torch.distributed.launch --nproc_per_node=4 --master_port=4321 basicsr/test.py -opt options/test/Ensemble/CAIR_Ensemble.yml --launcher pytorch

Results

name	PSNR	SSIM	pretrained models
CAIR-S	33.87	0.970	Synology drive
CAIR-M	34.39	0.971	Synology drive
CAIR-Ensemble(CAIR*)	34.42	0.972	Synology drive

Citation

@article{yeo2022cair,
  title={CAIR: Fast and Lightweight Multi-Scale Color Attention Network for Instagram Filter Removal},
  author={Yeo, Woon-Ha and Oh, Wang-Taek and Kang, Kyung-Su and Kim, Young-Il and Ryu, Han-Cheol},
  journal={arXiv preprint arXiv:2208.14039},
  year={2022}
}

Contacts

If you have any question, please contact mm074111@gmail.com

Acknowledgements

Expand

hnvlab-syu/CAIR