/ERRNet

Single Image Reflection Removal Exploiting Misaligned Training Data and Network Enhancements (CVPR 2019)

Primary LanguagePython

ERRNet

The implementation of CVPR 2019 paper "Single Image Reflection Removal Exploiting Misaligned Training Data and Network Enhancements"

News (19/09/2019): Fix the broken link; our pretrained model and collected unaligned dataset are now available at OneDrive

Highlights

  • Our network can extract the background image layer devoid of reflection artifacts, as in the example:

  • We captured a new dataset containing 450 unaligned image pairs that are considerably easier to collect. Image samples from our unaligned dataset are shown below:

  • We introduce a simple but powerful alignment-invariant loss function to facilitate exploiting misaligned real-world training data. Finetuning on unaligned image pairs with our loss leads to sharp and reflection-free results, in contrast to the blurry ones when using a conventional pixel-wise loss (L1, L2, e.t.c.). The resulting images finetuned by different losses are shown below: (Left: Pixel-wise loss; Right: Ours)

Prerequisites

  • Python >=3.5, PyTorch >= 0.4.1
  • Requirements: opencv-python, tensorboardX, visdom
  • Platforms: Ubuntu 16.04, cuda-8.0

Quick Start

1. Preparing your training/testing datasets

Training dataset

  • 7,643 cropped images with size 224 × 224 from Pascal VOC dataset (image ids are provided in VOC2012_224_train_png.txt, you should crop the center region with size 224 x 224 to reproduce our result).

  • 90 real-world training images from Berkeley real dataset

Testing dataset

Once the data are downloaded, you must organize the dataset according to our code implementation (see the source code of datasets.CEILDataset, e.t.c.)

2. Playing with aligned data

Testing

  • Download our pretrained model from OneDrive and move errnet_060_00463920.pt to checkpoints/errnet/.
  • Evaluate the model performance by python test_errnet.py --name errnet -r --icnn_path checkpoints/errnet/errnet_060_00463920.pt --hyper

Training

  • Reproduce our results by python train_errnet.py --name errnet --hyper
  • Check options/errnet/train_options.py to see more training options.

3. Playing with unaligned data

  • Reproduce our finetuned model by python train_errnet_unaligned.py --name errnet_unaligned_ft --hyper -r --icnn_path checkpoints/errnet/errnet_060_00463920.pt --unaligned_loss vgg

Citation

If you find our code helpful in your research or work please cite our paper.

 @inproceedings{wei2019single,
   title={Single Image Reflection Removal Exploiting Misaligned Training Data and Network Enhancements},
   author={Wei, Kaixuan and Yang, Jiaolong and Fu, Ying and David, Wipf and Huang, Hua},
   booktitle={IEEE Conference on Computer Vision and Pattern Recognition},
   year={2019},
 }

Contact

If you find any problem, please feel free to contact me (kxwei at princeton.edu kaixuan_wei at bit.edu.cn). A brief self-introduction is required, if you would like to get an in-depth help from me.

Acknowledgments