This repository provides the official implementation for the following two papers:
Flare7K++, the first comprehensive nighttime flare removal dataset, consists of 962 real-captured flare images (Flare-R) and 7,000 synthetic flares (Flare7K). Flare7K is generated based on the observation and statistic of real-world nighttime lens flares. It offers 5,000 scattering flare images and 2,000 reflective flare images, consisting of 25 types of scattering flares and 10 types of reflective flares. Flare-R is captured by smartphone rear cameras with different lens contaminants in the dark room. These flare patterns can be randomly added to the flare-free images, forming the flare-corrupted and flare-free image pairs.
- 2023.06.12: We released the training codes and checkpoints for our Flare7K++ [Baidu Netdisk/Google Drive]. More training details can be found at our new technical report: arxiv.
- 2023.06.08: We create a mixing dataset called Flare7K++ that augments the synthetic Flare7K dataset with a new real-captured Flare-R dataset.
- 2023.02.09: Our training code is released.
- 2022.12.28: The MIPI Workshop 2023 is released now. Our dataset serves as a track in this challenge. Please check the CodaLab page to find more details about our challenge.
- 2022.10.12: Upload a flare-corrupted test dataset without ground truth.
- 2022.10.11: Upload the dataset and pretrained model in Baidu Netdisk.
- 2022.10.09: Update baseline inference code for flare removal.
- 2022.09.16: Our paper Flare7K: A Phenomenological Nighttime Flare Removal Dataset is accepted by the NeurIPS 2022 Track Datasets and Benchmarks. 🤗
- 2022.08.27: Update dataloader for our dataset.
- 2022.08.25: Increase the number of test images from 20 to 100. Please download the latest version of our Flare7K dataset.
- 2022.08.19: This repo is created.
-
Clone the repo
git clone https://github.com/ykdai/Flare7K.git
-
Install dependent packages
cd Flare7K pip install -r requirements.txt -
Install Flare7K
Please run the following commands in the Flare7K root path to install Flare7K:python setup.py develop
| Baidu Netdisk | Google Drive | Number | Description | |
|---|---|---|---|---|
| Flare7K++(new) | link | link | 7,962 | Flare7K++ consists of Flare7K and Flare-R. Flare7K offers 5,000 scattering flare images and 2,000 reflective flare images, consisting of 25 types of scattering flares and 10 types of reflective flares. Flare-R offers 962 real-captured flare patterns. |
| Background Images | link | link | 23,949 | The background images are sampled from [Single Image Reflection Removal with Perceptual Losses, Zhang et al., CVPR 2018]. We filter our most of the flare-corrupted images and overexposed images. |
| Flare-corrupted images | link | link | 645 | We offer an extra flare-corrupted dataset without ground truth. It contains 645 images captured by different cameras and some images are very challenging. |
We provide a on-the-fly dataloader function and a flare-corrupted/flare-free pairs generation script in this repository. To use this function, please put the Flare7K dataset and 24K Flickr dataset on the same path with the generate_flare.ipynb file.
If you only want to generate the flare-corrupted image without reflective flare, you can comment out the following line:
# flare_image_loader.load_reflective_flare('Flare7K','Flare7k/Reflective_Flare')
The inference code based on Uformer is released Now. Your can download the pretrained checkpoints from the following links. Please place it under the experiments folder and unzip it, then you can run the test.py for inference. We provide two checkpoints for models training on Flare7K, the model in the folder uformer can help remove both the reflective flares and scattering flares. The uformer_noreflection one can only help remove the scattering flares but is more robust. Now, we prefer the users to test our new model trained on Flare7K++, it can achieve better results and more realistic light source.
| Training Data | Baidu Netdisk | Google Drive |
|---|---|---|
| Flare7K | link | link |
| Flare7K++ (new) | link | link |
To estimate the flare-free images with our checkpoint pretrained on Flare7K++, you can run the test.py or test_large.py (for image larger than 512*512) by using:
python test_large.py --input dataset/Flare7Kpp/test_data/real/input --output result/test_real/flare7kpp/ --model_path experiments/flare7kpp/net_g_last.pth --flare7kpp
If you use our checkpoint pretrained on Flare7K, please run:
python test_large.py --input dataset/Flare7Kpp/test_data/real/input --output result/test_real/flare7k/ --model_path experiments/flare7k/net_g_last.pth
To calculate different metrics with our pretrained model, you can run the evaluate.py by using:
python evaluate.py --input result/blend/ --gt dataset/Flare7Kpp/test_data/real/gt/ --mask dataset/Flare7Kpp/test_data/real/mask/
Training with single GPU
To train a model with your own data/model, you can edit the options/uformer_flare7k_option.yml and run the following codes. You can also add --debug command to start the debug mode:
python basicsr/train.py -opt options/uformer_flare7k_option.yml
If you want to use Flare7K++ for training, please use:
python basicsr/train.py -opt options/uformer_flare7kpp_baseline_option.yml
Training with multiple GPU
You can run the following command for the multiple GPU tranining:
CUDA_VISIBLE_DEVICES=0,1 bash scripts/dist_train.sh 2 options/uformer_flare7k_option.yml
If you want to use Flare7K++ for training, please use:
CUDA_VISIBLE_DEVICES=0,1 bash scripts/dist_train.sh 2 options/uformer_flare7kpp_baseline_option.yml
├── Flare7K
├── Reflective_Flare
├── Scattering_Flare
├── Compound_Flare
├── Glare_with_shimmer
├── Core
├── Light_Source
├── Streak
├── Flare-R
├── Compound_Flare
├── Light_Source
├── test_data
├── real
├── input
├── gt
├── mask
├── synthetic
├── input
├── gt
├── mask
In our Flare7K++ dataset, we also update light source annotations for Flare7K dataset, new version contains a tiny glare around the light source to increase the reality. If you want to use the old version, please use the core annotations.
This project is licensed under S-Lab License 1.0. Redistribution and use of the dataset and code for non-commercial purposes should follow this license.
If you find this work useful, please cite:
@inproceedings{dai2022flare7k,
title={Flare7K: A Phenomenological Nighttime Flare Removal Dataset},
author={Dai, Yuekun and Li, Chongyi and Zhou, Shangchen and Feng, Ruicheng and Loy, Chen Change},
booktitle={Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track},
year={2022}
}
@inproceedings{dai2023nighttime,
title={Nighttime Smartphone Reflective Flare Removal using Optical Center Symmetry Prior},
author={Dai, Yuekun and Luo, Yihang and Zhou, Shangchen and Li, Chongyi and Loy, Chen Change},
booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
year = {2023}
}
@article{dai2023flare7kpp,
title={Flare7K++: Mixing Synthetic and Real Datasets for Nighttime Flare Removal and Beyond},
author={Yuekun Dai and Chongyi Li and Shangchen Zhou and Ruicheng Feng and Yihang Luo and Chen Change Loy},
year={2023}
}
If you have any question, please feel free to reach me out at ydai005@e.ntu.edu.sg.