/nighttime_dehaze

[ACMMM2023] "Enhancing Visibility in Nighttime Haze Images Using Guided APSF and Gradient Adaptive Convolution", https://arxiv.org/abs/2308.01738

Primary LanguagePython

nighttime_dehaze (ACMMM'2023)

Introduction

Enhancing Visibility in Nighttime Haze Images Using Guided APSF and Gradient Adaptive Convolution
ACM International Conference on Multimedia (ACMMM2023)
Yeying Jin, Beibei Lin, Wending Yan, Yuan Yuan, Wei Ye, and Robby T. Tan

arXiv [Paper]

Prerequisites

git clone https://github.com/jinyeying/nighttime_dehaze.git
cd nighttime_dehaze/
conda create -n dehaze python=3.7
conda activate dehaze
conda install pytorch=1.10.2 torchvision torchaudio cudatoolkit=11.3 -c pytorch
python3 -m pip install scipy==1.7.3
python3 -m pip install opencv-python==4.4.0.46

Nighttime Haze Data

Data Dropbox BaiduPan Number & Type
RealNightHaze Dropbox BaiduPan code:r5mi 443, Haze
Internet_night_clean1 Dropbox BaiduPan code:m7k1 411, Clean Reference
Internet_night_clean2 Dropbox BaiduPan code:8f13 50, Clean Reference
GTA5 nighttime fog Dropbox BaiduPan code:67ml Train:787,Test:77, Synthetic

Synthetic Nighttime Haze and Clean Ground Truth

Nighttime Dehazing Results Dropbox | BaiduPan code:oovt

Model Dropbox BaiduPan Model Put in Path Results Dropbox Results BaiduPan
dehaze.pt dehaze.pt dehaze.pt code:n3t8 results/dehaze/model RealNightHaze RealNightHaze code:i43f
GTA5.pt GTA5.pt GTA5.pt code:fk29 results/GTA5/model GTA5 GTA5 code:ufen
NHR.pt NHR.pt NHR.pt code:dnhf results/NHR/model NHR NHR code:0nma
NHM.pt NHM.pt NHM.pt code:d7oj results/NHM/model NHM NHM code：4gt0
NHC.pt NHC.pt NHC.pt code:yryp results/NHC/model NHC NHC code：njf9

We provide the visualization results in 0_ACMMM23_RESULTS/NHR/index.html,
inside the directory 0_ACMMM23_RESULTS/NHR/img_0/ are hazy inputs, img_1 are ground truths, img_2 are our results.
For results corresponding to GTA5, NHM or NHC, please refer to the respective directories.

  • For the RealNightHaze Dataset
  1. Set the datasetpath to RealNightHaze,
  2. Download the checkpoint dehaze.pt Dropbox| BaiduPan code:n3t8 put in results/dehaze/model,
  3. Run the Python code, results are in results/dehaze/output.
CUDA_VISIBLE_DEVICES=1 python main_test.py --dataset dehaze --datasetpath /diskc/yeying/night_dehaze/dataset/Internet_night_fog/

  • For the Synthetic Dataset
  1. Set Line18 --have_gt to True, set the datasetpath to GTA5 or NHR or NHM or NHC,
  2. Download the checkpoint GTA5.pt, put in results/GTA5/model. Similarly, for NHR.pt, NHM.pt, NHC.pt,
  3. Run the Python code,
CUDA_VISIBLE_DEVICES=1 python main_test.py --dataset NHM --datasetpath /diskc/yeying/night_dehaze/dataset/middlebury/testA/ 
CUDA_VISIBLE_DEVICES=1 python main_test.py --dataset NHC --datasetpath /diskc/yeying/night_dehaze/dataset/Cityscape/testA/ 
CUDA_VISIBLE_DEVICES=1 python main_test.py --dataset NHR --datasetpath /diskc/yeying/night_dehaze/dataset/NHR/testA/ 
CUDA_VISIBLE_DEVICES=1 python main_test.py --dataset GTA5 --datasetpath /diskc/yeying/night_dehaze/GTA5/testA/
  • Evaluation: Set the dataset_name GTA5 or NHR or NHM or NHC, and run the Python code:
python calculate_psnr_ssim_NH_GTA5.py
Dataset PSNR SSIM
GTA5 30.383 0.9042
NHR 26.56 0.89
NHM 33.76 0.92
NHC 38.86 0.97

APSF-Guided Nighttime Glow Rendering

Run the Matlab code to obtain the clean and glow pairs:

APSF_GLOW_RENDER_CODE/synthetic_glow_pairs.m

Change the datapath nighttime_dehaze/paired_data/clean_data/,
the paired clean and glow results are saved in nighttime_dehaze/paired_data/clean/ and nighttime_dehaze/paired_data/glow/,
the visualization of light source maps are in nighttime_dehaze/paired_data/glow_render_visual/light_source/.

Run the Matlab code to visualize Fig.3 in the main paper:

APSF_GLOW_RENDER_CODE/synthetic_glow_fig3_visualization.m

APSF and Alpha Matting are the implementations of the papers:

  • CVPR03 Shedding Light on the Weather [Paper]
  • CVPR06 A Closed-Form Solution to Natural Image Matting [Paper]

Edge

Run the Python code to visualize Fig.6, the environment is Pytorch 1.9 with cuda 10.1 and cudnn 7.5, results are in EDGE/results.

conda install pytorch==1.9.0 torchvision==0.10.0 torchaudio==0.9.0 cudatoolkit=10.1 -c pytorch

python main.py --sa --dil --gpu 1 --datadir ./Input/ --evaluate-converted

Enhancement

Run the Matlab code to visualize Fig.8, results are in ENHANCEMENT/attention_map.

Run the Matlab code to visualize Fig.10.

License

The code and models in this repository are licensed under the MIT License for academic and other non-commercial uses.
For commercial use of the code and models, separate commercial licensing is available. Please contact:

Citation

If this work or the Internet data is useful for your research, please cite our paper.

@inproceedings{jin2023enhancing,
  title={Enhancing visibility in nighttime haze images using guided apsf and gradient adaptive convolution},
  author={Jin, Yeying and Lin, Beibei and Yan, Wending and Yuan, Yuan and Ye, Wei and Tan, Robby T},
  booktitle={Proceedings of the 31st ACM International Conference on Multimedia},
  pages={2446--2457},
  year={2023}
}

@inproceedings{jin2022unsupervised,
  title={Unsupervised night image enhancement: When layer decomposition meets light-effects suppression},
  author={Jin, Yeying and Yang, Wenhan and Tan, Robby T},
  booktitle={European Conference on Computer Vision},
  pages={404--421},
  year={2022},
  organization={Springer}
}