Adverse Weather Image Translation with Asymmetric and Uncertainty-aware GAN (AU-GAN)

Official Tensorflow implementation of Adverse Weather Image Translation with Asymmetric and Uncertainty-aware GAN (AU-GAN)
Jeong-gi Kwak, Youngsaeng Jin, Yuanming Li, Dongsik Yoon, Donghyeon Kim and Hanseok Ko
British Machine Vision Conference (BMVC), 2021

Intro

Night → Day (BDD100K)

Rainy night → Day (Alderdey)

Architecture

Our generator has asymmetric structure for editing day→night and night→day. Please refer our paper for details

Envs

git clone https://github.com/jgkwak95/AU-GAN.git
cd AU-GAN

# Create virtual environment
conda create -y --name augan python=3.6.7
conda activate augan

conda install tensorflow-gpu==1.14.0   # Tensorflow 1.14
pip install --no-cache-dir -r requirements.txt

Preparing datasets

Night → Day
Berkeley DeepDrive dataset contains 100,000 high resolution images of the urban roads for autonomous driving.

Rainy night → Day
Alderley dataset consists of images of two domains, rainy night and daytime. It was collected while driving the same route in each weather environment.

Please download datasets and then construct them following ForkGAN

Pretrained Model

Download the pretrained model for BDD100K(256x512) here and unzip it to ./check/bdd_exp/bdd100k_256/

Training

# Alderley (256x512)
python main_uncer.py --dataset_dir alderley
                     --phase train
                     --experiment_name alderley_exp
                     --batch_size 8 
                     --load_size 286 
                     --fine_size 256 
                     --use_uncertainty True

# BDD100k (256x512)
python main.py --dataset_dir bdd100k  --phase train --experiment_name my_train_5_epoch --batch_size 2  --load_size 286 --fine_size 128 --epoch 5 --save_freq 5 --print_freq 2 --use_uncertainty True

# AJDATASET01 (256x512)
python main.py --dataset_dir AJDATASET01  --phase train --experiment_name pure_data_same_model_as_BDD --batch_size 2  --load_size 286 --fine_size 128 --epoch 10 --save_freq 5 --print_freq 2 --use_uncertainty True --continue_train 1

python main.py --dataset_dir AJDATASET01  --phase train --experiment_name pure_data_same_model_as_BDD_NU --batch_size 2  --load_size 286 --fine_size 128 --epoch 10 --save_freq 5 --print_freq 2 --use_uncertainty False --continue_train 1

python main.py --dataset_dir AJDATASET02  --phase train --experiment_name augmented_ajdataset --batch_size 2  --load_size 286 --fine_size 128 --epoch 10 --save_freq 10 --print_freq 2 --use_uncertainty True --continue_train 1

python main.py --dataset_dir AJDATASET02  --phase train --experiment_name augmented_ajdataset_NU --batch_size 2  --load_size 286 --fine_size 128 --epoch 10 --save_freq 10 --print_freq 2 --use_uncertainty False --continue_train 1

python main.py --dataset_dir AJDATASET01  --phase train --experiment_name same_load_and_fine_size --batch_size 2  --load_size 150 --fine_size 128 --epoch 20 --save_freq 10 --print_freq 2 --use_uncertainty True --continue_train 0

python main.py --dataset_dir AJDATASET02  --phase train --experiment_name same_load_and_fine_size02 --batch_size 2  --load_size 143 --fine_size 128 --epoch 20 --save_freq 20 --print_freq 5 --use_uncertainty True --continue_train 0

python main.py --dataset_dir AJDATASET01  --phase train --experiment_name same_load_and_fine_size_better_resol --batch_size 1  --load_size 286 --fine_size 256 --epoch 20 --save_freq 10 --print_freq 10 --use_uncertainty True --continue_train 0

Test

# Alderley (256x512)
python main_uncer.py --dataset_dir alderley
                     --phase test
                     --experiment_name alderley_exp
                     --batch_size 1 
                     --load_size 286 
                     --fine_size 256

# BDD100k (256x512)
python main.py --dataset_dir bdd100k --phase test --experiment_name bdd_exp --batch_size 1 --load_size 286 --fine_size 256

Additional results

More results in paper and supplementary

Uncertainty map

Citation

If our code is helpful your research, please cite our paper:

@article{kwak2021adverse,
  title={Adverse weather image translation with asymmetric and uncertainty-aware GAN},
  author={Kwak, Jeong-gi and Jin, Youngsaeng and Li, Yuanming and Yoon, Dongsik and Kim, Donghyeon and Ko, Hanseok},
  journal={arXiv preprint arXiv:2112.04283},
  year={2021}
}

Acknowledgments

Our code is bulided upon the ForkGAN implementation.

jirayuwat12/AU-GAN