IRVLab/funie-gan

Resources

• Implementations of FUnIE-GAN for underwater image enhancement
• Simplified implementations of UGAN and its variants (original repo)
• Modules for quantifying image quality based on UIQM, SSIM, and PSNR
• Implementation: TensorFlow >= 1.11.0, Keras >= 2.2, and Python 2.7
• This repository contains modules at the time of publication; sub-sequent updates can be found here

Perceptual enhancement	Color and sharpness	Hue and contrast

Enhanced underwater imagery	Improved detection and pose estimation

FUnIE-GAN Features

• Provides competitive performance for underwater image enhancement
• Offers real-time inference on single-board computers
  • 48+ FPS on Jetson AGX Xavier, 25+ FPS on Jetson TX2
  • 148+ FPS on Nvidia GTX 1080
• Suitable for underwater robotic deployments for enhanced vision

FUnIE-GAN Pointers

• Paper: https://ieeexplore.ieee.org/document/9001231
• Preprint: https://arxiv.org/pdf/1903.09766.pdf
• Datasets: http://irvlab.cs.umn.edu/resources/euvp-dataset
• Bibliography entry for citation:

   @article{islam2019fast,
       title={Fast Underwater Image Enhancement for Improved Visual Perception},
       author={Islam, Md Jahidul and Xia, Youya and Sattar, Junaed},
       journal={IEEE Robotics and Automation Letters (RA-L)},
       volume={5},
       number={2},
       pages={3227--3234},
       year={2020},
       publisher={IEEE}
   }
  

Usage

• Download the data, setup data-paths in the training-scripts
• Use paired training for FUnIE-GAN or UGAN, and unpaired training for FUnIE-GAN-up
  • Sample checkpoints: checkpoints/model-name/dataset-name
  • Data samples: data/samples/model-name/dataset-name
• Use the test-scripts for evaluating different models
  • A few test images: data/test/A (ground-truth: GTr_A), data/test/random (unpaired)
  • Output: data/output
• Use the measure.py for quantitative analysis based on UIQM, SSIM, and PSNR
• A few saved models are provided in saved_models/

Constraints and Challenges

• Issues with unpaired training (as discussed in the paper)
  • Inconsistent coloring, inaccurate modeling of sunlight
  • Often poor hue rectification (dominant blue/green hue)
  • Hard to achieve training stability
• Much better enhancement performance can be obtained
  • With denser models at the cost of speed
  • By exploiting optical waterbody properties as prior

Underwater Image Enhancement: Recent Research and Resources

2019

Paper	Theme	Code	Data
Multiscale Dense-GAN	Residual multiscale dense block as generator
Fusion-GAN	FGAN-based model, loss function formulation		U45
UDAE	U-Net denoising autoencoder
VDSR	ResNet-based model, loss function formulation
JWCDN	Joint wavelength compensation and dehazing
AWMD-Cycle-GAN	Adaptive weighting for multi-discriminator training
WAug Encoder-Decoder	Encoder-decoder module with wavelet pooling and unpooling	GitHub
Water-Net	Dataset and benchmark	GitHub	UIEBD

2017-18

Paper	Theme	Code	Data
UGAN	Several GAN-based models, dataset formulation	GitHub	Uw-imagenet
Underwater-GAN	Loss function formulation, cGAN-based model
LAB-MSR	Multi-scale Retinex-based framework
Water-GAN	Data generation from in-air image and depth pairings	GitHub	MHL, Field data
UIE-Net	CNN-based model for color correction and haze removal

Non-deep Models

• Sea-Thru (project page)
• Haze-line-aware Color Restoration (code)
• Local Color Mapping Combined with Color Transfer (code)
• Real-time Model-based Image Color Correction for Underwater Robots (code)
• All-In-One Underwater Image Enhancement using Domain-Adversarial Learning (code)
• Difference Backtracking Deblurring Method for Underwater Images
• Guided Trigonometric Bilateral Filter and Fast Automatic Color correction
• Red-channel Underwater Image Restoration (code)

Reviews, Metrics, and Benchmarks

• Real-world Underwater Enhancement: Challenges, Benchmarks, and Solutions
• Human-Visual-System-Inspired Underwater Image Quality Measures
• An Underwater Image Enhancement Benchmark Dataset and Beyond
• An Experimental-based Review of Image Enhancement and Restoration Methods (code)
• Diving Deeper into Underwater Image Enhancement: A Survey
• A Revised Underwater Image Formation Model

Acknowledgements

• https://github.com/floodsung/Deep-Learning-Papers-Reading-Roadmap
• https://github.com/PacktPublishing/Advanced-Deep-Learning-with-Keras
• https://github.com/cameronfabbri/Underwater-Color-Correction
• https://github.com/eriklindernoren/Keras-GAN
• https://github.com/phillipi/pix2pix
• https://github.com/wandb/superres
• https://github.com/aiff22/DPED
• https://github.com/roatienza/Deep-Learning-Experiments
• https://github.com/CMU-Perceptual-Computing-Lab/openpose