This project gathers together and packages various image restoration techniques that follow various criteria:
- State-of-the-art (they are all based on Deep Learning; as of today (25/07/2019 at time of writing), NLRN and ESRGAN are leaders in various leaderboards maintained by paperswithcode.com, see here and here ).
- Open source (the selected implementations are under MIT or Apache licenses)
- Usable (a pretrained model is available, and the code does not need painfull1 dependencies)
- Python implementation (easier to use together, to share, and especially to use in Google Colab).
The project is a work in progress. However, it is already functional and can be tested on your own images through this demo in Google Colab.
The algorithms currently included in the packages are directly replicated or slightly adapted from external github repositories (see below). These methods were selected based on the above criteria, and after a comparison with other methods (comparison colab notebooks are coming soon).
NLRN - Liu et al. 2018. Non-Local Recurrent Network for Image Restoration (NeurIPS 2018) - MIT License
Denoising methods comparison
WDNN - Guan et al. 2019. Wavelet Deep Neural Network for Stripe Noise Removal. (IEEE Access, 7, 2019) - Apache-2.0 License
(no leaderboard associated... yet).
The original algorithm has been adapted to allow removal of both horizontal and vertical stripes. Note that in some cases, multiple steps of this technique can provide better results (see image below).
Multiple steps of stripe removal with adapted WDNN (image credits: Jean Vandendries. Reuse of this image is prohibited).
DeOldify ("NoGAN" algorithm) - Jason Antic, 2019 - MIT License
(no leaderboard associated... yet).
Comparison of image colorization methods (image credits: Jean Vandendries. Reuse of this image is prohibited).
ESRGAN - Wang et al. 2018. ESRGAN: Enhanced super-resolution generative adversarial networks (ECCV 2018) - Apache-2.0 License
Comparison of image super-resolution methods.
Many thanks to the authors of these awesome contributions to image restoration research, and for sharing them as open-source projects.
(more detailed info coming soon)
You need a CUDA-compatible GPU.
Tested with:
- Python 3.*
- CUDA
- Tensorflow
- Pytorch >= 1.1
- Torchvision >= 0.3.0
In a Google Colab notebook, you can use this:
import os
from os.path import *
basedir = "/content"
# Import library
repodir = join(basedir,"open-image-restoration")
if not exists(repodir):
os.chdir(basedir)
!git clone https://github.com/titsitits/open-image-restoration {repodir}
os.chdir(repodir)
#Todo: create a pip module
#Specific dependencies needed for colorization
!pip install --quiet -r requirements.txt
import ImagePipeline_utils as IP
import ImageRestorer
restorer = ImageRestorer.ImageRestorer()(more detailed info to come)
inputdir = "path/to/your/images"
outputdir = "path/to/output" #(can be inputdir)
#Todo: make restorer available from outside repo directory
os.chdir(repodir)
# Restore images
restorer.preprocess(inputdir, outputdir, gray=True) #resize if needed (by default limit to 1000x1000px), convert to 1-channel image
restorer.remove_stripes() #reduce image moire (WDNN)
restorer.denoise() #remove image grain (NLRN)
restorer.remove_stripes(process_args="-n 2") #reduce remaining image moire (n iterations)
restorer.colorize() #colorize image (DeOldify) (first time takes a long time as large models must be downloaded)
restorer.super_resolution() #upsample image (ESRGAN)
#Compare input and output folders
IP.compare_folders([inputdir, outputdir])These results can be reproduced in this notebook in Google Colab: https://github.com/titsitits/open-image-restoration/blob/master/Colab_Notebooks/Christmas_Truce_Restoration.ipynb
Examples of old image restoration (Christmas Truce, 1914).
A more detailed comparison of State-of-the-art super-resolution algorithms can be found in this Google Colab Notebook. A blog post (in french) presents an overview on the subject here
This research was conducted at CETIC, as part of the FEDER Digistorm project, co-financed by the European Union and the Wallonia Region.
1: In my research of IR algorithms, I subjectively considered Matlab and Caffe as painfull dependencies. Matlab is simply not free (and simpy not usable in Google colab), and Caffe hard to install, especially in Google Colab. Both of these issues make the algorithms hard to share to the community.