This personal project has been made for my last year of engineering school at IMAC.
My goal is to denoise an image using deep learning. After the denoising, NLP technologies are used to automatically describe the content of the image. Indeed, if my image is denoised enough, I could describe it well without any problem. That's the point of my project : denoise an image enough in order to describe the content of it.
pip install scipy
pip install tensorflow
pip install keras
pip install numpy
pip install scikit-image
pip install opencv-python
For PyTorch, it is better to follow the installation on their website, as it depends on CUDA and Linux/Windows : Installing PyTorch
Flicker dataset is used on both denoising and describing image.
To get the Flicker dataset, you must run the following command :
wget https://drive.google.com/uc?export=download&id=1yBlN1CnaMwFvwoW_jjwqPsNDVOZVw4nu
You must put is in the directory ./dataset
If you want to get trained model, run the following commands :
wget https://drive.google.com/uc?export=download&id=1UnjL6HGLsxGgaRYbLoPYq1RI0xh8VixD
It will give you the trained model for the NLP image descriptor, so you must put it in ./NLP_description
wget https://drive.google.com/uc?export=download&id=1yBar0ywvinwf_ZfjtntxYNqmLw-cxzzu
It will give you the trained model for the denoising, so you must put it in ./denoise/model
You'll find every file in the directory denoise/
As I wanted to have total control on my data, the first thing I did is apply noise to my dataset. As the model of denoising was trained with different data, it's no problem if I proceed that way.
You will first need to run : preprocess_images.py to apply a gaussian noise to your dataset if it is a cleared one. If it is already noised, there is no need.
Then you can denoise it with denoising.py
After that, you can check the SSIM and PSNR between the original image and the denoised one with compare.py to check how similar they are
You'll find every file in the directory NLP_description/
If you want to use the descriptor directly, skip the preprocessing and the training step.
First, you need to get dataset files with get_dataset_files.py
Then you need to prepare the photo data with prepare_data.py
After that it's the turn of the text data to be prepared with prepare_text_data.py
You can train the model with train_model.py
If you want to run all in once you can use the bash script : run_NLP_preprocess.sh. This script run on Windows only !
Once the model is trained, you can use it to generate a new description of an image by launching generate_new_desc.py
| Name of the article | Authors | Link |
|---|---|---|
| Deep Learning on Image Denoising: An Overview | Chunwei Tiana, Lunke Feic, Wenxian Zhengd, Yong Xua, Wangmeng Zuof Chia-Wen Lin | https://arxiv.org/pdf/1912.13171.pdf |
| Image Denoising and Inpainting with Deep Neural Networks | Junyuan Xie, Linli Xu, Enhong Chen | https://papers.nips.cc/paper/4686-image-denoising-and-inpainting-with-deep-neural-networks.pdf |
| Toward Convolutional Blind Denoising of Real Photographs | Shi Guo, Zifei Yan, Kai Zhang, Wangmeng Zuo, Lei Zhang | https://arxiv.org/pdf/1807.04686v2.pdf |
| Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising | Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, Lei Zhang | https://arxiv.org/pdf/1608.03981v1.pdf |
| Robust and interpretable blind image denoising via bias-free convolutional neural networks | Sreyas Mohan, Zahra Kadkhodaie, Eero P. Simoncelli, Carlos Fernandez-Granda | https://openreview.net/attachment?id=HJlSmC4FPS&name=original_pdf |
| Real Image Denoising with Feature Attention | Saeed Anwar, Nick Barnes | https://arxiv.org/pdf/1904.07396v2.pdf |
| Image denoising using deep CNN with batch renormalization | Chunwei Tian, Yong Xu, Wangmeng Zuo | https://www.sciencedirect.com/science/article/abs/pii/S0893608019302394 |
| Name of the repository | Authors | Link |
|---|---|---|
| CBDNet-tensorflow | IDKiro | https://github.com/IDKiro/CBDNet-tensorflow |
| Deep learning denoise | Shibui Yusuke | https://github.com/shibuiwilliam/DeepLearningDenoise |
| High-Quality Self-Supervised Deep Image Denoising - Official TensorFlow implementation of the NeurIPS 2019 paper | Samuli Laine, Tero Karras, Jaakko Lehtinen, Timo Aila | https://github.com/NVlabs/selfsupervised-denoising |
| Tensorflow tutorials | Parag K Mital | https://github.com/pkmital/tensorflow_tutorials/blob/master/python/08_denoising_autoencoder.py |
| BRDNet | Chunwei Tian, Yong Xu, Wangmeng Zuo | https://github.com/hellloxiaotian/BRDNet |