An end-to-end video restoration project by using the-state-of-art deep neural network models. Only pretrained and open-source repositories on Github were used to produce the end result, without doing any training task.
In this repository, video restoration methods are reviewed and implemented by using cited repositories and papers with the demonstration of the results. Sample video was used for the restoration example.
- Final vs Original
- Final Postprocessing (4K + 50FPS)
- Fully Automatic Video Colorization with Self-Regularization and Diversity
- Google Colab
- Anaconda
- Local:
OS: Windows 10 Home 64-bit
GPU: GeForce GTX 1060 Max-Q
CPU: Intel i7-7700HQ
RAM: 16GB
- Length : 00:01:49
- Width : 640
- Height : 480
- Frame per second: 25
- Number of frames: 2726
- Format : .mp4
EDSR, WDSR and SRGAN for single image super-resolution that requires the Keras Tensorflow backend.
Depth-Aware Video Frame Interpolation (CVPR 2019)
DeOldify: A Deep Learning based project for colorizing and restoring old images and videos
Natron, an open-source node-based compositing software application
Fully Automatic Video Colorization with Self-Regularization and Diversity
Note: This model was used only for comparison, DeOldify was primary colorization method.
Following procedure was followed in this project:
1 - Analysis of the sample video properties for the objective and parsing it to frames
2 - Original frames are colorized with DeOldify to improve results of the other GAN models as a preprocessing step.
3 - Super resolution was used in local environment with Keras, 640x480 resolution was increased by factor 4 to 2560x1920.
4 - Due to memory limitations, 2560x1920 frames were resized to 1920x1080 for video frame interpolation on Google Colab that is prepared by styler00dollar which uses Depth-Aware Video Frame Interpolation. Please note that a P100 GPU in Colab is needed to allocate memory for 1080p videos.
5 - After frame interpolation with 1920x1080 images, resulting frames were downsized to 720x540 resolution for Super Resolution with scale 4 to achieve 4K specs(3840x2160) with horizontal padding(480x2160 + 2880x2160 + 480x2160).
6 - Finally, resulting frames are used to generated 4K 50fps end result, after the application of color correction & supression by using Natron.
- Title: Efkarlı Sosyetede
- Director: Türker İnanoğlu
- Cast: Sadri Alışık, Filiz Akın
- Production Company: Erler Film
- Producer: Türker İnanoğlu
- Year: 1968
1 - Bao, W., Lai, W.S., Ma, C., Zhang, X., Gao, Z., & Yang, M.H. (2019). Depth-Aware Video Frame Interpolation. In IEEE Conference on Computer Vision and Pattern Recognition.
2 - Bao, W., Lai, Zhang, X., Gao, Z., & Yang, M.H. (2018). MEMC-Net: Motion Estimation and Motion Compensation Driven Neural Network for Video Interpolation and EnhancementIEEE Transactions on Pattern Analysis and Machine Intelligence.
3 - Lei, C., & Chen, Q. (2019). Fully Automatic Video Colorization With Self-Regularization and Diversity. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
-
This repository is prepared for the review of recent the state-of-art methods that are available for video restoration on Github with nonprofit educational purposes only. The sample video was used to reproduce the whole pipeline while respecting the copyright holders in terms of fair use.
-
Yapılan çalışma Github üzerinde yalnızca kar amacı gütmeyen eğitim amaçları göz edilerek video restorasyonu için mevcut olan en son teknolojik yöntemlerin gözden geçirilmesi için hazırlanmıştır. Kullanılan eser, adil kullanım kapsamında telif hakkı sahiplerine saygı gösterilerek işlenmiştir.