A collection of state-of-the-art video or single-image super-resolution architectures, reimplemented in tensorflow.
Pretrained weights is uploading now.
Several referenced PyTorch implementations are also included now.
Quick Link:
The hyperlink directs to paper site, follows the official codes if the authors open sources.
All these models are implemented in ONE framework.
| Model | Published | Code* | VSR (TF)** | VSR (Torch) | Keywords | Pretrained |
|---|---|---|---|---|---|---|
| SRCNN | ECCV14 | -, Keras | Y | Y | Kaiming | √ |
| RAISR | arXiv | - | - | - | Google, Pixel 3 | |
| ESPCN | CVPR16 | -, Keras | Y | Y | Real time | √ |
| VDSR | CVPR16 | - | Y | Y | Deep, Residual | √ |
| DRCN | CVPR16 | - | Y | Y | Recurrent | |
| DRRN | CVPR17 | Caffe, PyTorch | Y | Y | Recurrent | |
| LapSRN | CVPR17 | Matlab | Y | - | Huber loss | |
| EDSR | CVPR17 | - | Y | Y | NTIRE17 Champion | √ |
| SRGAN | CVPR17 | - | Y | - | 1st proposed GAN | |
| VESPCN | CVPR17 | - | Y | Y | VideoSR | √ |
| MemNet | ICCV17 | Caffe | Y | - | ||
| SRDenseNet | ICCV17 | -, PyTorch | Y | - | Dense | √ |
| SPMC | ICCV17 | Tensorflow | T | Y | VideoSR | |
| DnCNN | TIP17 | Matlab | Y | Y | Denoise | √ |
| DCSCN | arXiv | Tensorflow | Y | - | ||
| IDN | CVPR18 | Caffe | Y | - | Fast | √ |
| RDN | CVPR18 | Torch | Y | - | Deep, BI-BD-DN | |
| SRMD | CVPR18 | Matlab | T | - | Denoise/Deblur/SR | |
| DBPN | CVPR18 | PyTorch | Y | Y | NTIRE18 Champion | √ |
| ZSSR | CVPR18 | Tensorflow | - | - | Zero-shot | |
| FRVSR | CVPR18 | T | Y | VideoSR | √ | |
| DUF | CVPR18 | Tensorflow | T | - | VideoSR | |
| CARN | ECCV18 | PyTorch | Y | Y | Fast | √ |
| RCAN | ECCV18 | PyTorch | Y | Y | Deep, BI-BD-DN | |
| MSRN | ECCV18 | PyTorch | Y | Y | √ | |
| SRFeat | ECCV18 | Tensorflow | Y | Y | GAN | |
| NLRN | NIPS18 | Tensorflow | T | - | Non-local, Recurrent | |
| SRCliqueNet | NIPS18 | - | - | - | Wavelet | |
| FFDNet | TIP18 | Matlab | Y | Y | Conditional denoise | |
| CBDNet | arXiv | Matlab | T | - | Blind-denoise | |
| SOFVSR | ACCV18 | PyTorch | - | Y | VideoSR | √ |
| ESRGAN | ECCVW18 | PyTorch | - | Y | 1st place PIRM 2018 | √ |
| TecoGAN | arXiv | Tensorflow | - | T | VideoSR GAN | √ |
| RBPN | CVPR19 | PyTorch | - | Y | VideoSR | √ |
*The 1st repo is by paper author.
**Y: included; -: not included; T: under-testing.
You can download pre-trained weights through prepare_data, or visit the hyperlink at √.
(please contact me if any of links offend you or any one disabled)
| Name | Usage | # | Site | Comments |
|---|---|---|---|---|
| SET5 | Test | 5 | download | jbhuang0604 |
| SET14 | Test | 14 | download | jbhuang0604 |
| SunHay80 | Test | 80 | download | jbhuang0604 |
| Urban100 | Test | 100 | download | jbhuang0604 |
| VID4 | Test | 4 | download | 4 videos |
| BSD100 | Train | 300 | download | jbhuang0604 |
| BSD300 | Train/Val | 300 | download | - |
| BSD500 | Train/Val | 500 | download | - |
| 91-Image | Train | 91 | download | Yang |
| DIV2K | Train/Val | 900 | website | NTIRE17 |
| Waterloo | Train | 4741 | website | - |
| MCL-V | Train | 12 | website | 12 videos |
| GOPRO | Train/Val | 33 | website | 33 videos, deblur |
| CelebA | Train | 202599 | website | Human faces |
| Sintel | Train/Val | 35 | website | Optical flow |
| FlyingChairs | Train | 22872 | website | Optical flow |
| DND | Test | 50 | website | Real noisy photos |
| RENOIR | Train | 120 | website | Real noisy photos |
| NC | Test | 60 | website | Noisy photos |
| SIDD(M) | Train/Val | 200 | website | NTIRE 2019 Real Denoise |
| RSR | Train/Val | 80 | download | NTIRE 2019 Real SR |
| Vimeo-90k | Train/Test | 89800 | website | 90k HQ videos |
Other open datasets: Kaggle ImageNet COCO
This package offers a training and data processing framework based on TF. What I made is a simple, easy-to-use framework without lots of encapulations and abstractions. Moreover, VSR can handle raw NV12/YUV as well as a sequence of images as inputs.
-
Prepare proper tensorflow and pytorch(optional). For example, GPU and CUDA10.0 (recommend to use
conda):conda install tensorflow-gpu==1.12.0 # optional # conda install pytorch==1.0.1
-
Install VSR package
# For someone see this doc online # git clone https://github.com/loseall/VideoSuperResolution && cd VideoSuperResolution pip install -e .
-
Download pre-trained weights and (optinal) training datasets. For instance, let's begin with VESPCN and vid4 test data:
python prepare_data.py --filter vespcn vid4
-
Evaluate
cd Train python run.py --model vespcn --test vid4 -
Train
python prepare_data.py --filter mcl-v cd Train python run.py --model vespcn --dataset mcl-v --memory_limit 1GB --epochs 100
OK, that's all you need. For more details, use --help to get more information.
To train/test/infer any other model in VSR.Models, please see README. To write and train your own model via VSR, please see Docs.