Video Super Resolution

A collection of state-of-the-art video or single-image super-resolution architectures, reimplemented in tensorflow.

Pretrained weights is uploading now.

Several referenced PyTorch implementations are also included now.

Quick Link: Installation Getting Started

Network list and reference (Updating)

The hyperlink directs to paper site, follows the official codes if the authors open sources.

All these models are implemented in ONE framework.

Model	Published	Code*	VSR (TF)**	VSR (Torch)	Keywords	Pretrained
SRCNN	ECCV14	-, Keras	Y	-	Kaiming	√
RAISR	arXiv	-	-	-	Google, Pixel 3
ESPCN	CVPR16	-, Keras	Y	Y	Real time	√
VDSR	CVPR16	-	Y	-	Deep, Residual	√
DRCN	CVPR16	-	Y	-	Recurrent
DRRN	CVPR17	Caffe, PyTorch	Y	-	Recurrent
LapSRN	CVPR17	Matlab	Y	-	Huber loss
EDSR	CVPR17	-	Y	Y	NTIRE17 Champion	√
SRGAN	CVPR17	-	Y	-	1st proposed GAN
VESPCN	CVPR17	-	Y	Y	VideoSR	√
MemNet	ICCV17	Caffe	Y	-
SRDenseNet	ICCV17	-, PyTorch	Y	-	Dense	√
SPMC	ICCV17	Tensorflow	-	-	VideoSR
DnCNN	TIP17	Matlab	Y	-	Denoise	√
DCSCN	arXiv	Tensorflow	Y	-
IDN	CVPR18	Caffe	Y	-	Fast	√
RDN	CVPR18	Torch	Y	-	Deep, BI-BD-DN
SRMD	CVPR18	Matlab	T	-	Denoise/Deblur/SR
DBPN	CVPR18	PyTorch	Y	Y	NTIRE18 Champion	√
ZSSR	CVPR18	Tensorflow	-	-	Zero-shot
FRVSR	CVPR18	PDF	T	Y	VideoSR
DUF	CVPR18	Tensorflow	T	-	VideoSR
CARN	ECCV18	PyTorch	Y	Y	Fast	√
RCAN	ECCV18	PyTorch	Y	Y	Deep, BI-BD-DN
MSRN	ECCV18	PyTorch	Y	Y		√
SRFeat	ECCV18	Tensorflow	Y	T	GAN
NLRN	NIPS18	Tensorflow	T	-	Non-local, Recurrent
SRCliqueNet	NIPS18	-	-	-	Wavelet
FFDNet	TIP18	Matlab	Y	-	Conditional denoise
CBDNet	arXiv	Matlab	T	-	Blind-denoise
ESRGAN	ECCVW18	PyTorch	-	Y	1st place PIRM 2018
TecoGAN	arXiv	Tensorflow	-	T	VideoSR GAN
RBPN	CVPR19	PyTorch	-	-	VideoSR

*The 1st repo is by paper author.

**Y: included; -: not included; T: under-testing.

You can download pre-trained weights through prepare_data, or visit the hyperlink at √.

Link of datasets

(please contact me if any of links offend you or any one disabled)

Name	Usage	#	Site	Comments
SET5	Test	5	download	jbhuang0604
SET14	Test	14	download	jbhuang0604
SunHay80	Test	80	download	jbhuang0604
Urban100	Test	100	download	jbhuang0604
VID4	Test	4	download	4 videos
BSD100	Train	300	download	jbhuang0604
BSD300	Train/Val	300	download	-
BSD500	Train/Val	500	download	-
91-Image	Train	91	download	Yang
DIV2K	Train/Val	900	website	NTIRE17
Waterloo	Train	4741	website	-
MCL-V	Train	12	website	12 videos
GOPRO	Train/Val	33	website	33 videos, deblur
CelebA	Train	202599	website	Human faces
Sintel	Train/Val	35	website	Optical flow
FlyingChairs	Train	22872	website	Optical flow
DND	Test	50	website	Real noisy photos
RENOIR	Train	120	website	Real noisy photos
NC	Test	60	website	Noisy photos
SIDD(M)	Train/Val	200	website	NTIRE 2019 Real Denoise
RSR	Train/Val	80	download	NTIRE 2019 Real SR
Vimeo-90k	Train/Test	89800	website	90k HQ videos

Other open datasets: Kaggle ImageNet COCO

VSR package

This package offers a training and data processing framework based on TF. What I made is a simple, easy-to-use framework without lots of encapulations and abstractions. Moreover, VSR can handle raw NV12/YUV as well as a sequence of images as inputs.

Install

Prepare proper tensorflow and pytorch(optional). For example, GPU and CUDA10.0 (recommend to use conda):
```
conda install tensorflow-gpu==1.12.0
# optional
# conda install pytorch==1.0.1
```

Install VSR package

# For someone see this doc online
# git clone https://github.com/loseall/VideoSuperResolution && cd VideoSuperResolution
pip install -e .

Getting Started

Download pre-trained weights and (optinal) training datasets. For instance, let's begin with VESPCN and vid4 test data:
```
python prepare_data.py --filter vespcn vid4
```

Evaluate

cd Train
python run.py --model vespcn --test vid4

Train

python prepare_data.py --filter mcl-v
cd Train
python run.py --model vespcn --dataset mcl-v --memory_limit 1GB --epochs 100

OK, that's all you need. For more details, use --help to get more information.

To train/test/infer any other model in VSR.Models, please see README. To write and train your own model via VSR, please see Docs.

hqleeUstc/VideoSuperResolution