This is an implementation of "Accurate Image Super-Resolution Using Very Deep Convolutional Networks" (CVPR 2016 Oral Paper) in caffe.
VDSR (Very Deep network for Super-Resolution) is an end-to-end network with 20 convolutional layers for single image super-resolution. The performance of VDSR is better than other state-of-the-art SISR methods, such as SRCNN, A+ and CSCN (My implementation of CSCN).
I trained my "VDSR_170000" model by about 300000 training samples (1638 images augumented from 91 images). So, please do data augumentation if you want to get good performances.
I trained a new model by multi scale data augumentation. But there are not obvious improvemets between multi scale model and my single scale model. Replies welcome, if anyone has experience on multi scale training.
Recommend to use "VDSR_Official.mat" if you just want to do some test. "VDSR_Official.mat" achieve better performance than "VDSR_170000.mat" and it supports multi-scale test (you can test different scales super-resolution in only one model).
Recommend to train VDSR following the setting of paper. Train it about 80 epoches and decrease learning rate by dividing 10 every 20 epoches. It means that you should modify the solver_prototxt in Train folder like this:
stepsize: numberoftrainingsamples / batchsize * 20
max_iter: numberoftrainingsamples / batchsize * 80
-
Place the "Train" folder into "($Caffe_Dir)/examples/", and rename "Train" to "VDSR"
-
Open MATLAB and direct to ($Caffe_Dir)/example/VDSR, run "generate_train.m" and "generate_test.m" to generate training and test data (Code from SRCNN)
-
To train VDSR, run ./build/tools/caffe train --solver examples/VDSR/VDSR_solver.prototxt
-
Set clip_gradients in VDSR_solver.prototxt to solve gradient explosion problem, 0.1 or 1 is a good choice
-
Change the learning rate when the error plateaus
-
After training, run "caffemodel2mat.m" to convert caffemodel to mat for testing (matcaffe is required)
-
"Demo_SR_Conv.m" is a simple test code. Just run it and you will get the result
-
"VDSR_170000.mat" is a model trained by myself
-
"VDSR_Official.mat" is an official model converted from Official Test Code
Because of the limitation of hardware conditions, I didn't do complete training. So there are some differences between this implementation ("VDSR_170000.mat") and original paper ("VDSR_Official.mat").
This implementation: 91 images (with data augumentation and only factor 2)
Original paper: 291 images (with data augumentation and factor 2, 3 and 4)
This implementation: Casade of 2x to generate 3x and 4x result
Original paper: Multi scale in one model
This implementation: about 30 epoch
DataSet | VDSR_Official | VDSR_170000 |
---|---|---|
Set5 | 37.53 | 37.46 |
Set14 | 33.03 | 32.83 |
BSD100 | 31.90 | 31.65 |
DataSet | VDSR_Official | VDSR_170000 |
---|---|---|
Set5 | 33.66 | 33.52 |
Set14 | 29.77 | 29.55 |
BSD100 | 28.82 | 28.62 |
DataSet | VDSR_Official | VDSR_170000 |
---|---|---|
Set5 | 31.35 | 31.14 |
Set14 | 28.01 | 27.81 |
BSD100 | 27.29 | 27.13 |
Please cite [1] if you use this code in your work, thank you!
[1] Jiwon Kim, Jung Kwon Lee and Kyoung Mu Lee, "Accurate Image Super-Resolution Using Very Deep Convolutional Networks", Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016