twtygqyy/pytorch-LapSRN

How to modify your code LapSRN_WGAN for same size image processing?

kirill-pinigin opened this issue · 7 comments

kirill-pinigin commented

Hello. Please help me how to modify your code LapSRN_WGAN for same size image processing without upsampling input image? I want use LapSRN_WGAN for image processing with same size

Should I modify this line of code in https://github.com/twtygqyy/pytorch-LapSRN/blob/master/lapsrn_wgan.py

Change " nn.ConvTranspose2d(in_channels=64, out_channels=64, kernel_size=4, stride=2, padding=1, bias=False)", to nn.Conv2d(in_channels=64, out_channels=64, kernel_size=4, stride=1, padding=1, bias=False),

Or may be I can just remove this layer?

twtygqyy commented

Hi @kirill-pinigin, you are right, but you will have to change the size of the input training samples. Check my VDSR repo which same size input and output are used for training.

kirill-pinigin commented

Thank you I understood. I chahge size of input image. But I don't understand what I have to do with nn.ConvTranspose2d layer? Just remove or replace on simple Conv2D?

twtygqyy commented

@kirill-pinigin you can just remove nn.ConvTranspose2d layers.

kirill-pinigin commented

May I ask you? If you do not mind I need to know your expert opinion. Can I use LapSRN_WGAN for image inpainting? I want to eliminate artefacts from grayscale image? What should I do? Can LapSRN_WGAN reduce or even elminate scratches or other artefacts from grayscale photo? What can you advise me?

twtygqyy commented

Hi @kirill-pinigin Of course the code can be modified for image in-painting. Basically there are two steps you need to do:

  1. Remove deconvolutional layers, keep input and output in same shape. You can check my VDSR repo for reference.
  2. Create input and output training datasets. I guess input could be image with mask, and output should be the raw image.
kirill-pinigin commented

@twtygqyy
I have already used your VDSR model in same maner like iitem 2 . unfortunately qualiy of restored images is not appropriate. May you advise some method to improve perceptual quality of restored images.
bad_samples_230
reconst_230_psnr 30 8107676076

May be I should change kernel Size or quantity of layer in VDSR model or may be method of learning rate adjusting?

Or may be I strat to train LapSRN_WGAN in same maner?

twtygqyy commented

@kirill-pinigin I will suggest you to use GAN instead of MSE-loss for training the network. It usually works better for this task. Check out http://hi.cs.waseda.ac.jp/~iizuka/projects/completion/en/