FSRNet-pytorch
This repository is a PyTorch implementation of FSRNet: End-to-End Learning Face Super-Resolution with Facial Priors. Official demo code and other re-implementations available at:
Note: This repository has not yet reproduced the performance of the original implementation.
Demo
python demo.py --checkpoint ./e099_psnr22.53.pth
Datasets
Link]
Helen Dataset [Helen Dataset provides 194 facial-landmark points by default, and this allows you to construct the landmark heatmap. To obtain the parsing-map from Helen Dataset, you can either go through a separate masking process or simply reuse the landmark points (connect the dots, and color them!). For more information, go to datasets/README.md.
Link]
CelebAMask-HQ [CelebAMask-HQ provides 19 classes masking images by default, and this allows you to construct the parsing-map. To obtain the landmark heatmap from CelebAMask-HQ, you may have to carry out a separate facial-landmark detection process. For more information, go to datasets/README.md.
*Note that the original implementation in the paper used CelebA dataset (Liu et al., 2015), not CelebAMask-HQ dataset (Lee et al., 2019), and gained parsing map through GFC (Li et al., 2017).
FSRNet
Overview
- Train Coarse SR Network to minimize (1).
- Train Coarse SR Network, and Prior Estimation Network to minimize (2).
- Train whole network to minimize (3).
Train
python train.py --dataset {celeb, helen}
Evaluation
[BICUBIC]
python eval.py
>> * Mean PSNR = 21.7142
>> * Mean SSIM = 0.6045
[FSRNet]
python eval.py --checkpoint ./e099_psnr22.53.pth
>> * Mean PSNR = 22.5537
>> * Mean SSIM = 0.7008
Dataset | # Images (train/test) |
BICUBIC (PSNR/SSIM) |
FSRNet (PSNR/SSIM) |
PSNR (increased) |
SSIM (increased) |
|
---|---|---|---|---|---|---|
FSRNet Paper | CelebA | 18000/100 | 23.75/0.6423 | 26.31/0.7522 | +10.78% | +17.11% |
Implementation | CelebAMask-HQ | 20951/100 | 21.71/0.6045 | 22.55/0.7008 | +3.87% | +15.93% |
TODO : reproduce the results presented in the paper...
FSRGAN
Overview
- Train Discriminator to distinguish between ground truth and generated SR image.
- Train Generator in the same way as before (FSRNet).
- Train Generator to minimize perceptual loss using Feature Extractor (fixed).
- Train Generator to surpass Discriminator.
Train
python train_GAN.py --dataset {celeb, helen}
TODO : reproduce the results presented in the paper...
References
-
FSRNet: End-to-End Learning Face Super-Resolution with Facial Priors
Yu Chen, Ying Tai, Xiaoming Liu, Chunhua Shen, Jian Yang
https://arxiv.org/abs/1711.10703 -
Interactive facial feature localization
Vuong Le, Jonathan Brandt, Zhe Lin, Lubomir Bourdev, Thomas S. Huang
http://www.ifp.illinois.edu/~vuongle2/helen/ -
MaskGAN: Towards Diverse and Interactive Facial Image Manipulation
Cheng-Han Lee, Ziwei Liu, Lingyun Wu, Ping Luo
https://arxiv.org/abs/1907.11922