Face-Renovation

HiFaceGAN: Face Renovation via Collaborative Suppression and Replenishment

Lingbo Yang, Chang Liu, Pan Wang, Shanshe Wang, Peiran Ren, Siwei Ma, Wen Gao

Project | arXiv | Supplementary Materials(TODO)

Usage
Benchmark
Remarks
License
Citation
Acknowledgements

Usage

Environment

Ubuntu/CentOS
PyTorch 1.0+
CUDA 10.1
python packages: opencv-python, tqdm,
Data augmentation tool: image_augmentor
Face Recognition Toolkit for evaluation
tqdm to make you less anxious when testing:)

Dataset Preparation

Download FFHQ, resize to 512x512 and split id [65000, 70000) for testing. We only use first 10000 images for training, which takes 2~3 days on a P100 GPU, training with full FFHQ is possible, but could take weeks.

After that, run degrade.py to acquire paired images for training. You need to specify the degradation type and input root in the script first.

Configurations

The configurations is stored in options/config_hifacegan.py, the options should be self-explanatory, but feel free to leave an issue anytime.

Training and Testing

python train.py            # A fool-proof training script
python test.py             # Test on synthetic dataset
python test_nogt.py        # Test on real-world images
python two_source_test.py  # Visualization of Fig 5

Evaluation

Please find in metrics_package folder:

main.py: GPU-based PSNR, SSIM, MS-SSIM, FID
face_dist.py: CPU-based face embedding distance(FED) and landmark localization error (LLE).
PerceptualSimilarity\main.py: GPU-based LPIPS
niqe\niqe.py: NIQE, CPU-based, no reference

Note:

Read the scripts and modify result folder path(s) before testing (do not add / in the end), the results will be displayed on screen and saved in txt.
At least 10GB is required for main.py. If this is too heavy for you, reducebs=250 at line 79
Initializing Inception V3 Model for FID could take several minutes, just be patient. If you find a solution, please submit a PR.
By default face_dist.py script runs with 8 parallel subprocesses, which could cause error on certain environments. In that case, just disable the multiprocessing and replace with a for loop (This would take 2~3 hours for 5k images, you may want to wrap the loop in tqdm to reduce your anxiety).

Benchmark

Please refer to benchmark.md for benchmark experimental settings and performance comparison.

Remarks

Face Renovation is not designed to create a perfect specimen OUT OF you, but to bring out the best WITHIN you.

The Philosophy of Face Renovation | Understanding of HiFaceGAN

License

Citation

Please kindly cite our paper when using this project for your research.

@journal{Yang2020HiFaceGANFR,
  title={HiFaceGAN: Face Renovation via Collaborative Suppression and Replenishment},
  author={Lingbo Yang and Chang Liu and Pan Wang and Shanshe Wang and Peiran Ren and Siwei Ma and Wen Gao},
  journal={Arxiv},
  url={https://arxiv.org/abs/2005.05005},
  year={2020}
}

Acknowledgements

The replenishment module borrows the implementation of SPADE.

machanic/Face-Renovation