/FaceMorphing

This project aims at using a generative adversarial approach to perform face morphing

Primary LanguagePython

Face Morphing with StyleGAN

Python 3.6 TensorFlow 1.15 cuDNN 7.3.1

This project aims at using a generative adversarial approach to create face morphs.

Example image

Left columns: original and reconstructed images. Other images in rows and columns: faces morphs produced

Usage

Usage: The simplest way to get started and to perform face morphing is to run this colab notebook which uses the scripts in this repo. You can also find this notebook here.

Face morphing approaches

Face morphing by encoding and interpolating the latent space of a pretrained StyleGAN

Algo:1 Face morphing by latent space interpolation Face morphing by latent space interpolation

Face morphing by concurrent optimization in the latent space of a pretrained StyleGAN

Algo:2 Face morphing by concurrent optimization in latent space Face morphing by concurrent optimization in latent space

Folder Structure

FaceMorphing/
  │
  ├── README.md
  ├── requirements.txt
  │
  ├── align_images.py                            - align faces from input images
  ├── encode_images.py                           - find latent representation of reference images using perceptual losses
  ├── face_morphing_latent_interpolation.py      - perform face morphing by interpolating the latent space of StyleGAN
  ├── face_morphing_concurrent_optimization.py   - perform face morphing with concurrent optimization in the latent space of a pretrained StyleGAN
  ├── train_resnet.py                            - train a ResNet to predict latent representations of images in a StyleGAN model from generated examples
  │
  ├── notebooks/
  │   └── FaceMorphing.py   - Colab notebook containing instructions to perform face morphing
  │
  ├── figures/  - saved figures of face morphing results
  │
  ├── encoder/  - folder containing the models for the generator and for the perceptual loss
  │   ├── generator_model.py                 - model for styleGAN's generator
  │   ├── perceptual_model.py                - model for the perceptual loss in the case of face embedding
  │   └── perceptual_model_concurrent.py     - model for the perceptual loss in the case of concurrent optimization
  │
  ├── utils/ - utils scripts and functions
  │   ├── face_utilities/   - folder containing scripts, functions and models for face alignement, recognition, etc.
  │   └── utils.py 
  │
  └── dnnlib/   - helper library by NVIDIA for deep neural networks

Resources

Pretrained models used by our face morphing scripts are available via the following links:

Link Description
url_styleGAN Pretrained StyleGAN generator
url_resnet Pretrained ResNet encoder
url_VGG_perceptual Pretrained VGG network for the perceptual model

System requirements

  • Both Linux and Windows are supported, but we strongly recommend Linux for performance and compatibility reasons.
  • 64-bit Python 3.6 installation. We recommend Anaconda3 with numpy 1.14.3 or newer.
  • TensorFlow 1.10.0 or newer with GPU support.
  • One or more high-end NVIDIA GPUs with at least 11GB of DRAM. We recommend NVIDIA DGX-1 with 8 Tesla V100 GPUs.
  • NVIDIA driver 391.35 or newer, CUDA toolkit 9.0 or newer, cuDNN 7.3.1 or newer.

Acknowledgements

Thanks to @Puzer and @pbaylies for the original styleGAN encoder, of which this is a fork and to @SimJeg for the initial code that formed the basis of the ResNet model used here!