SC-FEGAN

SC-FEGAN : Face Editing Generative Adversarial Network with User's Sketch and Color

Youngjoo Jo, Jongyoul Park

arXiv: https://arxiv.org/abs/1902.06838

Overview

Edit face images using a a deep neural network. Users can edit face images using intuitive inputs such as sketching and coloring, from which our network SC-FEGAN generates high quality synthetic images. We used SN-patchGAN discriminator and Unet-like generator with gated convolutional layers.

Dependencies

tensorflow
numpy
Python3
PyQt5
opencv-python
pyyaml

Setup

First, download the model from Google drive.

Run these commands to start the program.

mv /${HOME}/SC-FEGAN.ckpt.* /${HOME}/ckpt/
python3 demo.py

Select the number of GPUs you want to use by editing demo.yaml file (multi-GPUs are not supported).

GPU_NUM: 1 (the number you want to use)
#GPU_NUM: (if you want to use only CPU, erase the number)

How to Use

Edit face images using a simple GUI. Only erased regions of the image are filled in by the network.

Explanation of the buttons:

Open Image: Open the image you want to edit.
Mask: Draw a mask on the desired regions of the face on the left viewer.
Sketches: Sketch the desired lines on the left viewer.
Color: Draw colored lines. If you click this button the first time, you have to choose a color from the palette.
Palette: Change color. After choosing a color, click the Color button to apply the change.
Save Img: Save the results. It is saved as 'name.jpg'.
Arrange: Arrange the editing works.
Undo: Undo previous editing work.
Complete: Generate the image and show it on the right viewer.

We recommend using the following workflow:

1. Draw the sketch plausibly referring to the original image.
2. Draw the mask on the sketched region.
3. Click the `Arrange` button.
4. Draw the color on the masked region.
5. Click `Complete'.

Example Results

Face editing

Edit earring

Face restoration

Face restoration (with only sketch and color)

License

CC 4.0 Attribution-NonCommercial International

The software is for educational and academic research purpose only.

Notes

This is developed on Linux machine running Ubuntu 18.04.1
Provided model and sample code is under a non-commercial creative commons license.

Citing

@article{jo2019sc,
  title={SC-FEGAN: Face Editing Generative Adversarial Network with User's Sketch and Color},
  author={Jo, Youngjoo and Park, Jongyoul},
  journal={arXiv preprint arXiv:1902.06838},
  year={2019}
}

Update training code

Acknowledgement

We acknowledge the official code DeepFillv1. We thank all the researchers at ETRI, especially Kimin Yun and Jinwoo Jung, for insightful discussions.

lx120/SC-FEGAN