COFW sample images segmented using our method.
Yuval Nirkin, Iacopo Masi, Anh Tuan Tran, Tal Hassner, and Gerard Medioni.
This project provides an interface for face segmentation using Caffe with a fully connected convolutional neural network. The network was trained on IARPA Janus CS2 dataset (excluding subjects that are also in LFW) using a novel process for collecting ground truth face segmentations, involving our tool for semi-supervised Face video segmentation. Additional synthetic images were generated by augmenting hands from the EgoHands dataset, and augmenting 3D models of glasses and microphones.
If you find this code useful, please make sure to cite our paper in your work:
Yuval Nirkin, Iacopo Masi, Anh Tuan Tran, Tal Hassner, Gerard Medioni, "On Face Segmentation, Face Swapping, and Face Perception", arXiv preprint arXiv:1704.06729, 22 Apr 2017
Please see project page for more details, more resources and updates on this project.
| Library | Minimum Version | Notes |
|---|---|---|
| Boost | 1.47 | Optional - For command line tools |
| OpenCV | 3.0 | |
| Caffe | 1.0 | ☕️ |
- Use CMake and your favorite compiler to build and install the library.
- Download the face_seg_fcn8s.zip and extract to "data" in the installation directory.
- Add "bin" in the installation directory to path.
- For using the library's C++ interface, please take a look at the Doxygen generated documentation.
- For python go to "interfaces/python" in the installation directory and run:
python face_seg.py- For running the segmentation on a single image:
cd path/to/face_segmentation/bin
face_seg_image ../data/images/Alison_Lohman_0001.jpg -o . -m ../data/face_seg_fcn8s.caffemodel -d ../data/face_seg_fcn8s_deploy.prototxt- For running the segmentation on all the images in a directory:
cd path/to/face_segmentation/bin
face_seg_batch ../data/images -o . -m ../data/face_seg_fcn8s.caffemodel -d ../data/face_seg_fcn8s_deploy.prototxt- For running the segmentation on a list of images, first prepare a file "img_list.txt", in which each line is a path to an image and call the following command:
cd path/to/face_segmentation/bin
face_seg_batch img_list.txt -o . -m ../data/face_seg_fcn8s.caffemodel -d ../data/face_seg_fcn8s_deploy.prototxtIn our paper we used a different network for our face segmentation. In the process of converting it to the Caffe model used in our end-to-end face swap distribution we notices some performance drop. We are working to fix this. We therefore ask that you please check here soon for updated on this Caffe model.
- End-to-end, automatic face swapping pipeline, example application using out face segmentation method.
- Interactive system for fast face segmentation ground truth labeling, used to produce the training set for our deep face segmentation.
- CNN3DMM, estimation of 3D face shapes from single images.
- ResFace101, deep face recognition used in the paper to test face swapping capabilities.
Copyright 2017, Yuval Nirkin, Iacopo Masi, Anh Tuan Tran, Tal Hassner, and Gerard Medioni
The SOFTWARE provided in this page is provided "as is", without any guarantee made as to its suitability or fitness for any particular use. It may contain bugs, so use of this tool is at your own risk. We take no responsibility for any damage of any sort that may unintentionally be caused through its use.