This page contains DCNN model and python code to robustly estimate 6 degrees of freedom, 3D face pose from an unconstrained image, without the use of face landmark detectors. The method is described in the paper:
F.-J. Chang, A. Tran, T. Hassner, I. Masi, R. Nevatia, G. Medioni, "FacePoseNet: Making a Case for Landmark-Free Face Alignment", in 7th IEEE International Workshop on Analysis and Modeling of Faces and Gestures, ICCV Workshops, 2017 [1].
This release bundles up our FacePoseNet (FPN) with the Face Renderer from Masi et al. [2,5], which is available separately from this project page.
The result is an end-to-end pipeline that seamlessly estimates facial pose and produces multiple rendered views to be used for face alignment and data augmentation.
- 6DoF 3D Head Pose estimation + 3D rendered facial views.
- Does not use fragile landmark detectors
- Robustness on images landmark detectors struggle with (low rez., occluded, etc.)
- Extremely fast pose estimation
- Both CPU and GPU supported
- Provides better face recognition through better face alignment than alignment using state of the art landmark detectors [1]
The code has been tested on Linux only. On Linux you can rely on the default version of python, installing all the packages needed from the package manager or on Anaconda Python and install required packages through conda.
Note: no landmarks are used in our method, although you can still project the landmarks on the input image using the estimated pose. See the paper for further details.
- Important: In order to download both FPN code and the renderer use
git clone --recursive - Important: Please download the learned models from https://www.dropbox.com/s/r38psbq55y2yj4f/fpn_new_model.tar.gz?dl=0 and make sure that the FPN models are stored in the folder
fpn_new_model.
The alignment and rendering can be used from the command line in the following, different ways.
To run it directly on a list of images (software will run FPN to estimate the pose and then render novel views based on the estimated pose):
$ python main_fpn.py <input-list-path>We provide a sample input list available here.
<ID, FILE, FACE_X, FACE_y, FACE_WIDTH, FACE_HEIGHT>where <FACE_X, FACE_y, FACE_WIDTH, FACE_HEIGHT> is the face bounding box information, either obtained manually or by the face detector.
Please see the input images here and rendered outputs here.
FPN is currently trained with a single 3D generic shape, without accounting for facial expressions. Addressing these is planned as future work.
Please cite our paper with the following bibtex if you use our face renderer:
@inproceedings{chang17fpn,
title={{F}ace{P}ose{N}et: Making a Case for Landmark-Free Face Alignment},
booktitle = {7th IEEE International Workshop on Analysis and Modeling of Faces and Gestures, ICCV Workshops},
author={
Feng-ju Chang
and Anh Tran
and Tal Hassner
and Iacopo Masi
and Ram Nevatia
and G\'{e}rard Medioni},
year={2017},
}[1] F.-J. Chang, A. Tran, T. Hassner, I. Masi, R. Nevatia, G. Medioni, "FacePoseNet: Making a Case for Landmark-Free Face Alignment", in 7th IEEE International Workshop on Analysis and Modeling of Faces and Gestures, ICCV Workshops, 2017
[2] I. Masi*, A. Tran*, T. Hassner*, J. Leksut, G. Medioni, "Do We Really Need to Collect Million of Faces for Effective Face Recognition? ", ECCV 2016, * denotes equal authorship
[3] I. Masi, S. Rawls, G. Medioni, P. Natarajan "Pose-Aware Face Recognition in the Wild", CVPR 2016
[4] T. Hassner, S. Harel, E. Paz and R. Enbar "Effective Face Frontalization in Unconstrained Images", CVPR 2015
[5] I. Masi, T. Hassner, A. Tran, and G. Medioni, "Rapid Synthesis of Massive Face Sets for Improved Face Recognition", FG 2017
- August 2017, First Release
The SOFTWARE PACKAGE provided in this page is provided "as is", without any guarantee made as to its suitability or fitness for any particular use. It may contain bugs, so use of this tool is at your own risk. We take no responsibility for any damage of any sort that may unintentionally be caused through its use.
If you have any questions, drop an email to fengjuch@usc.edu, anhttran@usc.edu, iacopo.masi@usc.edu or hassner@isi.edu or leave a message below with GitHub (log-in is needed).