Code for the CNN-based Eye Landmark Estimation project under the supervision of Roberto Valenti at SightCorp.
Robust facial landmark estimation in-the-wild is a challenging task due to many variations in image conditions, pose and occlusion. Traditional research in landmark estimation uses feature-based methods and the extension of such approaches towards robust performance against large variations is challenging and computationally expensive. In this work we focus on estimating eye landmarks and present a convolutional neural network model for 2D and 3D landmark estimation. Furthermore, we define custom a evaluation metric and objective function which are more suitable for the task of landmark estimation. We show that through data augmentation, the network is able to generalize from synthetic images to real-world images and handle a variety of image, pose and gaze conditions, achieving a 93% 2D landmark estimation accuracy in test data. The quality of the 3D eye shapes reconstructed from estimated landmarks is acceptable despite the lack of depth information.
| Synthetic Data | Real Data |
|---|---|
 |
 |
Yellow: true labels - Red: estimated landmarks
python src/live_demo.pySee requirements.txt.
Copyright © 2017 Dana Kianfar and Jose Gallego.
This project is distributed under the MIT license. This was developed as part of the Project AI course coordinated by Maarten van Someren at the University of Amsterdam. Please review the UvA regulations governing Fraud and Plagiarism in case you are a student at the UvA.