deep-face-alignment: A Python repository from lightwind002

Robust 2D and 3D Face Alignment Implemented in MXNet

This repository contains several heatmap based approaches like stacked Hourglass and stacked Scale Aggregation Topology (SAT) for robust 2D and 3D face alignment. Some popular blocks such as bottleneck residual block, inception residual block, parallel and multi-scale (HPM) residual block and channel aggregation block (CAB) are also provided for building the topology of the deep face alignment network. All the codes in this repo are implemented in Python and MXNet.

The models for 2D face alignment are verified on IBUG, COFW and 300W test datasets by the normalised mean error (NME) respectively. For 3D face alignment, the 3D pre-trained models are compared on AFLW2000-3D with the most recent state-of-the-art methods.

The training/validation dataset and testset are in below table:

Data	Download Link	Description
train_testset2d.zip	BaiduCloud or GoogleDrive, 490M	2D training/validation dataset and IBUG, COFW, 300W testset
train_testset3d.zip	BaiduCloud or GoogleDrive, 1.54G	3D training/validation dataset and AFLW2000-3D testset

The performances of 2D pre-trained models are shown below. Accuracy is reported as the Normalised Mean Error (NME). To facilitate comparison with other methods on these datasets, we give mean error normalised by the diagonal of the ground truth bounding box and the eye centre distance. Each training model is denoted by Topology^StackBlock (d = DownSampling Steps) - BlockType - OtherParameters.

Model	Model Size	IBUG	COFW	300W	Download Link
Hourglass2(d=4)-Resnet	26MB	2.051/7.819	2.276/7.094	1.891/6.640	BaiduCloud or GoogleDrive
Hourglass2(d=3)-HPM	38MB	1.970/7.499	2.116/6.587	1.785/6.256	BaiduCloud or GoogleDrive
Hourglass2(d=4)-CAB	46MB	1.912/7.289	1.992/6.216	1.658/5.816	BaiduCloud or GoogleDrive
SAT2(d=3)-CAB	40MB	1.875/7.154	1.939/6.047	1.640/5.751	BaiduCloud or GoogleDrive
Hourglass2(d=3)-CAB	37MB	1.874/7.140	1.926/6.006	1.640/5.748	BaiduCloud or GoogleDrive

The performances of 3D pre-trained models are shown below. Accuracy is reported as the Normalised Mean Error (NME). The mean error is normalised by the square root of the ground truth bounding box size.

Model	Model Size	AFLW2000-3D	Download Link
SAT2(d=3)-CAB-3D	40MB	3.360	BaiduCloud or GoogleDrive
Hourglass2(d=3)-CAB-3D	37MB	3.307	BaiduCloud or GoogleDrive

Note: More pre-trained models will be added soon.

Environment

This repository has been tested under the following environment:

Python 2.7
Ubuntu 18.04
Mxnet-cu90 (==1.3.0)

Installation

Prepare the environment.
Clone the repository.
Type make to build necessary cxx libs.

Training

Download the training/validation dataset and unzip it to your project directory.
You can define different loss-type/network topology/dataset in config.py(from sample_config.py).
You can use CUDA_VISIBLE_DEVICES='0' train.py --network satnet to train stacked Scale Aggregation Topology (SAT) networks or CUDA_VISIBLE_DEVICES='0' train.py --network hourglass to train stacked Hourglass models. Instead, you can also edit train.sh and run sh train.sh to train your models.

Testing

Download the ESSH model from BaiduCloud or GoogleDrive and place it in ./essh-model/.
Download the pre-trained model and place it in ./models/.
You can use python test.py to test your models for 2D and 3D face alignment.

Results

Results of 2D face alignment (inferenced from model Hourglass2(d=3)-CAB) are shown below.

Results on ALFW-2000 dataset (inferenced from model SAT2(d=3)-CAB-3D) are shown below.

License

MIT LICENSE

Reference

@article{guo2018stacked,
  title={Stacked Dense U-Nets with Dual Transformers for Robust Face Alignment},
  author={Guo, Jia and Deng, Jiankang and Xue, Niannan and Zafeiriou, Stefanos},
  journal={arXiv preprint arXiv:1812.01936},
  year={2018}
}

@inproceedings{Deng2018Cascade,
  title={Cascade Multi-View Hourglass Model for Robust 3D Face Alignment},
  author={Deng, Jiankang and Zhou, Yuxiang and Cheng, Shiyang and Zaferiou, Stefanos},
  booktitle={2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018)},
  pages={399-403},
  year={2018},
}

@article{Bulat2018Hierarchical,
  title={Hierarchical binary CNNs for landmark localization with limited resources},
  author={Bulat, Adrian and Tzimiropoulos, Yorgos},
  journal={IEEE Transactions on Pattern Analysis & Machine Intelligence},
  year={2018},
}

@inproceedings{Jing2017Stacked,
  title={Stacked Hourglass Network for Robust Facial Landmark Localisation},
  author={Jing, Yang and Liu, Qingshan and Zhang, Kaihua and Jing, Yang and Liu, Qingshan and Zhang, Kaihua and Jing, Yang and Liu, Qingshan and Zhang, Kaihua},
  booktitle={IEEE Conference on Computer Vision & Pattern Recognition Workshops},
  year={2017},
}

Acknowledgment

The code is adapted based on an intial fork from the insightface repository.

lightwind002/deep-face-alignment