PFLD_68Points_Pytorch

Implementation of PFLD For 68 Facial Landmarks By Pytorch

DataSets

WFLW Dataset

Wider Facial Landmarks in-the-wild (WFLW) is a new proposed face dataset. It contains 10000 faces (7500 for training and 2500 for testing) with 98 fully manual annotated landmarks.

1.Training and Testing images[Google Drive][Baidu Drive], Unzip and put to ./data/WFLW/raw/

2.Have got list_68pt_rect_attr_train.txt and list_68pt_rect_attr_test.txt. If you want to get them by youself, please watch get68psFrom98psWFLW.py and run it before please get WFLW Face Annotations , unzip and put to ./data/WFLW/

3.Move Mirror68.txt to ./data/WFLW/annotations/
```
 $ cd ./data/WFLW 
 $ python3 WFLW_SetPreparation68.py
```
300W Dataset

300W is a very general face alignment dataset. It has a total of 3148+689 images, each image contains more than one face, but only one face is labeled for each image.File directory includes afw(337)，helen(train 2000+test 330)，ibug(135)，lfpw(train 811+test 224) with 68 fully manual annotated landmarks.

1.Training and Testing images[Databases][Baidu Drive], Unzip and put to ./data/300W/raw/

2.Have got list_68pt_rect_attr_train.txt and list_68pt_rect_attr_test.txt. If you want to get them by youself, please watch get68pointsfor300W.py and run it

3.Move Mirror68.txt to ./data/300W/annotations/
```
 $ cd ./data/300W 
 $ python3 300W_SetPreparation68.py
```
300VW Dataset

300VW is a video format, which needs to be processed into a single frame picture and corresponds to each key point pts file.

1.Training and Testing images[Databases], Unzip and put to ./data/300VW/raw/

2.Run get68psAndImagesFrom300VW.py to get list_68pt_rect_attr_train.txt

3.Move Mirror68.txt to ./data/300VW/annotations/
```
 $ cd ./data/300VW 
 $ python3 get68psAndImagesFrom300VW.py
 $ python3 300VW_SetPreparation68.py
```
Your Own Dataset

If you want to get facial landmarks for new face data, please use Detect API of face++. For specific operations,
please refer to API Document. And refer to ./data/getNewFacialLandmarksFromFacePP.py for using the api interface.
All Dataset

After completing the steps of each data set above, you can run the code merge_files.py directly .
```
 $ cd ./data
 $ python3 merge_files.py
```

training & testing

training :

 $ sh train.sh

reading images from a camera to test:

 $ python3 camera.py

reading images from a dir to test:

 $ python3 test.py

Result

Sample IMGs:

Details about the models are below:

tip: please install resnest to use ResNest models.

Name	# Params	Mean error	Failure rate	One iteration time(s)
`ResNest50`	122.27M	0.046	0.038	0.304
`MobileNetV2_0.25`	1.09M	0.075	0.174	0.154
`MobileNetV2_1.00`	7.28M	0.065	0.127	0.203
`BlazeLandmark`	7.52M	0.069	0.131	0.171
`HRNetV2`	545.07M	0.066	0.125	0.769
`efficientnet-b0`	16.67M	0.064	0.119	0.202
`efficientnet-b1`	26.37M	0.075	0.149	0.252
`efficientnet-b2`	30.85M	0.071	0.145	0.266
`efficientnet-b3`	42.29M	0.099	0.136	0.307
`efficientnet-b4`	68.34M	0.077	0.164	0.375
`efficientnet-b5`	109.34M	0.094	0.173	0.501
`efficientnet-b6`	156.34M	0.081	0.175	0.702
`efficientnet-b7`	244.03M	0.081	0.196	0.914

pytorch -> onnx -> ncnn

Pytorch -> onnx -> onnx_sim

Make sure pip3 install onnx-simplifier

 $ python3 pytorch2onnx.py
 $ python3 -m onnxsim model.onnx model_sim.onnx

onnx_sim -> ncnn

How to build :https://github.com/Tencent/ncnn/wiki/how-to-build

 $ cd ncnn/build/tools/onnx
 $ ./onnx2ncnn model_sim.onnx model_sim.param model_sim.bin

reference:

PFLD: A Practical Facial Landmark Detector https://arxiv.org/pdf/1902.10859.pdf

ResNest: Split-Attention Networks https://hangzhang.org/files/resnest.pdf

EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks https://arxiv.org/pdf/1905.11946.pdf

pytorch：https://github.com/lukemelas/EfficientNet-PyTorch

tensorflow：https://github.com/tensorflow/tpu/tree/master/models/official/efficientnet

keras: https://github.com/qubvel/efficientnet

Tensorflow Implementation for 98 Facial Landmarks: https://github.com/guoqiangqi/PFLD

geoffzhang/PFLD_68points_Pytorch