/PFLD_68points_Pytorch

Implementation of PFLD For 68 Facial Landmarks By Pytorch

Primary LanguagePython

PFLD_68Points_Pytorch

Implementation of PFLD For 68 Facial Landmarks By Pytorch

DataSets

  • WFLW Dataset

    Wider Facial Landmarks in-the-wild (WFLW) is a new proposed face dataset. It contains 10000 faces (7500 for training and 2500 for testing) with 98 fully manual annotated landmarks.

    1.Training and Testing images[Google Drive][Baidu Drive], Unzip and put to ./data/WFLW/raw/

    2.Have got list_68pt_rect_attr_train.txt and list_68pt_rect_attr_test.txt. If you want to get them by youself, please watch get68psFrom98psWFLW.py and run it before please get WFLW Face Annotations , unzip and put to ./data/WFLW/

    3.Move Mirror68.txt to ./data/WFLW/annotations/

     $ cd ./data/WFLW 
     $ python3 WFLW_SetPreparation68.py
  • 300W Dataset

    300W is a very general face alignment dataset. It has a total of 3148+689 images, each image contains more than one face, but only one face is labeled for each image.File directory includes afw(337),helen(train 2000+test 330),ibug(135),lfpw(train 811+test 224) with 68 fully manual annotated landmarks.

    1.Training and Testing images[Databases][Baidu Drive], Unzip and put to ./data/300W/raw/

    2.Have got list_68pt_rect_attr_train.txt and list_68pt_rect_attr_test.txt. If you want to get them by youself, please watch get68pointsfor300W.py and run it

    3.Move Mirror68.txt to ./data/300W/annotations/

     $ cd ./data/300W 
     $ python3 300W_SetPreparation68.py
  • 300VW Dataset

    300VW is a video format, which needs to be processed into a single frame picture and corresponds to each key point pts file.

    1.Training and Testing images[Databases], Unzip and put to ./data/300VW/raw/

    2.Run get68psAndImagesFrom300VW.py to get list_68pt_rect_attr_train.txt

    3.Move Mirror68.txt to ./data/300VW/annotations/

     $ cd ./data/300VW 
     $ python3 get68psAndImagesFrom300VW.py
     $ python3 300VW_SetPreparation68.py
  • Your Own Dataset

    If you want to get facial landmarks for new face data, please use Detect API of face++. For specific operations,
    please refer to API Document. And refer to ./data/getNewFacialLandmarksFromFacePP.py for using the api interface.

  • All Dataset

    After completing the steps of each data set above, you can run the code merge_files.py directly .

     $ cd ./data
     $ python3 merge_files.py

training & testing

training :

 $ sh train.sh

reading images from a camera to test:

 $ python3 camera.py

reading images from a dir to test:

 $ python3 test.py

Result

Sample IMGs:

Image text Image text Image text Image text Image text Image text Image text Image text Image text

Details about the models are below:

tip: please install resnest to use ResNest models.

Name # Params Mean error Failure rate One iteration time(s)
ResNest50 122.27M 0.046 0.038 0.304
MobileNetV2_0.25 1.09M 0.075 0.174 0.154
MobileNetV2_1.00 7.28M 0.065 0.127 0.203
BlazeLandmark 7.52M 0.069 0.131 0.171
HRNetV2 545.07M 0.066 0.125 0.769
efficientnet-b0 16.67M 0.064 0.119 0.202
efficientnet-b1 26.37M 0.075 0.149 0.252
efficientnet-b2 30.85M 0.071 0.145 0.266
efficientnet-b3 42.29M 0.099 0.136 0.307
efficientnet-b4 68.34M 0.077 0.164 0.375
efficientnet-b5 109.34M 0.094 0.173 0.501
efficientnet-b6 156.34M 0.081 0.175 0.702
efficientnet-b7 244.03M 0.081 0.196 0.914

pytorch -> onnx -> ncnn

Pytorch -> onnx -> onnx_sim

Make sure pip3 install onnx-simplifier

 $ python3 pytorch2onnx.py
 $ python3 -m onnxsim model.onnx model_sim.onnx

onnx_sim -> ncnn

How to build :https://github.com/Tencent/ncnn/wiki/how-to-build

 $ cd ncnn/build/tools/onnx
 $ ./onnx2ncnn model_sim.onnx model_sim.param model_sim.bin

reference:

PFLD: A Practical Facial Landmark Detector https://arxiv.org/pdf/1902.10859.pdf

ResNest: Split-Attention Networks https://hangzhang.org/files/resnest.pdf

EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks https://arxiv.org/pdf/1905.11946.pdf

pytorch:https://github.com/lukemelas/EfficientNet-PyTorch

tensorflow:https://github.com/tensorflow/tpu/tree/master/models/official/efficientnet

keras: https://github.com/qubvel/efficientnet

Tensorflow Implementation for 98 Facial Landmarks: https://github.com/guoqiangqi/PFLD