RealSmileNet pytorch Implementation

Dependency

  • python 3.7
  • numpy 1.18.1
  • Pillow 7.0.0
  • dlib 19.19.0
  • opencv-python 4.2.0.32
  • torch 1.4.0
  • torchvision 0.5.0

Dataset

The sample data is contained in processed_data folder

Train RealSmileNet

  • test.py contains the test program for deep smile net. We do not provide the test program for model performance as it is time-consuming. Only the basic test are provided. run python test.py

  • run python demo.py for the sample training of UVA-NEMO databases, the verbose.txt will contains the log of training. Note only one public data from UVA-NEMO are provided.

  • The model weight will be saved in database_training folder, for example if the model is trained on UVA-NEMO database, the folder names uvanemo_training

               [--label_path LABEL_PATH] [--frame_path FRAME_PATH]
               [--frequency FREQUENCY] [--sub SUB] [--epochs EPOCHS] [--lr LR]

DeepSmileNet training, optional arguments for demo.py :

  -h, --help            show this help message and exit
  --database DATABASE   select the database to run, please selected from
                        UVANEMO,SPOS,MMI and BBC
  --batch_size BATCH_SIZE
                        the mini batch size used for training
  --label_path LABEL_PATH
                        the path contains training labels
  --frame_path FRAME_PATH
                        the path contains processed data
  --frequency FREQUENCY
                        the frequency used to sample the data
  --sub SUB             the subsitution for the model, please selected from
                        org,LSTM,GRU,resnet,miniAlexnet,minidensenet
  --epochs EPOCHS       number of total epochs to run
  --lr LR, --learning-rate LR
                        learning rate
  • data_preprocessin.py contains the function used to preprocess UVA-NEMO,BBC,SPOS and MMI database. The pre trained shape_predictor_68_face_landmarks model can be downloaded from http://dlib.net/files/

  • model.py contains the implementation of different model structures.

Notice

  • The current version of code only tested on MacOS.
  • We refactor the current implementation from our original training code. If you find any implementation error, please do not hesitate to contact us.
  • Code for domain adaptation will be added quickly.

License