deepface

deepface is a lightweight face recognition and facial attribute analysis (age, gender, emotion and race) framework for python. It is a hybrid face recognition framework wrapping state-of-the-art models: VGG-Face, Google FaceNet, OpenFace, Facebook DeepFace, DeepID and Dlib. The library is mainly based on Keras and TensorFlow.

Installation

The easiest way to install deepface is to download it from PyPI.

pip install deepface

Face Recognition

A modern face recognition pipeline consists of 4 common stages: detect, align, represent and verify. deepface handles all these common stages in the background.

Face Verification - Demo

Verification function under the deepface interface offers to verify face pairs as same person or different persons. You should pass face pairs as array instead of calling verify function in a for loop for the best practice. This will speed the function up dramatically and reduce the allocated memory.

from deepface import DeepFace
result  = DeepFace.verify("img1.jpg", "img2.jpg")
#results = DeepFace.verify([['img1.jpg', 'img2.jpg'], ['img1.jpg', 'img3.jpg']])
print("Is verified: ", result["verified"])

Large scale face recognition - Demo

Face recognition requires to apply face verification several times. Herein, deepface offers an out-of-the-box find function to handle this action. You can apply face recognition on a large scale data set as well.

from deepface import DeepFace
import pandas as pd
df = DeepFace.find(img_path = "img1.jpg", db_path = "C:/workspace/my_db")
#dfs = DeepFace.find(img_path = ["img1.jpg", "img2.jpg"], db_path = "C:/workspace/my_db")

Face recognition models - Demo

Deepface is a hybrid face recognition package. It currently wraps the state-of-the-art face recognition models: VGG-Face , Google FaceNet, OpenFace, Facebook DeepFace, DeepID and Dlib. The default configuration verifies faces with VGG-Face model. You can set the base model while verification as illustared below.

models = ["VGG-Face", "Facenet", "OpenFace", "DeepFace", "DeepID", "Dlib"]
for model in models:
   result = DeepFace.verify("img1.jpg", "img2.jpg", model_name = model)

FaceNet, VGG-Face and Dlib overperforms than OpenFace, DeepFace and DeepID based on experiments.

Similarity

Face recognition models are regular convolutional neural networks and they are responsible to represent faces as vectors. Decision of verification is based on the distance between vectors. We can classify pairs if its distance is less than a threshold.

Distance could be found by different metrics such as Cosine Similarity, Euclidean Distance and L2 form. The default configuration finds the cosine similarity. You can alternatively set the similarity metric while verification as demostratred below.

metrics = ["cosine", "euclidean", "euclidean_l2"]
for metric in metrics:
   result = DeepFace.verify("img1.jpg", "img2.jpg", distance_metric = metric)

Euclidean L2 form seems to be more stable than cosine and regular Euclidean distance based on experiments.

Facial Attribute Analysis - Demo

Deepface also offers facial attribute analysis including age, gender, facial expression (including angry, fear, neutral, sad, disgust, happy and surprise) and race (including asian, white, middle eastern, indian, latino and black) predictions. Analysis function under the DeepFace interface is used to find demography of a face.

from deepface import DeepFace
obj = DeepFace.analyze("img4.jpg", actions = ['age', 'gender', 'race', 'emotion'])
#objs = DeepFace.analyze(["img1.jpg", "img2.jpg", "img3.jpg"]) #analyzing multiple faces same time
print(obj["age"]," years old ",obj["dominant_race"]," ",obj["dominant_emotion"]," ", obj["gender"])

Face Detectors - Demo

Face detection and alignment are early stages of a modern face recognition pipeline. OpenCV haar cascade, SSD, Dlib and MTCNN methods are wrapped in deepface as a detector. You can optionally pass a custom detector to functions in deepface interface. OpenCV is the default detector if you won't pass a detector.

backends = ['opencv', 'ssd', 'dlib', 'mtcnn']
for backend in backends:
   #face detection and alignment
   detected_face = DeepFace.detectFace("img.jpg", detector_backend = backend)
   
   #face verification
   obj = DeepFace.verify("img1.jpg", "img2.jpg", detector_backend = backend)
   
   #face recognition
   df = DeepFace.find(img_path = "img.jpg", db_path = "my_db", detector_backend = backend)
   
   #facial analysis
   demography = DeepFace.analyze("img4.jpg", detector_backend = backend)

MTCNN seems to overperform in detection and alignment stages but it is slower than SSD.

Streaming and Real Time Analysis - Demo

You can run deepface for real time videos as well.

Calling stream function under the DeepFace interface will access your webcam and apply both face recognition and facial attribute analysis. Stream function expects a database folder including face images. VGG-Face is the default face recognition model and cosine similarity is the default distance metric similar to verify function. The function starts to analyze if it can focus a face sequantially 5 frames. Then, it shows results 5 seconds.

from deepface import DeepFace
DeepFace.stream("C:/User/Sefik/Desktop/database")

Even though face recognition is based on one-shot learning, you can use multiple face pictures of a person as well. You should rearrange your directory structure as illustrated below.

user
├── database
│   ├── Alice
│   │   ├── Alice1.jpg
│   │   ├── Alice2.jpg
│   ├── Bob
│   │   ├── Bob.jpg

Ensemble learning for face recognition - Demo

A face recognition task can be handled by several models and similarity metrics. Herein, deepface offers a special boosting and combination solution to improve the accuracy of a face recognition task. This provides a huge improvement on accuracy metrics. Human beings could have 97.53% score for face recognition tasks whereas this ensemble method passes the human level accuracy and gets 98.57% accuracy. On the other hand, this runs much slower than single models.

resp_obj = DeepFace.verify("img1.jpg", "img2.jpg", model_name = "Ensemble")
df = DeepFace.find(img_path = "img1.jpg", db_path = "my_db", model_name = "Ensemble")

API - Demo

Deepface serves an API as well. You can clone /api/api.py and pass it to python command as an argument. This will get a rest service up. In this way, you can call deepface from an external system such as mobile app or web.

python api.py

The both face recognition and facial attribute analysis are covered in the API. You are expected to call these functions as http post methods. Service endpoints will be http://127.0.0.1:5000/verify for face recognition and http://127.0.0.1:5000/analyze for facial attribute analysis. You should pass input images as base64 encoded string in this case. Here, you can find a postman project.

Passing pre-built face recognition models

You can build models once and pass to deepface functions as well. This speeds you up if you are going to call deepface several times.

#face recognition
from deepface.basemodels import VGGFace, OpenFace, Facenet, FbDeepFace, DeepID
model = VGGFace.loadModel() #all face recognition models have loadModel() function in their interfaces
DeepFace.verify("img1.jpg", "img2.jpg", model_name = "VGG-Face", model = model)

#facial analysis
import json
from deepface.extendedmodels import Age, Gender, Race, Emotion
models = {}
models["emotion"] = Emotion.loadModel()
models["age"] = Age.loadModel()
models["gender"] = Gender.loadModel()
models["race"] = Race.loadModel()
DeepFace.analyze("img1.jpg", models=models)

E-Learning

Deepface package for python is mentioned in this playlist as video lectures. Subscribe the channel to stay up-to-date and be informed when a new lecture is added.

Translations

You can also read a translated version of deepface tutorials in Chinese (深臉) or Turkish.

Disclaimer

Reference face recognition models have different type of licenses. This framework is just a wrapper for those models. That's why, licence types are inherited as well. You should check the licenses for the face recognition models before use.

Herein, OpenFace is licensed under Apache License 2.0. FB DeepFace and Facenet is licensed under MIT License. Dlib is licensed under Boost Software License. They all allow you to use for personal and commercial purpose free.

On the other hand, VGG-Face is licensed under Creative Commons Attribution License. That's why, it is restricted to adopt VGG-Face for commercial use.

Support

There are many ways to support a project - starring⭐️ the GitHub repos is just one.

Citation

Please cite deepface in your publications if it helps your research. Here is an example BibTeX entry:

@inproceedings{serengil2020lightface,
  title={LightFace: A Hybrid Deep Face Recognition Framework},
  author={Serengil, Sefik Ilkin and Ozpinar, Alper},
  booktitle={2020 Innovations in Intelligent Systems and Applications Conference (ASYU)},
  year={2020},
  organization={IEEE}
}

Licence

Deepface is licensed under the MIT License - see LICENSE for more details.

Logo is created by Adrien Coquet. Licensed under Creative Commons: By Attribution 3.0 License.

meet-ai/deepface