/Face-recognition-via-SVD-and-PCA

Program will detect the face from the web-camera and try to recognize it among existing faces in the DB.

Primary LanguagePython

Face Recognition Project

Overview

This project implements a face detection and recognition in Python (based on Eigenfaces, SVD, and PCA).

Notice: the part of the code was taken from the source [4] and extended.

Technologies and devices used:

  • Python 2.7
  • Python libraries:
    • OpenCV v.2.4.12
    • NumPy
    • Tkinter
    • Os (to scan files)
  • Ordinary Web Camera
  • Tested on the device "Banana Pro"

Testing the script

Recognize the image from the web camera

  1. Run "main.py" in the terminal: 
    python main.py
  2. Press on the "Space" button on your keyboard. Here, the program will try to recognize the face from the web camera among the existing faces in the DB. During the first recognition, the program will compute eigenfaces, SVD, and PCA. Once it's computed, it'll work much faster.

Recognized face from the DB

Add new image to the face DB

  • either add this image to the folder "target_imgs"
  • or while running "main.py" in the terminal (command "python image_preprocessing.py") or run this script from your IDE press on "Space" to capture new image from a web camera.

Capture and save image from web camera

After a new image is added, you will need to stop the script "main.py" and run pre-processing step again (to grayscale the image, detect a face, and resize it) by typing command in the terminal "python image_preprocessing.py":

python image_preprocessing.py

Project Description in details

  • Face Detection

    The OpenCV library includes Cascade Classification for object recognition, which can be used for real-time face detection. You can find All available cascades in the data folder in the compiled OpenCV files.

    Briefly, The Haar feature-based cascade classifiers created by Paul Viola and Michael Jones to detect different types of objects that rely on edge and specific features in the images (e.g. each face contains at least by 3 features: nose, mouth, eyes) [1]. Also, the Haar classifier is based on assumption that regions with eyes and mouth are darker than cheeks and forehead [2].

    We used already pre-trained Haar cascade classifiers for face detection "haarcascade_frontalface.xml".

  • Image Preprocessing

    • Grayscale image
    • Detect and extract face
    • Resize extracted face to the target Width x Height
    • Save preprocessed image

  • Face Recognition

    • Transform the image into a vector. For example, we have an image 100x100 pixels. As a result, we will have a vector of size 1x10,000 (flatten image). Read all pre-processed train images and flatten them (they must have the same size)
    • Compute the mean face. The example of the computed mean face is below (~5 images in the face DB):

    Computed mean face

    • Subtract the mean face from each image before performing SVD and PCA

    • Compute the SVD for the matrix from the previous step.

      As a result, we got:

      U, S, Vt = SVD(A)

      Where the matrix U represents the eigenfaces. To reconstruct our initial matrix A we multiply U, S, Vt:

      A' = U x S x Vt

      On this step, we can reduce the dimensionality of the initial matrix A by sorting only the most important features and slicing matrices U, S, Vt accordingly, then multiplication of all these 3 matrices will give an approximation of the initial matrix of images but with reduced dimensionality.

    • Project onto PCA space In order to select the most similar face to the input face, we will need to project the received features onto the PCA space where each feature will be in a new dimension, so the number of features = the number of dimensions. Feature representation Image source [3].

      To do the PCA projections, we need to obtain weights of these features for each dimension by multiplying the reconstructed array of images with subtracted mean face by eigenfaces.

    • Project test image onto PCA space and find the most similar face in the face database.

      We need to grayscale the target image, flatten it (convert to vector), and project onto PCA space by multiplying the obtained eigenfaces from step #3 by target image (flatten and grayscale).

      To find the identical face we will compute and compare the Euclidian distance between feature weights of test image and weights of all other images obtained in step #3. The image with the smallest score (sum of distances in all feature dimentions) will be classified as the result.

Limitations of this approach

  • Currently, the algorithm has an assumption that there is only 1 face photo in database instead of having several photos of 1 face, but made from different angles or/and in different conditions.

  • PCA relies on linear assumptions.

  • We do not take into account different variations of the face position.

    The photo with face can be taken from different angles. That is why it is better either to use several photos of the same face which were made from different angles or using 3D face model.

  • Face detection does not take into account alignment of the face and recognizes only 1 face from the picture (even if there are several faces in one image).

  • We do not know how the algorithm will behave with the same face, but with differences in age (e.g. photo that was made 10 years ago)

Plans for future improvements

  • Test the accuracy of the algorithm
  • Extract only K important features to reduce dimensions during projecting onto PCA space
  • Add histogram equalization to frame from web camera/image to improve the quality of the recognition
  • Measure execution time of the script and detect the most time-consuming parts of the code (by using timeit or similar tools)
  • Add threshold to detect non-existing image in the face DB

Alternative and similar methods

References

  1. Face Detection using Haar Cascades
  2. Mastering OpenCV with Practical Computer Vision Projects (Chapter 8) by D.Baggio, S.Emami, D.Escrivá, K.Ievgen, N.Mahmood, J.Saragih, R.Shilkrot
  3. Geometric explanation of PCA
  4. Eigenfaces and Forms
  5. Original paper about eigenfaces:
    "Turk, M., and Pentland, A. "Eigenfaces for recognition.". Journal of Cognitive Neuroscience"
  6. Face recognition datasets
  7. Principal component analysis in Python
  8. PCA in Python (Jupiter notebook)
  9. PCA via SVD (Matlab example)
  10. Matlab to Numpy syntax
  11. Yale public face dataset
  12. Code for the book "Mastering OpenCV with Practical Computer Vision Projects" by Packt Publishing 2012.
  13. Data-Driven Modeling & Scientific Computation. Chapter 15: Linear Algebra and Singular Value Decomposition (SVD)
  14. Description of eigenfaces algorithm in OpenCV doc.