A real time face recognition system is capable of identifying or verifying a person from a video frame. To recognize the face in a frame, first you need to detect whether the face is present in the frame. If it is present, mark it as a region of interest (ROI), extract the ROI and process it for facial recognition.
1.OpenCV installed :- for that install conda which include all libraries -Conda is an open source package management system and environment management system that runs on Windows, macOS and Linux. Conda quickly installs, runs and updates packages and their dependencies.
- use Spyder IDE for Python Programming Python
-install the python from this link : https://www.python.org/downloads/
-install numpy by typing on commnad promt :-
pip install "numpy‑1.14.2+mkl‑cp36‑cp36m‑win32.whl"
-install opencv by typing on commnad promt:-
pip install "opencv_python‑3.4.1+contrib‑cp36‑cp36m‑win32.whl'
-Note the Successfully installed … message after each command. You can safely ignore the upgrade pip suggestion.
after installation check allrequired libraries install or not for that check following in cmd : Start the Python interpreter and write:
import cv2
print(cv2.__version__)
and confirm that you are running the 3.5 (or above) version
If no error messages appear, the OpenCV is correctly installed ON YOUR PYTHON
OpenCV install, in my case this was 3.4.1.
2.in face recognition reequired image dataset so, go to following link :- The sources of images are:
http://www.imageprocessingplace.com/r...
http://sipi.usc.edu/database
-
Once you have OpenCV installed in your laptop or pc let's test to confirm that your camera is working properly.
import numpy as np import cv2 cap = cv2.VideoCapture(0) cap.set(3,640) # set Width cap.set(4,480) # set Height while(True): ret, frame = cap.read() frame = cv2.flip(frame, -1) # Flip camera vertically if your cam is not flip just comment this line gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) cv2.imshow('frame', frame) cv2.imshow('gray', gray) k = cv2.waitKey(30) & 0xff if k == 27: # press 'ESC' to quit break cap.release() cv2.destroyAllWindows()
*The above code will capture the video stream that will be generated by your webCam, displaying both, in BGR color and Gray mode. *note:-To finish the program, you must press the key [ESC] on your keyboard
**This project was done with this fantastic "Open Source Computer Vision Library", the OpenCV
Object Detection using Haar feature-based cascade classifiers is an effective object detection method proposed by Paul Viola and Michael Jones in their paper, "Rapid Object Detection using a Boosted Cascade of Simple Features" in 2001. It is a machine learning based approach where a cascade function is trained from a lot of positive and negative images. It is then used to detect objects in other images.
The most common way to detect a face (or any objects), is using the "Haar Cascade classifier"
OpenCV comes with a trainer as well as detector. If you want to train your own classifier for any object like car, planes etc. you can use OpenCV to create one. Its full details are given here: Cascade Classifier Training.
Here we will deal with detection. OpenCV already contains many pre-trained classifiers for face, eyes, smile etc. Those XML files are stored in opencv/data/haarcascades/ folder. Let's create face and eye detector with OpenCV.
First we need to load the required XML classifiers. Then load our input image (or video) in grayscale mode. Then, we will set our camera and inside the loop, load our input video in grayscale mode.
import numpy as np
import cv2
face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
eye_cascade = cv2.CascadeClassifier('haarcascade_eye.xml')
img = cv2.imread('imageName.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
Now we find the faces in the image. If faces are found, it returns the positions of detected faces as Rect(x,y,w,h). Once we get these locations, we can create a ROI for the face and apply eye detection on this ROI (since eyes are always on the face !!! ).
faces = face_cascade.detectMultiScale(gray, 1.3, 5)
for (x,y,w,h) in faces:
cv2.rectangle(img,(x,y),(x+w,y+h),(255,0,0),2)
roi_gray = gray[y:y+h, x:x+w]
roi_color = img[y:y+h, x:x+w]
eyes = eye_cascade.detectMultiScale(roi_gray)
for (ex,ey,ew,eh) in eyes:
cv2.rectangle(roi_color,(ex,ey),(ex+ew,ey+eh),(0,255,0),2)
cv2.imshow('img',img)
cv2.waitKey(0)
cv2.destroyAllWindows()
Here we will work with face detection. Initially, the algorithm needs a lot of positive images (images of faces) and negative images (images without faces) to train the classifier. Then we need to extract features from it. The good news is that OpenCV comes with a trainer as well as a detector. If you want to train your own classifier for any object like car, planes etc. you can use OpenCV to create one. Its full details are given here:
https://docs.opencv.org/3.3.0/dc/d88/tutorial_traincascade.html
If you do not want to create your own classifier, OpenCV already contains many pre-trained classifiers for face, eyes, smile, etc. Those XML files can be download from haarcascades directory at
https://github.com/opencv/opencv/tree/master/data/haarcascades
When you compare with the last code used to test the camera, you will realize that few parts were added to it. Note the line below:
faceCascade = cv2.CascadeClassifier('Cascades/haarcascade_frontalface_default.xml')
*This is the line that loads the "classifier" (that must be in a directory named "Cascades/", under your project directory).
Now we must call our classifier function, passing it some very important parameters, as scale factor, number of neighbors and minimum size of the detected face.
faces = faceCascade.detectMultiScale(
gray,
scaleFactor=1.2,
minNeighbors=5,
minSize=(20, 20)
)
Where, .gray is the input grayscale image. .scaleFactor is the parameter specifying how much the image size is reduced at each image scale. It is used to create the scale pyramid. .minNeighbors is a parameter specifying how many neighbors each candidate rectangle should have, to retain it. A higher number gives lower false positives. .minSize is the minimum rectangle size to be considered a face.
The function will detect faces on the image. Next, we must "mark" the faces in the image, using, for example, a blue rectangle. This is done with this portion of the code:
for (x,y,w,h) in faces:
cv2.rectangle(img,(x,y),(x+w,y+h),(255,0,0),2)
roi_gray = gray[y:y+h, x:x+w]
roi_color = img[y:y+h, x:x+w]
If faces are found, it returns the positions of detected faces as a rectangle with the left up corner (x,y) and having "w" as its Width and "h" as its Height ==> (x,y,w,h).
Once we get these locations, we can create an "ROI" (drawn rectangle) for the face and present the result with imshow() function.
You can also include classifiers for "eyes detection" or even "smile detection". On those cases, you will include the classifier function and rectangle draw inside the face loop, because would be no sense to detect an eye or a smile outside of a face.
On my GitHub you will find examples:- these code you could find in the faceDetectionProject 1.faceEyeDetection.py
2.faceSmileDetection.py
3.faceSmileEyeDetection.py
*on my Github you will find the other folder :faceRecognitionProject In this folder we would be starting with the training and recognising the image from webcam
1.Face Detection and Data Gathering
2.Train the Recognizer
3.Face Recognition
So now we rare ready to code the trainner,
Just like all the other opencv script we need:
import the opencv / cv2 library,
we will need the os to accress the file list in out dataset folder, we also need to import the numpy library, and we need to import the pillow / PIL library we installed before,
It will look something like this:
import cv2,os
import numpy as np
from PIL import Image
Python
Now we need to initialize the recognizer and the face detector
recognizer = cv2.createLBPHFaceRecognizer()
detector= cv2.CascadeClassifier("haarcascade_frontalface_default.xml");
Python
Load The Training Data Ok, now we will are going to create a function which will grab the training images from the dataset folder, and will also get the corresponding Ids from its file name, (remember we formatted the filename to be like User.id.samplenumber in our previous script)
So I am going to name this function “getImagesAndLabels” we need the path of the dataset folder so we will provide the folder path as argument. So the function will be like this
def getImagesAndLabels(path):Python
So now inside this function we are going to do the following
Load the training images from dataset folder capture the faces and Id from the training images Put them In a List of Ids and FaceSamples and return it To load the image we need to create the paths of the image
imagePaths=[os.path.join(path,f) for f in os.listdir(path)] Python
this will get the path of each images in the folder. now we need to create two lists for faces and Ids to store the faces and Ids
faceSamples=[]
Ids=[]
Python Now we will loop the images using the image path and will load those images and Ids, we will add that in your lists
for imagePath in imagePaths:
pilImage=Image.open(imagePath).convert('L')
imageNp=np.array(pilImage,'uint8')
Id=int(os.path.split(imagePath)[-1].split(".")[1])
faces=detector.detectMultiScale(imageNp)
for (x,y,w,h) in faces:
faceSamples.append(imageNp[y:y+h,x:x+w])
Ids.append(Id)Python
In the above code we used used “Image.open(imagePath).convert(‘L’)” is loading the image and converting it to gray scale, but now its a PIL image we need to convert it to numpy array. for that we are converting it to numpy array “imageNP=np.array(pilImage,’uint8′)”. To get the Id we split the image path and took the first from the last part (which is “-1” in python) and that is the name of the imagefile. now here is the trick, we saved the file name in our previous program like this “User.Id.SampleNumber” so if we split this using “.” the we will get 3 token in a list “User”, “Id”, “SampleNumber” so to get the Id we will choone 1st index (index starts from 0)
So we get:
Id=int(os.path.split(imagePath)[-1].split(".")[1])Python
Now we are using the detector to extract the faces and append them in the faceSamples list with the Id
which looks like:
for (x,y,w,h) in faces:
faceSamples.append(imageNp[y:y+h,x:x+w])
Ids.append(Id)Python
So we are done now we just have to return that value
return faceSamples,IdsPython
Now the entire function will look like this def getImagesAndLabels(path):
#get the path of all the files in the folder
imagePaths=[os.path.join(path,f) for f in os.listdir(path)]
#create empth face list
faceSamples=[]
#create empty ID list
Ids=[]
#now looping through all the image paths and loading the Ids and the images
for imagePath in imagePaths:
#loading the image and converting it to gray scale
pilImage=Image.open(imagePath).convert('L')
#Now we are converting the PIL image into numpy array
imageNp=np.array(pilImage,'uint8')
#getting the Id from the image
Id=int(os.path.split(imagePath)[-1].split(".")[1])
# extract the face from the training image sample
faces=detector.detectMultiScale(imageNp)
#If a face is there then append that in the list as well as Id of it
for (x,y,w,h) in faces:
faceSamples.append(imageNp[y:y+h,x:x+w])
Ids.append(Id)
return faceSamples,IdsPython
Almost Done!! We are almost finished, now we just have to call that function and feed the data to the recognizer to train
faces,Ids = getImagesAndLabels('dataSet')
recognizer.train(faces, np.array(Ids))
recognizer.save('trainner/trainner.yml')Python
Thats it!! Now if we run this code it will create a “trainner.yml” file inside the trainner folder, We will use this file in our next post to actually recognize the faces that we trained the face recognizer to recognize,
The Complete Code is given below :-
import cv2,os
import numpy as np
from PIL import Image
recognizer = cv2.createLBPHFaceRecognizer()
detector= cv2.CascadeClassifier("haarcascade_frontalface_default.xml");
def getImagesAndLabels(path):
#get the path of all the files in the folder
imagePaths=[os.path.join(path,f) for f in os.listdir(path)]
#create empth face list
faceSamples=[]
#create empty ID list
Ids=[]
#now looping through all the image paths and loading the Ids and the images
for imagePath in imagePaths:
#loading the image and converting it to gray scale
pilImage=Image.open(imagePath).convert('L')
#Now we are converting the PIL image into numpy array
imageNp=np.array(pilImage,'uint8')
#getting the Id from the image
Id=int(os.path.split(imagePath)[-1].split(".")[1])
# extract the face from the training image sample
faces=detector.detectMultiScale(imageNp)
#If a face is there then append that in the list as well as Id of it
for (x,y,w,h) in faces:
faceSamples.append(imageNp[y:y+h,x:x+w])
Ids.append(Id)
return faceSamples,Ids
faces,Ids = getImagesAndLabels('dataSet')
recognizer.train(faces, np.array(Ids))
recognizer.save('trainner/trainner.yml')
To make sure the trainner doesn’t take any file other that the jpg files we will add an if condition in the method
import cv2,os
import numpy as np
from PIL import Image
recognizer = cv2.createLBPHFaceRecognizer()
detector= cv2.CascadeClassifier("haarcascade_frontalface_default.xml");
def getImagesAndLabels(path):
#get the path of all the files in the folder
imagePaths=[os.path.join(path,f) for f in os.listdir(path)]
#create empth face list
faceSamples=[]
#create empty ID list
Ids=[]
#now looping through all the image paths and loading the Ids and the images
for imagePath in imagePaths:
# Updates in Code
# ignore if the file does not have jpg extension :
if(os.path.split(imagePath)[-1].split(".")[-1]!='jpg'):
continue
#loading the image and converting it to gray scale
pilImage=Image.open(imagePath).convert('L')
#Now we are converting the PIL image into numpy array
imageNp=np.array(pilImage,'uint8')
#getting the Id from the image
Id=int(os.path.split(imagePath)[-1].split(".")[1])
# extract the face from the training image sample
faces=detector.detectMultiScale(imageNp)
#If a face is there then append that in the list as well as Id of it
for (x,y,w,h) in faces:
faceSamples.append(imageNp[y:y+h,x:x+w])
Ids.append(Id)
return faceSamples,Ids
faces,Ids = getImagesAndLabels('dataSet')
recognizer.train(faces, np.array(Ids))
recognizer.save('trainner/trainner.yml')
Face Recognition Using OpenCV | Loading Recognizer In my previous trainnner we learnt to train a recognizer using a dataset, in this post we are loading recognizer to see how we can use that recognizer to recognize faces.
If you are following my above line of the trainner then you already have the trained recognizer with you inside a folder named “trainner” and “trainner.yml” file inside it. Now we are going to use that training data to recognize some faces we previously trained .
Lets Start By Importing The Libraries
import cv2
import numpy as np
Python
Yes that’s it, thats all we need for this projects.
Now Loading Recognizer
next we create a recognizer object using opencv library and load the training data (before that just save your script in the same location where your “trainner” folder is located)
recognizer = cv2.createLBPHFaceRecognizer()
recognizer.load('trainner/trainner.yml')
Python
Now we will create a cascade classifier using haar cascade for face detection, assuming u have the cascade file in the same location,
cascadePath = "haarcascade_frontalface_default.xml"
faceCascade = cv2.CascadeClassifier(cascadePath);Python
Now we will create the video capture object
cam = cv2.VideoCapture(0)Python
Next we need a “font” that’s because we are going to write the name of that person in the image so we need a font for the text
font = cv2.cv.InitFont(cv2.cv.CV_FONT_HERSHEY_SIMPLEX, 1, 1, 0, 1, 1) Python
Okay so the first parameter is the font name, 2nd and 3rd is the horizontal and the vertical scale, 4rth is shear (like italic), 5th is thickness of line, 6th is line type
So we have all setup Lets Start the main Loop Lets start the main loop and do the following basic steps
Starts capturing frames from the camera object Convert it to Gray Scale Detect and extract faces from the images Use the recognizer to recognize the Id of the user Put predicted Id/Name and Rectangle on detected face
while True:
ret, im =cam.read()
gray=cv2.cvtColor(im,cv2.COLOR_BGR2GRAY)
faces=faceCascade.detectMultiScale(gray,1.2,5)
for(x,y,w,h) in faces:
cv2.rectangle(im,(x-50,y-50),(x+w+50,y+h+50),(225,0,0),2)
Id, conf = recognizer.predict(gray[y:y+h,x:x+w])
cv2.cv.PutText(cv2.cv.fromarray(im),str(Id), (x,y+h),font, 255)
cv2.imshow('im',im)
if cv2.waitKey(10) & 0xFF==ord('q'):
breakPython
So its pretty similar to the face detection code the only difference is the following lines
Id, conf = recognizer.predict(gray[y:y+h,x:x+w])
cv2.cv.PutText(cv2.cv.fromarray(im),str(Id), (x,y+h),font, 255)Python
in the above two line the recognizer is predicting the user Id and confidence of the prediction respectively in the next line we are writing the User ID in the screen below the face, which is (x, y+h) coordinate
Just Little Finishing Touch (For Unknown Faces)
Now with this we are pretty much done we can add some more finishing touch like its showing user Id instead of the name, and it cant handle unknown faces,
So to add this additional features we can do the following,
Id, conf = recognizer.predict(gray[y:y+h,x:x+w])
if(conf<50):
if(Id==1):
Id="Anirban"
elif(Id==2):
Id="name you want to add "
else:
Id="Unknown"
cv2.cv.PutText(cv2.cv.fromarray(im),str(Id), (x,y+h),font, 255)Python
Now Some Cleanup
cam.release()
cv2.destroyAllWindows()Python
Now that everything is done, we need to close the camera and the windows. and we are done!!!! Andyou will see the results.
Loading recognizer which is previously trained, and using it in face recognition system, names are being displayed on your screen where there faces.
The Complete Face Recognition Code is written bellow :=>
import cv2
import numpy as np
recognizer = cv2.createLBPHFaceRecognizer()
recognizer.load('trainner/trainner.yml')
cascadePath = "haarcascade_frontalface_default.xml"
faceCascade = cv2.CascadeClassifier(cascadePath);
cam = cv2.VideoCapture(0)
font = cv2.cv.InitFont(cv2.cv.CV_FONT_HERSHEY_SIMPLEX, 1, 1, 0, 1, 1)
while True:
ret, im =cam.read()
gray=cv2.cvtColor(im,cv2.COLOR_BGR2GRAY)
faces=faceCascade.detectMultiScale(gray, 1.2,5)
for(x,y,w,h) in faces:
cv2.rectangle(im,(x,y),(x+w,y+h),(225,0,0),2)
Id, conf = recognizer.predict(gray[y:y+h,x:x+w])
if(conf<50):
if(Id==1):
Id="rahul"
elif(Id==2):
Id="irfan"
else:
Id="Unknown"
cv2.cv.PutText(cv2.cv.fromarray(im),str(Id), (x,y+h),font, 255)
cv2.imshow('im',im)
if cv2.waitKey(10) & 0xFF==ord('q'):
break
cam.release()
cv2.destroyAllWindows()Python