: In this repository, a program was developed to identify the driver's drowsiness based on real-time camera image and image processing techniques, and this program makes warning alarms go off for each level of drowsiness when it detects drowsiness driving.
: Based on the real-time Vision System, drivers' face and eye detection techniques were added, as well as removing lighting effects due to the eye detection false positives, drowsiness detection techniques, and supervised learning algorithms to identify drowsiness level.
The Histogram of Oriented Gradients technology and the learned Face Landmark estimation techniques were used to detect faces and eyes.
In order to eliminate the effects of lighting, the light channels of the original images were separated and reversed, and then composed with the grayscale images of original images.
Furthermore the concept of Eye Aspect Ratio was used to detect drivers' drowsiness.
Finally, the KNN algorithm was used to divide the drivers' level of drowsiness into three stages, and differential alarms go off for each stages.
Through these works, we could research and make technology of intelligent vehicle systems and vision computing, which is gaining much attention recently.
This code is in Python 3.6
Get face images from the camera -> Grayscaling -> Light processing -> HOG & find face -> Face Landmark Estimation -> Detect drowsiness driving.
- In detail
0 : The filming.
10 : Lightness preprocessing.
100 : Detecting drowsiness.
110 : Getting face's image.
120 : Finding eyes region.
130 : Determining the value of the EAR normally.
140 : Determining drowsiness driving.
141 : Calculating the value of the EAR.
142 : Calculating the amount of time eyes are closed.
143 : Calculating the amount of time eyes are opened.
144 : Determining the level of the drowsiness.
- Using the HOG face pattern, to find the face from the Grayscaled-HOG-input-image.
- Use the Face Landmark Estimation algorithm to locate the landmarks on the face.
: Invert the lightness channel detached from the original image and composed it with the original grayscale image to produce a clear image.
- Converting color to grayscale using Luma Coding
- There are many different models in Color Space, the LAB color space model is the best way to separate Lightness. Median filtering is applied to convert the value of lightness(L) obtained by using the LAB color space to match the actual lighting conditions because it differs from the actual lighting conditions.
- The pictures below are the original image, image that separates L channel, image with Median filter applied, and inverted images from left to right. Drowsiness detection method
- Results of preprocessing
- Each eye is represented by 6 (x, y)-coordinates
- The EAR equation
- Calculated EAR
- The calculated EAR will have a value more than zero when the eyes are open, and a value close to zero when the eyes are closed.
- This program has set a 50% value from the average EAR value to the threshold value. So, 1) measures the average EAR value when the eyes are open, 2) measures the average EAR value when the driver is closing his eyes, and 3) sets the threshold using the above two results.
- .1) == 과정 1), 2) == 과정 2), 3) == 과정 3) (in drowsiness_detector code)
- Conditions :
- 30 FPS
- Prescribed speed : 100km/h, Retention distance between vehicles >= 100m
- The time which takes a person to push the brakes 0.45 (response time) + 0.2 (brake pushing time) + 0.05 (time to start braking) = 0.7 seconds
- The braking distance of a vehicle running at 100 km/h is 56 meters (the driver has 44 meters of free distance)
- Under the above conditions, the drivers has almost 0.9 seconds of free time (100km/h -> 27m/s == 1.63s of free time. 1.63 - 0.7 = 0.9 s).
- 30 FPS -> 27 frame = 0.9s.
- if EAR < threshold for 27 frame? then going alarm off.
- Now I separated the drowsiness phase into three steps.
- Drowsiness levels are identified by the following conditions.
- The first alarm will sound(approximately 0.9 seconds) between level 1 and 2 of the drowsy phase.
- If you are dozing (sleeping and waking again and again) in less than 15 seconds, the drowsiness phase starts at level 1 and then the next alarm goes up to 0.
- The first alarm is level 2 and the second alarm is level 1 and the third alarm makes level 0 sound when driving drowsy between 15 and 30 seconds.
- To distinguish drowsiness level, I used K-Nearest Neighbor(KNN) supervised learning algorithm.
. 1. Create arrays with random (x, y)-coordinates.
- Labeling
- Define K value.
- Test KNN algorithm.
- Before applying preprocessing
- After applying preprocessing
- I run drowsiness_detector.ipynb just typing CTRL+ENTER.