Driver Drowsiness Detection System using Machine Learning

Shivangi Srivastava1, Swekcha Srivastava2, Sanjeevani Srivastava3, Sakshi Mishra4, Sakil

Ahmad Ansari5
1,2,3,4
(Computer Science Engineering, Institute of Technology and Management, Gorakhpur, India.)
5
(Asst.Prof.at ITM Gorakhpur)

Shivangi Srivastava: [email protected]

Swekcha Srivastava: [email protected]
Sanjeevani Srivastava: [email protected]
Sakshi Mishra: [email protected]
Sakil Ahmad Ansari: [email protected]

ABSTRACT

In today's world, tiredness is one of the leading factors in traffic accidents, many of which result in fatalities.
According to statistics, drowsy driving causes the majority of traffic accidents, which often result in fatalities and
serious injuries. Because of this, numerous experiments have been conducted on creating software that can
identify driver drowsiness and warn them before a serious mistake is made. Some of the more popular techniques
create their own systems using techniques from the automotive industry. But other elements like the design of the
road, the type of vehicle, and the ability to drive using the driver's wheel heavily influenced these conventional
criteria. In order to monitor a driver's drowsiness, certain techniques use psychological methods, which frequently
yield the most precise and reliable results. These methods are expensive, though, because electrodes must be
applied to the head and torso. To generate results, the model is provided with a sizable database of closed and
open eyes. Every time the driver is observed to be sleepy, Buzz alerts the driver. In our model, we employ a typical
forward-facing smartphone camera and use the data we have collected to provide results for our website.

Keywords: Drowsiness, OpenCV, MobileNet architecture, Convolutional Neural Network (CNN), HAAR
cascade classifier, webcam.

INTRODUCTION

Drowsy driving is a common factor in fatal car accidents. Most drivers are aware of the risks associated with
texting while driving and driving while intoxicated, but many people underestimate the risks associated with
drowsy driving. According to the National Safety Council, drowsy driving causes around 100,000 collisions,
71,000 injuries, and 1,550 fatalities each year (NSC). Drowsiness was also discovered to be a contributing factor
in up to 9.5 percent of all crashes and 10.8 percent of crashes involving airbag deployment, injuries, or major
property damage, according to a study from the AAA Foundation for Traffic Safety. Drivers need to be aware of
the risks because driving while fatigued is extremely risky. Detecting driver drowsiness and warning the driver
are both crucial.

To sum up, there are several challenges that are needed to be resolved:

• The ability to recognise a driver's drowsiness while they are driving and sound an alarm or warning at the
appropriate moment is a key difficulty because any delay will increase the likelihood of an accident or failure.
• A portable webcam's low-quality video capturing capabilities can occasionally prevent it from detecting
drowsiness, which compromises the model's accuracy.
• As a result, there is a need for more, and there is also a demand for more affordable, adaptable equipment,
such as a webcam, that can take better pictures.
• The likelihood of an accident or failure will increase if the webcam feed occasionally shows a loss in frames
or self-pressing, which means that the output frame is given to the model with a delay of a few seconds and
a portion of the alarm sound or warning delay.
 LITERATURE REVIEW

Several methods have been suggested in an effort to improve drowsiness detection's accuracy and speed. The
previous drowsiness detection techniques and strategies are summarized in this section. A smartphone (either an
Android or an iPhone smartphone) serves as the proposed mechanism in [1]'s driver sleepiness detection system.
A smartphone's front camera records photos of the driver, which are subsequently sent to the device's CPU for
image processing. The main objective in [2] is to develop a drowsiness detection system for drivers based on the
measurement of yawning using eye and mouth detection. In order to prevent car accidents, it aims to ensure the
accurate detection of yawning expression in the presence of changing lighting conditions and facial occlusions.
The work displayed in [3] is based on the Transfer Learning tenets. To separate the images, it used the MobileNet
architecture. The architecture used by CNN for mobile vision and image segmentation is modelled after
MobileNet. [4] uses image processing to identify faces and eyes. For face detection, a cascade classifier based on
HAAR is used. The eyes are continuously tracked using an algorithm designed to track moving objects. The
PERCLOS algorithm was used to determine whether the driver was sleepy. The system will use a camera to track
the driver's eyes. When the driver is showing signs of fatigue, output in the form of sound and seat belt vibration
is provided. There will be no automatic deactivation of the warning. The final idea is to create drowsiness detection
wearable hardware, like smart watches. The driver's face is captured on camera in [5] and converted into numerous
image frames. The image frame is used as input, and facial features like the eyes and mouth are extracted from
the input image through pre-processing for face detection. The system uses the LBPH technique to detect features
like the mouth and eyes. The LBPH technique is used to extract the features of the mouth and eyes. Using the
HAAR detection method, the features of the eye (blink) and mouth (yawn) are identified. Check the blink rate of
each eye to see if it is open or closed. If the blink rate is higher than the normal threshold, the warning beep system
is alerted, and a red signal is displayed. If not, the steps continue to be executed, and a green signal is displayed.
Multilayer Perceptron Classifier16, also known as MLP, was used in [6] for data processing. MLP is a simple
neural network made up of entangled nodes (neurons) that represent the output from the input class. The artificial
neuron takes one or more inputs that resemble dendrites, adds them based on connection weights, and then
generates a class of outputs.

METHODOLOGY

1. System Configuration: The suggested driver sleepiness detection system is built up for real-world driving
in Figure 1. A webcam can be installed on a vehicle's dashboard. The webcam is pointed at the driver's face,
and it is held horizontally.

2. Algorithm Framework: The foundation of the model is transfer learning theory. Transfer learning is a
machine learning method in which a model designed for a task in the past is also used as the starting point
for a second activity model. They produce very accurate results on related problems. In order to divide up
photos, we used MobileNet architecture. The architecture used by CNN for mobile vision and image
segmentation is modelled around MobileNet.

3. HAAR Training: The OpenCV library provides functions for face and feature (eyes, mouth, sunglasses, etc)
detection. Classifiers can be trained using some of these functions. The method of training the classifiers to
recognize faces is known as HAAR training. Here, a cascade function is trained using both positive and
negative images. Each feature consists of a single value that is created by subtracting the sum of the pixels
under various regions of the images. For each characteristic, a different set of pixels are used for extraction.
All the extracted features will not be useful for the required process.

4. Face Detection: By analysing video stream visuals, this system can determine whether a user is sleepy.
Multiple CNNs-kernelized correlation filters, a more advanced in-video face-tracking technique, is used for
face detection. The eyes and lips were among the important areas in the driver's face that were located using
68 key points. To determine the driver's level of drowsiness, the system measured the ratio of closed-eye
frames to total frames, the duration of continuous eye closure, the frequency of blinking. As a last step, the
DDD system warns the driver.

5. Eye Detection: The proposed method makes use of HAAR Cascades to identify objects in real time, including
the driver's face and eyes. The OpenCV library is used by the model. Then, in order to keep track of the
number of frames for both open and closed eyes as well as the alarm status, we set the frame counter
"COUNTER" to 0 and the Boolean "ALARM ON" to OFF. If the recognized eyes are closed for five
consecutive frames, we raise the alarm, else the counter is restarted at 0.
6. Drowsiness Detection: In the suggested technique, HAAR Cascades are used to recognize things in real
time, such as the driver's face and eyes. Face identification is performed using multiple CNNs-kernelized
correlation filters, a more sophisticated in-video face-tracking method. The system calculates the length of
continuous eye closure, the frequency of blinking, and the ratio of closed-eye frames to total frames. The
DDD system finally alerts the driver. In the Setting tab, users may adjust settings including alert sound and
volume.

Figure 1: Flowchart showing entire process of Drowsiness Detection System

 CONCLUSION

Driver Drowsiness Detection was created to help a driver stay awake while driving in order to reduce car accidents
caused by drowsiness. The topic of this research was drowsy drivers and their potential to cause vehicle accidents.
The driver drowsiness detection system calculates the driver's tiredness level from the driver using a combination
of webcam, HAAR cascade classifier, drowsiness detection is used to calculate whether or not a driver is drowsy.
It retrieves photos from the camera at the same time, which is enough to determine a driver's features in real-time.
The system uses open-source software known as OpenCV image processing libraries to process the captured
pictures. The combination of a webcam and an OpenCV transforms the total system into a low-cost drowsiness
detection system.

REFERENCES

[1] Suraj Mali, Kiran Bangale, Rohan Walekar, Prof. P .T Shitole (2022). "Driver Drowsiness Detection Using
Smartphone".
[2] Piyush Ravi Kumar, Dr. Neeraj Dahiya (2019). "Driver Distraction and Drowsiness Detection System".
[3] Daksh Khetan, Arun Nawani, Anshul Aggarwal, Ms. Surinder Kaur (2022). "Driver Drowsiness Detection in
Real-time".
[4] V B Navya Kiran, Raksha R, Anisoor Rahman, Varsha K N, Dr. Nagamani N P (2020). "Driver Drowsiness
Detection".
[5] Naga Ramyatha K, Bhavya Sree K, Swathi Krishna K, Manjunath C R (2020). "Driver Drowsiness Detection
Alert System using Haar Method".
[6] Rateb Jabbara, Khalifa Al-Khalifaab, Mohamed Kharbechea, Wael Alhajyaseena, Mohsen Jafaric, Shan
Jiangc (2018). "Real-time Driver Drowsiness Detection for Android Application Using Deep Neural
Networks Techniques".
[7] Yaman Albadawi, Maen Takruri, Mohammed Awad (2022). "A Review of Recent Developments in Driver
Drowsiness Detection Systems".
[8] Miss. Ankita M. Bhoyar, Prof. S. N. Sawalkar (2019). "Implementation on Visual Analysis of Eye State
Using Image Processing for Driver Fatigue Detection".
[9] H. Varun Chand, J. Karthikeyan (2021). "CNN Based Driver Drowsiness Detection System Using Emotion
Analysis".
[10] Jay D. Fuletra and Dulari Bosamiya, "A Survey on Driver's Drowsiness Detection Techniques", International
Journal on Recent and Innovation Trends in Computing and Computation, Volume: 1, Issue: 1
[11] M. Ramzan, H. U. Khan, S. M. Awan, A. Ismail, M. Ilyas and A. Mahmood, "A Survey on State-of-the-Art
Drowsiness Detection Techniques," in IEEE Access, vol. 7, pp. 61904-61919, 2019.
[12] Mohamad-Hoseyn Sigari, Muhammad Reza Pourshahabi, Mohnsen Soryani and Mahmood Fathy, "A Review
on Driver Face Monitoring Systems for Fatigue and Distraction Detection", International Journal of
Advanced Science and Technology Vol.64.pp 73-100
[13] Bappaditya Mandal, Liyuan Li, Gang Sam Wang and Jle Lin, "Towards Detection of Bus Driver Fatigue Base
on Robust Visual Analysis of Eye State", IEEE Transactions on Intelligent Transportation Systems, vol 18,
No. 3, March 2017.
[14] Tereza Soukupova and Jan Cech, "Real-Time Eye Blink Detection using Facial Landmarks", 21st Computer
Vision Winter Workshop, Luke Cehovin, Rok Mandeljic, Vitomir Struc (eds.) Rimskke Toplice, Slovenia,
February 3-5 2016.
[15] S. Pandey, "Study Tonight," 12 August 2021. [Online]. Available: https://www.studytonight.com/post/dlib-
68-points-face-landmark detection-with-opencv-and-python. [Accessed 2021].
[16] D. Pandey, "Medium.com," Analytics Vidhya, April 2021. [Online]. Available:
https://medium.com/analytics-vidhya/eye-aspect-ratioear-and-drowsiness-detector-using-
dliba0b2c292d706. [Accessed 2021].