TRENDS IN SCIENCES, 2021; 18(xx): xxxxx RESEARCH ARTICLE

https://doi.org/

Computer Vision based Real-Time Car Driver’s Drowsiness Detection

system with Notifications
First name Last name* (Times New Roman, 12 pt, Bold)
Full Affiliation (Times New Roman, 10 pt, Italic)

(*Corresponding author’s e-mail: e-mail) (Times New Roman, 10 pt, Bold)

Received: xxx, Revised: xxx, Accepted: xxx (Times New Roman, 10 pt, Italic)

Highlights
Your highlights are the first thing about your paper that anyone online will read. Use the highlights
to tell that potential reader what's so great about your paper and why they want to read it. Highlights are
three to five bullet points that capture the novel results of your research as well as new methods that were
used during the study.

Graphical abstract
A graphical abstract is a single, concise, pictorial and visual summary of the main findings of the
article. This could either be the concluding figure from the article or a figure that is specially designed for
the purpose, which captures the content of the article for readers at a single glance.
1. Image size: Please provide an image with a minimum of 531 x 1328 pixels (hxw) using a
minimum resolution of 300 dpi. If you are submitting a larger image, then please use the same ratio (200
high x 500 wide).
2. File type: preferred file types are TIFF, EPS, PNG, JPEG
3. No additional text, outline or synopsis should be included. Any text or label must be part of the
image file.

Abstract
The Accidents are becoming very common nowadays. It’s caused due to lack of the driver's
attention while driving. This makes heavy losses and a lot of death. Drowsiness and sleeping are vital
cause for traffic accidents as a subsequent may occur injuries and also cause of deaths. Accidents related
to sleepiness often occur on highways, but also on major roads, even within localities. Considering this
problem, this paper proposed an actual Drowsiness Detection System applicable in vehicles with the help
of conventional Computer Vision applications. This system is used to detect drowsiness with a camera
that is located in forward-facing of the car driver and alerts the driver if there is an act of drowsiness. The
system easily detects drowsiness by means of the face and eyes from the specific area of the image and
eye blinks of the driver. A real-time webcam captures the driver's face and dlib, is a machine learning
model, also a pre- trained based model for facial landmark detection has been used to detect the faces.
The Convolutional Neural Network (CNN) is used for the detection of drowsiness. Then it recognizes the
eye boundaries of the eye from this actual video stream. Whereas every eye is symbolized by six
coordinates that will be started from the left corner of the eye and then work clockwise around the eye.
The Ear Aspect Ratio (EAR) is calculated for 20 consecutive frames that trigger an alarm when an
adaptive threshold value is reached. In the proposed system, following the face detection, the facial
components that are more important and considered as the most effective for drowsiness, are extracted
and tracked in video sequence frames. The system has been tested and implemented in a real
environment.
Keywords: component, formatting, style, styling, insert (key word)

Introduction
Text (Times New Roman, 10 pt, Tab 0.8 cm) Every year about 7,855 people die and 13,330 people
are injured due to road accidents in Bangladesh. It means more than 21 people lost their lives on roads
Trends Sci. 2021; 18(xx): xxxxx 2 of 17

each day. The main cause of road accidents is reckless driving, risky overtaking, fault in road design,
unfit vehicles, unskilled drivers, passengers and pedestrians’ unawareness, use of mobile phones or
headphones while driving vehicles, drunk driving, occupied footpath and poor enforcement of law for the
road crashes. Traffic accidents caused due to drowsiness and sleeping is more dangerous than drunk or
rush driving accidents. Because a driver may be unconscious, the drowsiness can cause serious traffic
accidents. Drowsiness and fatigue are major safety issues that cannot be measured directly. Drowsiness is
a state where a person sleeps or almost likely. It alludes to a failure to keep awake or a drive to rest [2].
Drowsiness and sleepiness considered in this paper as equivalent words. Although, in this paper
sleepiness is utilized as a part of a distinctive term as fatigue, which is a great tiredness because of
physical and mental action. Although people who fall asleep for more than a few minutes are often aware
of those lapses in wakefulness, drivers may not be aware of shorter lapses and microsleeps, which can
also have serious consequences when a quick reaction is needed to avoid high-speed crashes. Drowsiness
can likewise be depicted by the evaluation of alertness or vigilance. Attentiveness is the same as
sharpness or a condition of sleep inability, while vigilance can be depicted as watchfulness or a state
where one is readied for something to happen [3]. The steering response in the frequency domain of an
active driver and a drowsy driver is shown in Figure 1.

It is hard to give a definite measure of drowsiness in the way that is possible with for instance
alcohol. Moreover, drowsiness is a transient state, which additionally makes the discovery hard. We can
only imagine what it is like for Bangladesh which has a higher road accident rate compared to the USA.
We have discussed several Machine Learning algorithms that were proposed by other researchers to
automatically detect the drowsiness of the driver. However, these algorithms do not utilize all the relevant
features for classification such as the vertical distances of eyes and mouth. Automatic processes invented
to know about driver weariness and detect driver sleepiness will be an intrinsic part of the upcoming
automatic vehicle inventions. There are many features to consider for detecting driver fatigue and some of
them which are used till now are lane detection, pulse and heart rate, steering wheel movement[9] pattern
detection etc. . These are the ways in which the driver would not get any obstruction because of the
system. As described earlier, researchers also used physical and mental features [10] for detecting fatigue
while driving. In this work, we will be developing a Machine Learning, Pattern Recognition and
Computer Vision-based algorithm that will use all the relevant features for fast and accurate classification
of drowsiness in a driver. We will also develop an algorithm that will determine the level of drowsiness of
the driver called the rate of drowsiness.

Background and Related Work

It is very much tricky work to detect the driver’s drowsiness using a camera while driving. But, this
is a crucial factor to detect their drowsiness. In fact, road accidents during midnight time occurred
because of driver’s drowsiness.
Hu, S. et.al [1], discussed the detection of drowsiness of the driver using the Support Vector
Machine algorithm of Machine learning. The algorithm is implemented in the driving simulation device
to detect the abnormality in the driving of a driver. This paper is based on the eyelid observation of the
driver. Christopher Streiffer, created DarNet, a multi-modular information accumulation and investigation
framework intended to identify and characterize distracted driving behavior. Driver diversion is
distinguished as a critical supporter of motor vehicle accidents and wounds. DarNet, combined
convolutional & intermittent neural system that can break down driving pictures. IMU sensor information
to figure out how to recognize up to 6 classes of driving practices with expanded precision [2].
Trends Sci. 2021; 18(xx): xxxxx 3 of 17

In [11], traditional techniques are elaborated which are based on physiological measurements
including brain waves, heart rate, pulse rate and respiration. However, these techniques are intrusive in
nature. This paper[12] is based on Rowley’s eye detection code from the STASM library. However, the
presence of glasses adversely affects the performance of the system. This research [13] monitors only
yawning patterns of the driver using two separate cameras to acquire information of the upper part of the
body in order to track the driver’s mouth. However, the hardware dependency is higher.
Almost all the published researches which rest on machine learning approaches are image-based
real-time systems for drowsiness supervising using classic facial attributes. Singh[3] invented the
machine learning-based system that works on eye blink duration. Saito[4] used the driver’s psychological
and physical conditions to detect drowsiness. Horng[5] used edge detection for detecting eyes and
dynamical matching for eye-tracking and also for driver drowsiness recognition.
Smith[6] published a machine algorithm that mainly focuses on trailing a person’s head by using
optical flow. Kartik Dwivedi[7] invented a new drivers fatigue detection by anchoring multi-layer
convolutional neural networks. It is proved in Zutao zang[8] paper that if the person closes his/her eyes
for 5 seconds then he is considered as drowsy but in reality, we need more time to decide drowsiness of a
person and the rate at which the person is drowsy is also not considered. Vehicle-based measures provide
a full evaluation on the driving performance by analyzing the changes in the vehicle’s environment
including changes in speed, steering wheel movement, etc. [17, 14]. But the changes in the vehicle’s
environment are often the result of the last stage of drowsiness when the driver sleeps and loses control of
his car, which does not help to prevent the accident. Another problem that limits the use of this measure
in real world is the difficulty of adapting these systems to the changing nature of methods [15, 16].

Our Proposed System

The proposed work is depicted in fig. 3. It takes the front face image of the driver using a camera
which is already fixed in front of him. It contains two parts: Working of Night vision camera and
Prediction of Eye blinking rate, the frames are extracted to obtain 2-D images. A Convolutional Neural
Network (CNN) is being used for face detection through frame, actually is the first procedure in
automatic face recognition. It’s a very significant progression to the precise detection of human faces. The
face detection technique is a computer vision tool to detect the anterior faces of the person from images or
videos. This paper narrates computer vision practice for anterior faces detection through detecting facial
landmarks of eyes. Then it identifies the eye borders of the eye from the real-time video stream. An
equation that represents the eye aspect ratio in order to calculate the Eye Blinking occurrences, also
calculate the rate of eye opening events. Eye blinks can be distinguished by orienting major facial
landmarks, where The Eye Aspect Ratio (EAR) value is constant whenever eye is open, but swiftly drops
to 0 whenever eye is closed. The individual’s eye blinks are tangible. The Eye Aspect Ratio (EAR) drops
underneath a certain threshold then a program can calculate in the position where an individual’s eyes are
totally closed. From the facial landmarks, eye aspect ratio, and position of the eye are enumerated and
consuming these features and machine learning methods, a decision is obtained about the active mode,
drowsiness and sleeping mode of the vehicle driver. A database is maintained regarding all aspects of the
eye blinks detection. If drowsiness or sleepiness is detected, an alarm will be sent to the driver to alert
him/her and send a SMS through mobile to driver and vehicle owner as well as.
Trends Sci. 2021; 18(xx): xxxxx 4 of 17

In the proposed work, pre-existing features for facial landmark detection is implemented to identify
the state of drowsiness and fatigue. 68- facial landmark predefined landmark helps in shape prediction to
clearly identify the various regions of the face like eyebrows, eye, mouth region etc. Various changes in
parameters of these distinguished points reports various expressions of the person. Here, the video
captured from the camera for detecting the eye. The system was embedded with programs which get the
data and identify the eye by the points (p1 and p2) were set with limits at two ends of the eyes.

The blink rate calculation is determined with the blinking rate calculation center and it help to
determine the blink rate with the two points associated with the programming. Now the blink rate is
determined with frequent blinking of the driver. For a particular time thus, he has a short blink rate or he
may close the eyes more than 30 second, then it is alerted as danger and the control system was embedded
with the blink rate calculation function may expose and provide signal to the connecting system for
generating alarm.
Trends Sci. 2021; 18(xx): xxxxx 5 of 17

Keep your text and graphic files separate until after the text has been formatted and styled. Do not
use hard tabs, and limit use of hard returns to only one return at the end of a paragraph. Do not add any
kind of pagination anywhere in the paper.

Implementation Process
The entire design of the process can be seen in Fig. 1. All the important steps in the diagram are
explained in detail as follows.
A. Programming for the proposed system
We use “PYTHON “and different python modules coding for our implementation. To do most of the
image processing functions, we relied upon OpenCV library in python. This language gave high accuracy
on face detection. Thus, we have two main functions on that. First one for detecting the eye blinking and
the second one is for reading the blinking. This calculation is invoked into the complete set of programs.
The camera system continuously monitors and sends the video file to the programming. The function
which is for getting the data to observe it and the blinking detection function reads the file if it detects
then it completely makes reading with that corresponding function and the signals are sent to the
speedometer which are connected through the wires.
B. Getting the Input Video
In our method, the setup is quite simple. An USB camera may be set up in front of the car. Here, a laptop
computer and Logitech HD webcam C270 was used for processing the data. But for industrial purposes
single board computers such as raspberry pi and USB camera may be used.
Identify applicable funding agency here. If none, delete this text box.
Trends Sci. 2021; 18(xx): xxxxx 6 of 17

C. Facial Landmark Detection

In video input, we used dlib library for facial landmark detection and ensemble of regression trees to
train the facial landmarks. The camera used in the device gives a constant flow of frames i.e. video
streaming. Every frame is fixed with contours at the face as given in the figure to detect the driver’s face
and eyelids position. The cv library is used to put the frontal face and haar cascade is used to put the
contour. The contour is provided with numbers and we specified the eyes parts.

The scipy package used in the image to get the distance between the contours such that we will get the
Euclidean distance between certain contours. The cv2 package first converts to grayscale for easy usage
then we use the FPS package for resizing the image at 500 pixels for faster processing the cascading
image for the following ways. Using this method, the boundary points of eyes, mouth and the central line
of the nose are marked and the number of points shown in Fig 2. The red points are the detected
landmarks for for eye, mouth and nose are given in Table I. The facial landmarks are further processing.

We have a car driver at our institute who has driven 250 km, the system has been applied on it for 8
hours. Work has been done on 3 states i) Active ii) Drowsiness iii) Full Sleep Eyes. Detects the eyelid,
predicts the condition of the eyelid by multiplying the EAR value and mentions any one of the 3 states
mentioned above. We have a car driver at our institute who has driven 250 km, the system has been
applied on it for 8 hours. Work has been done on 3 states 1 Active 2 Sleep 3 Full Sleep Eyes. Detects the
eyelid, predicts the condition of the eyelid by multiplying the EAR value and mentions any one of the 3
states mentioned above
Trends Sci. 2021; 18(xx): xxxxx 7 of 17

Figure : Active

Figure Drowsiness with alarm system

Figure: Sleeping with alarm system

D. Real Time Face Tracking

Experimental Result
We have contacted a car drive Mr. Kabir for our project implementation. Every Monday and
Wednesday at 7:00AM he has to travel from Dhaka to Chittagong for his official job. We set up a cc
camera in front of Mr. Kabir to save his bip records. After completing eight travel, we have collected and
inserted the all bip records into a table and have observed that,
Trends Sci. 2021; 18(xx): xxxxx 8 of 17

The first hour of his driving, he was in active mode. After three hours of his driving, his EAR record
was very satisfactory. During the fifth hour of his driving, several times he was in a drowsiness or sleepy
mood and the rest of the time during his driving, he was in an active mood.

Background and Related Work

Text (Times New Roman, 10 pt, Tab 0.8 cm)
Materials and methods
Text (Times New Roman, 10 pt, Tab 0.8 cm)

Results and discussion

Text (Times New Roman, 10 pt, Tab 0.8 cm)

Conclusions
Text (Times New Roman, 10 pt, Tab 0.8 cm)

Acknowledgements
Text (Times New Roman, 10 pt, Tab 0.8 cm)

References
[1] W Pralomkarn, C Supakorn and D Boonsanit. Knowledge in goats in Thailand. Trends Sci. 2021;
18, 24750.
Trends Sci. 2021; 18(xx): xxxxx 9 of 17

[2] Articles with a DOI should be referenced as follows: D Zhu, I Thapa, I Korobkov, S Gambarotta
and PHM Budzelaar. Redox-active ligands and organic radical chemistry. Inorg. Chem. 2021, DOI:
10.1021/ic2002145.
[3] LI Iverson, SD Iverson and SH Snyder. Handbook of psychopharmacology. Vol II. Plenum Press,
New York, 2020, p. 99-115.
[4] S Chumpawadee, K Sommart, T Vongpralub and V Paatarajinda. Effect of synchronizing the rate of
dietary energy and nitrogen release on ruminal fermentation, microbal protein synthesis and blood
urea nitrogen in beef cattle. New dimensions and challenges for sustainable livestock farming
volume III. In: Proceedings of the 11th AAAP Animal Science Congress, Kuala Lumpur, Malaysia.
2019, p. 364-6.
[4] AL Hecker. Nutrition and physical performance. In: RH Strauss (Ed.). Drugs & performance in
sport. 2nd ed. WB Saunders, Philadelphia, 2018, p. 23-40.
[5] DJ Harding. 2020, Redox-active group 6 transition metal alkyne complexes. Ph. D. Dissertation.
University of Bristol, Bristol, England.
[6] Trends in Sciences, Available at: https://tis.wu.ac.th, accessed October 2021.

Names of 5-7 referees

We encourage authors to submit the names of 5-7 referees (not from your own country) suitable to
review the work to aid in the peer review process.
1. Name, Affiliation, Country, e-mail
2. Name, Affiliation, Country, e-mail
3. Name, Affiliation, Country, e-mail
4. Name, Affiliation, Country, e-mail
5. Name, Affiliation, Country, e-mail
6. Name, Affiliation, Country, e-mail
7. Name, Affiliation, Country, e-mail
Trends Sci. 2021; 18(xx): xxxxx 10 of 17
Trends Sci. 2021; 18(xx): xxxxx 11 of 17
Trends Sci. 2021; 18(xx): xxxxx 12 of 17
Trends Sci. 2021; 18(xx): xxxxx 13 of 17
Trends Sci. 2021; 18(xx): xxxxx 14 of 17
Trends Sci. 2021; 18(xx): xxxxx 15 of 17
Trends Sci. 2021; 18(xx): xxxxx 16 of 17
Trends Sci. 2021; 18(xx): xxxxx 17 of 17