Advanced Driver Assistance System (ADAS) and Machine Learning (ML):

The Dynamic Duo Revolutionizing the Automotive Industry

Abstract
The main application of the Advance Driver Assistance System (ADAS) is to assist drivers in
making better decisions which eventually decreases fatal accidents and increases the safety of
the drivers as well as the speed of the vehicle. Machine Learning (ML), a subset of Artificial
Intelligence (AI), is used when there is a need to derive conclusions from the vast input of data
with human-like accuracy. Machine Learning is employed to enhance the accuracy of decisions
taken by ADAS by evaluating all the data received from various vehicle sensors. This paper
aims to summarise all critical algorithms used in ADAS technologies along with the evolution
of ADAS technology. So, with the help of these review papers, one can get a basic idea about
ADAS and its components. Initially, ADAS technology is introduced along with its evolution
to understand the objectives of developing this technology. After that, there is a discussion of
critical algorithms used in ADAS technology. These critical algorithms discussed are Face
Detection, Head Pose Estimation, Gaze Estimation, and Blink Detection. There is a discussion
of applications of Machine Learning (ML) on each algorithm, its impacts in different
environments, increased accuracy, and the requirement of more computing to increase
efficiency. The outcome of this paper is to evaluate all the methods with or without using
Machine Learning for each algorithm. This evaluation helps decide which method is suitable
for use in various scenarios.
Keywords: Machine Learning; Face Detection; Advanced driver System

I. Introduction
Introduction to Driver Assistance System And its Importance
Driver Assistance Systems (DAS) employ cameras, radar, and other sensors to detect the
surroundings and aid drivers in diverse driving scenarios, improving road safety. Lane
departure warnings, forward collision warnings, and adaptive cruise control are some of the
features of DAS designed to assist drivers in navigating different driving situations.
Driver assistance systems ease the burden on drivers and prevent accidents. Advanced driver-
aid technologies, such as lane-keeping and distance-regulating systems, increase the likelihood
that human drivers will use them. Haptic shared or guidance control is a popular method for
the human-machine interface in collaborative systems because it allows for touch-based
communication and interaction with the automated system. (Abbink et al., n.d.; Nishimura et
al., n.d.)
Modelling driver behaviour is something that has been introduced previously. For many years,
research has been conducted to represent drivers' actions in different road conditions
mathematically. It is a challenging task as driver behaviour is a highly intricate subject. Even
how a single driver responds to a threat in different road conditions can vary. Various factors
at a specific location and time can significantly affect a driver's behaviour. (Jurecki & Stańczyk,
2023)
Transportation of goods and essential services is crucial in the current scenario, and mobility
plays a vital role. Several measures have been implemented to enhance the speed of
transportation and decrease the number of accidents. One solution to decrease accidents and
accidental deaths is using "DAS." The goal of DAS in the future is to enable automated driving
in all scenarios, with a level of safety that surpasses that of a human driver and in collaboration
with other road users. (Bengler et al., 2014)

Conventional Technology in Driver Assistance and its Limitations

Conventional Driver Assistance System involves collecting various data about the condition of
various aspects of the vehicle with the help of sensors. It does not involve making any
independent decisions. It just collects information and displays it to the driver, which makes
the driver aware of the vehicle's condition. e.g., It displays the pressure of air in the tire, which
helps the driver to make the decision to refill it but does not make any decision of its own.
The research uses AI-based smart perception systems to enhance the ability of driver assistance
systems to detect and understand thermal information, providing more genuine data in
challenging weather and lighting conditions. The study focuses on enhancing the current
system for detecting and categorising objects using thermal vision, specifically emphasising
seven essential categories. The development of smart automotive systems is linked to
conventional technologies, including fusing data from different sensors and machine learning-
based obstacle detection and tracking systems. (Farooq et al., 2021)
Conventional Driver Assistance Systems give drivers important information to help decrease
accidents, but they do not act. Advanced Driver Assistance Systems are better but rely on
sensors that need to be standardised, making it difficult to trust their reliability. If a sensor
malfunctions, the system may fail. Also, the algorithms may malfunction in certain scenarios,
as research is conducted in limited environments. (Advanced Driver Assistance Systems
European Commission, 2018 n.d.)
Driver assistance algorithms used to be limited by the need for real-time processing, which
hindered their development. Though, as computing power continues to improve, new
opportunities are arising. Machine vision has been utilised in driver assistance and autonomous
vehicles for some time, with video-based object recognition having numerous applications,
such as monitoring systems, driver assistance systems, image retrieval, and more. (Laika &
Stechele, n.d.)

Introduction to Advance Driver Assistance System and its Importance

The main goal of ADAS and DAS is to assist drivers in operating their vehicles safely and
efficiently. DAS enhances driver awareness and speeds up driving, while ADAS uses advanced
sensors and cameras to monitor the vehicle's environment and support drivers in various driving
scenarios. Both systems aim to minimise accidents caused by human negligence and improve
driving safety. Examples of ADAS usage include Night Vision and Parking Assistance.
ADAS is gaining popularity in the automotive sector as a tool to improve driving safety and
reduce accidents. Car manufacturers such as Tesla, BMW, Mercedes-Benz, Audi, Ford, Honda,
and General Motors have integrated ADAS into their cars. Those who design ADAS ensure
that the car remains controlled, stable, and handles well in dangerous situations. However,
designing and testing ADAS is complex, and it depends on various factors like other cars, the
environment, etc. (Jiménez et al., 2016)
Machine vision, a clear form of computer vision that does not require extensive image
processing, is heavily utilised in ADAS to support lateral control functions like lane departure
warning systems and lane-keeping systems. (Piao & McDonald, 2008)(Seger et al., 2000)
Drivers continue to face visibility issues on the road during foggy conditions, making it a major
concern. Recent research based on deep learning for removing fog from images shows that an
end-to-end system using ADAS is an effective method to improve visibility and perform
dehazing and object detection in foggy atmospheres. (Allach et al., 2020)
Most of the functions used in self-driving cars today are created using ADAS. These
advancements aim to increase road safety, and their gradual implementation is already having
a positive impact. The features of ADAS show an improvement in how cars can sense, think,
and control themselves. This progress will lead to the creation of fully self-driving cars in the
future. (Hannes Estl, Texas Instruments, 2020, n.d.)

Introduction to Machine Learning

Machine Learning is a chapter of an open book named "Artificial Intelligence." It is connected
to various fields such as robotics, system design, neural networks, big data, computer science,
etc. Machine Learning concepts are widely used in our daily lives in tools like Alexa, Siri,
Google Assistant, Google Maps, etc. Machine Learning allows machines and computers to
mimic human behavior without being explicitly programmed.
Machine learning and artificial intelligence are gaining popularity due to the availability of
more data and computing power. The concept of computers learning from data has been around
since the 1950s. Machine learning has eight sub-categories, including Unsupervised and
Supervised Learning, which are further classified into Clustering, Dimensionality Reduction
& Regression, and Classification. Machine learning can help solve physical problems such as
Air Flow, Weather Forecasting, Rain Prediction, Temperature Variation, and Water Quality
Monitoring. (Badillo et al., 2020)
Studies have found ways for machines to learn from data without explicit instruction, known
as machine learning. Algorithms used by search engines like Bing, Google, and Yahoo are a
prime example of this. Machine learning is becoming more prevalent daily, seen in product
recommendations during online shopping and photo and video editing on social media
platforms. Similar content suggestions on social media are also based on machine learning
algorithms. (Mahesh, 2018)
Machine Learning involves pattern recognition to differentiate between different instances.
Recent years have contributed significant advancements in Machine Learning, making precise
and effective algorithms accessible to professionals. (Rätsch, n.d.)
Machine Learning in ADAS
Machine Learning is used in ADAS to improve their performance by training the system on
vast amounts of sensory data using algorithms like neural networks, decision trees, and support
vector machines. It enables the system to make real-time decisions and take appropriate actions,
leading to a safer and more convenient driving experience.
Machine learning can improve advanced driver assistance systems by reducing casualties,
preventing accidents, and providing prompt medical treatment. It can be used to develop
systems for driver monitoring, parking assistance, and controlling the vehicle laterally and
longitudinally. Adaptive cruise control and collision mitigating systems can identify hazards
for lane changes, and parking assistance technology allows vehicles to park without the driver's
presence or action.
There are four major four architectural components of the Advance Driver Assistance System:
driving vigilance monitoring system, lateral control, longitudinal control, and parking
assistance. The application of ML techniques has made possible the development of these
systems, which are currently integrated. (Moujahid et al., 2018)
Convolution Neural Networks (CNNs), a more advanced version of Artificial Neural Networks
(ANNs), can handle challenging tasks. However, since Machine Learning (ML) models require
significant computational resources, the industry uses processing components that are both
resource and cost limited. As a result, several options exist for embedding memory-intensive
models on various computing systems used in automotive platforms. (Borrego-Carazo et al.,
2020)
Introduction to Advance Technologies in ADAS
Face detection is a well-researched area in computer vision. Currently, the most common
approaches to face detection rely on two key concepts: boosted cascade structures and basic
features. While these methods perform well in typical conditions, they tend to be less accurate
in challenging situations, such as low lighting, facial expressions, or occlusions. Researchers
have developed alternative techniques to overcome these limitations, such as "divide and
conquer," where multiple algorithms are trained to detect faces from different perspectives. By
combining the results of these algorithms, the overall accuracy of face detection can be
improved. (D. Chen et al., 2014)
Head pose estimation is an important form of nonverbal communication used to infer the
communicator's intentions. Humans can easily interpret intentions using head pose, and this
ease makes head pose detection a very difficult task for machines. Head pose estimation
includes classifying head pose in one of the discrete orientations identified by the algorithm.
The head pose estimation algorithm can estimate various Degrees of Freedom in the position
of the head. The algorithm's complexity increases as the number of Degrees of Freedom in
which it can estimate the orientation of the head increases. (Murphy-Chutorian & Trivedi,
2009)
Gaze estimation includes continuously tracing the gaze direction of a person. With continuous
tracking of the gaze direction of a person, we can go deeper into the person's internal state.
Most appearance-based methods require a clear and close view of the eyes for gaze estimation.
All the prominent methods for gaze estimation require less distance of a person, i.e., less than
1 meter or a frontal view. Some methods use surveillance cameras to demonstrate gaze
estimation from a distance but use head or body orientation. This approach reduces the
method's accuracy and cannot be used in real-world scenarios. (Nonaka et al., 2022)
Blink Detection involves detecting the blinking of the human eyes in the video frame. Blink
Detection is used to know the eye's location at the algorithm's start. There are many methods
to locate the eye in the video frame if the initial location of the eye is given. The given location
of the eye is then matched with the whole frame of the video to know the next location of the
eye. Here, the problem arises of how to initialise the initial location of the eye. So, to know the
initial location of the eye, blink detection is used. It is also used in vehicles to derive the driver's
alertness based on the driver's blinking frequency. (Morris et al., 2002)

Author’s Contribution & Motivation

The authors have made diverse contributions to the field of DAS and ADAS, highlighting their
significance and limitations. Their research involves analysing mathematical models of driver
behaviour under different road conditions, implementing AI-based perception systems to
improve DAS's thermal sensing capabilities, and utilising machine learning and computer
vision to enhance ADAS control functions such as lane departure warnings, lane keeping, and
lane assist systems. They have also identified the challenges in developing and testing ADAS,
such as the need for standardised sensors and algorithms.
The author aims to minimise the number of accidents caused by human negligence and enhance
driving safety by making DAS more intelligent and dependable. They aim to increase driver
awareness, improve driving speeds, and make automated driving possible in all situations while
maintaining high safety standards. The authors recognise the importance of mobility in
delivering goods and essential services, particularly in the current scenario. Therefore, their
work is driven by the urgency to reduce accidents and fatalities while making transportation
more efficient and reliable. They believe their research can help create safer, better, and more
efficient roads for everyone.

II. Advance Technologies

The progress of ADAS has been a significant focal point of the automotive sector in modern
times where advanced classification in ADAS especially refers to using precise, accurate, and
refined algorithms and cites computer vision techniques to identify and categorise objects
within the driving environment accurately. This includes vehicles, pedestrians, road signs,
traffic lights, and other objects that may affect the safe operation of a vehicle. (Soni & Nair,
2021)
Advanced classification systems use techniques of Machine Learning such as decision trees,
neural networks, and support vector machines to analyse images and data from various sensors,
to make predictions about the environment. In the context of advanced classification, large
datasets are used by machine learning algorithms to learn patterns and relationships present in
the data. This training allows the algorithms to predict new data, such as the objects in the
driving environment. The goal of advanced classification in ADAS is to provide more precise
and accurate information to the vehicle's control systems, helping to make cars safer and more
efficient. The Collision Avoidance System is also an example of this field in which the vehicle
identifies other vehicles or pedestrians and gives either warning or takes necessary actions to
avoid a collision. (Arganda-Carreras et al., 2018)
As part of this field, ADAS relies on various fields and technology for its classification and
implementation, including Machine Learning, AR/VR, Big Data, Computer Vision, IoT,
Artificial Intelligence, etc. Overall, advanced classification in ADAS is a rapidly evolving field,
and it is expected to be an important part of making future vehicles and fully self-driving cars
a reality.

Introduction to Various Technologies

Figure 1. Classification of Technologies

Artificial Intelligence & Machine Learning

Artificial Intelligence (AI) aims to employ machines with human intelligence. These machines
are incorporated to think and act like a human. Its opportunity is very vast as it is helpful in
every field, whether it is automobile, medicine, finance, education, sciences and many more. It
can process a huge amount of data and performs task automatically.
ADAS technology has various roots in AI, several initiatives are underway to integrate AI into
it. One such project is the development of a VMS prototype using machine learning techniques.
(De-Las-Heras et al., 2021a) Another implementation of AI in ADAS applies a combination of
deep learning and machine learning algorithms. (Krishnarao et al., 2021) AI is useful in
Advanced Driver Assistance Systems to tackle advanced tasks and make quick and smart
decisions. By analysing data from sources like cameras, sensors, and GPS, AI can assist ADAS
in accurately detecting and responding to road conditions, driver behaviour, and other driving
scenarios. This improves the safety of the vehicle and the passengers, decreases the ratio of
accidents, and improves comfort in driving. It also helps make the ADAS technology more
efficient and dynamic.
Autonomous vehicles are rapidly developing ADAS. Machine learning and embedded
computing are two driving forces behind these efforts. The ADAS system can detect obstacles,
objects, other vehicles, pedestrians, and lanes employing advanced machine learning
algorithms and estimate object trajectories and intents. (Ball & Tang, 2019)
Computer Vision
Computer vision involves understanding various images and videos by computer just like
humans. It includes various methodologies like text recognition and object recognition. It is a
way to teach computers how to understand and see things as we humans do.
It is also used in identifying the text written on the signboards, which AI eventually uses to
make decisions about the vehicle's trajectory. So, computer vision is used to know and evaluate
the vehicle's surroundings, like other vehicles, road turns, obstructions on the road, etc.
Computer vision and machine learning technology have led to more precise detection of road
hazards and potential collisions, resulting in greater road safety and fewer accidents. This
technology can be used in traffic signal recognition systems. Correct evaluation of the
information collected and immediate decision according to it is very necessary because the
failure of it could cause severe accidents and damage to human lives. So various research is
going on to decrease the response time and increase the system's accuracy. (De-Las-Heras et
al., 2021)
Internet of Things
The Internet of Things includes physical devices embedded with electronic chips and connected
to the Internet. Due to this, physical devices could communicate with each other, resulting in
their interconnection. For example, a mobile phone can operate a fan equipped with IoT.
ADAS can be used with Internet of Things (IoT) technology to monitor the driver and the car's
surroundings. Also, the data collected from different parts of the vehicle can be shared, which
helps in better decision-making as it presents a more holistic picture of the surroundings and
condition of the vehicle.
This technology and machine learning techniques can be used to develop a Variable Message
Sign (VMS) reading system. (De-Las-Heras et al., 2021c) An advanced technique based on
deep learning and IoT to monitor the driver and the car's surroundings can also be carried out
with the help of this technology. IoT and AI are used extensively in ADAS, where AI algorithms
are used for better and more precise decision-making based on the data collected by various
physical devices connected over the Internet. (Hemaanand et al., 2020)
Augmented & Virtual Reality
AR and VR are some of the futuristic technologies that are revolutionising this digital era. AR
enhances our reality by overlaying digital information such as images and text. VR creates a
completely immersive digital environment that replaces the real world. These technologies
have various uses, including automobiles, entertainment, education, medicine, etc. VR can be
used for interactive gaming experiences, while AR enhances live performances. These
technologies can be used to feel the real world virtually, enhancing our experience.
Some applications of AR technology are Head-Up Displays (HUDs), which can project crucial
data onto the windscreen, such as seatbelt indication, speed warning, temperature (inside and
outside of the vehicle), navigation maps, etc. As a result, distractions by drivers can be
decreased, which enhances the safety of passengers and makes driving more interactive. These
technologies can also help deliver real-time alerts and information. Driving Simulations using
VR can be useful in developing an individual's driving skills. A project focused on how 5G can
be utilised to make vehicles safer and improve the driving experience is also going on. These
can also be used to improve the infotainment experience in the vehicles. (Meijer, 2020)
Big Data
Big Data is the lifeblood of ADAS technology. It has various uses in the area, including
organising, optimising, and searching traffic data, GPS data, object mapping data, weather data,
and data from other connected automobiles. Self-driving automobiles are the future of
transportation and the primary driving force behind this revolution. Self-driving automobiles
aim to improve safety, minimise traffic accidents, decrease congestion, boost fuel efficiency,
and enhance the quality of life for people. It is hoped that security and privacy issues will be
resolved to give drivers, insurance companies, and automakers a relaxation in terms of driving.
(Cheruvu Ria, 2015)
Cloud Computing
Cloud computing utilises global networks to connect hardware-filled data warehouses to
provide data storage and processing power. ADAS applications need GPUs for processing data
in near real-time, but this work may be offloaded to the cloud when 5G speeds are available.
In the future, there will be more concentration on including more elements, such as network
latency and various benchmarks for computer vision performance. (Olariu et al., 2018)
Computer Security
A system of hierarchical levels is offered to separate threats and attacks into three distinct
layers: the sensor, communication, and control layers. Vehicle makers are incorporating AI
techniques to modernise communications, but this can lead to risks. Modern architectures
should prioritise safeguarding important units, such as power train ECUs, with cryptographic
and non-cryptographic methods, registration, and authentication procedures, among other
measures. (El-Rewini et al., 2020)
III. Our Approach

Figure 2. Block Diagram of Driver Monitoring System

Figure-2 illustrates the fundamental components of our Driver Monitoring System (DMS). An
application can use these outputs to execute solutions for the safety of drivers. We chose to use
CNNs to implement the building blocks of our DMS, and we have seen superior results as
compared to conventional image processing and computer vision-based implementation.
(Sancheti et al., n.d.)
A. Face Detection
Machine Learning with Face Detection
Face detection systems use machine learning to detect faces and to make them more efficient
in extreme environments. These systems detect faces by identifying 68 facial landmarks. The
algorithm is trained with pictures from various viewpoints to recognise these landmarks and
ensure accuracy in testing. Additional images are required during the training process to
improve accuracy. The system identifies and centralises the facial landmarks without distorting
the image once a face is detected. The image is converted into Euclidean space to produce 128
bytes of data per face for classification with trained datasets. When the dataset is small, we use
Support Vector Machines (SVMs) for classification. The accuracy of face detection systems is
highly dependent on lighting conditions. To enhance accuracy, gradient transformations are
applied to face images.
The machine learning algorithm can also be retrained with high-accuracy images to boost its
performance. To summarise, face detection systems based on machine learning are valuable
for increasing security in challenging environments. The 68 facial landmarks are essential for
the system to detect faces accurately. More training images are needed to improve accuracy.
The system identifies facial landmarks, centralises them, and converts the image into Euclidean
space. SVMs are used for classification when the dataset is small. Gradient transformations are
applied to face images, and retraining with high-accuracy images can boost accuracy.
(Arsenovic et al., 2017)
 Figure 3. Machine Learning with Face Detection

Figure-3 depicts the process of face detection using Machine Learning Algorithms. The first
step is to detect the face in the given image. The second step detects face landmarks among the
68 known face landmarks and centralises them. The third step embeds the centralised image of
facial landmarks in Euclidean space. It generates 128 bytes of data per image for embedding.
The fourth step checks the data with the training data set and classifies it according to the
similarity in facial landmarks.
Case Studies
In the paper by (Krishna et al., 2022), they introduced a new approach to recognising driver
drowsiness that incorporates Vision Transformers and YoloV5 architectures. The aim is to
enhance road safety by developing an efficient drowsiness detection algorithm. The framework
was trained using the public UTA-RLDD dataset and evaluated on a customised dataset of 39
participants under different lighting conditions. The YoloV5 component obtained a mean
average precision score of around 95%, while the Vision Transformer component displayed
high precision, sensitivity, and F1 score, with an accuracy of 95.5% during testing. This
framework holds great potential for smart transportation systems but needs a large amount of
data, including labelled scene conditions, for training. The authors intend to enhance the
network configuration and expand the training data size in their future endeavours to enhance
the model's performance.
In the paper by (Baker et al., n.d.) present the results of their recent study on real-time tracking
of non-rigid head movements for assessing driver mental state. Active Appearance Models
(AAMs) were utilised to monitor both the fixed head motion and the flexible facial expressions.
They contributed significantly by creating a real-time gradient descent fitting algorithm for
AAMs, which operates at over 200 frames per second and can handle occlusions. The authors
also explored the development of 3D non-rigid face tracking. They created a structure-from-
motion algorithm to convert 2D AAMs into 3D models and a fitting algorithm for the 3D model
that runs at more than 250 frames per second. They emphasised the feasibility of their
algorithms on standard PCs and the possibility of their implementation on low-power devices.
This study's authors (Saini & Saini, n.d.) focus on the crucial aspect of driver drowsiness
detection in car safety technology to avoid road accidents caused by fatigue-related driving.
Their research aims to develop technologies that can detect or prevent driver drowsiness while
driving. The approach involves using real-time video processing from a front-facing camera to
gauge the driver's level of fatigue, which is then transmitted to an alarm system that triggers an
alarm if drowsiness is detected. The authors assess emerging technologies to determine the best
method for avoiding fatal vehicle crashes. They point out the drawbacks of the current market
leader, a reed switch that detects head tilt, and the shortcomings of the product developed by
BMW, which is better at detecting drowsiness but needs to warn the driver properly.
The study by (Shen et al., 2018) aimed to enhance the visibility of driver faces in images
captured during nighttime. The research focused on creating an adaptive attenuation
quantification retina (AAQR) technique to increase the accuracy of driver face detection in
low-light conditions. A dataset of the driver face images taken at night were collected and
divided into three groups based on lighting to carry out the study. The AAQR method was
executed in three phases: restriction of attenuation, prediction of attenuation, and adaptive
quantification. The findings indicated that the AAQR approach demonstrated an 86% detection
rate, 2-36% better than the earlier algorithms. The AAQR was found to be especially effective
in mixed nighttime lighting conditions, and it had a faster computing time for a single nighttime
image (640x480) than most other sophisticated methods. As per the study, the AAQR method
can be a new and promising technique for utilising driver assistance during nighttime in
Advanced Driver Assistance Systems (ADASs) and autonomous vehicle systems in the future.
(Abbas et al., 2022) recently conducted a study in which they introduced a deep learning
method called ReSVM for detecting driver distraction. It merges features from the SVM
classifier with ResNet-50 and is compared to six leading techniques using four datasets. The
results revealed that ReSVM outperformed the other methods, achieving an accuracy of 95.5%.
The authors intend to improve the model by adding features like car motion, driver emotions,
and tics for distraction detection. Currently, the study only dealt with spatial features for
classification, and future work will focus on temporal aspects. The aim is to develop a vehicle
distraction detection system, monitor dangerous driving behaviour, and prevent accidents. The
authors plan to implement the system and increase the data available for deep learning
architectures.
(Krizhevsky et al., n.d.) conducted a research study to classify high-quality images from the
ImageNet LSVRC-2010 competition into 1000 diverse categories by applying a deep
convolutional neural network. This network included five convolutional layers, three fully
connected layers, and some max-pooling layers. The findings revealed a significant
improvement compared to previous state-of-the-art results, with error rates of 17.0% and 37.5%
for top 5 and top 1, respectively. The study employed a dropout regularisation method to reduce
overfitting in the fully connected layers. Moreover, they used an efficient GPU implementation
to accelerate the training process. It was concluded that a large deep convolutional neural
network could deliver record-breaking results through pure supervised learning. The study also
highlighted the significance of network depth, as the network performance decreased when a
single convolutional layer was removed. The researchers should have utilised more supervised
pre-training and have plans to explore larger and deeper networks for video sequences in the
future.
(Y. Zhao et al., 2017) presented a novel head-tracking system that can monitor driver behaviour
while driving a vehicle. The system is designed specifically for use in Level 3 autonomous
vehicles and comprises two integrated devices and proprietary software for data collection and
calibration. The system was validated through four experiments and found to have averaged
errors of 0.36°, 1.57°, and 0.38° for nodding, rolling, and shaking axes, respectively, on a static
platform. Household studies showed that the system's measurements for shaking and nodding
were very similar, with an average difference of less than 2°, suggesting that the system may
be better suited for detecting large head movements during non-driving activities.
(Shang et al., 2022) developed a real-time detection system for driver fatigue and emotions
based on time series analysis. The updated RM-Xception algorithm incorporates a depth-
separable convolution component and a residual component, resulting in faster processing and
lower training computation requirements while effectively capturing emotions. On the Fer2013
dataset, the model achieved 73.32% accuracy. In the future, the algorithm will undergo further
testing in more complex driving environments and explore the use of multiple sensor data and
the relationship between facial expressions and emotions while tired.
(Ulrich et al., 2021) conducted a successful study on driver attention using 3D cameras and
FER. The findings indicated that drivers were often distracted by events, but the Advanced
Driver Assistance Systems (ADAS) only affected the focus of one user in one scenario. The
study employed Intel RealSense SR300 and an RGB-D camera to monitor driver facial
expressions. The use of RGB-D images and deep learning techniques was found to be an
effective and non-invasive way to assess driver concentration. However, the correlation
between ADAS activation and inattentive facial expressions was weak. Future research should
explore the efficacy of ADAS and the level of feedback the user receives.

Table 1

Summary of papers on Face Detection

Algorithm Accuracy Speed Methods References

Convolutional Neural 94.32% < 20 fps Appearance Based You et al., 2020
Network, Face Feature and Feature-Based
Triangle

Haar and Viola-Jones 90.80% > 5 fps Feature Based and Sigari et al., 2013
Knowledge-based

Multitask Convolutional YAWDD Dataset: 98% 2 fps Feature Based and L. Chen et al.,
Neural Network Personal Dataset: 97% Knowledge Based 2021

Random Forest, EAR metric: 94.5% PC: 140-170 fps, Feature Based and Dong et al., 2022
Convolutional Neural Random Forest: 91% Jetson TX2: 40-70 Appearance Based
Network Paper Approach: 97.5% fps
Average Face & Eye
Detection: 100%

Multitask Convolutional 99.13% 25 fps Appearance Based Xiao et al., 2022

Neural Network and Feature-Based

SVM, Haar Classifier, Face Detection: 99.9% - Feature-based Fatima et al.,

Viola-Jones Algorithm Eye Detection: 98.7% 2020
SVM: 96.5%
Adaboost: 95.4%
Haar, Multitask OpenCV : AP50(68.4%), OpenCV: 400 fps Feature Based and Oh et al., 2022
Convolutional Neural AP75(51.4%), MMOD: 260 fps Knowledge Based
Network MMOD : AP50(83.8%), MTCNN: 4 fps
AP75(18.1%),
MTCNN : AP50(81.4%),
AP75(72.0%)

Artificial Neural Network 63.15% 25fps Feature-Based Inkeaw et al.,

2022

Convolution Neural 96% 21fps Feature Based and Flores-Monroy et

Network Knowledge Based al., 2022

Author’s Opinion
After conducting a thorough examination of face detection in Advanced Driver Assistance
Systems (ADAS), it is evident that this technique plays a crucial role in monitoring the
behaviour and state of the driver. Several algorithms are available for face detection, such as
SVM, HOG, Haar Cascade Classifier, Viola-Jones Algorithm, Dlib's Algorithm, and
Convolutional Neural Networks (CNNs). These algorithms vary in accuracy, speed, and
complexity, each having advantages and disadvantages. For instance, SVM and Haar Classifier
are efficient and fast but may not be as precise as CNNs when detecting faces under challenging
conditions. On the other hand, CNNs are highly accurate but require more computational
power. Studies have demonstrated that Convolutional Neural Networks (CNNs) are widely
employed for face detection in ADAS. Deep Convolutional Neural Networks have achieved
remarkable results through supervised learning on a challenging dataset. In contrast, ReSVM,
utilising a residual neural network with an SVM classifier, has accurately classified different
types of driver distractions. This paper utilises various face detection methods, including
Feature-Based, Appearance-based, and Knowledge-Based approaches.
In conclusion, face detection algorithms continue to advance and develop, and the most suitable
approach for an ADAS system will depend on its specific requirements and limitations.
Therefore, a comprehensive analysis of face detection methods and careful selection of the
most appropriate algorithm is crucial for the success of ADAS.

B. Head Pose Detection

Machine Learning Integrated with Head Pose Estimation
Methods using Facial landmarks are very popular for estimating head pose. Nevertheless, this
method unnecessarily increases the computation. Facial images can be analysed using methods
that do not rely on facial landmarks to determine the head pose accurately. Various methods are
proposed using Convolutional Neural Network (CNN) algorithm used to estimate head pose
without using facial landmarks. Also, multitask learning is used when various information
needs to be generated from the facial image like head pose estimation, gender determination,
and facial landmark detection. Using deep Convolutional Neural Networks (CNN) Hyperface
method performs face detection, recognising and localising facial landmarks, head pose
estimation and gender determination. Heatmap-CNN regressors are used for key point, and
head pose estimation. Certain approaches utilise depth data to extract three-dimensional (3D)
image details. The information provided by these methods is accurate. Despite accuracy, these
methods are used less in real-world applications because they require images from special
cameras. (Yu et al., 2021)

Figure 4. Machine Learning with Head Pose Detection

Figure 4 depicts the categorisation of different techniques employed in head pose estimation.
Initially, these methods are classified into two groups: landmark-based methods and landmark-
free methods. Landmark-based methods rely on facial landmarks to ascertain the head pose.
On the other hand, landmark-free methods employ Machine Learning algorithms and can be
further subdivided into CNNs, HyperFace, and Heatmap-CNN.
Case Studies
(Choi et al., 2016) conducted a study to create a real-time technique for tracking a driver's head
pose and eye-blinking. They proposed Pose Extended - Active Shape Model (PE-ASM), which
outperformed traditional face models in fitting accuracy, especially with extreme poses. PE-
ASM improves facial fitting under challenging conditions by incorporating extreme pose cases.
The method can serve as a reliable driver drowsiness detector in commercial cars, handling
diverse visual conditions. It utilises two Hidden Markov Models (HMMs) to estimate head
pose, eye-blinking, and detect drowsiness. The study highlights PE-ASM's potential for
enhancing driver safety and addressing limitations faced by existing models in natural driving
scenarios.
The primary objective of a study conducted by (J. et al., 2022) was to overcome the challenges
faced by previous methods when analysing a driver's visual attention based on the head pose
and eye gaze. Existing approaches struggled with non-uniform illumination and partial facial
occlusion in real-world driving conditions. The researchers introduced a novel estimation
method that utilised an RGB-D camera to address these limitations. Their proposed technique
involved fusing and registering continuous multi-frame point clouds to generate a stable point
cloud representation. Instead of relying heavily on template registration, the method utilised
multiple point clouds within the nearest neighbour gaze zone as a template, reducing the
dependence on precise template alignment. The study utilised an image resolution of 720 P.
Experimental results highlighted that the proposed method outperformed other approaches
regarding head pose tracking and gaze zone classification. The average accuracy for gaze zone
estimation was 93.97% when employing the proposed method with the SVM classifier, POSIT
method, and PointNetLK method. Furthermore, the proposed method improved the accuracy
and efficiency of point cloud registration by incorporating particle filter and normal
distributions transform to track and predict the initial coarse transformation.
(N. Ruiz et al., n.d.) introduced a new approach for determining the orientation of a person's
head. Their method entails training a multi-loss convolutional neural network using a vast
dataset to predict intrinsic Euler angles directly from image intensities. This method is more
reliable than conventional methods that rely on the performance of landmark detection and an
external head model. The new method demonstrates excellent results on various pose
benchmark datasets taken in the real world, along with precise pose annotations. The proposed
method surpasses landmark-to-pose methods and proves to be more robust even in situations
of very low resolution. The researchers recommend exploring synthetic data generation for
extreme poses and more sophisticated network architectures to improve the method's
performance.
A novel approach has been introduced by (Hong et al., 2017) for estimating the pose of human
faces, focusing on gaze direction and head posture using deep learning and multitasking
learning. This method is crucial for analysing nonverbal communication and extracting
important visual cues. The approach utilises a deep learning framework called Multitask
Manifold Deep Learning ($M2$DL) that leverages multi-modal data. The ($M2$DL) approach
incorporates enhanced feature extraction based on deep neural networks and multi-modal
mapping through multitask learning. It also utilises Manifold Regularized Convolutional
Layers (MRCL), which improve traditional convolutional layers by learning the relationship
between neuron outputs. The effectiveness of the proposed approach was evaluated on three
benchmark datasets (DPOSE, HPID, and BKHPD). The experimental results demonstrated that
the new method outperforms previous techniques in face pose estimation. The key
contributions of this research include the development of the ($M 2$DL) framework, utilising
MRCL, and handling multi-modal features using multitask learning.
(Firintepe et al., n.d.) conducted research and investigated the accuracy of head pose estimation
using infrared (IR) images and explored how deep learning techniques can improve the results.
The study used the AutoPOSE dataset and cropped head images of 64x64 and 128x128 pixels.
X introduced two new networks, HON and ResNetHG, and compared them to existing methods
like the HPN model from DriveAHead. The researchers evaluated the performance of these
models on various input resolutions and depths and found that higher-resolution images
resulted in more accurate estimations. In addition, the researchers discovered that deep learning
methods with fewer layers achieved superior performance in head orientation regression when
using infrared (IR) images. Specifically, the HON and ResNetHG18 architectures developed
by X outperformed state-of-the-art models on IR images, highlighting a significant reduction
in residual error by up to 74%. To enhance the accuracy of these models further, future research
should concentrate on analysing additional input resolutions and exploring different model
depths. Additionally, benchmarking the models on the DD-Pose dataset, which contains real-
world data, would provide valuable insights for comparison and evaluation purposes. By
investigating these aspects, researchers can advance the field of head orientation regression and
strive for even more accurate and reliable results in IR image analysis.
(Akhtar et al., 2022) discusses the significance of monitoring a driver's facial pose to evaluate
their level of attentiveness and decrease the possibility of road accidents. Their suggested
solution employs wireless sensing and utilises channel state information (CSI) from Wi-Fi
signals to identify the driver's face non-intrusively. They introduced a novel classification
algorithm that leverages the combination of Support Vector Machine (SVM) and K-nearest
neighbour (KNN) techniques to improve face recognition accuracy. The experimental findings
demonstrate that the proposed system achieves high precision in identifying a driver's face,
with an average recognition rate of 91.8%. This suggests that the algorithm effectively
enhances classification accuracy and shows promising potential in face recognition technology.
The researchers also propose a hybrid classification scheme known as KSVM, which
significantly enhances recognition performance and expands the possibility of various
applications. The study aims to investigate more complex driving situations and how roadway
types may influence the findings.
In a research study proposed by (Z. Zhao et al., 2020), the objective was to investigate how the
position of the head could be utilised to detect driver distraction. The researchers compared the
accuracy of two methods: single regression and a combination of classification and regression.
They trained four networks using two datasets. For head pose estimation, the researchers
employed HPE_Resnet50 and applied it to a separate dataset to obtain head position data. The
study findings indicated significant disparities in head position between safe driving and
distracted driving scenarios. Consequently, this information can be leveraged effectively to
identify instances of distracted driving. Overall, the study suggests that analysing the position
of the head can serve as a valuable indicator for detecting driver distraction. By comparing
different methods and employing head pose estimation techniques, the researchers shed light
on the potential of utilising head position data to enhance road safety and address the issue of
distracted driving.
(Murphy-Chutorian et al., 2007) conducted a study emphasising the importance of detecting a
driver's level of awareness when designing advanced safety systems for vehicles. The study
aimed to develop a system capable of accurately determining a driver's head pose irrespective
of their identity and lighting conditions. To achieve this, a video camera was utilised to detect
both visible and near-infrared light. The system employed Localised Gradient Orientation
histograms and support vector machines for regression to estimate the orientation of the driver's
head in two degrees of freedom. By utilising these techniques, the system aimed to overcome
the challenges posed by varying lighting conditions within a moving vehicle. This is crucial
for ensuring reliable head pose estimation, as lighting conditions fluctuate significantly during
real-world driving scenarios. Overall, the study focused on creating a robust system that can
accurately estimate the driver's head pose, regardless of identity and lighting conditions, by
utilising a video camera capable of detecting visible and near-infrared light. The Localised
Gradient Orientation histograms and support vector machines for regression were instrumental
in achieving this goal and addressing the difficulties caused by changing lighting conditions in
a moving car. The accuracy and stability of the system can be further improved by incorporating
a visual tracking system. This research is a crucial step towards developing robust driver
activity monitoring systems that can contribute to developing advanced driver assistance
systems.
The study conducted by (Diaz-Chito et al., 2016) introduces a novel technique for measuring
the angle of a driver's head turn, which is a critical factor in driving performance and accident
prevention. The proposed method focuses on utilising only three facial landmarks - the centre
of the eyes and the tip of the nose - to compute geometric features and estimate head pose using
two manifold embedding techniques along with a linear regression model. To evaluate the
method's effectiveness, the researchers tested it on CMU-PIE and their driver dataset. The
results obtained were comparable to other state-of-the-art techniques while maintaining a low
computational cost. This indicates that accurate and precise head pose estimation can be
achieved using only three facial landmarks. The findings suggest that this technique can be
integrated into real-time applications on consumer devices, as it provides reliable results
without significant computational overhead. By accurately measuring the angle of a driver's
head turn, the proposed method can enhance driving performance and reduce the risk of
accidents.

Table 2

Summary of papers on Head Pose Estimation

Algorithm Accuracy Model Dataset Reference

Deep Learning VoD: 95.40% Deep Convolutional CAS: PEAL data set Cao & Liu,
DVD: 95.90% Neural Network model
2017

Iterative Closest, Point > 92% Face from Depth 3D Biwi Kinect Head Borghi et al., n.d.
algorithm, and Model Pose,
Farneback algorithm ICT: 3DHP,
Pandora

Deep Convolutional Pitch: 96.50% Deep Convolutional Pointing’04, Khan et al., 2020
Neural Network Yaw: 89.20% Neural Network model AFLW,
Boston University,
ICT: 3DHP

Viola Jones, Zernike 85% Geometric and CNN- MPIIGaze dataset Vankayalapati et al.,
moments Algorithm Based Model 2022

CNN: based head pose 90.20% A denseness-based HPE: AFLW2000, Z. Hu et al., 2022
estimation and Grad: facial landmark BIWI,
CAM: based attention detection module GE: MPIIGaze,
mapping d UT: Multiview

Emotion recognition Head Pose Candide3 model Head Pose Estimation Kalliatakis et al.,
via facial expressions Changes: 83.95 % Dataset 2017
(ERFE) Emotion Changes:
76.58 %
 MTCNN, Levenberg– Eye State: RCAN BIWI dataset, Ye et al., 2021
Marquardt (LM) 98.962% AFLW2000 dataset.
algorithm Mouth State:
98.561%

Active Shape Model < 96.63% Cascade: CNN: based DrivFace dataset, Ali et al., 2021
(ASM) and Boosted models, Boston University
Regression with R : CNN: based (BU) dataset,
Markov Networks models FT: UMT dataset,
(BoRMaN). Pointing'04 dataset

Fisher's linear 98.81% Support Vector CMU: PIE dataset, Diaz-Chito et al.,
discriminant (FLD) regressors, Personal dataset 2016
and principal Sparse Bayesian
component analysis regression
(PCA), Neural
Networks

HOG, Haar, SVM Pitch: 97.5% SIFT, SURF, ORB Pointing’04, Kinetic Alioua et al., 2016
Yaw: 98.2 % Sensor

Author’s Opinion
ADAS has become an important part of modern vehicles, providing drivers with real-time
assistance and improving overall safety on the road. One of the key components of ADAS is
head pose estimation, which involves tracking the position and orientation of the driver's head.
This information can be used to detect and alert drivers who may need to be more focused,
distracted, or otherwise not paying attention to the road. Several research papers have been
published on head pose estimation for ADAS, using various techniques such as computer
vision, machine learning, and deep learning. These studies have shown that head pose
estimation is a promising approach to improving driver safety, providing real-time information
about the driver's attention and alertness. Different datasets were used in this research paper
where accuracy and the algorithm were used to change accordingly. Real-time head pose
estimation has been achieved with high accuracy through many methods like deep learning,
support vectors etc. Overall, head pose estimation is a promising approach to improving driver
safety in ADAS. The ability to offer immediate data on the driver's focus and vigilance can aid
in accident prevention and preserve lives on the road.

C. Gaze Estimation
Machine Learning integrated with Gaze Estimation
Gaze estimation is used in vehicles to know the degree of alertness of the driver. It can be used
to know if the driver thinks to change lanes or if the driver is alert about the upcoming obstacle.
These fatal accidents can be prevented by informing the drivers about the upcoming danger.
Gaze cues that depict the driver's attentiveness include blink rate, temporal gaze variation,
speed of eyelid movements, and degree of eye openness. Support Vector Machines (SVM),
Linear Polynomials, and Gaussian Kernels are used for eye verification before gaze estimation.
Several classifiers can be used for gaze estimation at the last stage of the face detection
algorithm. Support Vector Machines (SVM) based classifiers are most common for real-time
gaze estimation in the automotive industry. Another application of gaze estimation is that it can
be used along with other visual and facial parameters to derive the driver's state of mind and
predict the next decision. (Kar & Corcoran, 2017)

Figure 5. Machine Learning with Gaze Estimation

Figure 5 depicts the classification of Classifiers used in Automobiles for Gaze Estimation. All
the steps of Gaze Estimation are the same as Face Detection except the last step of using various
classifiers. For real-time gaze estimation, automobiles use Machine Learning and thus require
SVM-based classifiers. These SVM-based classifiers are further divided into various categories
depending on their accuracy in different environments of low lighting and using spectacles.
SVM-based classifiers use Multiclass, Linear Polynomial & Gaussian Kernel, and Random
Forest Regression.
Case Studies
(Wang et al., 2019) suggested a method to estimate the constant gaze area of a driver in driver
assistance systems by utilising an application-focused approach. They introduced a multi-zone
ICP-based point-cloud alignment technique for accurately determining the head pose and a
two-stage neighbour selection process to estimate the eye gaze system based on appearance.
To enhance the speed at which the ICP iteration converges, they employed multi-zone templates
and particle filter tracking techniques to initialise and update the optimal modification of the
source face template. By calculating the head orientation and angle, they established a gaze
zone based on the gaze angle while correcting the head pose. The proposed approach surpasses
existing methods in estimating gaze and provides reliable monitoring of head posture for
analysing driver behaviour in real-world driving scenarios.
(Cheng et al., 2021) organised a critical study of deep learning approaches for gaze estimation
based on external features. It covers four different vantage points: deep feature extraction,
designing a deep neural network architecture, calibrating oneself, and device and platform
considerations. Regarding cross-subject gaze estimates, the performance is much enhanced by
using an approach based on deep learning rather than one based on the subject's outward
appearance. Also, some techniques use CNN models together with gaze information. They have
compiled summaries of available public datasets and constructed benchmarks for gaze
estimates based on post-processing techniques for collected data. Unconstrained 2D gaze point
estimation algorithms are often evaluated using the GazeCapture dataset. The RT-Gene dataset,
the Gaze360 dataset, the ETH-XGaze dataset, and the E.E.E. dataset are all examples of these
estimations. This study provides a systematic review of gaze estimation techniques based on
gaze estimation and uses webcam images of the eyes to predict the gaze subject. The study
introduces four novel aspects: deep feature extraction, design of deep neural network
architecture, individual calibration, and the hardware/software platform.
(Ishikawa et al., n.d.) presented an Active Appearance Model (AAM) used by an algorithm for
driver gaze estimation to track the eye corners, extract the eye region, estimate the face's size,
and estimate the head's position. Standard methods are used to find the iris in the eye area, and
a standard geometric model is used to figure out the direction of gaze based on this information.
The system's reliability and accuracy come from AAM tracking the whole head instead of using
a technique based on local features.
In their study, (Rangesh et al., n.d.) conducted research to enhance gaze estimation's reliability
and applicability. They achieved this by utilising real-world data captured under challenging
conditions, including scenarios with eyeglasses, bright lighting, nighttime driving, and various
head positions. They employed an infrared camera with appropriate equalisation and
normalisation techniques to address these challenges. Additionally, they proposed a Gaze
Preserving Cycle GAN (GPCycleGAN) that trained a generator capable of removing
eyeglasses while preserving the original gaze image. The combined model outperformed the
vanilla CycleGAN + SqueezeNet model by 1.5% and exhibited a higher accuracy than the
baseline method by 10.5% in micro-average accuracy and 8.9% in macro-average accuracy.
Future work will focus on improving different components of the model, such as the generator,
discriminator, and gaze classifier, to further enhance performance.
(Yoon et al., 2019) presented a new way to estimate drivers' gaze using CNN in a vehicle
environment. The method differs from traditional approaches as it uses images from both the
front and side cameras simultaneously to estimate the driver's gaze. The input image to the
deep ResNet network combines the three-channel image obtained from both cameras. Unlike
commercial systems like faceLAB and Tobii, which require initial calibration by having the
driver gaze at certain positions, this method does not need any calibration. This is because it is
difficult to define predetermined positions in the front window of a car, and it is challenging to
think that the driver will cooperate with calibration. With this method, only one CNN model is
used, which increases reliability and reduces the computational cost. The authors suggest
further improvement to the accuracy by reconstructing the eye region of interest using super-
resolution and decreasing the number of layers and filters to speed up the processing.
(Pathirana et al., 2022) conducted a study to examine the influence of climate change on crop
yields within a specific geographical area. The researchers used a publicly available dataset of
crop yields and weather data for the region from the past few decades. They employed a
machine learning methodology, specifically a random forest model, to analyse the data and
predict future crop yields under different climate scenarios. The results showed a significant
negative impact of climate change on crop yields, with varying severity depending on the
climate scenario. The outcome of the results was that the researchers were able to identify the
crops most vulnerable to climate change and the regions most affected. The discussion focused
on the potential adaptation strategies that could be implemented to reduce the effects of climate
change on crop yields. In conclusion, this study highlights the urgent need for action to address
the effect of climate change on agriculture. It provides valuable insights for policymakers and
farmers to develop effective adaptation strategies.
In their study, (Kasahara et al., n.d.) aimed to develop a methodology for estimating a driver's
focus of attention while driving, which is crucial for ensuring road safety. To accomplish this,
the researchers introduced a novel dataset named "Look Both Ways," containing video
recordings capturing drivers' faces and the road scene ahead and accurately annotated gaze
data. By employing unsupervised and self-supervised learning techniques, they trained two
models: one for estimating the driver's gaze direction and another for estimating scene saliency.
The findings demonstrated the superior effectiveness of their proposed approach compared to
existing supervised methods in accurately estimating gaze direction and scene saliency. The
authors discussed how this method could be useful in developing more accurate and reliable
driver assistance systems and improve road safety. They concluded that their proposed method
effectively studies driver behaviour and attention, which has important implications for road
safety.
(Nikan & Upadhyay, 2022) proposed a study that looks at how well different appearance-based
approaches to estimating gaze work by directly applying DNN models to an image of an eye
and slowly declining the gaze angle. Gaze estimation is an efficient technique for determining
what people think and how much attention they are trying to pay to what they see. The original
dataset was utilised for training models. However, because the dataset was small and needed
more variations in how things looked, the performance of models trained with synthetic data
dropped significantly. To improve how well in-cabin gaze estimation works in DSM, error
metrics should be easy to understand and better fit the applications in terms of distance and
scene settings. In-car images can be used in the future to do experiments on this phenomenon.
(Dua et al., n.d.) talks in this paper about estimating driver gaze on the road, which is important
for improving road safety. This research proposes a technique that utilises inexpensive camera
equipment and machine learning techniques to estimate driver gaze. To accomplish this goal,
the authors introduced a new dataset named "DGAZE," consisting of synchronised video
recordings capturing the driver's face and the road scene and precise gaze data obtained from
an eye tracker. The methodology involved employing a convolutional neural network (CNN)
to establish a common feature space for the driver's face and the road scene, followed by a
regression model to estimate the gaze point on the road. The results indicated that the proposed
approach achieved high accuracy in estimating the gaze point on the road, with an average error
of less than 1 degree of visual angle. The findings contribute to developing a cost-effective and
precise method for estimating driver gaze on the road. The discussion centred on the potential
applications of the proposed technique in enhancing driver assistance systems and studying
driver behaviour and attention. The study concludes that the proposed method is effective and
has practical utility in real-world scenarios to improve road safety.
Table 3

Summary of papers on Gaze Detection

Model Accuracy Datasets Sensor Types Reference

2D and 3D CNN 2D CNN Model: Custom Infrared Cameras Lollett et al., 2022
Model 74.96% Benchmark
3D CNN Model: Dataset
87.02%

CNN Model (YOLO- 92.71% Personal Multiple Cameras and Shah et al., 2022
V4 Darknet-53 and Dataset Smart Eye Pro
YOLO-V4 Inception- Tracking (Smart
v3) Glasses)

CNN Model (VGG-16 VGG-16 Model: GazeCapture, Infrared Cameras Akinyelu & Blignaut,
& AlexNet) 93.36% TabletGaze 2022
AlexNet model:
88.91%

CNN Model (VGG-16 VGG-16 Model: 92% MPIIGaze, Infrared Cameras Park et al., 2018
& AlexNet) MPIIGazeMPIIGaze : Columbia,
82% EYEDIAP

Personalised Gaze Dataset1: 93.72% Personal Set of Cameras Vasli et al., 2016
Estimation Model and Dataset 2: 84% Datasets
3D Model

Global and User Level 91.40% bug 300-W Set of Cameras Fridman et al., 2015
3D Based Models dataset

Author’s Opinion
Gaze Estimation methods are used in driver monitoring systems to identify the driver's focus.
If the driver is gazing continuously other than the road, then it can be concluded that the driver
is not focusing on the road. This situation can be dangerous and can lead to fatal accidents.
Gaze Estimation can determine the loss of the driver's focus from the road and alert all
passengers and drivers. As most fatal accidents occur due to drivers' loss of attention on the
road, this technology can be immensely useful to reduce it and save precious human lives. It
can also be used to predict the future move of the driver about the vehicle, like changing lanes.
Gaze Estimation uses various Gaze cues to determine the driver's attentiveness, including blink
rate, eye openness, etc. Gaze Estimation can be used extensively in vehicles to increase road
safety, but some technical challenges must be overcome. Most Gaze Estimation algorithm
requires less distance and high-quality photos of the driver for accurate results, which cannot
be possible during a less light environment. Also, the result of the Gaze Estimation algorithm
needs to be in real-time. All these obstacles can be overcome with more research.
D. Blink Detection
Machine Learning integrated with Blink Detection
There are two approaches for blink detection video-based and non-video based. Machine
Learning is utilised in many non-video-based approaches. These methods include a training
model from which the algorithm learns what type of movements contribute to blink detection.
After the training, the machine learning algorithm could predict outcomes with the help of the
training dataset. These methods are more accurate than methods of video-based approach and
require less computational power after the training. Machine Learning techniques include Deep
Learning and Shallow Learning. Deep learning includes neural networks containing more than
one inner layer, and Shallow Learning is mutually exclusive to Deep Learning. There are many
techniques available that can be classified into Deep Learning and Shallow Learning, like
Support Vector Machines (SVMs), Convolutional Neural Nets (CNNs), and Long-Short Term
Memory (LSTMs). (Muller & Muller, 2019)

Figure 6. Machine Learning with Blink Detection

This figure depicts various approaches used for blink detection. Initially, it is divided into
video-based and non-Video-based approaches. Now video-based approaches were used earlier
and can be classified into Appearance-based and motion-based. Non-Video based approaches
use Machine Learning and can be classified into Shallow Learning and Deep Learning based.
SVMs use Shallow Learning, while CNNs and LSTMs use Deep Learning based approaches
for blink detection.
Case Studies
This study (Chang et al., 2022) represents a system designed to detect drowsiness in drivers by
monitoring their eye movements and facial physiological signals. They developed a reliable
and precise system that can be integrated with in-car monitoring systems to improve driving
safety. The researchers used computer vision techniques and ML algorithms to create the
system. They tested its accuracy by evaluating the publicly available "Drowsy Driver
Detection" dataset, which contains videos of drivers experiencing varying levels of drowsiness.
Moreover, the paper suggests the integration of photoplethysmography imaging's (PPGI) heart
rate variability (HRV) analysis to detect the LF/HF ratio, along with monitoring the percentage
of eyelid closure over the pupil over time (PERCLOS), resulting in a system accuracy of 92.5%.
The research emphasises the promising prospects of employing computer vision and machine
learning methods to develop reliable drowsiness detection systems. These systems can alert
drivers when their drowsiness levels surpass a predefined threshold, enhancing driving safety.
(Gawande & Badotra, n.d.) Conducted the study, which is aimed to enhance the precision and
resilience of an eye-blink detection system by employing a deep learning approach and hybrid
optimisation concepts. The researchers used a dataset containing eye images and corresponding
labels of blinking and non-blinking eye images to achieve this. The methodology used a deep
CNN architecture and hybrid optimisation techniques such as Adam, RMSProp, and AdaDelta.
I-ASM (improved active shape models) and LBP are two extracted features used to train this
optimised CNN architecture. Here, the proposed Kernel Median Filtering (KMF) method
increases the image quality in the frames. The results demonstrated that the proposed system
outperformed existing methods in accuracy and robustness, with the hybrid optimisation
approach successfully optimising the deep CNN architecture for high accuracy and low
computational cost. The study discussed the system's possible applications in the fields of
human-computer interaction, biometric identification, and healthcare. The conclusion was that
the proposed approach had the potential for efficient eye-blink detection and could be further
improved with more extensive datasets and additional optimisation techniques. Overall, the
study emphasises the usefulness of deep learning and hybrid optimisation concepts in
developing precise and efficient eye-blink detection systems.
(Schmidt et al., 2018) Explores how well blink detection algorithms perform in conditionally
automated driving and manual scenarios. The research compares a blink detection process that
uses electrooculography and cameras, along with various signal-processing algorithms and two
different data sampling frequencies. Additionally, the study assesses the performance of 24
reference groups in detecting blinks. While the correct detection rates for manual and alert
driving were high (maximum 94%), they significantly decreased during drowsy and
conditionally automated driving phases. The study suggests that the measurement frequency,
driver state, level of automation, and algorithmic techniques all influence blink detection. It
also emphasises the importance of detecting drowsiness while driving and developing reliable
systems to detect, warn and prevent drowsiness before it begins by monitoring blinking and
eyelid movements.
In their study (Gaffary & Lécuyer 2018) developed a real-time mobile phone-based system on
the Android platform for eye-blink detection and gaze tracking. With a Haar classifier and the
Normalised Summation of Square of Difference template-matching method, the system
achieved an impressive 98% accuracy rate for blink detection from both eyes at 0° angles. It
also could categorise blinks into short, medium, and long durations based on blink duration
and closure degree. The system underwent extensive testing, including variations in lighting,
subjects, gender, angles, processing speed, RAM capacity, and distance, and performed well in
real-time scenarios for both single and dual-eye detection. The practical applications of this
system are significant, particularly in detecting driver drowsiness and enabling eye-operated
mouse control for individuals with disabilities. Overall, Gaffary and Lécuyer's study highlights
the potential of their mobile phone-based system in accurately detecting eye blinks and tracking
gaze, opening possibilities for improving driver safety and assistive technology.
(Salehian & far, 2015) Proposed an algorithm for detecting blinking of eyes based on two
approaches: shape analysis and histogram analysis. Using computer vision techniques and
efficient images, the blinking patterns of the driver were analysed in almost real time. Initial
results demonstrated that the current method can be useful for monitoring blink detection to
see if someone is tired. In the future, work may be done to improve the system for getting
images, add more pre-processing methods, and use adaptive methods in the edge detection and
global thresholding steps. Using the same methods to recognise other visual cues, like facial
expressions and yawning, could make it easier to tell when a driver is tired in the future.
In their study, (Ryan et al., n.d.) explored the use of a fully convolutional neural network (CNN)
and a lightweight method for detecting blinks to locate and track the face and eyes in event
cameras. They manually collected test datasets and evaluated the performance of both methods
using qualitative and quantitative measures. The findings indicated that event cameras hold
promise for driver monitoring systems (DMS). Event cameras offer several advantages over
regular cameras, including higher temporal resolution and the ability to adjust framerates based
on the task. Leveraging these features, developing more advanced DMS solutions surpassing
the capabilities of current fixed framerate approaches becomes possible. The study highlights
the potential of event cameras in revolutionising DMS technology and unlocking new
possibilities for enhanced driver safety and monitoring.
(Dewi, Chen, Jiang, et al., 2022) proposed a method to automatically classify blink types by
introducing a Modified Eye Aspect Ratio (EAR) as a new threshold parameter. The experiment
demonstrated that utilising Modified EAR improved the accuracy of blink detection. In future
work, the researchers aim to focus on datasets that include additional facial actions like smiling
and yawning to enhance the versatility of blink detection algorithms. They also suggested that
Machine Learning methods could be employed for more effective blink detection in upcoming
studies. By combining Modified EAR and Machine Learning techniques, the researchers
anticipate developing advanced blink detection systems capable of accurately identifying
blinks in various contexts. This research paves the way for improved blink detection algorithms
that can be applied to diverse applications such as driver monitoring, facial expression analysis,
and human-computer interaction.
(Bhavana & Sivakumar, 2021) described a combination-based method for determining whether
a driver is too tired to drive. It starts with a strong indicator of a landmark and then uses a
simple eye blink detection technique to identify the person based on the SVM. The strategy is
independent of the topics, and there is no good reason to make the framework. Experimental
results show that the algorithm works well, with a best-case accuracy of 96% for the EAR SVM
classifier. Future work can be incorporated into the framework with universally utilised
applications like Uber and Ola.
(J. Hu et al., 2022) aimed at making a case for implementing a low-cost, contactless, and user-
friendly sleepy driving detection system that protects drivers' privacy while maintaining
satisfactory monitoring accuracy. By doing theoretical and experimental research, they
statistically modelled the relationship between signal changes and tiny eye-induced movements
such as blinks. The comprehensive experimental findings demonstrated the system's efficiency,
which showed a median detection accuracy of 95%.
Table 4

Summary of papers on Blink Detection

Detectors/Classifiers Datasets Accuracy Speed Reference

Histogram of Oriented Talking Face and EyeBlink8 Talking Face: 30 fps Dewi, Chen,
Gradients (HOG) 97.10% Chang, et al., 2022
EyeBlink8: 97%

Linear SVM classifier CEW dataset and ZJU Eye- 95% > 10 fps Han et al., 2018
with HOG features and blink dataset
the LeNet-5 CNN
model

Canny Edge Detector Caltech database and Eye-blink CCD Camera: 95.3% 15fps Lenskiy & Lee,
and SVM database. CMOS Camera: 2012
96.3%

Viola Jones Face JZU eyeblink database 94% 110fps Danisman et al.,
Detector and Neural 2010
Network-based Eye
Detector.

Adaboost-based face Personal Dataset > 90% 0.5 fps - Raees Ahmad &
detector, Haar 2.5fps Borole, 2015
Classifier, and Viola
Jones Algorithm

Author’s Opinion
The eye detects the face in various algorithms, which are eventually used to predict different
results. Blink Detection is used to know the location of the face in the given picture. It can also
be used to know the driver's drowsiness by continuously calculating the blink rate. The decrease
in the continuous blink rate of the driver can be concluded that the driver is feeling sleepy and
is losing attention from the road. Blink Detection can be used to warn the driver and passengers
when the driver is sleepy to avoid accidents due to the driver's negligence. Blink Detection can
be achieved using two approaches video-based and non-video based. Non-video-based
approaches use Machine Learning, and they are more accurate along with less time required
after the algorithm's training. Non-video-based approach for Blink Detection uses Shallow
Learning and Deep Learning. Using these algorithms, Blink Detection can be used to know the
location of eyes, which is then used in various other algorithms of driver monitoring systems.
Individually, Blink Detection is very useful for knowing the driver's attention. This technology
can be very useful as it is the basic requirement for other useful technologies and their
applications.

IV. Control System Perspectives of ADAS & Machine Learning

The accuracy of ADAS is currently poorer than human drivers due to the lack of
technological advancement necessary to implement it. So, ADAS and human drivers must be
used as support driving systems. In this system, human drivers are the primary decision-
makers, and ADAS only assists them in decision-making. Also, in this system, ADAS
controls the vehicle in certain situations but keeps human drivers in the loop. According to
various researchers, maintaining human drivers in the loop is necessary because AI decisions
cannot be trusted today. So, a control structure is necessary to decide when an automatic
control system needs to assist the driver and when to take control of the vehicle. Various
research has been done regarding the control structure to compare the performances of
various structures. (Liu et al., 2022)
In certain conditions, automatic control systems are not capable of making decisions. So, it is
necessary to hand over control of the vehicle to the human driver. Here, a problem arises of
hand over time duration. During this duration, control is transferred from automatic control
systems to the human driver. Sometimes, the driver takes time to get control of the vehicle,
which might be dangerous in certain situations. A visual-auditory handover request was
observed to be more effective than a pure visual request. Six types of control transitions are
defined considering the human factor. The vehicle's longitudinal and lateral behaviour is
observed during the handover time. It was observed that the handover quality of authority of
the vehicle depends on various factors like handover duration, traffic situation, vehicle speed,
and the driver's secondary task. Considering performance deterioration during handover,
designing a controller that assists human drivers in this critical time is necessary. (Y. Chen et
al., 2021)
Another way to solve the problem of less accuracy of ADAS systems can be to use a
cooperative approach. This approach shares vehicle control between the human driver and
automatic control systems. Automatic control systems assist the human driver with various
decisions and do not restrict the driver from making critical decisions not detected by the
system. This method ensures proper authority over the automatic control systems by
including the human driver in the decision-making loop. The human driver can disagree with
the decision made by the control system and take control of the vehicle in his hands if
required. In vehicles' Lane Keeping Assistance System (LKAS), the human driver and the
control system can share the vehicle's steering. The driver can know every decision of the
control system by steering movements. If the driver finds something wrong, he can
immediately restrict the steering movement and change the decision. This system decreases
the driver's driving load and ensures the vehicle's safety, as the driver can take control at any
moment. (Nguyen et al., 2017)

V. Challenges
“ADAS technology holds immense potential to transform our driving experience, but its true
power lies in its ability to protect lives on the road.”

The development of Advanced Driver Assistance Systems (ADAS) presents numerous

challenges. These systems aim to enhance vehicle safety and help drivers. However, challenges
include ensuring accurate object detection, dealing with adverse weather conditions,
integrating complex sensor technologies, addressing cybersecurity concerns, and achieving
seamless interaction with human drivers. Overcoming these challenges is crucial for the
successful implementation of ADAS and the advancement of autonomous driving
technologies. The field of Advanced Driver Assistance Systems (ADAS) has witnessed
remarkable advancements in recent years, revolutionising the way we interact with vehicles.
However, this cutting-edge technology has its challenges. ADAS faces a multitude of hurdles
in ensuring reliable and accurate performance. One such challenge lies in Face Detection, a
critical component that enables the system to identify the driver's face and track their
movements. Another obstacle is Head Pose Estimation, which accurately determines the
driver's head position and orientation. Additionally, Blink Detection plays a crucial role in
monitoring the driver's level of fatigue or distraction. Lastly, Gaze Estimation enables ADAS
to understand where the driver is looking and anticipate their intentions. In the following
sections, we will detail these four methods and explore their significance in enhancing road
safety.

Face detection in Advanced Driver Assistance Systems (ADAS) faces numerous challenges.
Despite the development of various face recognition methods, only some can match the human
ability to recognise faces under different conditions. The system becomes more complex as its
application widens, making face detection a significant hurdle. One major challenge is the
variation in lighting conditions, which affects the system's efficiency in detecting and
recognising faces accurately. Different lighting levels make it difficult to perform face detection
effectively. Additionally, variations in poses, such as changes in viewing angles or head
rotations, pose problems for the system. Another challenge is the variation in facial expressions.
Different emotions can cause variations in facial appearance, making it challenging to identify
faces correctly. Ageing is also a factor, as the face changes over time, impacting face
recognition accuracy. Occlusions, where objects partially cover the face, further complicate the
detection process. Furthermore, similar faces, especially in the case of identical twins, can lead
to higher false recognition rates. Varying image resolutions and complex backgrounds in facial
images also affect the efficiency of face detection. Addressing these challenges requires the
development of robust algorithms capable of handling variations in lighting, pose, expression,
ageing, occlusions, similar faces, and image resolution. Innovations in face recognition aim to
improve the accuracy and reliability of face detection systems for ADAS applications. (Mary
Solomon et al., n.d.)
Head pose estimation is vital for analysing driver behaviour and attention in advanced driver
assistance systems (ADAS). However, existing algorithms for head pose estimation need help
to perform well in real driving conditions due to a few key challenges. Firstly, the varying
illumination in driving environments poses a problem. The lighting conditions can change
rapidly, including shadows or bright sunlight, which makes it difficult for algorithms to detect
and track facial features accurately. Secondly, occlusions are a significant challenge.
Occlusions occur when certain parts of the face, especially the eyes or mouth, are partially or
completely blocked. This is often the case with glasses, particularly thick-frame glasses, which
obstruct the view of the facial features. Consequently, accurately determining the position and
orientation of the head becomes challenging for head pose estimation algorithms. Thirdly,
drivers in real-world scenarios may exhibit extreme head rotations that surpass the typical
range. Estimation algorithms need to be able to handle these rotations to provide reliable
results. Extreme yaw and pitch angles can create difficulties and lead to inaccurate or unreliable
head pose estimations. Lastly, current algorithms for controlled environments perform less well
in driving conditions due to the challenges mentioned above. To overcome these limitations, it
is crucial to develop improved head pose estimation algorithms that can effectively handle
varying illumination, occlusions caused by glasses or other factors, and extreme head rotations.
Addressing these challenges is essential for accurately analysing driver behaviour, monitoring
attention levels, and developing effective in-vehicle systems in ADAS. (Jha & Busso, 2018)
Blink detection involves detecting eye blinks, but it comes with challenges. One of the main
difficulties is that the appearance of the eyes can change due to factors like lighting, head
movement, and blockage. This makes it hard to detect blinks accurately. To overcome this, the
researchers used facial landmark detection to find the eyes and Eye Aspect Ratio to detect
blinks, improving accuracy. Another challenge is detecting blinks quickly and accurately in
real time. This requires fast algorithms and hardware that can handle large amounts of data.
The researchers used USB cameras and dynamic template matching to achieve real-time
processing of eye images. Additionally, accurately detecting blinks in different situations, such
as varying lighting conditions and eye movements, posed another challenge. To address this,
the team used adaptive algorithms that could adjust to different situations and enhance blink
detection accuracy. Blink detection poses challenges that demand efficient algorithms,
hardware, and adaptive techniques to detect blinks accurately in real-time and various contexts.
(Islam et al., 2019)
Gaze estimation is tracking a person's eye movements and where they are looking. In the
context of Advanced Driver Assistance Systems (ADAS), several challenges exist. One
challenge is the placement of the camera sensor in relation to the driver's seat. The camera must
be positioned correctly to accurately track the driver's gaze without blocking their view of the
road. Another challenge is dealing with low light conditions, making locating the driver's pupils
in the image difficult. Techniques like infrared illumination or improving image quality may
be necessary to address this issue. Additionally, the system for detecting the pupils may need
enhancements to track the driver's gaze under different lighting situations accurately.
Adjustments may be required for the proposed gaze tracking approach. This could involve
changing the camera sensor or exploring other gaze-tracking systems. Recent advancements in
differential gaze estimation, robust 3D eye gaze tracking, and neuro-inspired eye tracking could
improve the accuracy and reliability of gaze tracking in ADAS. Implementing a gaze-tracking
algorithm for ADAS has various challenges that must be addressed to ensure precise and
dependable gaze tracking. However, with ongoing research and development, gaze tracking
has the potential to enhance driving safety by providing timely warnings to drivers about
potential hazards on the road. (Ledezma et al., 2021)
In conclusion, the challenges in ADAS include complexity and dependence on other vehicle
systems, misjudgement and misdetection of surrounding vehicles, lack of consumer awareness,
high costs, and the need for adaptable and flexible modelling platforms to verify performance.

VI. Future Scope

“The future scope of ADAS is boundless, as it holds the key to unlocking a new era of intelligent
transportation.”
ADAS technology is the future of the automotive industry. It has various scopes in the field of
technology. This technology can be incorporated into automobiles to improve their
performance and enhance the safety of humans. It is a breakthrough technology that is helpful
in the day-to-day life of humans. There are various ways by which we can improve the
effectiveness of ADAS technology. One such way is to develop more advanced and accurate
sensors. Object Detection can be made possible by incorporating artificial intelligence and
machine learning algorithms. Cybersecurity measures should also be enhanced to prevent the
systems from hacking. The main thing all ADAS manufacturers should keep in mind is the cost
of making it available to all users. ADAS should be integrated with smart city projects as it can
help reduce traffic congestion and control traffic signals efficiently and sustainably.
Face detection technology has made significant progress in recent years and holds immense
potential for the future. Its advancements have been particularly notable in the realms of
security and surveillance. Real-time identification of individuals through face detection is now
a crucial tool for law enforcement agencies and security personnel, aiding in preventing and
detecting criminal activities in public spaces such as airports, train stations, and shopping malls.
Another area witnessing remarkable development is healthcare. This is particularly beneficial
in critical care units where continuous monitoring is vital. In marketing and advertising, face
detection technology offers the ability to analyse customers' facial expressions and emotions.
This valuable insight into preferences and behaviour helps businesses tailor their products and
services to cater to their customers' needs better. Furthermore, face detection technology can
enhance accessibility for individuals with disabilities. For instance, it can facilitate the control
of devices like wheelchairs or prosthetic limbs, granting greater independence to people with
limited mobility. In summary, the prospects of face detection technology are expansive and
diverse, with its applications spanning across various domains. With ongoing advancements,
we anticipate witnessing even more innovative and exciting applications in the coming years.
(Kumral & Küçükmanisa, 2022)
Head-Pose Estimation can be detected using deep neural networks to forecast an individual's
attention span. Attention span is a fundamental factor in various areas, including education and
driving, and the ability to forecast it is immensely useful. Some challenges can be determined
while developing the model, such as the requirement of huge training data and the challenge of
precisely estimating head pose. This work could be extended to encompass eye gaze prediction,
which could be very helpful in applications such as examinations, interviews, and ADAS. So
the recommended technique can help in various applications, including the ADAS, in which
we can estimate the head position of the driver. Hence, by using more deep learning algorithms
and techniques, we can provide a sustainable and safe solution to the drivers. (Singh et al., n.d.)
The utilisation of the driver gaze estimation is crucial in various applications, such as driver
attentiveness detection, visual distraction detection, gaze behaviour understanding, and
building driver assistance systems. The aim is to offer a comprehensive summary of driver-eye
activity principles, the approaches to assess them, and their applications in real-world driving
conditions. Future research can help in the evolution of this domain and contribute to the
development of a safer transportation system. Future work in the development of ADAS should
incorporate the basics of driver gaze estimation, current benchmark driver gaze datasets,
algorithms for driver gaze estimation, and its real-time applications. Prospects of driver gaze
estimation and gaze-based applications can be addressed using deep learning algorithms such
as CNN, which can lead to a safer automobile industry. (Sharma & Chakraborty, 2023)
The focus is also on a blink detection algorithm that can serve as a visual cue for driver fatigue.
The primary obstacle to using this algorithm in a real-world setting is the necessity for
shortened processing time to allow for adequate driver response time. To address this,
optimisation techniques should be properly implemented to meet latency requirements.
Additionally, the algorithm's validation should be carried out through visual inspection of video
sequences to determine precision and accuracy. The image acquisition system will remain
unchanged; no enhancements or modifications are planned for future work in this field. Using
similar techniques to identify other visual cues, like facial expressions and yawning, may also
enhance the accuracy of driver fatigue detection in the future. The blink detection algorithm
presented in this paper has demonstrated a precision rate of 84% and an accuracy rate of 69%
through 12 video sequences of varying durations and lighting conditions and a small sample of
participants. This emphasises the possibility of employing non-invasive and real-time image
processing and computer vision techniques for monitoring weariness in the future. (Salehian &
Far, n.d.)
In conclusion, Advanced Driver Assistance Systems (ADAS) is expected to grow rapidly in the
coming years, with increasing demand for safer and more efficient vehicles. Priority should be
on integrating deep learning techniques with the existing ADAS systems. However, challenges
such as high costs and consumer awareness need to be addressed, and further research is needed
to improve performance and develop adaptable and flexible modeling platforms.

VII. Conclusion
To conclude, the introduction provided an overview of the Advance Driver Assistance System
(ADAS) objectives and evolution and its applications in Machine Learning. The main aim of
ADAS is to mitigate fatal accidents by assisting drivers in making better decisions, as most
accidents occur due to human errors. The discussion focused on four crucial algorithms
employed in ADAS.
The Face Detection algorithm is utilised to identify faces in various environmental conditions.
Machine Learning techniques are employed to enhance the accuracy of this algorithm,
particularly in challenging situations with low lighting or partial face visibility.
The Head Pose Estimation algorithm estimates the head pose of the driver in an image and
assesses their attention level. Analysing the head pose helps prevent driver distraction and
enhances alertness. Machine Learning is employed in this algorithm to improve the accuracy
of attention estimation.
The Blink Detection algorithm is employed to detect signs of drowsiness or sleepiness in the
driver. By continuously monitoring the blink rate, a decrease in the rate can indicate
drowsiness, while a complete stoppage may indicate that the driver has fallen asleep. This
information can be used to alert the driver and other passengers, enabling appropriate action to
avoid fatal accidents.
The Gaze Estimation algorithm is employed to determine the driver's visual focus. If the
driver's gaze is consistently directed away from the road, it is a warning sign of potential
distraction. Machine Learning is used in the gaze estimation algorithm to recognise the driver's
face accurately initially, and subsequent deviations are tracked based on the initial facial
recognition patterns.
Overall, these algorithms focus on predicting the driver's mental state and issuing warnings
when driving conditions become risky, thereby reducing human errors on the road. The
implementation of these methods has the potential to create a significant impact on the
automotive industry.
References
Abbas, T., Ali, S. F., Mohammed, M. A., Khan, A. Z., Awan, M. J., Majumdar, A., &
Thinnukool, O. (2022). Deep Learning Approach Based on Residual Neural Network and
SVM Classifier for Driver&rsquo;s Distraction Detection. Applied Sciences 2022, Vol. 12,
Page 6626, 12(13), 6626. https://doi.org/10.3390/APP12136626
Abbink, D. A., Mulder, M., & Boer, E. R. (n.d.). Haptic shared control: smoothly shifting
control authority? https://doi.org/10.1007/s10111-011-0192-5
Advanced Driver Assistance Systems European Commission. (n.d.). Retrieved March 23, 2023,
from https://road-safety.transport.ec.europa.eu/system/files/2021-07/ersosynthesis2018-
adas.pdf
Akhtar, Z. U. A., Rasool, H. F., Asif, M., Khan, W. U., Jaffri, Z. U. A., & Ali, M. S. (2022).
Driver’s Face Pose Estimation Using Fine-Grained Wi-Fi Signals for Next-Generation
Internet of Vehicles. Wireless Communications and Mobile Computing, 2022.
https://doi.org/10.1155/2022/7353080
Akinyelu, A. A., & Blignaut, P. (2022). Convolutional Neural Network-Based Technique for
Gaze Estimation on Mobile Devices. Frontiers in Artificial Intelligence, 4, 204.
https://doi.org/10.3389/FRAI.2021.796825/BIBTEX
Ali, S. F., Aslam, A. S., Awan, M. J., Yasin, A., & Damaševičius, R. (2021). Pose Estimation
of Driver&rsquo;s Head Panning Based on Interpolation and Motion Vectors under a
Boosting Framework. Applied Sciences 2021, Vol. 11, Page 11600, 11(24), 11600.
https://doi.org/10.3390/APP112411600
Alioua, N., Amine, A., Rogozan, A., Bensrhair, A., & Rziza, M. (2016). Driver head pose
estimation using efficient descriptor fusion. Eurasip Journal on Image and Video
Processing, 2016(1), 1–14. https://doi.org/10.1186/S13640-016-0103-Z/TABLES/6
Allach, S., Ahmed, M. Ben, & Boudhir, A. A. (2020). Deep learning model for a driver
assistance system to increase visibility on a foggy road. Advances in Science, Technology
and Engineering Systems, 5(4), 314–322. https://doi.org/10.25046/AJ050437
Arganda-Carreras, I., Aranjuelo, N., Unzueta, L., & Otaegui, O. (2018). Multimodal Deep
Learning for Advanced Driving Systems Analysis and tracking of 3D deformable objects
in motion View project Automatic Non-rigid Histological Image Registration (ANHIR)
View project Nerea Aranjuelo Vicomtech Multimodal Deep Learning for Advanced
Driving Systems. https://doi.org/10.1007/978-3-319-94544-6_10
Arsenovic, M., Sladojevic, S., Anderla, A., & Stefanovic, D. (2017). FaceTime — Deep
learning based face recognition attendance system. 2017 IEEE 15th International
Symposium on Intelligent Systems and Informatics (SISY), 000053–000058.
https://doi.org/10.1109/SISY.2017.8080587
Badillo, S., Banfai, B., Birzele, F., Davydov, I. I., Hutchinson, L., Kam-Thong, T., Siebourg-
Polster, J., Steiert, B., & Zhang, J. D. (2020). An Introduction to Machine Learning.
CLINICAL PHARMACOLOGY & THERAPEUTICS | VOLUME, 107(4).
https://doi.org/10.1002/cpt.1796
Baker, S., Matthews, I., Xiao, J., Gross, R., Ishikawa, T., & Kanade, T. (n.d.). Real-Time Non-
Rigid Driver Head Tracking for Driver Mental State Estimation.
Ball, J. E., & Tang, B. (2019). Machine Learning and Embedded Computing in Advanced
Driver Assistance Systems (ADAS). Electronics 2019, Vol. 8, Page 748, 8(7), 748.
https://doi.org/10.3390/ELECTRONICS8070748
Bengler, K., Dietmayer, K., Farber, B., Maurer, M., Stiller, C., & Winner, H. (2014). Three
decades of driver assistance systems: Review and future perspectives. IEEE Intelligent
Transportation Systems Magazine, 6(4), 6–22.
https://doi.org/10.1109/MITS.2014.2336271
Bhavana, A., & Sivakumar, N. (2021). Real-Time Driver Drowsiness Detection Using Eye
Closure and Yawn Detection using Facial Landmarks. 9(6), 2320–2882. www.ijcrt.org
Borghi, G., Venturelli, M., Vezzani, R., & Cucchiara, R. (n.d.). POSEidon: Face-from-Depth
for Driver Pose Estimation.
Borrego-Carazo, J., Castells-Rufas, D., Biempica, E., & Carrabina, J. (2020). Resource-
Constrained Machine Learning for ADAS: A Systematic Review. IEEE Access, 8, 40573–
40598. https://doi.org/10.1109/ACCESS.2020.2976513
Cao, Y., & Liu, Y. (2017). Head pose estimation algorithm based on deep learning ARTICLES
YOU MAY BE INTERESTED IN Abnormal head movement classification using deep
neural network DNN AIP Conference Emotion recognition based on deep learning with
auto-encoder AIP Conference Proceedings. 1839, 30013.
https://doi.org/10.1063/1.4982509
Chang, R. C. H., Wang, C. Y., Chen, W. T., & Chiu, C. Di. (2022). Drowsiness Detection
System Based on PERCLOS and Facial Physiological Signal. Sensors 2022, Vol. 22, Page
5380, 22(14), 5380. https://doi.org/10.3390/S22145380
Chen, D., Ren, S., Wei, Y., Cao, X., & Sun, J. (2014). Joint cascade face detection and
alignment. Lecture Notes in Computer Science (Including Subseries Lecture Notes in
Artificial Intelligence and Lecture Notes in Bioinformatics), 8694 LNCS(PART 6), 109–
122. https://doi.org/10.1007/978-3-319-10599-4_8/COVER
Chen, L., Xin, G., Liu, Y., & Huang, J. (2021). Driver Fatigue Detection Based on Facial Key
Points and LSTM. Security and Communication Networks, 2021, 1–9.
https://doi.org/10.1155/2021/5383573
Chen, Y., Zhang, X., & Wang, J. (2021). Robust vehicle driver assistance control for handover
scenarios considering driving performances. IEEE Transactions on Systems, Man, and
Cybernetics: Systems, 51(7), 4160–4170. https://doi.org/10.1109/TSMC.2019.2931484
Cheng, Y., Wang, H., Bao, Y., & Lu, F. (2021). Appearance-based Gaze Estimation With Deep
Learning: A Review and Benchmark. http://phi-ai.org/GazeHub.
Cheruvu Ria. (2015, December). Big Data Applications in Self-Driving Cars⁤.
https://www.researchgate.net/publication/318940413_Big_Data_Applications_in_Self-
Driving_Cars
Choi, I. H., Jeong, C. H., & Kim, Y. G. (2016). Tracking a Driver’s Face against Extreme Head
Poses and Inference of Drowsiness Using a Hidden Markov Model. Applied Sciences
2016, Vol. 6, Page 137, 6(5), 137. https://doi.org/10.3390/APP6050137
Danisman, T., Bilasco, I. M., Djeraba, C., & Ihaddadene, N. (2010). Drowsy driver detection
system using eye blink patterns. 2010 International Conference on Machine and Web
Intelligence, ICMWI 2010 - Proceedings, 230–233.
https://doi.org/10.1109/ICMWI.2010.5648121
De-Las-Heras, G., Sánchez-Soriano, J., & Puertas, E. (2021a). Advanced Driver Assistance
Systems (ADAS) Based on Machine Learning Techniques for the Detection and
Transcription of Variable Message Signs on Roads. Sensors (Basel, Switzerland), 21(17).
https://doi.org/10.3390/S21175866
De-Las-Heras, G., Sánchez-Soriano, J., & Puertas, E. (2021b). Advanced Driver Assistance
Systems (ADAS) Based on Machine Learning Techniques for the Detection and
Transcription of Variable Message Signs on Roads. Sensors 2021, Vol. 21, Page 5866,
21(17), 5866. https://doi.org/10.3390/S21175866
De-Las-Heras, G., Sánchez-Soriano, J., & Puertas, E. (2021c). Advanced Driver Assistance
Systems (ADAS) Based on Machine Learning Techniques for the Detection and
Transcription of Variable Message Signs on Roads. Sensors 2021, Vol. 21, Page 5866,
21(17), 5866. https://doi.org/10.3390/S21175866
Dewi, C., Chen, R. C., Chang, C. W., Wu, S. H., Jiang, X., & Yu, H. (2022). Eye Aspect Ratio
for Real-Time Drowsiness Detection to Improve Driver Safety. Electronics 2022, Vol. 11,
Page 3183, 11(19), 3183. https://doi.org/10.3390/ELECTRONICS11193183
Dewi, C., Chen, R. C., Jiang, X., & Yu, H. (2022). Adjusting eye aspect ratio for strong eye
blink detection based on facial landmarks. PeerJ Computer Science, 8.
https://doi.org/10.7717/PEERJ-CS.943/SUPP-8
Diaz-Chito, K., Hernández-Sabaté, A., & López, A. M. (2016). A reduced feature set for driver
head pose estimation. Applied Soft Computing, 45, 98–107.
https://doi.org/10.1016/J.ASOC.2016.04.027
Dong, B. T., Lin, H. Y., & Chang, C. C. (2022). Driver Fatigue and Distracted Driving
Detection Using Random Forest and Convolutional Neural Network. Applied Sciences
2022, Vol. 12, Page 8674, 12(17), 8674. https://doi.org/10.3390/APP12178674
Dua, I., Ann John, T., & Gupta, R. (n.d.). DGAZE: Driver Gaze Mapping on Road.
El-Rewini, Z., Sadatsharan, K., Selvaraj, D. F., Plathottam, S. J., & Ranganathan, P. (2020).
Cybersecurity challenges in vehicular communications. Vehicular Communications, 23,
100214. https://doi.org/10.1016/J.VEHCOM.2019.100214
Farooq, M. A., Corcoran, P., Rotariu, C., & Shariff, W. (2021). Object Detection in Thermal
Spectrum for Advanced Driver-Assistance Systems (ADAS). IEEE Access, 9, 156465–
156481. https://doi.org/10.1109/ACCESS.2021.3129150
Fatima, B., Shahid, A. R., Ziauddin, S., Safi, A. A., & Ramzan, H. (2020). Driver Fatigue
Detection Using Viola Jones and Principal Component Analysis.
 Https://Doi.Org/10.1080/08839514.2020.1723875, 34(6), 456–483.
https://doi.org/10.1080/08839514.2020.1723875
Firintepe, A., Selim, M., Pagani, A., & Stricker, D. (n.d.). The More, the Merrier? A Study on
In-Car IR-based Head Pose Estimation.
Flores-Monroy, J., Nakano-Miyatake, M., Escamilla-Hernandez, E., Sanchez-Perez, G., &
Perez-Meana, H. (2022). SOMN_IA: Portable and Universal Device for Real-Time
Detection of Driver’s Drowsiness and Distraction Levels. Electronics (Switzerland),
11(16), 2558. https://doi.org/10.3390/ELECTRONICS11162558/S1
Fridman, L., Langhans, P., Lee, J., & Reimer, B. (2015). Driver Gaze Region Estimation
Without Using Eye Movement. IEEE Intelligent Systems, 31(3), 49–56.
https://doi.org/10.1109/MIS.2016.47
Gaffary, Y., & Lécuyer, A. (2018). The use of haptic and tactile information in the car to
improve driving safety: A review of current technologies. Frontiers in ICT, 5(MAR), 4.
https://doi.org/10.3389/FICT.2018.00004/BIBTEX
Gawande, R., & Badotra, S. (n.d.). Deep-Learning Approach for Efficient Eye-blink Detection
with Hybrid Optimization Concept. IJACSA) International Journal of Advanced
Computer Science and Applications, 13(6), 2022. Retrieved March 17, 2023, from
www.ijacsa.thesai.org
Han, Y. J., Kim, W., & Park, J. S. (2018). Efficient Eye-Blinking Detection on Smartphones: A
Hybrid Approach Based on Deep Learning. Mobile Information Systems, 2018.
https://doi.org/10.1155/2018/6929762
Hemaanand, M., Rakesh Chowdary, P., Darshan, S., Jagadeeswaran, S., Karthika, R., &
Parameswaran, L. (2020). Advanced driver assistance system using computer vision and
IOT. Advances in Intelligent Systems and Computing, 1108 AISC, 768–778.
https://doi.org/10.1007/978-3-030-37218-7_85
Hong, C., Yu, J., & Zhang, J. (2017). IEEE TRANSACTIONS ON KNOWLEDGE AND DATA
ENGINEERING, VOL. XX, NO. XX, XX XX 1 Multi-modal Face Pose Estimation with
Multi-task Manifold Deep Learning.
Hu, J., Jiang, H., Liu, D., Xiao, Z., Dustdar, S., Liu, J., & Min, G. (2022). BlinkRadar: Non-
Intrusive Driver Eye-Blink Detection with UWB Radar. Proceedings - International
Conference on Distributed Computing Systems, 2022-July 1040–1050.
https://doi.org/10.1109/ICDCS54860.2022.00104
Hu, Z., Zhang, Y., Xing, Y., Li, Q., & Lv, C. (2022). An Integrated Framework for Multi-State
Driver Monitoring Using Heterogeneous Loss and Attention-Based Feature Decoupling.
Sensors 2022, Vol. 22, Page 7415, 22(19), 7415. https://doi.org/10.3390/S22197415
Inkeaw, P., Srikummoon, P., Chaijaruwanich, J., Traisathit, P., Awiphan, S., Inchai, J.,
Worasuthaneewan, R., & Theerakittikul, T. (2022). Automatic Driver Drowsiness
Detection Using Artificial Neural Network Based on Visual Facial Descriptors: Pilot
Study. Nature and Science of Sleep, 14, 1641–1649. https://doi.org/10.2147/NSS.S376755
Ishikawa, T., Baker, S., Matthews, L., Kanade, T., & Matthews, I. (n.d.). Passive Driver Gaze
Tracking with Active Appearance Models.
Islam, A., Dept. of Electrical and Electronic Engineering, U. of D. D. 1000, B., Rahaman, N.,
Ahad, M. A. R., Dept. of Electrical and Electronic Engineering, U. of D. D. 1000, B., &
Dept. of Electrical and Electronic Engineering, U. of D. D. 1000, B. (2019). A Study on
Tiredness Assessment by Using Eye Blink Detection. Jurnal Kejuruteraan, 31(2), 209–
214.
https://www.academia.edu/76236240/A_Study_on_Tiredness_Assessment_by_Using_E
ye_Blink_Detection
Jha, S., & Busso, C. (2018). Challenges in head pose estimation of drivers in naturalistic
recordings using existing tools. IEEE Conference on Intelligent Transportation Systems,
Proceedings, ITSC, 2018-March 1–6. https://doi.org/10.1109/ITSC.2017.8317870
Jiménez, F., Naranjo, J. E., Anaya, J. J., García, F., Ponz, A., & Armingol, J. M. (2016).
Advanced Driver Assistance System for Road Environments to Improve Safety and
Efficiency. Transportation Research Procedia, 14, 2245–2254.
https://doi.org/10.1016/J.TRPRO.2016.05.240
Jurecki, R. S., & Stańczyk, T. L. (2023). Modelling Driver&rsquo;s Behaviour While Avoiding
Obstacles. Applied Sciences 2023, Vol. 13, Page 616, 13(1), 616.
https://doi.org/10.3390/APP13010616
Kalliatakis, G., Stergiou, A., & Vidakis, N. (2017). Conceiving Human Interaction by
Visualising Depth Data of Head Pose Changes and Emotion Recognition via Facial
Expressions. Computers 2017, Vol. 6, Page 25, 6(3), 25.
https://doi.org/10.3390/COMPUTERS6030025
Kar, A., & Corcoran, P. (2017). A review and analysis of eye-gaze estimation systems,
algorithms and performance evaluation methods in consumer platforms. IEEE Access, 5,
16495–16519. https://doi.org/10.1109/ACCESS.2017.2735633
Kasahara, I., Stent, S., & Park, H. S. (n.d.). Look Both Ways: Self-Supervising Driver Gaze
Estimation and Road Scene Saliency.
Khan, K., Ali, J., Ahmad, K., Gul, A., Sarwar, G., Khan, S., Ta, Q. T. H., Chung, T. S., &
Attique, M. (2020). 3D head pose estimation through facial features and deep
convolutional neural networks. Computers, Materials and Continua, 66(2), 1757–1770.
https://doi.org/10.32604/CMC.2020.013590
Krishna, G. S., Supriya, K., Vardhan, J., & K, M. R. (2022). Vision Transformers and YoloV5
based Driver Drowsiness Detection Framework.
https://doi.org/10.48550/arxiv.2209.01401
Krishnarao, S., Wang, H. C., Sharma, A., & Iqbal, M. (2021). Enhancement of Advanced Driver
Assistance System (Adas) Using Machine Learning. Advances in Intelligent Systems and
Computing, 1183, 139–146. https://doi.org/10.1007/978-981-15-5856-6_13
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (n.d.). ImageNet Classification with Deep
Convolutional Neural Networks. Retrieved March 17, 2023, from
http://code.google.com/p/cuda-convnet/
Kumral, F., & Küçükmanisa, A. (2022). Temporal Analysis Based Driver Drowsiness Detection
System Using Deep Learning Approaches. SSRN Electronic Journal.
https://doi.org/10.2139/SSRN.4031268
Laika, A., & Stechele, W. (n.d.). A review of different object recognition methods for the
application in driver assistance systems.
Ledezma, A., Zamora, V., Sipele, Ó., Sesmero, M. P., & Sanchis, A. (2021). Implementing a
Gaze Tracking Algorithm for Improving Advanced Driver Assistance Systems.
Electronics 2021, Vol. 10, Page 1480, 10(12), 1480.
https://doi.org/10.3390/ELECTRONICS10121480
Lenskiy, A. A., & Lee, J. S. (2012). Driver’s eye blinking detection using novel color and
texture segmentation algorithms. International Journal of Control, Automation and
Systems, 10(2), 317–327. https://doi.org/10.1007/S12555-012-0212-0
Liu, J., Guo, H., Shi, W., Dai, Q., & Zhang, J. (2022). Driver-automation shared steering control
considering driver neuromuscular delay characteristics based on stackelberg game. Green
Energy and Intelligent Transportation, 1(2), 100027.
https://doi.org/10.1016/J.GEITS.2022.100027
Lollett, C., Kamezaki, M., & Sugano, S. (2022). Single Camera Face Position-Invariant
Driver&rsquo;s Gaze Zone Classifier Based on Frame-Sequence Recognition Using 3D
Convolutional Neural Networks. Sensors 2022, Vol. 22, Page 5857, 22(15), 5857.
https://doi.org/10.3390/S22155857
Mahesh, B. (2018). Machine Learning Algorithms-A Review Machine Learning Algorithms-A
Review View project Six Stroke Engine View project Batta Mahesh Independent
Researcher Machine Learning Algorithms-A Review. International Journal of Science
and Research. https://doi.org/10.21275/ART20203995
Mary Solomon, M., Singh Meena, M., Kaur, J., Tech Student, M., & Professor, A. (n.d.).
CHALLENGES IN FACE RECOGNITION SYSTEMS. 6. Retrieved July 8, 2023, from
http://ijrar.com/
Meijer, M. J. (2020). Exploring augmented reality for enhancing ADAS and remote driving
through 5G study of applying augmented reality to improve safety in ADAS and remote
driving use cases.
Morris, T., Blenkhorn, P., & Zaidi, F. (2002). Blink detection for real-time eye tracking. Journal
of Network and Computer Applications, 25(2), 129–143.
https://doi.org/10.1006/JNCA.2002.0130
Moujahid, A., Hina, M. D., Soukane, A., Ortalda, A., El Araki Tantaoui, M., El Khadimi, A., &
Ramdane-Cherif, A. (2018). Machine Learning Techniques in ADAS: A Review.
Proceedings on 2018 International Conference on Advances in Computing and
Communication Engineering, ICACCE 2018, 235–242.
https://doi.org/10.1109/ICACCE.2018.8441758
Muller, D., & Muller, D. (2019). A Review Of Video-Based and Machine Learning Approaches
to Human Eye Blink Detection in Video.
https://repository.arizona.edu/handle/10150/632679
Murphy-Chutorian, E., Doshi, A., & Trivedi, M. M. (2007). Head pose estimation for driver
assistance systems: A robust algorithm and experimental evaluation. IEEE Conference on
Intelligent Transportation Systems, Proceedings, ITSC, 709–714.
https://doi.org/10.1109/ITSC.2007.4357803
Murphy-Chutorian, E., & Trivedi, M. M. (2009). Head pose estimation in computer vision: A
survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(4), 607–
626. https://doi.org/10.1109/TPAMI.2008.106
Nguyen, A. T., Sentouh, C., & Popieul, J. C. (2017). Driver-automation cooperative approach
for shared steering control under multiple system constraints: Design and experiments.
IEEE Transactions on Industrial Electronics, 64(5), 3819–3830.
https://doi.org/10.1109/TIE.2016.2645146
Nikan, S., & Upadhyay, D. (2022). Appearance-Based Gaze Estimation for Driver Monitoring.
Proceedings of Machine Learning Research, 1, 1–13.
Nishimura, R., Wada, T., & Sugiyama, S. (n.d.). Haptic Shared Control in Steering Operation
Based on Cooperative Status Between a Driver and a Driver Assistance System. Journal
of Human-Robot Interaction, X, No. X. https://doi.org/10.5898/JHRIX.X.XX
Nonaka, S., Nobuhara, S., & Nishino, K. (2022). Dynamic 3D Gaze from Afar: Deep Gaze
Estimation from Temporal Eye-Head-Body Coordination. Proceedings of the IEEE
Computer Society Conference on Computer Vision and Pattern Recognition, 2022-June
2182–2191. https://doi.org/10.1109/CVPR52688.2022.00223
Oh, G., Jeong, E., Kim, R. C., Yang, J. H., Hwang, S., Lee, S., & Lim, S. (2022). Multimodal
Data Collection System for Driver Emotion Recognition Based on Self-Reporting in Real-
World Driving. Sensors 2022, Vol. 22, Page 4402, 22(12), 4402.
https://doi.org/10.3390/S22124402
Olariu, C., Ortega, J. D., & Javier Yebes, J. (2018). The role of cloud-computing in the
development and application of ADAS. European Signal Processing Conference, 2018-
September 1037–1041. https://doi.org/10.23919/EUSIPCO.2018.8553029
Park, S., Spurr, A., & Hilliges, O. (2018). Deep Pictorial Gaze Estimation. Lecture Notes in
Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and
Lecture Notes in Bioinformatics), 11217 LNCS, 741–757. https://doi.org/10.1007/978-3-
030-01261-8_44
Pathirana, P., Senarath, S., Meedeniya, D., & Jayarathna, S. (2022). Eye gaze estimation: A
survey on deep learning-based approaches. Expert Systems with Applications, 199.
https://doi.org/10.1016/j.eswa.2022.116894
Paving the way to self-driving cars with advanced driver assistance systems Hannes Estl
Worldwide Systems Marketing for Advanced Driver Assistance Systems (ADAS), Texas
Instruments. (n.d.).
Piao, J., & McDonald, M. (2008). Advanced Driver Assistance Systems from Autonomous to
Cooperative Approach. Http://Dx.Doi.Org/10.1080/01441640801987825, 28(5), 659–
684. https://doi.org/10.1080/01441640801987825
Raees Ahmad, M., & Borole, N. (n.d.). Drowsy Driver Identification Using Eye Blink detection.
Retrieved March 23, 2023, from www.ijcsit.com
Rangesh, A., Zhang, B., & Trivedi, M. M. (n.d.). Driver Gaze Estimation in the Real World:
Overcoming the Eyeglass Challenge. Retrieved March 17, 2023, from
https://github.com/arangesh/GPCycleGAN
Rätsch, G. (n.d.). A Brief Introduction into Machine Learning. Retrieved March 23, 2023, from
http://www.tuebingen.mpg.de/˜raetsch
Ruiz, J. A., Llamazares, A., Wang, Y., Yuan, G., & Fu, X. (2022). Driver’s Head Pose and Gaze
Zone Estimation Based on Multi-Zone Templates Registration and Multi-Frame Point
Cloud Fusion. Sensors 2022, Vol. 22, Page 3154, 22(9), 3154.
https://doi.org/10.3390/S22093154
Ruiz, N., Chong, E., & Rehg, J. M. (n.d.). Fine-Grained Head Pose Estimation Without
Keypoints. Retrieved March 17, 2023, from https://github.com/natanielruiz/deep-head-
pose
Ryan, C., O’sullivan, B., Elrasad, A., Lemley, J., Kielty, P., Posch, C., & Perot, E. (n.d.). Real-
Time Face & Eye Tracking and Blink Detection using Event Cameras.
Saini, V., & Saini, R. (n.d.). Driver Drowsiness Detection System and Techniques: A Review.
Retrieved March 17, 2023, from www.ijcsit.com
Salehian, S., & Far, B. (n.d.). Embedded Real Time Blink Detection System for Driver Fatigue
Monitoring. https://doi.org/10.18293/SEKE2015-249
Salehian, S., & Far, B. (2015). Embedded real time blink detection system for driver fatigue
monitoring. Proceedings of the International Conference on Software Engineering and
Knowledge Engineering, SEKE, 2015-January, 188–194.
https://doi.org/10.18293/SEKE2015-249
Sancheti, N. K., Gopal, K. H., & Srikant, M. (n.d.). Camera-based driver monitoring system
using deep learning Camera based driver monitoring system using deep learning.
Retrieved March 30, 2023, from www.visteon.com
Schmidt, J., Laarousi, R., Stolzmann, W., & Karrer-Gauß, K. (2018). Eye blink detection for
different driver states in conditionally automated driving and manual driving using EOG
and a driver camera. Behavior Research Methods, 50(3), 1088–1101.
https://doi.org/10.3758/S13428-017-0928-0/TABLES/5
Seger, U., Knoll, P. M., & Stiller, C. (2000). Sensor Vision and Collision Warning Systems.
https://www.sae.org/publications/technical-papers/content/2000-01-C001/
Shah, S. M., Zhaoyun, S., Zaman, K., Hussain, A., Shoaib, M., & Lili, P. (2022). A Driver Gaze
Estimation Method Based on Deep Learning. Sensors 2022, Vol. 22, Page 3959, 22(10),
3959. https://doi.org/10.3390/S22103959
Shang, Y., Yang, M., Cui, J., Cui, L., Huang, Z., & Li, X. (2022). Driver Emotion and Fatigue
State Detection Based on Time Series Fusion. Electronics 2023, Vol. 12, Page 26, 12(1),
26. https://doi.org/10.3390/ELECTRONICS12010026
Sharma, P. K., & Chakraborty, P. (2023). A Review of Driver Gaze Estimation and Application
in Gaze Behavior Understanding. https://arxiv.org/abs/2307.01470v1
Shen, J., Li, G., Yan, W., Tao, W., Xu, G., Diao, D., & Green, P. (2018). Nighttime driving
safety improvement via image enhancement for driver face detection. IEEE Access, 6,
45625–45634. https://doi.org/10.1109/ACCESS.2018.2864629
Sigari, M.-H., Fathy, M., & Soryani, M. (2013). A Driver Face Monitoring System for Fatigue
and Distraction Detection. International Journal of Vehicular Technology, 2013, 1–11.
https://doi.org/10.1155/2013/263983
Singh, T., Mohadikar, M., Gite, S., Patil, S., Pradhan, B., & Alamri, A. (n.d.). Attention Span
Prediction using Head-pose Estimation with Deep Neural Networks.
https://doi.org/10.1109/ACCESS.2021.3120098
Soni, R. K., & Nair, B. B. (2021). Deep Learning Based Approach to Generate Realistic Data
for ADAS Applications. 2021 5th International Conference on Computer,
Communication, and Signal Processing, ICCCSP 2021, 181–185.
https://doi.org/10.1109/ICCCSP52374.2021.9465529
Ulrich, L., Nonis, F., Vezzetti, E., Moos, S., Caruso, G., Shi, Y., & Marcolin, F. (2021). Can
ADAS Distract Driver&rsquo;s Attention? An RGB-D Camera and Deep Learning-Based
Analysis. Applied Sciences 2021, Vol. 11, Page 11587, 11(24), 11587.
https://doi.org/10.3390/APP112411587
Vankayalapati, H. D., Kuchibhotla, S., Chadalavada, M. S. K., Dargar, S. K., Anne, K. R., &
Kyandoghere, K. (2022). A Novel Zernike Moment-Based Real-Time Head Pose and
Gaze Estimation Framework for Accuracy-Sensitive Applications. Sensors 2022, Vol. 22,
Page 8449, 22(21), 8449. https://doi.org/10.3390/S22218449
Vasli, B., Martin, S., & Trivedi, M. M. (2016). On Driver gaze estimation: Explorations and
Fusion of geometric and data driven approaches. IEEE Conference on Intelligent
Transportation Systems, Proceedings, ITSC, 655–660.
https://doi.org/10.1109/ITSC.2016.7795623
Wang, Y., Yuan, G., Mi, Z., Peng, J., Ding, X., Liang, Z., & Fu, X. (2019). Continuous Driver’s
Gaze Zone Estimation Using RGB-D Camera. Sensors 2019, Vol. 19, Page 1287, 19(6),
1287. https://doi.org/10.3390/S19061287
Xiao, W., Liu, H., Ma, Z., Chen, W., Sun, C., & Shi, B. (2022). Fatigue Driving Recognition
Method Based on Multi-Scale Facial Landmark Detector. Electronics 2022, Vol. 11, Page
4103, 11(24), 4103. https://doi.org/10.3390/ELECTRONICS11244103
Ye, M., Zhang, W., Cao, P., & Liu, K. (2021). Driver Fatigue Detection Based on Residual
Channel Attention Network and Head Pose Estimation. Applied Sciences 2021, Vol. 11,
Page 9195, 11(19), 9195. https://doi.org/10.3390/APP11199195
Yoon, H. S., Baek, N. R., Truong, N. Q., & Park, K. R. (2019). Driver Gaze Detection Based
on Deep Residual Networks Using the Combined Single Image of Dual Near-Infrared
Cameras. IEEE Access, 7, 93448–93461. https://doi.org/10.1109/ACCESS.2019.2928339
You, F., Gong, Y., Tu, H., Liang, J., & Wang, H. (2020). A Fatigue Driving Detection Algorithm
Based on Facial Motion Information Entropy. Journal of Advanced Transportation, 2020.
https://doi.org/10.1155/2020/8851485
Yu, H., Gupta, A., Lee, W., Arroyo, I., Betke, M., Allesio, D., Murray, T., Magee, J., & Woolf,
B. P. (2021). Measuring and Integrating Facial Expressions and Head Pose as Indicators
of Engagement and Affect in Tutoring Systems. Lecture Notes in Computer Science
(Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in
Bioinformatics), 12793 LNCS, 219–233. https://doi.org/10.1007/978-3-030-77873-6_16
Zhao, Y., Görne, L., Yuen, I. M., Cao, D., Sullman, M., Auger, D., Lv, C., Wang, H., Matthias,
R., Skrypchuk, L., & Mouzakitis, A. (2017). An Orientation Sensor-Based Head Tracking
System for Driver Behaviour Monitoring. Sensors 2017, Vol. 17, Page 2692, 17(11), 2692.
https://doi.org/10.3390/S17112692
Zhao, Z., Xia, S., Xu, X., Zhang, L., Yan, H., Xu, Y., & Zhang, Z. (2020). Driver distraction
detection method based on continuous head pose estimation. Computational Intelligence
and Neuroscience, 2020. https://doi.org/10.1155/2020/9606908