SUBMITTED BY HARDEEP SINGH

Roll No. 0820808

Me (epdt)
Guide
Mr. J.S Bhatia
Director
CDAC Mohali.
Co-Guide
Mrs Jasbir Kaur
Astt. Professor
P.E.C University of Tech,
Chandigarh
IEEE India International Conference on Power Electronics, NSIT Dwarka, India
Objective:
The main aim of this project is to develope an Eye
Tracking based Driver Fatigue Monitoring and
Warning System.
The main idea behind this project is to develope a
non-intrusive real time system which can detect
fatigue of driver and issue a timely warning.
Why we need this system ?
Since large number of road accidents are caused by driver
drowsiness.
In real time dangerous behaviours which are related to
fatigue are eye-closing, head nodding and brain activity
etc.
Hence we can either measure change in physiological
signals, such as brain waves, heart rate and eye blinking or
measuring physical changes.
The previous technique, while more accurate, but is not
realistic since highly sensitive sensing electrodes would
have to be attached directly on the drivers body and hence
which can be annoying and distracting to the driver.
Technique of physical changes
The technique to measure physical changes (i.e
open/closed eyes of driver to detect fatigue).
It is well suited for real world driving conditions since it can
be non-intrusive by using a video cameras to detect
changes.
The eye blink frequency increases beyond the normal rate
in fatigue state. In addition micro sleeps that are the short
period of sleeps lasting 3 to 4 seconds are good indicator of
fatigue state.
This system will detect eye movement to detect the fatigue
state of driver.
How it will work
An algorithm is used for driver hypo-vigilance
detection based on eye-region processing and
without explicit eye detection stage.
The Algorithm used here is very much fast as
compared to PERCLOS [5] earlier used by other
hence the Processing time of this system is less
than half second hence it is quite fast and
issues timely warning to the driver.
After image acquisition, face detection is the
first stage of processing. Then symptoms of
hypo-vigilance are extracted from the eyes
If eyes are blinking normally no warning is
issued but when the eyes are closed for more
than half second this system issues warning to
the driver in form of alarm and vibration.
Calculation of criteria for judging
drowsiness/ fatigue
Face detection
Drivers Image
Recognition of eyes
whether open or closed
Is driver
drowsy
WARNING
Eye detection
Yes
No
Block diagram mainly consists of three input variables and one Image
processing unit i.e computer and two output units i.e warning components and
a microcontroller. Click here for Block diagram
Proposed Block Diagram
S. No INPUT VARIBALES S. No OUTPUT VARIABLES
1. Eye tracking unit 1. Sound alarm
2. Speed variation detection unit 2. Seat belt vibration
3. Steering wheel gripping pressure
variability detection unit
3. Graphical display of speed and
gripping pressure variation in
real time.
Input Units
This system will consists three input units to study the
drowsy behaviour of driver namely
1. Eye Tracking unit.
2. Steering wheel gripping
pressure variability
detection unit.
3. Speed variation detection unit.
Eye Tracking Unit
This is the most crucial unit as the
accuracy of warning issued will largely
depends on this unit.
Driver face monitoring system is a real-
time system that can detect driver
fatigue and driver distraction using
machine vision approaches.
In this method, horizontal projection of
top half-segment of facial image is used
to extract symptoms of fatigue and
distraction.
Percentage of eye closure (PERCLOS)
and eyelid distance changes during time
are used for fatigue detection; and eye
closure rate is used for distraction
detection.
Image processing
This approach analyzes the images captured by cameras to
detect physical changes of drivers, such as eyelid
movement, eye gaze, yawn, and head nodding.
We will use camera and imaging processing techniques to
measure the eyelid closure over the pupil over time.
This vision based method is
not intrusive and will not cause
annoyance to drivers.
Fig. Progression of drowsiness and concordance of predicted and
observed PERCLOS value. The three threshold values are also
drawn
P70: the proportion of time when the eyes were closed at least 70 percent.
P80: the proportion of time when the eyes were closed at least 80 percent
(the P80 metrics is usually referred asPERCLOS)
EYEMEAS (EM): the mean square percentage of the eyelid closure rating.
PERCLOS (PERcent eyelid CLOSure)
PERCLOS (PERcent eyelid
CLOSure) is a measure of driver
alertness, which was identified as
the most reliable and valid in a study
by the US Federal Highway
Administration.
The measure is the percentage of
eyelid closure over the pupil over
time and reflects slow eyelid
closures rather than blinks.
PERCLOS to be among the most
promising known real-time
measures of alertness for in-
vehicle drowsiness-detection
systems.
Gripping Pressure variability Unit
The technique used here is that human body conducts current.
A conducting wire is embedded on non conducting steering wheel of
Vehicle and an Analog to Digital Convertor (ADC) is connected
through a Transistor.
Whenever driver hold the steering
tightly more current flows through
base of Transistor as parallel resistances
made by our fingers add up in parallel
and as a result net resistance decreases
and base current increases.
Speed Variation Detection Unit
Speed variation detection unit consists of a sensor on accelerator and
whenever driver felt drowsy there is possibility of sudden acceleration
or de-acceleration.
Hence we can judge this by plotting a graph in time domain and when
all three input variables shows a possibility of fatigue.
This will directly give an indication of drowsiness/fatigue which can be
further used as record of driver performance or can be used by traffic
police which can take further action accordingly.
Output/Warning Unit
For indication of warning we will use two approaches i.e
one by blowing alarm and second by seat belt vibration
whose frequency will vary between 100 to 300 Hzs.
Moreover the warning will be deactivated manually rather
than automatically. So for this purpose a deactivation
switch will be used to deactivate warning.
RESULTS
The first window is used for driver eye detection so
that driver iris may be detected when eyes are kept
inside region between two imaginary lines drawn
horizontally.
Window 1
RESULTS
The Second window shows Driver iris and eye brows
when Eyes open
The Third window shows image of eye brows when Eyes
closed
Window 3
Window 2
RESULTS
Window 4
RESULTS
Window 5
Window 6
RESULTS with glasses
Complete MATLAB window
Window 7
Results without glasses
Complete MATLAB window
Window 8
Conclusion
So this project will be helpful in detecting driver fatigue in
advance and will gave a real time warning output in form of
sound and vibration.
Moreover if driver felt drowsy there is possibility of sudden
acceleration or de-acceleration hence we can judge this by
plotting a graph in time domain and when all three input
variables shows a possibility of fatigue at one moment.
This will directly give an indication of drowsiness/fatigue
which can be further used as record of driver performance
or can be used by traffic police which can take further
action accordingly.
Hardware Developed
Hardware of Eye Tracking based driver fatigue
monitoring and warning system
Future Scope
Research can be done to make system work more
accurately when driver is in motion with respect to camera
as here camera used is highly directional.
In order to make this system more accurate research can be
done in making this system working possible at night.
Moreover research can be pursued to record data
wirelessly and making use of internet, which will help in
detecting driver performance record at remote location i.e
police station.
References
[1] S. Jane, W. Jean, V. Bradley, Why Do People Have Drowsy Driving Crashes, Input
From People Who Just Did, AAA Foundation for Traffic Safety, 1999.
[2] Mc. Afferty, Kevin, The Safety Network, Rseau-Scurit, Canadian Association of
Road Safety Professionals (CARSP), 2000.
[3] M.H Sigari, Driver Hypo-Vigilance Detection based on Eyelid Behavior, in
Proceedings of the Seventh International Conference on Advances in Pattern Recognition,
IEEE Computer Society, 2009. pp. 426-429.
[4] H. Jonathan, D. Thomas, Hanowski, Richard, W. Walter, A. Christina, In-Vehicle
Information Systems Behavioural Model and Design Support: Final Report, U.S
Department of Transportation, Federal Highway Administration, 2000.
[5] W. W. Wierwille, S. S. Wreggit, C. L. Kirn, L. A. Ellsworth, R. J. Fairbanks III,
Research on vehicle-based driver status/performance monitoring: development,
validation, and refinement of algorithms for detection of driver drowsiness, National
Highway Traffic Safety Administration, U.S. DOT Tech Report No. DOT HS 808 247,
1994.
[6] Dr. P.R Bajaj, M.S Devi, Driver Fatigue Detection Based on Eye Tracking, in
Proceedings of the First International Conference on Emerging Trends in Engineering and
Technology, IEEE Computer Society, 2008, pp. 649-652.
[7] R. Grace, V.E Byrne, D.M Bierman, J.M Legrand, D Gricourt, R.K
Davis, J.J Staszewski, B Carnahan, A Drowsy Driver Detection System
for Heavy Vehicles, in Proceedings of the IEEE, 1998, pp. I36-1-8.
[8] K. Ambak, R Atiq, R Ismail, Intelligent Transport System for
Motorcycle Safety and Issues, in Proceedings of the European Journal
of Scientific Research, Vol. 28, No. 4, 2009, pp. 600-611.
[9] G. Richard, B. Vicky, B. Damian, L.J.Michel, G. David; Davis, R.S.
James; C. Brian, A Drowsy Driver Detection System For Heavy
Vehicles, IEEE 0-7803-5086-3, 1998.
[10] A. Kircher, M. Uddman, J. Sandin, Vehicle control and
drowsiness, Project Report in Proceedings of the Swedish National
Road Transport, Linkping Sweden, 2002
[11] C.D Wylie, T. Shultz, J.C Miller et al: Commercial Motor Vehicle
Driver Fatigue and Alertness Study, TP Report No. 12876E, July 2001.
[12] E. Bekiaris, S. Nikolaou, A. Mousadakou, System for effective
Assessment of driver vigilance and Warning According to traffic Risk
Estimation, Project Report in Proceedings of the Design Guidelines for
Driver drowsiness detection & avoidance, AWAKE Consortium, 2004.
T. DOrazio, M. Leo, G.Cicirelli, A. Distante, An Algorithm for real time
eye detection in face images, in Proceedings of the 17th International
Conference on Pattern Recognition, IEEE Computer Society, Vol. 4, 2004, pp.
1051-4651.
[14] R. Dawn, National Survey of Distracted and Drowsy Driving, The
Gallup Organisation and NHTSA, 2003.
[15] K. Lin, J. Huang, J. Chen, C. Zhou, Real-time Eye Detection in Video
Streams, in Proceedings of the Fourth International Conference on Natural
Computation, IEEE Computer Society, Vol. 9, No. 8, 2008, pp. 193-196.
[16] M. Pereira, H. Hamama, N. Dapzol, M.P. Bruyas, A. Simoes,
Simultaneous interaction with in-vehicle systems while turning left:
comparison among three groups of drivers, in Proceedings of the IET
Intelligent Transport Systems, Vol. 3, No. 4, 2009, pp. 369378.
[17] E. Rogado, J.L. Garca, R. Barea, L.M. Bergasa, E. Lpez, Driver Fatigue
Detection System, in the Proceedings of the IEEE International Conference on
Robotics and Biomimetics Bangkok, Thailand, 2009, pp. 1105-1110.
[18] J. F. May, C.L. Baldwin, Driver fatigue: The importance of identifying
causal factors of fatigue when considering detection and countermeasure
technologies, in the Proceedings of the Transportation Research Elsevier, Part F
12, 2009, pp. 218224.
[19] I. Park, J.H. Ahn, H. Byun, Efficient Measurement of Eye Blinking under
Various Illumination Conditions for Drowsiness Detection Systems, in
Proceedings of the18th International Conference on Pattern Recognition, IEEE
Computer Society, Vol. 6, 2006, pp. 111-120.
[20] L. Yunqi, Y. Meiling, S. Xiaobing, L. Xiuxia, O. Jiangfan, Recognition of
Eye States in Real Time Video, in Proceedings of the International Conference
on Computer Engineering and Technology, IEEE Computer Society, Vol. 10,
No.105, 2009, pp. 554-559.
[21] F. D. Torre, C. J.G. Rubio, E. Martinez, Subspace Eye Tracking for
Driver Warning, in Proceedings of the IEEE, Vol. 8, No. 3, 2003, pp 329-
332.
[22] Y. Jie, Y. DaQuan, W. WeiNa, X. XiaoXia, W. Hui, Real-Time
Detecting System of the Drivers Fatigue, in Proceedings of the IEEE,
Vol. 6, 2006, PP. 233-234.
[23] B. Roman, H. Karel, P. Lubomr, S. Pavel, S. Petr; V, Petr, Detection
of Fatigue States of a Car Driver, Czech Technical University in Prague,
Faculty of Electrical Engineering.
[24] G. Richard, S. Sonya, Drowsy Driver Monitor and Warning
System, Robotic Institute, Carnegie Mellon University, Pittsburgh
Pennsylvania, 2001
[25] K. Ron, R, Paul, PERCLOS: A Valid Psychophysiological Measure
of Alertness by Psychomotor Vigilance, Federal Highway
Administration, Office of Motor Carriers, 1998.
[26] Ji. Qiang, Y. Xiaojie, Real-Time Eye, Gaze, and Face Pose
Tracking for Monitoring Driving Vigilance, Elsevier Science Ltd, 2002.
[27] S. Ebenhard, Presentation and demonstration at Scania, Senso
Motoric Instruments (SMI), 2003
[28] W. Jian, L. Bing, Design and Simulated Implementation of
MATLAB-based Warning System for Fatigue Driving Driver, in
Proceedings of the Ninth International Conference on Intelligent Hybrid
Systems, IEEE Computer Society, Vol. 9, 2009, pp 467-470.
[29] K. Matti, Methods for Machine Vision Based Driver Monitoring
Applications, Espoo 2006. VTT Publications 621.82 p. + app. 79 p
[30] R. P. Hamlin, Three-in-one vehicle operator sensor, Transportation
Research Board,National Research Council, IDEA program project final
report ITS-18, 1995.
HARDEEP SINGH DHILLON
ME (EPDT), ID-0820808
PEC University of Technology, Chandigarh-160012
Email: [email protected]