ATTENTION BASED SPEED GOVERNOR FOR ELECTRIC

VEHICLES

DESIGN PROJECT REPORT

Submitted by
AUGUSTIN JOSE - TVE17EE034
GOPIKA B - TVE17EE048
MEENAKSHI SURESH - TVE17EE072
MEGHA SANTHOSH - TVE17EE073
to
the APJ Abdul Kalam Technological University
in partial fulfillment of the requirements for the award of the Degree
of
Bachelor of Technology
in
Electrical and Electronics

Department of Electrical Engineering

College of Engineering Trivandrum
Trivandrum – 695016
Kerala, India
NOVEMBER, 2019
 DEPARTMENT OF ELECTRICAL ENGINEERING
COLLEGE OF ENGINEERING, TRIVANDRUM

CERTIFICATE

This is to certify that this report entitled “Attention Based Speed Governor for Electric
Vehicles” submitted by AUGUSTIN JOSE, GOPIKA B, MEENAKSHI SURESH,
MEGHA SANTHOSH to the APJ Abdul Kalam Technological University in partial fulfill-
ment of the requirements for the award of the Degree of Bachelor of Technology in Electrical
and Electronics Engineering is a bonafide record of the design project work carried out by them
under my/our guidance and supervision. This report in any form has not been submitted to
any other University or Institute for any purpose.

Dr. N. Mayadevi Dr. P. Sreejaya

Associate Professor (Guide), Professor (HOD),
Department of Electrical Engineering, Department of Electrical Engineering,
College of Engineering Trivandrum. College of Engineering Trivandrum.

Dr. Savier J.S. Prof. Monish M.

U.G Professor, Assistant Professor,
Department of Electrical Engineering, Department of Electrical Engineering,
College of Engineering Trivandrum. College of Engineering Trivandrum.
 Acknowledgement

As we complete this project, we would like to thank the numerous hands who helped us, wholeheart-
edly. First and foremost, we thank the Almighty, without whose blessings we could have never reached
this far. We express our deepest gratitude to our guide on this project Dr N Mayadevi,Department of
Electrical Engineering for all her help and invaluable advice,support and constant encouragement. We
take this opportunity to express our profound gratitude to our Head Of the Department, Dr P Sreejaya
whose encouragement was a great source of inspiration. We would like to thank all the teachers of the
Department for their help on various areas of this project. We would also like to thank our friends and
family for their continued support and inspiration.
 Abstract

This project aims to design and develop an attention based speed governor for electric vehicles. This
project comes up with a design which is simple and can be easily implemented. It analyses the
driver’s facial features and determines whether the driver is paying attention on the road or not. This
system consists of two subsystems: Attention monitoring subsystem and Speed control subsystem.
The attention of the driver is assessed and a numerical value is derived which are checked against
threshold values of speed. If speed is high for low attention, the speed of the vehicle is controlled.

Keywords: Facial features, Attention Monitoring, Speed Control, Speed Governor.

Contents
1 Introduction 7
1.1 Problem Statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.3 Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.4 Expected Outcome . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.5 Outline of Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2 Literature Review 10
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.2 Existing System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.3 Detailed Discussion of Two Existing Systems . . . . . . . . . . . . . . . . . . . . 11
2.3.1 Eye Behavior Based Drowsiness System . . . . . . . . . . . . . . . . . . 11
2.3.2 Driver Drowsiness Detection Using Face Expression Recognition . . . . . 12
2.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

3 Proposed System 13
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.2 Block Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.3 Working . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

4 Methodology 15
 6

5 Design 17
5.1 Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
5.2 Design of Attention Monitoring Program . . . . . . . . . . . . . . . . . . . . . . 17
5.3 Design of entire attention monitoring subsystem . . . . . . . . . . . . . . . . . . 18
5.4 Design of speed control subsystem . . . . . . . . . . . . . . . . . . . . . . . . . 18
5.5 Block Diagram of Designed System . . . . . . . . . . . . . . . . . . . . . . . . . 19
5.6 Components and Specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
5.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

6 Results and Discussions 22

6.1 Hardware Components and Description . . . . . . . . . . . . . . . . . . . . . . 22
6.2 Hardware and Software Implementation . . . . . . . . . . . . . . . . . . . . . . 32
6.3 Complete Product . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
6.4 Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
6.5 Results and Observations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
6.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

7 Conclusions 44
7.1 Future Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

References 45
 7

1 Introduction
The number of accidents occurring on roads are increasing day by day. To overcome this
social issue a solution has been put forward by this design project. Keeping in mind the deplet-
ing sources of fuel, like petrol and diesel, and the upcoming electric vehicles, the solution is
to be implemented in electric vehicles. In the next 10-20 years, electric vehicles will dominate
the automobile industry and petrol and diesel vehicles will be completely eliminated.
In the 21st century, EVs saw a resurgence due to technological developments, and an
increased focus on renewable energy. The power of a vehicle’s electric motor is measured
in kilowatts(kW). Usually, direct current(DC) electricity is fed into a DC/AC inverter where
it is converted to alternating current(AC) electricity and this AC electricity is connected to a
3-phase AC motor.
Most electric vehicles use lithium ion batteries(Li-ions). These batteries have higher en-
ergy density than most other practical batteries.
Charging of an electric car can be carried out at various charging stations, these charging
stations can be installed in both houses and public areas.
Considering the current scenario of the automobile industry, to create a safe environment
on the roads, proper speed limiting precautions are necessary.

1.1 Problem Statement

The problem identified is one of social relevance: Increasing number of accidents on the
road due to the drowsiness of the driver. The solution identified can help in solving this
problem up to some extent.

1.2 Motivation
• The evolution of electric vehicles in recent years.

• To reduce the manual errors arising due to carelessness.

• To reduce the number of accidents due to drowsiness of drivers.

 8

1.3 Objective
To design and develop a system that controls the speed of the vehicle by monitoring the
attention of the driver.

1.4 Expected Outcome

• The system is expected to capture the image of the driver continuously and assess the
facial features in order to calculate a numerical value of attention.

• The threshold values of speed for each value of attention is checked against the calcu-
lated value of attention.

• If the attention of the driver is low(eg: he is drowsy) then the speed of the vehicle will
be controlled accordingly.

1.5 Outline of Report

The chapter organization of the report is as follows:
Chapter 1: This chapter gives a brief introduction about the current unsafe environment
prevalent on our roads and the upcoming electric vehicle industry. The motivation behind
creating this project are clearly stated and the expected outcomes are well-defined.
Chapter 2: This chapter provides a brief overview on the literature survey carried out for
the identification of the best and optimal solution of the identified problem. The existing
systems have been analyzed and their merits and demerits have been noted accordingly.
Chapter 3: The proposed system is explained fully with a block diagram. Each of the
components in the block diagram has been explained in terms of their general working.
Chapter 4: The methodology of the proposed system is explained with the help of a flow
chart. The steps followed by the system in order to get the expected outcome is also clearly
explained. The steps are given in the order in which they are being executed by the system.
Chapter 5: The design of the entire system is explained. Design of each subsystem: At-
tention monitoring subsystem and speed control subsystem has been explained in detail. The
block diagram of the designed system has been included.
 9

Chapter 6: This chapter gives a brief overview on the results obtained by testing the com-
pleted product. It also includes the scope of improvement of the designed system according
to the conclusions drawn from the testing.
 10

2 Literature Review
2.1 Introduction
Drowsiness detection and speed control technologies are described in the journals re-
ferred. The systems developed figured the advantage of recognizing drowsiness before it
happens and to have the capacity to caution the driver in time. Electronic controller unit
(ECU) used for controlling sub-systems of complex real life applications like automotive ve-
hicles can pose several challenges. In such case, hardware in the loop (HIL) approach for
testing such controllers emulates the external application environment and allows concurrent
development of the electronic controllers. The systems proposed a novel low cost testing
approach.

2.2 Existing System

In ref [1] , non-intrusive computer vision based ideas has been utilized for the develop-
ment of a Drowsy Driver Detection System. The small camera has been used by system that
focuses straight towards the face of the driver and checks the driver’s eyes with a specific
end goal to recognize fatigue. This paper illustrates the process of locating the eyes of driver,
and to decide whether the eyes of driver are open or closed. The system manages utilizing
data gained for the image which is in binary form to locate the face edges, which gets the
location where the eyes of a person may exist. If the eyes of the driver are found close for
five successive frames, the proposed system assumes that the driver is nodding off and a sig-
nal of warning has been issued. The camera based drowsiness measures give appreciated
contribution.
In ref [2], unlike conventional drowsiness detection methods, which are based on the
eye states alone, facial expressions are used to detect drowsiness. There are many challenges
involving drowsiness detection systems. Among the important aspects are: change of intensity
due to lighting conditions, the presence of glasses and a beard on the face of the person. In
this project, we propose and implement a hardware system which is based on infrared light
and can be used in resolving these problems. In the proposed method, following the face
detection step, the facial components that are more important and considered as the most
 11

effective for drowsiness, are extracted and tracked in video sequence frames. The system has
been tested and implemented in a real environment.
In ref [3], an improved Fuzzy PID controller to control the speed of Brushless DC motor
was used. The proposed controller is called proportional-integral-derivative (PID) controller
and Fuzzy proportional-integral-derivative controller. This paper provides an overview of per-
formance conventional PID controller and Fuzzy PID controller. It is difficult to tune the pa-
rameters and get satisfied control characteristics by using normal conventional PID controller.
As the Fuzzy has the ability to satisfied control characteristics and it is easy for computing, In
order to control the BLDC motor, a Fuzzy PID controller is designed as the controller of the
BLDC motor. The experimental results verify that a Fuzzy PID controller has better control
performance than the conventional PID controller. The modeling, control and simulation of
the BLDC motor has been done using the software package MATLAB/SIMULINK.
In ref [4], a case-study of how HIL testing approach was gainfully employed for an elec-
tronic road-speed limiting application was described. The control system uses an electronic
throttle controller as its heart. The HIL system can simulate vehicle dynamics for the end
vehicle as well as road-grade, vehicle load etc. This allowed the rapid development of the
core control algorithm, which minimized vehicle testing time. Another novel feature of this
particular testing was, that the HIL system was implemented on a low cost hardware based
on the target controller to be tested.

2.3 Detailed Discussion of Two Existing Systems

2.3.1 Eye Behavior Based Drowsiness System

Presented at the 2015 12th International Computer Conference on Wavelet Active Media
Technology and Information Processing(ICCWAMTIP) in 2015 [1] exhibits non-intrusive com-
puter vision based ideas has been utilized for the development of a Drowsy Driver Detection
System. The small camera has been used by system that focuses straight towards the face of
the driver and checks the driver’s eyes with a specific end goal to recognize fatigue.
This paper [1] illustrates the process of locating the eyes of driver, and to decide whether
the eyes of driver are open or closed. The system manages utilizing data gained for the image
which is in binary form to locate the face edges, which gets the location where the eyes of
 12

a person may exist. If the eyes of the driver are found close for five successive frames, the
proposed system assumes that the driver is nodding off and a signal of warning has been
issued. The camera based drowsiness measures give appreciated contribution.

2.3.2 Driver Drowsiness Detection Using Face Expression Recognition

Presented at the 2011 IEEE International Conference on Signal and Image Processing
Applications(ICSIPA) Proceeding in 2011 [2] exhibits illumination of face with IR LED for
better eye detection. In this paper, unlike conventional drowsiness detection methods, which
are based on the eye states alone, we used facial expressions to detect drowsiness.
There are many challenges involving drowsiness detection systems. Among the important
aspects are: change of intensity due to lighting conditions, the presence of glasses and beard
on the face of the person. In this project, we propose and implement a hardware system
which is based on infrared light and can be used in resolving these problems. In the proposed
method, following the face detection step, the facial components that are more important and
considered as the most effective for drowsiness, are extracted and tracked in video sequence
frames. The system has been tested and implemented in a real environment.

2.4 Conclusion
Literature referred have contributed refinement towards the problems faced on the road
due to the drowsiness of the driver. The proposal has been developed based on its merits and
aimed to overcome the demerits of past solutions.
 13

3 Proposed System
3.1 Introduction
The proposed system consists of a camera, microcontroller, speed controller, motor and
a speed sensor. The facial features of the driver is assessed from the images captured. The
program calculates a numerical value for attention and a trigger is sent by the microcontroller
which determines a value of speed for the particular value of attention. The trigger is sent to
the next microcontroller which checks the current speed of the motor which is being measured
by the rotary encoder. The measured speed is checked against a set threshold value and if the
speed is higher than that value it is accordingly controlled.

3.2 Block Diagram

F IGURE 3.2.1
Block diagram of the project
 14

3.3 Working
The attention monitoring subsystem consists of a camera module and a microcontroller.
The user’s attention is converted into a suitable scale and the value produced is checked
against a threshold value and a trigger is sent to the speed control subsystem. The speed
controller subsystem consists of a microcontroller, an electronic speed controller(ESC) and
a rotary encoder coupled with an electric motor as shown in fig 3.2.1. The rotary encoder
measures the speed and the trigger received by the microcontroller decides whether to control
the speed or not. If speed is more than the threshold value, speed control is carried out. Once
the speed equals threshold value, the output frequency of the electronic speed controller is
kept constant. Hence, the speed is limited to that value.

3.4 Conclusion
The main system consists of two subsystems for executing the required actions. Thus, the
system ensures a reliable detection of attention of the driver and the efficient speed control
of the vehicle. The system is to be implemented in an electric vehicle which is the future of
the automobile industry.
 15

4 Methodology

F IGURE 4.0.2
Flowchart depicting the working of the project
 16

• The image of the driver is captured and the facial features are assessed.

• A numerical value of attention is calculated from the extracted facial features

• If the attention is below a threshold value(the numerical value is higher for lower at-
tention) then the speed is limited.

• A trigger is sent to the Electronic speed controller which checks the current speed of the
vehicle and compares it to the threshold value set for that particular attention.

• If the speed is higher than the set threshold value, then it is limited to the threshold
value.
 17

5 Design
5.1 Requirements
• Python 3.6.8

• OpenCV

• TensorFlow

• Haarcascades

• Arduino IDE

5.2 Design of Attention Monitoring Program

The Attention Monitoing Program is written in Python. This program is continuously
run on the Raspberry Pi microcontroller. The VGA camera module is connected to USB port
number 1 and it is accessed by the program using OpenCV functions. The images of the driver
is extracted in the above way and following process are done.

• The images are converted into grayscale and low level noise reduction algorithms are
applied

• Facial features in the images are compared with the weights in the Haarcascades mod-
ule.

• After the above comparison, a region of interest(ROI) is drawn around the driver’s face.

• Rectangular boxes are drawn around the driver’s eyes.

The number of consecutive frames in which the driver’s eyes remain closed are calculated tak-
ing into account of the frame rate of the camera. The average of these values are calculated
and are tested against a set of trained values. The trained values are calculated using Tensor-
Flow based machine learning program running on a dataset of driver images. The percentage
drowsiness is thus evaluated after the above testing is done.
 18

The position of the pupil within the rectangular box drawn around the eye is evaluated.
For a highly attentive driver, the variation in this value from its mean position will have
smooth slope and have periodic characteristics. On the other hand, for a less attentive driver,
this variation will be a have steep slope and non periodic characteristics. This comparison is
done using a pretrained dataset.
By considering the above two values, the percentage attention is generated. The limiting
speed corresponding to this speed is calculated and transmitted to the Arduino using one of
the GPIO pins.

5.3 Design of entire attention monitoring subsystem

Amplitude vs velocity ratio eyelid blinking is calculated in order to measure the attention
value. Yawning detection is also enabled and thus is added with the previous value to get
the total mathematical value of attention. Storing attention patterns of recognized faces and
loading it afterwards. The image obtained is compressed to the pixel size is 64x64px.

5.4 Design of speed control subsystem

A 1000kv brushless DC motor is selected with a no load current of 0.5A at 10V. The
electronic speed controller is selected according to the following equation
Equation 5.4.1 : ESU rating = (1.2 or 1.5) × max ampere rating of BLDC motor
Equation 5.4.2 : RP M = kv rating × motor input
A feedback loop is present in the system consisting of the rotary encoder and the electronic
speed controller. The output frequency of ESU is changed in order to control the speed.
 19

5.5 Block Diagram of Designed System

F IGURE 5.5.3
Block diagram of the designed project
 20

5.6 Components and Specifications

Serial Hardware Name Specification Price
No

1 Arduino Uno Microcontroller : ATmega328P 550

Operating Voltage : 5V
Flash Memory : 32KB
Clock Speed : 16MHz
Analog Pins : 6
Digital I/O Pins : 14
2 Raspberry Pi Model 3B SoC : BCM2837 3000
CPU : Quad Cortex A53
Clock Speed : 1.2GHz
SDRAM : 1GB
GPIO Pins : 40
3 Breadboard 90
4 Lithium Polymer Battery Output Voltage : 11.1V 1500
Capacity : 2200mAh
5 BLDC Motor Model No : A2212/13T 500
KV Ratio : 1000KV
No Load Current : 0.5A at 10V
Resistance : 0.090Ω
Poles : 14
6 ESC Model : Simonk 30A 250
Constant Current : 30A
7 Rotory Encoder Model : M274 KY-040 150
Operating Voltage : 5V
8 VGA Camera Module Resolution : 640x480 1200
Connector Type : USB
9 Potentiometer Resistance : 10k 10
 21

5.7 Conclusion
• Solution can be easily implemented with the hardware that has been selected.

• Solution is cost effective.

• The selected solution will help in reducing road accidents due to the carelessness of the
driver.
 22

6 Results and Discussions

6.1 Hardware Components and Description

F IGURE 6.1.4
Arduino Uno

ARDUINO UNO

• Microcontroller: ATmega328

• Operating voltage: 5V

• Input Voltage (limits): 6-20V

• Digital I/O Pins: 14 (of which 6 provide PWM output)

• Analog Input Pins: 6DC

 23

• Current per I/O Pin: 40 mA

• DC Current for 3.3V Pin: 50 mA

• Flash Memory: 32 KB of which 0.5 KB used by boot loader

• Clock speed:16MHz
 24

F IGURE 6.1.5
Raspberry Pi Model 3B

Raspberry PI MODEL 3B

• SoC: Broadcom BCM2837

• CPU: Quad Cortex-A53,

• Clock speed: 1.2GHz

• GPU: Broadcom VideoCore IV

• Storage: microSD, 1GB

• GPIO: 40 pins

• Ports: HDMI, 3.5mm analogue audio-video jack, 4× USB 2.0, Ethernet, Camera Serial
Interface (CSI), Display Serial Interface (DSI)
 25

F IGURE 6.1.6
VGA Camera Module with USB Connector

VGA Camera Module with USB Connector

• USB Connector

• 30fps

• Resolution : 640x480
 26

F IGURE 6.1.7
Lithium Polymer Battery

LiPo Battery

• 11.1V

• 2200mAh

• Optimum operating temperature : 25◦ C

 27

F IGURE 6.1.8
Brushless DC Motor

BRUSHLESS DC MOTOR

• Model No: A2212/13T

• Kv Ratio: 1000Kv

• No load current: 0.5A at 10V

• Resistance: 0.090 ohm

• Poles:14

A brushless DC electric motor (BLDC motor or BL motor), also known as electronically com-
mutated motor (ECM or EC motor) and synchronous DC motors, are synchronous motors
 28

powered by direct current (DC) electricity via an inverter or switching power supply which
produces an alternating current (AC) electric current to drive each phase of the motor via a
closed loop controller. The controller provides pulses of current to the motor windings that
control the speed and torque of the motor.
The construction of a brushless motor system is typically similar to a permanent magnet
synchronous motor (PMSM), but can also be a switched reluctance motor, or an induction
(asynchronous) motor [1]
The advantages of a brushless motor over brushed motors are high power-to-weight ra-
tio, high speed, electronic control, and low maintenance. Brushless motors find applications
in such places as computer peripherals (disk drives, printers), hand-held power tools, and
vehicles ranging from model aircraft to automobiles
 29

F IGURE 6.1.9
Rotory Encoder

Rotory Encoder

• Model : M274 KY-040

• Operating Voltage : 5V
 30

F IGURE 6.1.10
Breadboard
 31

F IGURE 6.1.11
Potentiometer

Potentiometer

• Total Resistance : 1kΩ

 32

6.2 Hardware and Software Implementation

F IGURE 6.2.12
Final Hardware Implementation

The source codes for Raspberry Pi and Arduino are loaded into the respective devices.
Raspberry Pi is powered by a USB cable and the Arduino is powered by the LiPo battery. The
Raspberry Pi and the Arduino are interfaced by two wire digital connection. The USB camera
is connected to the Raspberry Pi. The BLDC motor and the rotory encoder is mechanically
coupled. The BLDC is connected to the ESU which is powered by the LiPo and its signal
pin is connected to Arduino. The rotory encoder is interfaced to the Arduino using serial
connection. A potentiometer is connected to the Arduino to manually control the speed of the
BLDC motor.
 33

Code Snippet 1: Arduino code(speed.ino)

1 #include <Servo.h>
2 #define inputCLK 8
3 #define inputDT 10
4 int counter = 0;
5 int currentStateCLK;
6 int previousStateCLK;
7 Servo s1;
8 int x=0;
9 int y=0;
10 int pushButton = 12;
11 int rps = 0;
12 void setup()
13 {
14 Serial.begin(9600);
15 pinMode(A0,INPUT);
16 pinMode(pushButton, INPUT);
17 pinMode (inputCLK,INPUT);
18 pinMode (inputDT,INPUT);
19 s1.attach(5);
20 s1.write(80);
21 delay(2000);
22 s1.write(0);
23 delay(2000);
24 s1.write(0);
25 delay(2000);
26 s1.write(0);
27 previousStateCLK = digitalRead(inputCLK);
28 }
29 int sp = 0;
 34

30 int count1 = 0;
31 int count2 = 0;
32 int limit = 80;
33 void loop()
34 {
35 x=analogRead(A0);
36 y = map(x, 0, 1023, 0, 180);
37 if(y>=limit)
38 {
39 y=limit;
40 }
41 if(y < limit)
42 {
43 sp = y;
44 }
45 sp = y;
46 s1.write(sp);
47 int buttonState = digitalRead(pushButton);
48 if(buttonState==1)
49 {
50 count2 = 0;
51 count1+=1;
52 limit = limit-(limit*0.1*(count1/5));
53 if(limit<=10)
54 {
55 limit=10;
56 }
57 }
58 if(buttonState<0)
59 {
 35

60 count1 = 0;
61 count2 = rpm();
62 }
63 else if(buttonState==0)
64 {
65 count1 = 0;
66 count2+=1;
67 limit = limit+(limit*0.1*(count2/5));
68 if(limit>=80)
69 {
70 limit=80;
71 }
72 }
73 }
74 int rpm()
75 {
76 currentStateCLK = digitalRead(inputCLK);
77 if (currentStateCLK != previousStateCLK)
78 {
79 if (digitalRead(inputDT) != currentStateCLK)
80 {
81 counter --;
82 }
83 else
84 {
85 counter ++;
86 }
87 rps = counter;
88 Serial.println(rps);
89 delay(1000);
 36

90 }
91 previousStateCLK = currentStateCLK;
92 return rps;
93 }

Code Snippet 2: Face Detection code(server.py)

1 import socket
2 from scipy.spatial import distance as dist
3 from imutils.video import VideoStream
4 from imutils import face_utils
5 from threading import Thread
6 import numpy as np
7 import playsound
8 import argparse
9 import imutils
10 import time
11 import dlib
12 import cv2
13 serv = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
14 serv.bind((’192.168.1.17’, 8089))
15 serv.listen(5)
16 path=’’
17 flag=0
18 def sound_alarm(path):
19 print("Drowsiness detected")
20 def eye_aspect_ratio(eye):
21 A = dist.euclidean(eye[1], eye[5])
22 B = dist.euclidean(eye[2], eye[4])
23 C = dist.euclidean(eye[0], eye[3])
 37

24 ear = (A + B) / (2.0 * C)
25 return ear
26 EYE_AR_THRESH = 0.3
27 EYE_AR_CONSEC_FRAMES = 15
28 COUNTER = 0
29 ALARM_ON = False
30 detector = dlib.get_frontal_face_detector()
31 predictor = dlib.shape_predictor(’
shape_predictor_68_face_landmarks.dat’)
32 (lStart, lEnd) = face_utils.FACIAL_LANDMARKS_IDXS["left_eye"
]
33 (rStart, rEnd) = face_utils.FACIAL_LANDMARKS_IDXS["right_eye
"]
34 threadprint("[INFO] starting video stream thread...")
35 vs = VideoStream("video1.mp4").start()
36 time.sleep(1.0)
37 while True:
38 try:
39 frame = vs.read()
40 frame = imutils.resize(frame, width=450)
41 gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
42 rects = detector(gray, 0)
43 for rect in rects:
44 #conn, addr = serv.accept()
45 shape = predictor(gray, rect)
46 shape = face_utils.shape_to_np(shape)
47 leftEye = shape[lStart:lEnd]
48 rightEye = shape[rStart:rEnd]
49 leftEAR = eye_aspect_ratio(leftEye)
50 rightEAR = eye_aspect_ratio(rightEye)
 38

51 ear = (leftEAR + rightEAR) / 2.0

52 leftEyeHull = cv2.convexHull(leftEye)
53 rightEyeHull = cv2.convexHull(rightEye)
54 cv2.drawContours(frame, [leftEyeHull], -1, (0,
255, 0), 1)
55 cv2.drawContours(frame, [rightEyeHull], -1, (0,
255, 0), 1)
56 if ear < EYE_AR_THRESH:
57 COUNTER += 1
58 if COUNTER >= EYE_AR_CONSEC_FRAMES:
59 if not ALARM_ON:
60 ALARM_ON = True
61 if path!= "":
62 t = Thread(target=sound_alarm,
args=(path,))
63 t.deamon = True
64 t.start()
65 flag=1
66 cv2.putText(frame, "DROWSINESS ALERT
!", (10, 30),cv2.
FONT_HERSHEY_SIMPLEX, 0.7, (0, 0,
255), 2)
67 else:
68 flag=0
69 COUNTER = 0
70 ALARM_ON = False
71 cv2.putText(frame, "EAR: {:.2f}".format(ear),
(300, 30),cv2.FONT_HERSHEY_SIMPLEX, 0.7, (0,
0, 255), 2)
72 cv2.imshow("Frame", frame)
 39

73 key = cv2.waitKey(1) & 0xFF

74 if key == ord("q"):
75 serv.send(b’1’)
76 break
77 except AttributeError:
78 print("End of file")
79 break
80 cv2.destroyAllWindows()
81 vs.stop()

Code Snippet 3: Raspberry Pi code(client.py)

1 import socketimport RPi.GPIO as GPIO
2 import time
3 GPIO.setmode(GPIO.BCM)
4 GPIO.setwarnings(False)
5 GPIO.setup(18,GPIO.OUT)
6 client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
7 client.connect((’192.168.137.1’, 8089))
8 while True:
9 data = client.recv(1)
10 if data==b’1’:
11 GPIO.output(18,GPIO.HIGH)
12 if data==b’0’:
13 GPIO.output(18,GPIO.LOW)

The speed.ino program is run in Arduino. The instantaneous resistance of the potentiometer
is read from the analog pins. The pulses generated by the Raspberry Pi is decoded and the
limit of speed is calculated. The ESU is then triggered to decrease the speed until, the speed
become less than the limit.
The server.py program monitors the driver’s face and determines whether the driver is
 40

drowsy or not. The weights for detecting facial features are used from a Haarcascades mod-
ule.The function dlib.get_frontal_face_detector() is used to detect all the faces present in the
given image frame. The dlib.shape_predictor() function extracts all the facial features from
the detected faces. The eyes are then mapped and eye contours are drawn. The aspect ratio
of the eye (ear) is calculated in each frame. If the ear value falls below a certain limit for
consecutive frames, the driver is deemed drowsy.
The client.py program runs in the Raspberry Pi. Whenever the driver becomes drowsy, it
sends a signal to the Arduino via the GPIO pins.
 41

6.3 Complete Product

F IGURE 6.3.13
Finished product

6.4 Testing
The testing of the final product is done in three ways
Test 1. :The accuracy of the drowsiness detection is tested
The Raspberry Pi program is run independently after removing connections to the Ar-
duino. Pre-recorded video files are used instead of real-time video recording. A random set of
videos of attentive and drowsy people are fed to the program and the ability of the program
to classify them is calculated
Test 2 :The manual control of the BLDC is tested using the potentiometer
 42

The product is run without connecting the Raspberry Pi. The Arduino, ESU and the BLDC
are powered on using the LiPo. The potentiometer is varied to change the speed of the motor.
Proper working of manual speed control is verified.
Test 3 :The speed limit for different drowsiness is verified
The product is powered on and the driver is made to show varying degree of attention.
The speed of the BLDC is also varied manually. The working of the speed limiter is verified.

6.5 Results and Observations

The mentioned three tests were conducted. While conducting Test 1, the striking obser-
vation was the poor performance of the Raspberry Pi. Maximum of 10fps for pre-recorded
videos and 3fps for real-time videos was observed. These values further decreased after the
interface with Arduino was reestablished. On the other hand, the program worked well and
the accuracy in classifying attentive and drowsy drivers was very high. The performance of
the program for drivers of different ethnicity was also very high. The manual control of the
BLDC motor using the potentiometer was done as per the Test 2. The speed of the BLDC motor
was varied with ease and without much latency. One recurring observation was that once the
speed of the motor fell below a certain value, the motor RPM suddenly fell to zero. This was
because once the motor speed became very small, the moment of inertia of the connected
system becomes large and thus the speed is not enough to sustain continuous revolution. An-
other observation was that the rotory encoder failed to accurately measure the motor speed at
very high speeds. The reason for this behavior is skipping encoder pulses at very high speeds.
At very high speeds, the encoder shaft turns more than its minimum angular position in a
single clock pulse. Hence, the encoder output will have one pulse instead of two. Test 3 was
conducted on the final product. The final product performed as it was expected to but with
the above mention issues. There was a high latency between the drowsiness detection time
and the speed limiting action. This is partially due to the poor performance of the Raspberry
Pi and partially due to the communication delay between the Raspberry Pi and the Arduino
The above observations and results is summarized as follows :

• The program was able to detect drowsiness in drivers with a very high level of accuracy.
 43

• The program did well with drivers of different ethnicity.

• The performance of the Raspberry Pi was poor.

• The manual control of the BLDC using potentiometer was verified.

• Below a value of speed, the BLDC was not able to sustain the rotation and the RPM fell
to zero.

• The rotory encoder failed to read speed accurately at very high RPMs

• The latency in the finished product was very high

6.6 Conclusion
The tests on the product was done successfully. The test results were closely observed, an-
alyzed and conclusions were made.The issues that arose can be fixed by making slight changes
to the hardware used. The Raspberry Pi used has to be replaced by a better microcontroller
with inbuilt GPU and faster processing speed. BLDC motor with good low speed performance
can be used to tackle the sudden fall to zero speed observed in the current BLDC. Hall effect or
optical rotory encoder can be used to measure very high speeds. The latency problem can be
fixed by using Arduino with faster CPU. The image processing programs can be run in Cloud
servers and high internet connectivity can considerably decrease the latency issue.
 44

7 Conclusions
The finished product performed as expected in the tests conducted. The attention of the
driver was successfully monitored and the speed of the BLDC limited as required.

7.1 Future Scope

There is a wide future scope for this project. This system can easily be installed in current
electric vehicles. Installation can also be done in the manufacturing state. With slight modifi-
cations, it can be implemented in non-electric vehicles too. If the driver is found to be drowsy,
self driving cars can switch to autopilot mode thus ensuring safety and time management.
Besides facial data, behavioral patterns, microvibrations from fingers and environmental
data can also be used which can increase the accuracy of the system. Personalised data can be
used to predict the possibility of sleepiness depending on the time of day and other physical
conditions and engage safety measures beforehand. Cloud based processing combined with
high speed internet connectivity can considerable decrease the processing time compared to
onboard processing.
 45

References
[1] Javed Ahmed, Jian-Ping Li, Saeed Ahmed Khan, Riaz Ahmed Shaikh, "Eye be-
haviour based drowsiness Detection System", 12th International Computer Conference
on Wavelet Active Media Technology and Information Processing (ICCWAMTIP), 18-
20 Dec. 2015, Chengdu, China, doi:10.1109/iccwamtip.2015.7493990, pp. 5, 6, 30.

[2] Mohammad Amin Assari, Mohammad Rahmati, "Driver Drowsiness Detection Us-
ing Face Expression Recognition", IEEE International Conference On Signal and Im-
age Processing Applications (ICSIPA), 16-18 Nov. 2011, Kuala Lumpur, Malaysia,
doi:10.1109/icsipa.2011.6144162, pp. 5, 6.

[3] R. Arulmozhiyal, R. Kandiban, "An intelligent speed controller for Brushless DC motor",
7th IEEE Conference on Industrial Electronics and Applications (ICIEA), 18-20 July
2012, Singapore, Singapore, doi:10.1109/iciea.2012.6360690, pp. 5.

[4] Abhijeet S Taksale, Priti Shahane, Vishwas Vaidya, Vivek Deulkar, Goutham Dron-
amraju, "Hardware in the loop for electronic speed limiter", IEEE International
Transportation Electrification Conference (ITEC), 27-29 Aug. 2015, Chennai, India,
doi:10.1109/itec-india.2015.7386899, pp. 6.