finalk
finalk
1
A Project report on
Bachelor of Technology
In
Computer Science and Engineering
Submitted By
BANOTH GANESH (20H51A05D6)
KETHAVATH SURESH (21H55A0511)
VANAPARTHI KEERTHANA (21H55A0524)
Under the esteemed guidance of
Mr. V. Narasimha
Assistant Professor
2
CMR COLLEGE OF ENGINEERING AND TECHNOLOGY
Kandlakoya, Medchal Road, Hyderabad-501401
CERTIFICATE
This is to certify that the Project report "Helmet & number plated detection of a Bike " being
Technology in Computer Science and Engineering of a record of bonafide work carried out his/her
The results embodies in this project report have not been submitted any other
3
Acknowledgment
With great pleasure We want to take this opportunity to express my heartfelt gratitude to all
the people who helped in making this project work a grand success.
Engineering for his valuable technical suggestions and guidance during the execution of this
project work.
We would like to thank Dr. S. Siva Skandha, Associate Professor Head of the Department of
Computer Science and Engineering, CMR College of Engineering and Technology, who is the
We are very grateful to Dr.K. Vijaya Kumar, Dean-Academic, CMR College of Engineering
and Technology, for his constant support and motivation in carrying out the project work
successfully.
We are highly indebted to Dr. V A Narayana, Principal, CMR College of Engineering and
Technology, for giving permission to carry out this project in a successful and fruitful way.
We would like to thank the Teaching & Non- teaching staff of Department of Computer
Finally we express our sincere thanks to Mr. Ch. Gopal Reddy, Secretary, CMR Group of
Institutions, for his continuous care. We sincerely acknowledge and thank all those who gave
NAME ROLLNO
BANOTH GANESH (20H51A05D6)
KETHAVATH SURESH (21H55A0511)
VANAPARTHI KEERTHAN (21H55A0524)
4
INDEX
Abstract i
1 Introduction 1-2
2 Background Work 3
Scope of Improvement 6
3 Existing System 7
8 Result 18-20
Conclusion 21
References 22
5
Figures
6
Abstract
Motorcycle accidents have been rapidly growing through the years in many countries.
Therefore, it is necessary to develop a system for automatic detection of helmet wearing for
road safety. We now face a number of issues with India's traffic restrictions, many of which
have workable solutions. Without a helmet, it is illegal to drive a motorbike or scooter in India,
which has led to a rise in accidents and fatalities. The current system largely uses CCTV records
to monitor traffic offences, requiring traffic police to zoom in on the license plate in the event
that the rider is not wearing a helmet and look into the frame where the infringement is taking
place. However, given the high frequency of traffic offences and the rising daily use of
motorbikes, this demands a significant amount of labour and time. What if there was a system
that would check for traffic violations like riding a motorbike or moped without a helmet and,
if found, would automatically extract the license plate number of the offending car. Recent
studies have successfully completed this task using features from CNN, R-CNN, LBP, HoG,
HaaR, etc. However, these efforts have limitations in terms of effectiveness, precision, or how
quickly objects may be identified and categorized. A non-helmet rider identification system is
created in this research study in an effort to automate the process of identifying the traffic
infraction of not wearing a helmet and obtaining the license plate number of the offending
vehicle. Object Detection using Deep Learning at three layers is the key idea at play. People,
motorcycles or mopeds were discovered at the first level using YOLOv2, helmets were detected
at the second level using YOLOv3, and licence plates were detected at the final level using
YOLOv2. After that, OCR is used to retrieve the licence plate registration number (Optical
Character Recognition). All of these methods are subject to predetermined restrictions and
circumstances, particularly the component that extracts licence plate numbers. The efficiency
of the execution is essential since this job uses video as its input. We developed a
comprehensive solution for both helmet detection and licence plate number extraction using
the aforementioned approaches.
Key Words: ANPR, Machine Learning, Image Processing, OpenCV, Pytesseract , OCR.
7
CHAPTER 1
INTRODUCTION
Automation of this process is highly desirable for reliable and robust monitoring of these
violations as well as it also significantly reduces the amount of human resources needed[3].
Also, many countries are adopting systems involving surveillance cameras at public
8
places. So, the solution for detecting violators using the existing infrastructure is also cost-
effective.
However, in order to adopt such automatic solutions certain challenges need to be addressed:
2 Occlusion: In real life scenarios, the dynamic objects usually occlude each other due
to which object of interest may only be partially visible. Segmentation and classification
become difficult for these partially visible objects[5].
4) Temporal Changes in Conditions: Over time, there are many changes in environment
conditions such as illumination, shadows, etc. There may be subtle or immediate changes
which increase complexity of tasks like background modelling[6].
5) Quality of Video Feed: Generally, CCTV cameras capture low resolution video. Also,
conditions such as low light, bad weather complicate it further. Due to such limitations, tasks
such as segmentation, classification and tracking become even more difficult. As stated
in, successful framework for surveillance application should have useful properties such
as real-time performance, fine tuning, robust to sudden changes and predictive. Keeping
these challenges and desired properties in mind, we propose a method for automatic
detection of bike-riders without helmet using feed from existing security cameras, which works
in real time[7].
9
CHAPTER 2
BACKGROUND WORK
Literature Review
1. Machine Vision Techniques for Motorcycle Safety Helmet Detection
This paper presents a system which automatically detects motorcycle riders and
determines that they are wearing safety helmets or not. The system extracts moving objects
and classifies them as a motorcycle or other moving objects based on features extracted from
their region properties using K-Nearest Neighbor / (KNN) classifier. The heads of the riders
on the recognized motorcycle are then counted and segmented based on projection
profiling. The system classifies the head as wearing a helmet or not using KNN based on
features derived from 4 sections of the segmented head region. Experiment results show an
average correct detection rate for near lane, far lane, and both lanes as 84%, 68%, and 74%,
respectively[8].
In this, Author has discussed safety and security of the bikers against road accident.
Smart helmet has special idea which makes motorcycle driving safety than before, this is
implemented using GSM and GPs technology. Other advantages of this project is to measure
the alcohol level of the drunken people who is riding the bike. Whenever the alcohol level
crosses the predefined value, the alarm starts and get notification about the drunken driver. The
author have also discussed about the accident detector and the sensor will active the GPS and
find the location and further SMS will send to ambulance or family members.[9]
According to the recent Research paper in 2016 titled ‘2 Helmet using GSM and GPS
technology for accident detection and reporting system’, The author specially developed this
project to improve the safety of the bikers. The objective of this project is to study and
understand the concept of RF transmitter and RF receiver circuit. The project uses ARM7,
GSM and GPS module. The project also uses buzzer for indication purpose. Whenever
the accident will occur then accident spot will be note down and information will send out on
the registered mobile number. The major disadvantage of this project is they are not using any
10
display device for showing the current status. Also the cost of helmet is still high since helmet
is designed for only one purpose. According to the Research paper in 2015 titled
‘Microcontroller based smart wear for driver safety’, In this paper author has discussed on the
speed of the vehicle. In this application the project will be monitoring the areas in which the
vehicle will be passing. On entering any cautionary areas like schools, hospitals, etc the
speed of the vehicle will be controlled to a predefined limit. LCD is used for showing the
various types of messages after wearing the helmet. The author has worked only on the
phenomenon of accident which is generally happens due to drunk and drive. But as we know
that the accident in the area is not happens only due to consuming alcohol but also other
parameters like speed are also responsible[9].
Image processing has become a common technique for making images more
comprehensible to the human eye. Images acquired are found to be corrupted with noise
in many cases. There are many methods available to remove impulse noise in gray scale
and color images. But very little has been done for the removal of additive noise in color
images. Of the many filters presented, most of them are only for gray scale images. The filtering
techniques developed for gray scale images can be extended to color images by applying
it to the different color components separately but it is also evident that they can partially
destroy image details. The existing systems includes Conservative Smoothing, linear filters,
non-linear filters like median filter and fuzzy filter, adaptive filter, wavelet based filter etc.
This paper presents a brief overview of digital image processing techniques such as
Feature Extraction, Image Restoration and Image Enhancement. A brief history of OCR and
various approaches to character recognition is also discussed in this paper. Handwritten
character recognition is always a frontier area of research in the field of pattern recognition.
There is a large demand for OCR on hand written documents in Image processing. Even
11
though, sufficient studies have performed in foreign scripts like Arabic, Chinese and
Japanese, only a very few work can be traced for handwritten character recognition mainly
for the south Indian scripts. OCR system development for Indian script has many application
areas like preserving manuscripts and ancient literatures written in different Indian scripts and
making digital libraries for the documents. Feature extraction and classification are essential
steps of character recognition process affecting the overall accuracy of the recognition
system[10].
Bikers do not wear helmets in the region where traffic checking is not done. There is
a tendency of the driver to wear helmet only where the anticipate checking may takes
place, they do not wear helmet where no checking is done. The vehicle may be turn on or
may be stolen by passing the ignition switch.
Testing alcohol content present in blood in each individual rider in big countries like
India is almost impossible. Accidents due phone calls as previous helmets do not contain
Bluetooth speakers.
This paper aims to explain and illustrate an automatic method for motorcycles
detection and classification of public roads and a system for automatic detection of
motorcyclists without the helmet. For this, a hybrid descriptor for features extraction is
proposed based on Local Binary Pattern, Histograms of Oriented Gradients and the Hough
Transform descriptors. Traffic images captured by cameras were used. The best result obtained
from classification was an accuracy rate of 0.9767, and the best result obtained from the helmet,
detection was an accuracy rate of 0.9423[4].
The method uses the visual length, visual width. and Pixel Ratio to detect the
classes of the motorcycle occlusions and segment the motorcycle from each occlusive class.
12
Because the motorcycle riders must put on their helmets, the helmet detection or search method
is used to make sure whether the helmet/motorcycle exits or not.
Experiments obtained by using complex road scenes are reported, which demonstrate
the validity of the method in terms of robustness, accuracy, and timely responses.
without helmet using surveillance videos in real time. The proposed approach first detects
bike riders from surveillance video using background subtraction and object segmentation.
Then it determines whether bike-rider is using a helmet or not using visual features and binary
classifier. Also, we present a consolidation approach for violation reporting which helps
in improving the reliability of the proposed approach. In order to evaluate our approach, we
have provided a performance comparison of three widely used feature representations namely
histogram of oriented gradients (HOG), scale-invariant feature transform (SIFT), and local
binary patterns (LBP) for classification[11].
Scope of Improvement
Any system always has a scope for improvements and more advancement. All the
systems studied under the literature survey have their own different features. All the systems
proposed till date are used only for sending message in case of accident. There could be such a
system where only alcohol detection is present. Here in this system many advanced features
are added and also the previous features are clubbed in a single system. It will send message
automatically when rider met an accident with helmet on. RF transmitter and receiver are used
for starting the two wheeler, if rider not wearing the helmet the bike will not get start. The
alcohol sensor will sense the alcohol and it will lock the ignition if drunk. The solar sense is
generating power for the system. It can tracked easily with location when bike is stolen.It can
also use to receive call while driving through wireless Bluetooth Speakers.
13
CHAPTER 3
Existing System
The system proposed isolates the bikes from the images by approximation crops the
most probable area where helmet might be present and then feeds it to the feature abstraction
and matching system. Automatic detection of bike-riders without helmet falls under broad
category of anomaly detection in surveillance videos. As explained in, effective automatic
surveillance system generally involve following tasks: environment modeling, detection,
tracking and classification of moving objects. It uses circle arc detection method based on the
Hough transform. The major limitation of this approach is that it tries to locate helmet in
the full frame which is computationally expensive and also it may often confuse other similar
shaped objects as helmet.
Existing System
14
CHAPTER 4
Proposed Methodology
15
3. Vehicle classification:
The next step is to classify the moving vehicle extracted in the last part. To classify
vehicle, we are going to use the number of machine learning algorithms, from classical machine
learning algorithms to modern deep neural networks, to see which approach works best in
vehicle classification with limited data. A vehicle can be classified into two categories two-
wheelers or four-wheelers. We are only interested in two wheelers Figure 1 since we want to
detect the presence of a helmet. The system proceeds further only if a two-wheeler is detected.
Else, it discards this vehicle and looks for other vehicles and the cycle continues. We will
collect the training data required for the classification of a vehicle on our own. We will
capture the images of various vehicles in various positions. Almost same number of images,
1000, were gathered for both the classes two-wheelers or four-wheelers. If there are equal
number of training images from both classes then it eliminates the problem of class imbalance
and leads to better performance of the classifier. The training images contain a vehicle
surrounded by other objects of interest such as trees, footpath, how a vehicle is normally seen
on roads. Although this dataset is not the most representative of the real world moving objects,
it is still enough to train and test the effectiveness of various machine learning algorithms to
check the feasibility of the approach. The images will be converted to grayscale. Raw pixel
values were fed to the classifier.
4. Helmet detection:
Using the same approach as applied to identify the type of vehicle, we detect
whether the rider is wearing a helmet. The images that will be used to train a helmet detector
will be cropped version of the two-wheeler images focusing on the head region of the rider.
Using this technique, we will be able to maintain the class balance, that is, there will be same
number of images where the rider was wearing a helmet and where the rider was not wearing
a helmet. We used numerous machine learning classifiers in order to select the best one for this
task.
16
CHAPTER 5
A total of five machine learning classifiers will be used to test which one performs better in our
This algorithm is based on decision trees. Here, instead of building one tree, a lot of
trees are grown in parallel. All these trees are fed only a subset of data points and a subset
of features. The sub setting ensures diversity among the trees. After training, each tree votes
for a class and a final class is chosen based on the highest number of votes.
This algorithm is also based on decision trees like the random forest. However, instead
of constructing a lot of trees in parallel, trees are constructed sequentially one after the other.
Each tree improves the loss by rectifying the error made by the previous tree while training.
An SVM [9] creates a hyperplane (a plane in n-dimensions) which divides all the classes
in the training data from one another in such a way that the difference between the two classes
is maximum. This algorithm takes a lot of computing resources to complete the training as
compared to the aforementioned classification techniques.
17
becomes a bottleneck for such networks. We used a 10-layer network with 50 nodes in each
layer.
TensorFlow:
The deep learning library in the world is Google's TensorFlow. Google product uses machine
learning in all of its products to improve the search engine, translation, image captioning or
recommendations. example, Google users can experience a faster and more refined the search with AI.
If the user types a keyword the search bar, Google provides a recommendation about what could be
the next word.
Google wants to use machine learning to take advantage of their massive datasets to
give users the best experience. Three different groups use machine learning:
Researchers
Data scientists
Programmers.
They can all use the same toolset to collaborate with each other and improve their efficiency.
Google does not just have any data; they have the world's most massive computer, so Tensor
Flow was built to scale. TensorFlow is a library developed by the Google Brain Team to
accelerate machine learning and deep neural network research.
It was built to run on multiple CPUs or GPUs and even mobile operating systems, and it has
several wrappers in several languages like Python, C++ or Java.
18
perform on that input. The input goes in at one end, and then it flows through this system of
multiple operations and comes out the other end as output.
OpenCV:
OpenCV is the leading open source library for computer vision, image processing and machine
learning, and now features GPU acceleration for real-time operation.
OpenCV is released under a BSD license and hence it’s free for both academic and
commercial use. It has C++, C, Python and Java interfaces and supports Windows, Linux,
Mac OS, iOS and Android. OpenCV was designed for computational efficiency and with a
strong focus on real-time applications. Written in optimized C/C++, the library can take
advantage of multi-core processing. Adopted all around the world, OpenCV has more than
47 thousand people of user community and estimated number of downloads exceeding 6
million.
Usage ranges from interactive art, to mines inspection, stitching maps on the web or
through advanced robotics.
OpenCV is being used for a very wide range of applications which include:
19
CHAPTER 6
Platform Requirement
1.Software Requirement
2. Hardware Requirement
20
CHAPTER 7
UML Diagrams
1. System Architecture
21
2. Use case Diagram
3.Component Diagram
22
4.State Chart diagram
5.Activity Diagram
23
6.Deployment Diagram
24
CHAPTER 8
RESULT
After setting path double click on ‘run.bat’ file to run project and to get below screen
I choose the picture labelled "5.png" in the screen above, then I clicked the "Open" button to
load the image. To determine if a picture includes a person and a motorbike, click the 'Detect
Motor Bike & Person' button now.
25
Fig3.Detecting image
Click the "Detect Helmet" button to determine if the individual riding the bike in the above
screen is wearing a helmet or not.
The programme on the top screen saw that the individual wasn't wearing a helmet, so it pulled
the person's number from the car and displayed it next to the text space. We will now verify
using a helmet picture.
I've uploaded image 4.png to the screen above; when you click the "Detect Motor Bike &
Person" button, the following result will appear.
26
Fig6.Helmet detected
Clicking the "Detect Helmet" button after Yolo identified a rider on a motorcycle in the
previous creen produced the results shown below.
The programme on the above screen saw that the guy was wearing a helmet and that the label
was visible around his head. The application stopped there without scanning the licence plate.
We have trained a few photographs to extract number plates for this project, but if you want to
extract for new images, email them to us so we may incorporate them in the yolo model to
extract number plates for new images as well.
27
Conclusion
A video file is used as the input for a system being built to detect non-helmeted riders. The
motorcycle's licence plate number is retrieved and shown if the rider in the video clip is not
wearing a helmet while operating the machine. Motorcycle, person, helmet, and licence plate
detection all employ the object detection concept with the YOLO architecture. In cases when
the rider is not wearing a helmet, OCR is utilised to retrieve the licence plate number. In order
to be utilised for various purposes, not only the characters but also the frame from which they
are retrieved are removed. The project's goals have all been properly met.
28
References
3. Romuere Silva, Kelson Aires, Thiago Santos, Kalyf Abdala, Rodrigo Veras, Andr´e Soares,
“Automatic Detection Of Motorcyclists without Helmet”, 2013 XXXIX Latin America
Computing Conference (CLEI).IEEE,2013.
6. Amir Mukhtar, Tong Boon Tang, “Vision Based Motorcycle Detection using HOG features”,
IEEE International Conference on Signal and Image Processing Applications (ICSIPA).IEEE,
2015.
29