Implementation of Various Machine Learning Algorithms for Traffic Sign Detection and Recognition

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 10 Issue: 01 | Jan 2023 www.irjet.net p-ISSN: 2395-0072
© 2022, IRJET | Impact Factor value: 7.529 | ISO 9001:2008 Certified Journal | Page 356
Implementation of Various Machine Learning Algorithms for Traffic
Sign Detection and Recognition
Omkar S. Kalange1, Rushikesh S. Kahat2, Atharv S. Kale3, Trupti R. Kale4, Pushkar S. Joglekar5
1,2,3,4 Student,
5 Professor,
Dept. of Computer Engineering, Vishwakarma Institute of Technology, Maharashtra, India
---------------------------------------------------------------------***---------------------------------------------------------------------
Abstract - The Traffic Sign Recognition System, abbreviated
to TSRS, could be useful in autonomous, i.e., self-driving cars
which is one of the most booming technologies in the domain
of transportation, artificial driver aid, trafficsurveillance, and
traffic safety. To handle traffic-related challenges, it is vital to
recognize traffic signs. There are twoaspectstothetrafficsign
recognition system: localization (or detection) and
recognition. The traffic sign from the input image is located
and recognized by constructing a rectangle area in the
localization section. The rectangle box then offered the result
for which traffic sign is positioned in that particular region in
the recognition part. We describe the implementation of ML
algorithms for traffic sign detection and recognition in this
paper. In this paper, we compared the accuracies obtained
from implementing various machine learningalgorithmssuch
as KNN, multinomial logistic regression, CNN and random
forest. The practical purpose of detecting and recognizing the
correct meaning of traffic signs is a bit difficult. For real time
object (traffic signs) detection the YOLO model has been used.
This helps to detect traffic signs while the vehicle is in speed.
This feature will definitely help the technologies of
autonomous vehicles for detection, recognition and
interpretation of traffic signs in real time.
Key Words: KNN, Image recognition, Logistics Regression,
YOLO v4, CNN, IOU, SOTA, ROI
1.INTRODUCTION
Analysis of traffic signs is a very crucial topic in the field of
computer vision (CV) and intelligentprogramming(artificial
intelligence). The Traffic signs along the roads are designed
to instruct and inform the drivers of the current status and
other extra important information on the road.
Traffic signs hold an important place in the road traffic
systems. The main function of the traffic signsistoshowcase
the contents that must be noticed in the current road
sections, to prompt the drivers in front of the road the
danger and difficulty in the environment, to warn the driver
to drive at the predefined speed, to provide a favorable
guarantee for safe driving. Hence, the detection and
recognition of traffic signs is a very important researchfield,
which is of great significance to prevent road traffic
accidents and protect the personal safety of drivers.
Autonomous vehicles are getting more popular day by day
and all leading automobile companies are competing to
produce the most efficient and secure automobile. Some
companies working in the autonomous transport field are
Tesla, Cruise, Ford, Waymo, etc. Traffic sign recognition and
detection are importantfunctionsthatmustbeperformed by
an autonomous vehicle. Detecting the traffic signs involves
locating the sign from its surrounding. Recognition involves
the actual classification of the detected sign. These two
operations come under the Image detection and recognition
field of AI & ML. Traffic sign recognition and detection gives
the autonomous vehicle a lot of information about the road
conditions like turns, speed breakers, construction work
going on, etc. and all of this information can help the car to
make decisions regarding changing speeds, stopping,
direction change, etc. Efficient detection andrecognition can
ensure a smooth and, more importantly, safe ride for the
passengers. For this, the autonomous vehicles are equipped
with several cameras which constantly monitor the road for
traffic signs.
In recent times, in-depth learning methodshaveprovedhigh
performances and results for many tasks such as image
classification. In this paper, we have discussed the
implementation of 4 algorithms namely KNN, multinomial
logistic regression, random forests, and CNN, for the
detection of traffic signs from their images. And also
discussed the object detection algorithm i.e Yolo v4 for
traffic signs detection from differentclassesofimages.These
algorithms come under the supervised learning paradigm of
ML in which the algorithm is trained upon a dataset to
predict / classify the test data.
In this paper, we discuss the implementation of four ML
algorithms viz. KNN, multinomial logistic regression, CNN
and random forests for the sake of traffic sign detection and
recognition.
2. LITERATURE REVIEW
Ilkay Cinar et al. in their paper [1] mention that it is critical
that the traffic signs used to maintain traffic order be
understood by vehicles. Traffic signs adhere tointernational
standards, allowing drivers to learn about the route and the
surroundings while on the road. To increase traffic safety,
traffic sign recognition technologies have lately been
installed in automobiles. Machine learning approaches are

employed in picture identification. The author of this study
has a dataset of 1500 photos of 14 distinct traffic signs that
are commonly seen on Turkey's roadways. Convolutional
neural networks fromdeeplearningarchitectureswereused
to extract picture features in this dataset. The 1000
characteristics acquired were categorized using machine
learning techniques' Random Forest approach. This
categorization technique yielded a success rate of 93.7%.
Mehdi Fartaj and Sedigheh Ghofrani in their paper [2]
mentioned that A robotic car that autonomously travels on
roadways must identify traffic signs. Because most traffic
road signs are blue and red, we initially employ the Hue-
Saturation- Intensity (HSI) color space for color-based
segmentation in this article. The road signs are identified
accurately using crucial geometrical properties. Following
segmentation, it is time to classify all identified road signs.
We use and compare the performance of three classifiersfor
this purpose: distance to border (DTB), FFT sample of
signature, and code matrix. For the first time, we apply the
code matrix as an efficient classifier in this paper. Although
the acquired accuracy usingcodematrixishigheronaverage
than the two mentioned classifiers, the major benefit is
simplicity and hence lower processing cost.
David Soendoro and Iping Supriana in their paper [3]
mentioned that To recognise traffic signs,thisstudypresents
a system that combines the Color-based Method with SVM.
The color-based method using CIELab + hue was chosen
because it produces good results when it comes to localizing
traffic signs. It is initially used to convert the picture to a
binary format. The binary picture is then processed with
canny to get more accurate traffic sign detection results.The
preprocessed picture is tested fortraffic signshapeusingthe
Ramer-Douglas-Peucker method, which will estimate each
closed object form in the preprocessed image. When it
detects a closed object, it is impossible to identify occluded
and connected road signs.
Nazmul Hasan et al. in their paper [4] mentioned TSRS
(Traffic Sign Recognition System) might be useful in self-
driving cars, artificial driver assistance, traffic surveillance,
and traffic safety. To address traffic-related challenges,
traffic sign recognition is required. The traffic sign
recognition system is divided into two components:location
and recognition. In the localization section, a rectangular
rectangle is used to find and identify the traffic sign zone.
Following that, in the recognition section, the rectangle box
offered the result for which traffic sign is placed in that
specific region. In this research, we offer a method for
recognising traffic signs. We workedon12differentsignsfor
traffic sign detection and identification. To do this, we
employed SupportVectorMachineandConvolutional Neural
Network separately to identify and recognise traffic signals.
Akshata V. S. and Subarna Panda in their paper [5]
mentioned that their study discusses a real-time system for
both recognition and classification of traffic signs. This
model is three-staged, the first step of which is picture
segmentation, the second stage is the detection/localization
of the traffic sign, and the last stageistheclassificationphase
that is based on the input image. The color enhancement
technique is used to extract red patches in the picture. To
identify the content of the traffic signs collected,
Convolutional Neural Networks (CNN) are used for
detection, classification, and identification. Introduction: In
recent years, the primary aim of AdvancedDriverAssistance
System (ADAS) has been to give the driver with critical
information concerning traffic signals and warning signs on
the road ahead.
Zsolt T. Kardkovacs et al. in their paper [6] mentioned that
Traffic Sign Recognition (TSR) is one of the most essential
background research issues for allowing self-driving
vehicles. There is no time for elaborate transformations or
advanced image processing algorithms in autonomous
driving systems; instead, they demand a solid and real-time
understanding of a scenario. This requirement becomes
more difficult to accomplish in a city-like setting, since
various traffic signs, advertisements, parked cars, and other
moving or background elementscomplicatedetection. While
various solutions have been published, they have all been
tested on highways, in the countryside, or at extremely low
speeds. In this work, we provide a brief summary of the
major difficulties and known solutions to these problems, as
well as a generic method to address real-time concerns in
metropolitan areas.
L. Breiman in his paper [7] mentioned that Random forests
are a tree predictor combination in which eachtreeisreliant
on the values of a random vector sampled separately and
with the same distribution for all trees in the forest. As the
number of trees in a forest grows enormous, the
generalization error approaches a limit. The generalization
error of a forest of tree classifiers is determined by the
strength of the individual trees in the forest as well as their
association. Using a random selection of features to divide
each node produces error rates that are comparable to
Adaboost but more resilient to noise.
Mr. Jean de Dieu TUGIRIMANA et al. in their paper [8]
mentioned that many years ago, the convolutional neural
network was used to identify andrecognise roadtrafficsigns
(CNN). CNN has played an important role in the field of
computer vision, and a variant of CNN has proven to be
successful in classification tasks across multiple domains;
however, CNN has two drawbacks: one is its failure to
consider importantspatial hierarchiesbetweenfeatures, and
the other is its lack of rotational invariance. They further
discuss the use of CapsNet (or Capsule Network) to
recognize and classify road traffic signs.

Table -1: Table for Accuracies of implemented algorithms
in the reviewed papers
Accuracies of algorithms in the reviewed papers
Paper no. SVM CNN Capsule Network
1 NA NA 97.60%
2 98.2% NA NA
3 NA 97% NA
4 96.40% 98.33% NA
5 96% NA NA
3. PROPOSED SYSTEM
Datasets used:
1. The first dataset is downloaded and extracted from
the German Traffic Sign Recognition Benchmark
(GTSRB), a multi-class, single-image classification
challenge which was given in 2011 at the (IJCNN)
International Joint Conference on Neural Networks
[10]. It contains more than 50,000 images. Over
12,000 images are available for testing and over
39,000 images are available for training. A total of
43 classes are available. Figure 1. shows some
images from this dataset.
2. The second dataset contains 877 images of various
different unique classes for the objective of road
sign recognition. Bounding box annotations in the
PASCAL VOC format and four classes are such as
traffic light, stop, speed limit and crosswalk.
(a) (b)
(c)
Fig -1: Some images from the first dataset: (a) 20 km/h
speed limit sign, (b) Pedestrian crossing sign, (c) Wild
animal crossing sign
Fig -2: Flowchart for traffic sign recognition
Data pre-processing:
1. Image resizing: The images had different sizes so
to make their sizes the same they were resized to
30x30 pixels.
2. Convert images to RGB arrays: Each oftheimages
were converted into a matrix with each element of
the matrix being a tuple of the RGB values of the
pixels present in the image.
3. Flattening the matrices: Thesematriceswerethen
flattened, i.e. reduced to one-dimensional arrays,so
that classification algorithms can be applied to
them.
Analysis:
1. Apply algorithms:
Algorithms applied for image recognition:
a. K-Nearest Neighbors (KNN): KNN is an
algorithm of machine learning that can be used
for both regressionandclassificationprocesses.
It always checks the labels for a selected
number of data points around the target data
point, to make predictions about the class the

data point falls into. KNN is a relatively simple
but powerful algorithm, and for those reasons,
it is one of the most popular machine learning
algorithms.
b. Multinomial logistic regression: Logistic
regression, like all regression analysis, is a
predictive analysis. It is used to define data and
to explain the relationship between single
binary dependent variables and one or more
independent variables, ordinal, interval, or
ratio-level independent variables. Multinomial
logistic regression is an extended version of
logistic regression that can be used for
classification purposes when the data hasmore
than two classes.
c. Random forest: Random forests are a set of
tree predictors in which the values of a random
vector, gathered independently and with the
same distribution for all trees in the forest, are
used to anticipate each tree's behavior [1]. A
random selection (without replacement) is
made from the instances in the training set to
construct each tree. They vote for the most
popular class once a huge number of trees have
been created. Random Forest algorithm is
useful for both regression and classification.
The number of trees used is 100 in our
experiment.
d. Convolutional Neural Networks (CNN): CNN
is used for all artistic situations to learn in-
depth neural network algorithms in many
image-related tasks. Inside the convolution
layer convolution captures image location
information by implementing the kernel
function. CNN contains input, output, and
hidden layers. The hidden layer also contains
layers which are convolutional, pooling, fully
connected, and normalization layers. We have
used Adam optimization and categorical cross-
entropy loss function in our experiment.
(The above four algorithms are implementedonthe
first dataset for traffic sign recognition only.)
2. Calculate Accuracy: The formula for accuracy is
mentioned in the “Results and discussions” section
of this paper.
3. Applying the models on test images.
 You Only Look Once (YOLO V4): YOLO v4 is a
SOTA (state-of-the-art) real-time Object Detection
model. YOLO is a one-stage detector. One of the two
primary cutting-edge techniques for the task of
object detection that prioritizes inference speeds is
the one-stage method. The classes and bounding
boxes for the entire image are predicted in one-
stage detector models; the ROI (Region of Interest)
is not chosen. An image and the actual valuesforthe
bounding boxes are provided as input.
The entire input image is partitioned into a square
grid, and each object is detected in the grid cell that
contains its center. The B bounding boxes and the
corresponding confidence ratings will be predicted
for each grid cell. This rating shows how accurate
the box is and how certain the model is that the
object is inside the box. IOU of predicted values and
the ground truth values of the bounding box is the
Confidence score. Till now we have tested Yolo v4
on only 4 classes of images i.e., Crosswalk, stop,
traffic signals, speed limit. Fig -3, Fig -5, Fig -6 and
Fig -7 are some resultant images of above classes.
(The above algorithm is implementedonthesecond
dataset for both localization / detection and
recognition of traffic signs.)
Fig -3: Crosswalk Sign detected by YOLOv4
Fig -4: Accuracies of algorithms used

Fig -5: Stop Sign detected by YOLOv4
Fig -6: Traffic Signals detected by YOLOv4
Fig -7: Speed Limit Sign detected by YOLOv4
4. RESULTS AND DISCUSSION
Accuracy:
The calculated accuracies for each algorithm are shown in
Fig -4 The bar graph helps to compare the performances of
the algorithms.
During the training phase, theproposedCNN model employs
an Adam optimizer. It is a very successful optimization
strategy for deep neural network training that combines the
benefits of AdaGrad (Adaptive Gradient Algorithm) and
RMSProp (Root Mean Square Propagation) After evaluating
the model on test data, a loss of 5.56% and an accuracy of
98.93% are obtained. The categorical cross entropy loss
function is used in themodel. MethodssuchasEarlyStopping
and ReduceLRonPlateauare employedtoprovidean efficient
and optimal training procedure. The EarlyStopping method
examines the value of the loss function and waits 12 epochs
before halting if the value of the model performance metrics
being monitored does not improve.
The accuracy of the model is calculated using formula:
Acc = (TP+TN)/(TP+TN+FN+FP)
Where:
TP = True Positive
TN = True Negative
FN = False Negative
FP = False Positive
According to the above formula we have calculated the
accuracy for various models. The True Positive and True
Negative elements are added together in the numerator of
Accuracy formula, and all of the confusion matrix entriesare
added together in the denominator. True Positives and True
Negatives are the items on the main diagonal of the
confusion matrix, which represent the items that the model
properly identified. The denominator additionallytakesinto
account all the elements that the model mistakenlyclassified
that are not on the main diagonal.[9]
5. CONCLUSIONS AND FUTURE SCOPE
Through this paper we demonstrated the efficiency of
different algorithms in detecting and recognizing Traffic
signs. Traffic sign recognition systems are an important
aspect that autonomous vehicles use to make a decision
based on the detected sign. An efficient TSRS helps to
provide the rider with smooth,safeandhalt-lessdriving. The
experiment consequences of this paper display that the
Convolutional neural network is an effective method for
image classification and recognition. It makes ita lotsimpler
for the researcher to work on it due to its simple
architecture, and a high-performance classifier for a tough
and lot-complicated task like traffic sign recognition. As
mentioned earlier, the KNN,Multinomial LR,Randomforests
and CNN fail to recognize the sign when the trafficsigninthe
image is titled or the sign is not the majority part of the
image. This can be highly ineffectual in real life scenarios in
which the image of the traffic sign will not alwaysbestraight
and will have several other things in the background. Thus,
YOLOv4 is implemented for localization of the sign from its
surroundings. However, since YOLOv4 is implemented on
the second dataset having only 4 classes, its efficiency can’t
be compared to the rest of the three algorithms. Our future
plans include implementation of YOLOv4 and other

localization algorithms on a larger dataset. The other three
recognition algorithms will be then implemented on the
traffic-sign-containing-sub-image given by the localization
algorithms and then a combined analysis can be done.
REFERENCES
[1] I. Cinar, Y. S. Taspinar, M. Mustafa andM.Koklu,“Feature
Extraction and Recognition on Traffic Sign Images”
(2020)
[2] M. Fartaj and S. Ghofrani, “Traffic Road Sign Detection
and Classification” (2012)
[3] D. Soendoro and I. Supriana, “Traffic Sign Recognition
with Color-based Method, Shape-arc Estimation and
SVM” (2011)
[4] N. Hasan, T. Anzum and N. Jahan, “Traffic Sign
Recognition System (TSRS): SVM and Convolutional
Neural Network” (2020)
[5] V. S. Akshata and S. Panda, “Traffic sign recognition and
classification using convolutional neural networks”
(2019)
[6] Z. T. Kardkovacs, Z. Paróczi and E. Varga, “Real-time
Traffic Sign Recognition System” (2011)
[7] L. Breiman, “Random Forests”, Machine Learning 45
(2001), pp. 5–32.
https://doi.org/10.1023/A:1010933404324
[8] J. de D. Tugirimana, J. Rulinda, K. L. Tresor and A.
Nzaramba, “Road traffic sign detectionandclassification
using Capsule Network”, International Journal of
Scientific & Engineering Research Volume 9, Issue 8,
ISSN 2229-5518 (August 2018)
[9] G. Margherita, B. Enrico and V. Giorgio, “Metrics for
Multi-Class Classification: an Overview” (2020)
https://doi.org/10.48550/arXiv.2008.05756
[10] J. Stallkamp, M. Schlipsing, J. Salmen and C. Igel, “Manvs.
computer: Benchmarking machine learning algorithms
for traffic sign recognition,” Neural Networks, Volume
32 (2012), pp. 323-332, ISSN 0893-6080.
https://doi.org/10.1016/j.neunet.2012.02.016

Implementation of Various Machine Learning Algorithms for Traffic Sign Detection and Recognition

Recommended

More Related Content

Similar to Implementation of Various Machine Learning Algorithms for Traffic Sign Detection and Recognition (20)

More from IRJET Journal (20)

Recently uploaded (20)

Implementation of Various Machine Learning Algorithms for Traffic Sign Detection and Recognition