ISSN: 2456-723X

International Journal of Volume 10, Issue 2, 2024

Embedded Systems and DOI (Journal): 10.37628/IJESET

Emerging Technologies
http://ecc.journalspub.info/index.php?journal=JESET&page=index

Review IJESET

CNN-Based Traffic Sign Recognition with Voice Alerts

Shilpa Shahapur1, Nyamatulla Patel2,*, Dr. Mohammed Ziaullah Choudhari3

Abstract
Traffic sign recognition is a crucial component of intelligent transportation systems, significantly
enhancing road safety by aiding both autonomous and human drivers. This paper presents a
convolutional neural network-based traffic sign recognition system integrated with voice alerts. Our
approach utilizes a deep learning model trained on a comprehensive dataset of traffic signs to
accurately identify various road signs in realtime. Upon recognition, the system issues corresponding
voice alerts to inform the driver, thus providing a hands-free method of ensuring road awareness. The
idea for a system is designed in a modular fashion, with distinct phases encompassing data collection,
preprocessing, model training, real-time recognition, and voice alerts. An extensive dataset of road
signals pictures is collected and pre-processed to standardize the data’s size and quality. A custom
convolutional neural network architecture is then developed to extra hierarchical features from the
images and performs accurate classification. The convolutional neural network model is meticulously
designed to balance high accuracy with computational efficiency, making it practical for real-world
applications. The results from experiments indicate the system’s high reliability and effectiveness
across various conditions, underlining its potential to reduce traffic incidents and enhance overall
traffic management. This integration of convolutional neural network -driven visual recognition with
auditory feedback offers a promising advancement in the field of driver assistance system.

Keywords: CNN, accident detection, road signals, visual recognition, vehicles, deep learning,
intelligent transportation systems

INTRODUCTION
Road signs and boardings that are printed along the side of also above roads give drivers accurate
directions or information about the signs. There will be disease on the roads if they do not exist, and
similar kinds of accidents will increase rapidly. This planned system will help to identify the signs of
the traffic logo that is signed and send an alert through a voice that will notify drivers of the signs’
meanings, thus improving awareness and reducing
*Author for Correspondence road accidents. The datasets are taken from the
Nyamatulla Patel Kaggle, and it has around 51,900 photos in 43
E-mail: [email protected] different sign types in the dataset. Nowadays,
1Student, Department of MCA, Secab Institute of Engineering Vehicles have become the greatest wanted and
and Technology, Vijayapur, Karnataka, India.
2Associate Professor, Department of MCA, Secab Institute of
significant assets a person may own in today’s
Engineering and Technology, Vijayapur, Karnataka, India.
modern world due to their wide range of benefits in
3Associate Professor, Department of Electronics and this fast-paced environment and are thus a vital
Communication Engineering, Secab Institute of Engineering component of today’s economy. Aside from that,
and Technology Vijayapur, Vijayapur, Karnataka, India.
many drivers from rural regions may face new
Received date: September 05, 2024 signs when visiting a big city and may find it
Accepted date: October 18, 2024 difficult to understand them [4–6].
Published date: October 28, 2024

Citation: Shilpa Shahapur, Nyamatulla Patel, Dr. Mohammed LITERATURE SURVEY

Ziaullah Choudhari. CNN-Based Traffic Sign Recognition
with Voice Alerts. International Journal of Embedded
De la Escalera, Moreno, salichs, and Armingol,
Systems and Emerging Technologies. 2024; 10(2): 1–9. the IEEE paper is published in 1997. The paper’s

© JournalsPub 2024. All Rights Reserved 1

CNN-Based Traffic Sign Recognition with Voice Alerts Shahapur et al.

name is Road Traffic Sign Detection and Classification the methodologies and techniques for
detecting and classifying road traffic signs for researching the pivotal developing intelligent
transportation systems and autonomous driving technologies for the in-depth of the paper [1].

The Traffic Sign Detection Approach Based on the Lightweight Network and the Multilayer
Propose Network by H. Nguyen, published in the Journal of the Sensors in the year 2020, presents an
Innovative approach to traffic sign detection using the neural network techniques focused on
achieving the fast and efficient detection suitable for the real-time application in the intelligent
transportation system and autonomous vehicles [3].

The traffic sign recognition and voice Alert System using CNN by Python in the details of the
building this kind of system will know traffic symbols using convolutional neural networks (CNNs) to
provide voice alerts to drivers the research is crucial for enhancing road safety by assisting drivers in
recognizing and responding to traffic-to-traffic of signs promptly and published in the year 2022 [7].

Mathias, Timofte, Benenson, and Van Gool presented in the year 2013 of the International Joint
Conference on the Neural Networks (IJCNN) by exploring the progress and challenges of the field of
traffic symbol appreciation of the research to give a complete overview of the state of the art
techniques at the time evaluates how the method is to achieve reliable and efficient traffic sign
recognition [2].

PROBLEM STATEMENT
Every day, a significant number of accidents happen reason is that a driver’s lack of kindness while
driving or, in some cases, planned disrespect to many road signs that they usually don’t understand.
External factors, such as weather, fog, and dew, along with the human error, affect the visibility of
road signs. Based on the book International Transport Forum, Malaysia had the highest number of
deaths on the road the given results of the factors listed above, with practically 24 deaths per 100,000
and it can above.

The optional system’s detach is to identify traffic signals in real-time and provide information on
what they signify and What group they fall within, allowing drivers to be extra cautious and, as a
result, lowering the average number of crashes on roads. Highway indicator identification requires a
method of identifying and recognizing routes and signs for traffic in traffic conditions taken by a
camera. It’s a method utilizing image-identifying.

MOTIVATION
The motivation behind the CNN-based traffic sign recognition with voice alerts project is to ensure
a safe and comfortable drive by recognizing traffic signs and alerting the driver through voice
messages. This project’s goal is to prevent road accidents caused by missing or misconstruing traffic
signs, which can lead to over-speeding and other dangerous activities. The system is designed to
support the driver in noticing the traffic signs near their eyes that are presentation screen, saving their
time in physical goals for progress the comfort of drivers by providing riders with real-time traffic
sign information and voice alerts, confirming that drivers follow traffic rules and regulations,
promoting road safety, and reducing the danger of accidents [8–10].

OBJECTIVES
The project is used to detect malware in mobile apps. This contains extracts the code after
extracting all the code from the app using backward engineering and then it is going to examine
whether this app can detect and engages in any malicious activities, such as sending SMS or accessing
contact details without proper permissions. To achieve this, more than 100 permissions need to be
extracted from the code to make a feature dataset. If the application has the necessary permissions,
then that must be done, if a value of 1 is assigned to the dataset, and if it is not, a value of 0 is

© JournalsPub 2024. All Rights Reserved 2

International Journal of Embedded Systems and Emerging Technologies
Volume 10, Issue 2
ISSN: 2456-723X

allocated. Based on this feature dataset, the app will be considered as either malware or safe. Given
Android’s open-source nature and the ease of publishing apps on Google Play Store, cyber criminals
feet this platform to develop and distribute malware. Such malicious apps can steal sensitive
information or grant unauthorized access to the user’s device [9–12].

PROPOSED SYSTEM
The suggested Road Signal Recognition & Speech Warning Network wants to look for the
drawbacks of existing systems by leveraging CNN and voice notifications to encourage traffic safety.
This system employs machine vision methods to perform real-time recognition and classification of
various warning signs using digital images or video streams. By applying CNNs, the system can
remove key characteristics on the input photos, enabling accurate and robust sign recognition even
under adverse conditions. Once a roadway signal is detected, the scheme resolves and creates timely
and strong voice warnings to inform the driver about the exact sign detected. This strategy ensures
that drivers receive immediate and easily understandable instructions without needing to divert their
attention from the road. The proposed system intends to improve overall traffic safety by reducing the
chance of accidents caused by missing or misinterpreted traffic signs, promoting adherence to traffic
regulations, and providing drivers with dependable assistance that enhances situational alertness.

TECHNOLOGIES USED
The project, python version 3.12.4 has been used. This language has been the biggest tool and is
considered the biggest well-balanced and very stable version with the most important advantage is its
easy-to-use functions. Most of the programmers are aware of the meaning of errors, and bugs in their
work and how annoying errors and particular error types may be raised. From time to time, there may
be some problems or issues with the syntax, before versions of Python language that would throw
error messages or notifications. The issues can be raised by poor syntax, omitted keywords, wrongly
spelled or wrong keywords, amongst other problems. This Python language allows developers to work
efficiently and with positive confidence while creating software, from the development part to the
deployment part and with maintenance. This kind of advantages that make this language that is
python allows the best option for developing projects on ML, that is, machine learning and
intelligence on artificial include freedom to use the platform, access to brilliant libraries and other
frameworks, and a large and better community.

SYSTEM DESIGN
Design Approach
Traffic sign detection with voice notification device utilizing CNN that is the neural network with
convolution. In this approach, we will break down the system into many types of functions or
modules, each one will be responsible for a specific task. By breaking down the system into these
functions or a module, it becomes so easier to learn, and think also to design each component
separately. This approach helps betterment of code groups, for modular development, and that allows
for scalability and flexibility in developing or replacing each signal function if needed [13–17].

DETAIL DESIGN
The creative method Functional-Oriented Traffic Sign Detection with voice notification device
utilizing neural networks with convolution (CNN):

Training Data
Dataset: A large dataset of signs for traffic images is required for the training of the CNN model.
The dataset should cover a broad spectrum of signs for traffic types, various lighting conditions, and
diverse backgrounds.

© JournalsPub 2024. All Rights Reserved 3

CNN-Based Traffic Sign Recognition with Voice Alerts Shahapur et al.

Labels: Each image in the training dataset needs to be labeled with the corresponding traffic sign
group for supervised learning. The labels should indicate the different types of traffic signs that are
featured at the time image.

Validation Data
Dataset: A separate dataset is needed for evaluating the trained CNN model. This dataset should be
distinct from the training dataset and contain traffic sign images anything the subject was not present
over training.

Labels: Like the training data, the validation dataset requires labeled images to determine the
performance and accuracy of the CNN model.

Image frames: The system needs a continuous video stream from a camera mounted on the vehicle.
In that, one of the frames of video stream is processed to detect and recognize traffic signs.

Frame resolution: The camera should provide video frames with sufficient resolution to capture the
traffic signs.

ARCHITECTURE OF CNN
If the structure is deep learning, CNN constitutes a single of the greatest popular and regularly
deployed deep neural networks. Around the 1980s, NNs initially emerged and were employed. At the
time, most networks were capable of identifying digits written by hand. It was frequently utilized in
the postal sector for reading postal codes, key values, and other associated data. The most crucial
point to know regarding any deep learning algorithm is the fact that it needs plenty of information and
a lot of computer resources to train. It was a huge drawback for CNN algorithms. At the time, they
were confined to the postal domain and never made items go into the field of artificial intelligence
(AI).

T comprehend the process a CNN model works, we need to be familiar with the basics, like what an
image is and the way it is represented.

A picture in grayscale is nothing more than an array of values for pixels with a single plane,
whereas an RGB picture has no significance more than an array of values for pixels with three planes.

Figure 1. Architecture of CNN.

Figure 1 shows the architecture of CNN. Artificial neurons or nodes in CNNs use inputs, process
them, and then transmit the result as output, just like a neuron in the brain processes and transmits
information throughout the body. An input layer, hidden layers, and an output layer make up an

© JournalsPub 2024. All Rights Reserved 4

International Journal of Embedded Systems and Emerging Technologies
Volume 10, Issue 2
ISSN: 2456-723X

ordinary neural network. CNNs were inspired by the brain’s structural design. Many hidden layers
may exist in CNNs, single of that which works featuring removal of the image by passing
calculations. Convolution, pooling, cured linear units, and fully connected layers are examples of this.
Convolution is the basic layer that draws features out of an input image. These image pixels are input
to provide input layering in the shape of arrays.

USE-CASE DIAGRAM
Use-case diagram is shown in Figure 2 of a CNN-based Traffic Sign Recognition System with
voice alerts that have many use cases to be considered. The main role is of a driver, who depends on
the system of traffic signs and that’s also gives real-time alerts through voice to develop safety and
ease. It utilizes CNN. This case involves a system analyzing the road environment with the help
camera and identifying traffic signs that may include speed limits, stop sign, turn sign, and more.

Figure 2. Shows use-case diagram.

METHODOLOGY
In general, word recognition denotes that a sign has been identified, while word classification
denotes that the sign has been allowed to a specific class based on features. Because recognition might
often imply classification, both phrases serve interchangeably in this chapter [18–20]. For the
identification and categorization of road and traffic signs, A wide range of different techniques are:
 Collecting dataset,
 Image pre-processing,
 Train CNN,
 Deploy CNN to detect traffic signs.

© JournalsPub 2024. All Rights Reserved 5

CNN-Based Traffic Sign Recognition with Voice Alerts Shahapur et al.

IMPLEMENTATION
User Interface Representation
Visual display: The user includes the interface of a visual display to show the real-time video
stream or images through the help vehicle’s camera. This display can overlay information, such as
detected traffic signs, their recognized classes, and any relevant warnings or alerts.

Traffic sign icons: To represent known traffic signs, you use icons or symbols that resemble the
actual signs. These icons can be displayed alongside the corresponding text labels for easy
identification.

Voice alerts: The user interface should include a section for voice alerts. When a traffic sign is
familiar, the corresponding voice alert should be displayed or played to notify the driver.

Alert log: A log or a history section can be added as to display a connection with of recent traffic
signs detected and the corresponding voice alerts generated. This allows the driver to review past
alerts and stay informed about the traffic signs met during the journey.

Brief Description of Several Modules of the Device

 Input module: This part receives information from the user’s vehicle’s camera or video stream.
It receives real-time images or video frames, which serve as the input for following processing.
 Road signals identification module: The road sign’s identify section uses vision-based methods
to identify potential traffic sign candidates within the input images. It identifies the image
regions and applies algorithms to detect objects that resemble traffic signs based on shape, color,
and other visual cues.
 Road signals identification module: Once potential road signal candidates are identified, the
recognition module uses a trained CNN to classify and recognize the specific traffic sign within
each candidate region. The CNN has been trained for a labeled dataset and can accurately
classify the road signs into different classes or categories.

Voice Alert Generation Module

After the road sign’s is recognized, the voice alert generation module converts the recognized sign
into an appropriate voice alert message. This module can utilize text-to-speech (TTS) technology to
convert the recognized traffic sign label into speech output. The generated voice alert message is
structured to inform the driver about the detected sign and provide relevant warnings or instructions.
The main page of the CNN-based traffic sign recognition system with voice alerts is shown in Figure
3.

Figure 3. Main page of CNN-based traffic sign recognition system with voice alerts.

© JournalsPub 2024. All Rights Reserved 6

International Journal of Embedded Systems and Emerging Technologies
Volume 10, Issue 2
ISSN: 2456-723X

Figure 4. Uploading page.

The page is used to add the image shown in Figure 4.

Figure 5. Output page.

The output page is shown in Figure 5. we are going to get a result of added image, after adding
image we are going to classify, and we will get the result with voice alert.

Figure 6. Camera output page.

© JournalsPub 2024. All Rights Reserved 7

CNN-Based Traffic Sign Recognition with Voice Alerts Shahapur et al.

The camera page is shown in Figure 6. We are going to add an image by clicking on the camera
button and showing an image of the in front of the camera after we need to classify the image it shows
output. The graphical visualization of the system efficiency is shown in Figure 7.

Figure 7. System efficiency.

CONCLUSION AND FUTURE SCOPE

The number of smashes in traffic that lead to due to confusion of road signals and to educate drivers
about the sign’s meaning. The idea of a road signals recognition tool that will recognize signs using
image processing and deep learning technology to display the sign’s category and meaning with a told
message to attention drivers. The primary goal is the outcome of the project; the second objective is
the functioning of the system; and the next objective is the methodology used for building the system.

To utilize CNN deep learning techniques to classify road signals images and test the accuracy of
image recognition. Therefore, it was a successful project, thus achieving the laid-out objectives. In
this portion, the personal experiences that were held throughout the project are discussed. The
objective was addressed in counting the journey, and challenges were satisfied during it. With this, the
project road signals identification and speech notice mechanism with CNN’s system is resolved.

REFERENCES
1. De La Escalera A, Moreno LE, Salichs MA, Armingol JM. Road traffic sign detection and
classification. IEEE Transac Indust Electron. 1997;44(6):848–59.
2. Mathias M, Timofte R, Benenson R, Van Gool L. Traffic sign recognition—how far are we from
the solution? In: 2013 International Joint Conference on Neural Networks (IJCNN). 2013, Aug 4.
pp. 1–8. IEEE.
3. Nguyen H. Fast traffic sign detection approach based on lightweight network and multilayer
proposal network. J Sens. 2020;2020(1):8844348.
4. Ansari S, Akhdar F, Mandoorah M, Moutaery K. Causes and effects of road traffic accidents in
Saudi Arabia. Public health. 2000;114(1):37–9.
5. Zhang J, Zou X, Kuang LD, Wang J, Sherratt RS, Yu X. CCTSDB 2021: a more comprehensive
traffic sign detection benchmark. Human-cent Comput Inform Sci. 2022;12.
6. Shao F, Wang X, Meng F, Rui T, Wang D, Tang J. Real-time traffic sign detection and
recognition method based on simplified Gabor wavelets and CNNs. Sensors. 2018;18(10):3192.

© JournalsPub 2024. All Rights Reserved 8

International Journal of Embedded Systems and Emerging Technologies
Volume 10, Issue 2
ISSN: 2456-723X

7. Sukhani K, Shankarmani R, Shah J, Shah K. Traffic sign board recognition and voice alert system
using convolutional neural network. In: 2021 2nd International Conference for Emerging
Technology (INCET). 2021, May 21. pp. 1–5. IEEE.
8. Kaushik B, Sharma R, Dhama K, Chadha A, Sharma S. Performance evaluation of learning
models for intrusion detection system using feature selection. J Comp Virol Hack Techn.
2023;19(4):529–48.
9. Manawadu M, Wijenayake U. Voice-assisted real-time traffic sign recognition system using
convolutional neural network. arXiv preprint arXiv:2404.07807. 2024 Apr 11.
10. Dean HN, Jabir KV. Real time detection and recognition of Indian traffic signs using Matlab. Int J
Scient Engin Res. 2013;4(5):684–90.
11. Shi JH, Lin HY. A vision system for traffic sign detection and recognition. In: 2017 IEEE 26th
International Symposium on Industrial Electronics (ISIE). 2017, Jun 19. pp. 1596–1601. IEEE.
12. Boujemaa KS, Berrada I, Bouhoute A, Boubouh K. Traffic sign recognition using convolutional
neural networks. In: 2017 International Conference on Wireless Networks and Mobile
Communications (WINCOM). 2017, Nov 1. pp. 1–6. IEEE.
13. Stanley KO, Miikkulainen R. Evolving neural networks through augmenting topologies. Evolut
Comp. 2002;10(2):99–127.
14. Yadav S, Patwa A, Rane S, Narvekar C. Indian traffic signboard recognition and driver alert
system using machine learning. Int J Appl Sci Smart Technol. 2019;1(1):1–0.
15. Hong CY, Lin CY, Shih TK. Automatic signboard detection and semi-automatic ground truth
generation. In: 2019 Twelfth International Conference on Ubi-Media Computing (Ubi-Media).
2019, Aug 5. pp. 256–61. IEEE.
16. Bi Z, Yu L, Gao H, Zhou P, Yao H. Improved VGG model-based efficient traffic sign recognition
for safe driving in 5G scenarios. Int J Mach Learn Cybernet. 2021;12:3069–80.
17. Cao J, Song C, Peng S, Xiao F, Song S. Improved traffic sign detection and recognition algorithm
for intelligent vehicles. Sensors. 2019;19(18):4021.
18. Wali SB, Abdullah MA, Hannan MA, Hussain A, Samad SA, Ker PJ, et al. Vision-based traffic
sign detection and recognition systems: current trends and challenges. Sensors. 2019;19(9):2093.
19. Santos A, Abu PA, Oppus C, Reyes RS. Real-time traffic sign detection and recognition system
for assistive driving. 2020;5(4):600–11.
20. Loce RP, Bala R, Trivedi M, Wiley J, editors. Computer vision and imaging in intelligent
transportation systems. Wiley; 2017.

© JournalsPub 2024. All Rights Reserved 9