Volume 8, Issue 3, March – 2023 International Journal of Innovative Science and Research Technology

ISSN No:-2456-2165

Sign Language Translator

[1]
V. Jyothi (Assistant Professor)
[2] [3]
A. Abilash ; Saripalli Shelsi; [4] G. Divij Reddy [5] R. Sreenidhi Reddy
Department of CSE
Anurag Group of Institutions

Abstract:- Individuals with hearing and speech providing text and audio output, this system falls short in
disabilities use sign language as their primary mode of terms of accuracy and speed.
communication to express their thoughts, ideas, feelings,
and opinions to the rest of the world. They use multiple III. PROPOSED SYSTEM
complementary channels to convey information as visual
languages. This includes manual characteristics like For hand sign sequences, our platform employs neural
hand shape, movement and pose, facial expression, lip networks with LSTM (long short-term memory) layers. It
movement, and so on. For someone who has never detects sign languages in real time using OpenCV. This
learned the language, the sign gestures are frequently platform converts recognised signs into written text and
mixed up and confused. Our project focuses on bridging audio. Another advantage of this approach is that it can be
this gap by recognizing hand gestures and converting used with any background and produces results faster. It is a
them into readable text and audio speech using machine complete sign language to audio and text translation model
learning algorithms, and it also allows written text to be that can also be used for learning. This platform offers a
converted into hand gestures. Sign language recognition user-friendly application that reduces the amount of time
and translation enable us to learn the spatial spent on communication. It allows you to reach a larger
representations, underlying language model, and audience and interact with them even from a distance,
mapping between sign and spoken language in real time. without making any noise.

I. INTRODUCTION

A sign language is a language that uses hand

movements, body orientation, and facial expressions for
communication rather than relying on acoustic waves.
According to a World Health Organization report, "a total of
466 million people worldwide have a hearing impairment,
and this number is expected to rise to 900 million people, by
the year 2050." As a result, it is now more than necessary to
develop a cost-effective and comprehensive system to assist
these individuals. This system aims to not only bridge the
communication gap, but also to improve individuals'
learning curves and make them more confident. There are
numerous data-sets and models available, but none of them
have been shown to be very accurate due to inconsistencies
in the data-set or the model. There have been hardware
innovations in the past that focused on phrase conversion,
but the scope is limited to the data-set. While in our
proposed system, each alphabet in American Sign Language
has been translated to text and speech and allows for bi- (Fig.1 Proposed System Architecture)
directional communication.
IV. EXPERIMENTAL TOOLS
II. EXISTING SYSTEM
 Deep learning:
To interpret sign language in the existing system, the Deep learning is a machine learning algorithm that
K-Nearest Neighbour (KNN) algorithm in Machine employs multiple layers to extract higher-level features from
Learning is used. The accuracy and speed with which this raw input. Learning can take place under supervision, semi-
system can generate text are insufficient. When the supervised, or unsupervised. Deep-learning architectures
background is not black, it is unable to interpret the text. such as deep neural networks, deep belief networks, deep
This system can only detect up to ten sign alphabets, which reinforcement learning, recurrent neural networks,
must be pre-trained before using a large number of images convolutional neural networks, and transformers have been
for a single alphabet, word, or phrase. When it comes to used in computer vision. Deep learning demands a vast
amount of data and computer power, but it has become more

IJISRT23MAR1660 www.ijisrt.com 1871

Volume 8, Issue 3, March – 2023 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
accessible in recent years thanks to cloud computing and  MySQL:
open-source frameworks like Tensor Flow and PyTorch. In MySQL is a well-known open-source relational
the upcoming years, it is anticipated to have a substantial database management system (RDBMS) that is used to store
impact on a variety of industries, including healthcare, and manage structured data. Because of its scalability,
banking, and transportation. Artificial neural networks are reliability, and ease of use, it is widely used in web
used in deep learning, a branch of machine learning, to development and is frequently the database of choice for
model and resolve complicated issues. many web applications. MySQL employs a client-server
architecture, in which data is stored on a server and
 Python: accessible via a network by multiple clients. MySQL is built
Python is an interpreted object-oriented, high-level on SQL (Structured Query Language), a database
programming language with dynamic semantics. Its built-in management and querying language. MySQL supports a
high-level data structures, combined with dynamic typing wide range of programming languages, including Python,
and dynamic binding, make it an excellent choice for use as Java, and PHP.
a scripting or glue language to connect existing components.
Python's straightforward, simple syntax emphasises  Visual studio code:
readability, lowering programme maintenance costs. Python Microsoft's Visual Studio Code (VS Code) is widely
supports modules and packages, which encourages used by programmers and developers for writing and
programme modularity and code reuse. Python's interpreter debugging code. Many features, such as code highlighting,
and extensive standard library are available in source or autocomplete, and debugging tools, make VS Code popular
binary form for all major platforms. among developers. It supports numerous programming
languages, including Java, Python, JavaScript, C++, and
 Flask: others. VS Code supports a wide range of programming
Flask is a Python-based micro web framework. languages, including CSS, Go, and Docker file in addition to
Because it does not require the use of any specific tools or Java, C++, and Python. In addition, VS Code enables you to
libraries, it is classified as a micro-framework. It lacks a add new extensions like code linters, debuggers, and support
database abstraction layer, form validation, and other for cloud and web development.
components that are provided by third-party libraries. Flask,
on the other hand, supports extensions that allow you to add V. OUTPUT
application features as if they were built into Flask. It is very
simple to learn and use. The tools and libraries required to
create web applications, like as routing, request processing,
and templates, are provided by Flask. Due to Flask's strong
extensibility, developers can quickly incorporate third-party
libraries and tools as required. The simplicity and
minimalism of Flask are two of its main advantages.
Overall, Flask is a strong and adaptable technology that is
suitable for Python web application development.

 HTML:
HTML is an abbreviation for Hypertext Markup
Language, which is the standard markup language for
creating web pages and applications. HTML allows
developers to structure content on a web page by defining
different elements such as headings, paragraphs, links,
images, and forms using tags and attributes. HTML
documents are composed of a series of tags enclosed in
angle brackets, such as <html>, <head>, <title>, <body>,
and many others.

 CSS:
CSS (Cascading Style Sheets) is a style sheet language
that is used to describe the appearance and layout of HTML (Fig.1 Home Page)
or XML documents. CSS allows developers to separate a
web page's content from its visual presentation, giving them Home page displays project title and Login button,
more flexibility and control over how the page is displayed. which takes you to the login page.
CSS applies rules or styles to specific HTML elements or
groups of HTML elements. These rules can be used to
define the font, colour, size, layout, and positioning of
various elements on a web page.

IJISRT23MAR1660 www.ijisrt.com 1872

Volume 8, Issue 3, March – 2023 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165

(Fig.3 Login Page) (Fig.5 Text as input)

It displays the login page, where they can register if it's In this page text is given as input and we will have
their first time and also login if they already registered. voice and sign language as the output.

The sign language is displayed in the other window.

(Fig.4 Registration Form)

It displays the user to fill the personal details to register (Fig.6 Image as input)
and can directly login next time.
In this page image is taken as input and we will have
the translated text of the sign image as the output.

IJISRT23MAR1660 www.ijisrt.com 1873

Volume 8, Issue 3, March – 2023 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165
VI. CONCLUSION [8]. Statistical Sign Language Translation (Jan Bungeroth,
Hermann Ney, Lehrstuhl fur Informatik VI, Computer
A sign language translator project is a significant Science Department, RWTH Aachen University, D-
technological advancement that has the potential to 52056 Aachen, Germany)
transform communication for the deaf and hard-of-hearing [9]. Sign Language Transformers: Joint End-to-end Sign
communities. The project's goal is to increase accessibility Language Recognition and Translation (Necati Cihan
and social inclusion by facilitating communication between Camgoz, Oscar Koller, Simon Hadfield and Richard
deaf and hard-of-hearing people and the hearing population. Bowden, CVSSP, University of Surrey, Guildford, UK,
A sign language translator can accurately recognise and Microsoft, Munich, Germany)
interpret sign language gestures and translate them into [10]. Hierarchical LSTM for Sign Language Translation
spoken language or written text using advanced artificial (Dan Guo,1 Wengang Zhou,2 Houqiang Li,2 Meng
intelligence and machine learning algorithms, and vice Wang1, 1School of Computer and Information, Hefei
versa. Education, public events, healthcare, customer University of Technology, Hefei, 230009, China,
service, online communication, and emergency services are 2EEIS Department, University of Science and
among the many applications of the project. A sign language Technology of China, Hefei, 230027, China)
translator project can improve communication and access to [11]. R. Harini, R. Janani, S. Keerthana, S. Madhubala and
information for all individuals, regardless of hearing ability, S. Venkatasubramanian, "Sign Language Translation,"
by developing a user-friendly interface that integrates with 2020 6th International Conference on Advanced
other technologies. Overall, the creation of a sign language Computing and Communication Systems (ICACCS),
translator project provides an opportunity to advance Coimbatore, India, 2020, pp. 883-886,
technology, promote inclusivity and accessibility, and raise doi:10.1109/ICACCS48705.2020.9074370.
awareness of the needs of the deaf or hard-of-hearing [12]. Real-Time Malaysian Sign Language Translation using
community. Colour Segmentation and Neural Network (Rini
Akmeliawatil, Melanie Po-Leen Ooi, Ye Chow Kuang,
REFERENCES Monash University, Jalan Universiti, Bandar Sunway,
46150 Petaling Jaya, Malaysia)
[1]. Recognition of Sign Language Symbols using [13]. Deep Learning Methods for Sign Language Translation
Templates (Deepika Pahuja, Sarika Jain, Amity (Tejaswini Ananthanarayana, Priyanshu Srivastava,
Institute of Information Technology, Amity Akash Chintha, Akhil Santha, Brian Landy, Joseph
University) Panaro, Andre Webste, Nikunj Kotecha, Shagan Sah,
[2]. Sign Language Interpreter System: An alternative Thomastine Sarchet, Raymond Ptucha, and Ifeoma
system for machine learning (Salma A. Essam El-Din1 Nwogu, Rochester Institute of Technology)
and Mohamed A. Abd El-Ghany 2,3 1Communication [14]. Sign Language Recognition, Generation, and
Engineering Department, German University in Cairo, Translation: An Interdisciplinary Perspective (Danielle
Cairo, Egypt 2Electronics Department, German Bragg, Oscar Koller, Mary Bellard, Larwan Berke,
University in Cairo, Cairo, Egypt 3 Integrated Patrick Boudreault, Annelies Braffort, Naomi Caselli,
Electronic Systems Lab, TU Darmstadt, Germany) Matt Huenerfauth, Hernisa Kacorri, Tessa Verhoef,
[3]. Sign Language Translator Using Machine Learning Christian Vogler, Meredith Ringel Morris)
(Swapna Johnnya, S Jaya Nirmala National Institute of [15]. Real-Time American Sign Language Recognition
Technology Tiruchirappalli) Using Skin Segmentation and Image Category
[4]. Live Sign Language Translation: A Survey (Aditya Classification with Convolutional Neural Network and
Dawda, Aditya Devchakke, Avdhoot Durgude, Dept. Deep Learning (Shadman Shahriar, Ashraf Siddiquee,
of Computer Science and Engineering, MIT School of Tanveerul Islam, Abesh Ghosh, Rajat Chakraborty,
Engineering, Maharashtra, India) Asir Intisar Khan, Celia Shahnaz, Shaikh Anowarul
[5]. American Sign Language Recognition System: An Fattah, Department of Electrical and Electronic
Optimal Approach (Shivashankara S, Srinath S, Engineering, Bangladesh University of Engineering
Department of Computer Science & Engineering, Sri and Technology, Dhaka-1205, Bangladesh)
Jayachamarajendra College of Engineering, Mysuru,
India)
[6]. Recognition of Single-Handed Sign Language Gestures
using Contour Tracing Descriptor (Rohit Sharma, Yash
Nemani, Sumit Kumar, Lalit Kane, Pritee Khanna,
Member, IAENG)
[7]. Neural Sign Language Translation (Necati Cihan
Camgoz, Simon Hadfield, Oscar Koller, Hermann Ney,
Richard Bowden; Proceedings of the IEEE Conference
on Computer Vision and Pattern Recognition (CVPR),
2018, pp. 7784-7793)

IJISRT23MAR1660 www.ijisrt.com 1874