0% found this document useful (0 votes)

23 views

Advances in Human-Computer Interaction - 2024 - Alam - Exploring Sign Language Detection On Smartphones A Systematic

research paper

Uploaded by

pranavijawalekar2005

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

0% found this document useful (0 votes)

23 views

Advances in Human-Computer Interaction - 2024 - Alam - Exploring Sign Language Detection On Smartphones A Systematic

research paper

Uploaded by

pranavijawalekar2005

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

You are on page 1/ 36

Hindawi

Advances in Human-Computer Interaction

Volume 2024, Article ID 1487500, 36 pages
https://doi.org/10.1155/2024/1487500

Review Article
Exploring Sign Language Detection on Smartphones: A Systematic
Review of Machine and Deep Learning Approaches

Iftikhar Alam ,1 Abdul Hameed,2 and Riaz Ahmad Ziar 3

1
Department of Computer Science, City University of Science and Information Technology, Peshawar 25000, Pakistan
2
Department of Computer Science, Islamia College University, Peshawar 25000, Pakistan
3
Department of Computer Science, Kardan University, Kabul 1001, Afghanistan

Correspondence should be addressed to Riaz Ahmad Ziar; [email protected]

Received 11 October 2023; Revised 28 February 2024; Accepted 4 March 2024; Published 11 March 2024

Academic Editor: Christos Troussas

Copyright © 2024 Iftikhar Alam et al. Tis is an open access article distributed under the Creative Commons Attribution License,
which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
In this modern era of technology, most of the accessibility issues are handled with the help of smart devices and cutting-edge
gadgets. Smartphones play a crucial role in addressing various accessibility challenges, including voice recognition, sign language
detection and interpretation, navigation systems, speech-to-text conversion, and vice versa, among others. Tey are compu-
tationally powerful enough to handle and run numerous machine and deep learning applications. Among various accessibility
challenges, speech disorders represent a disability where individuals struggle to communicate verbally. Similarly, hearing loss is
a disability that impairs an individual’s ability to hear, necessitating reliance on gestures for communication. A signifcant
challenge encountered by people with speech disorders, hearing loss, or both is their inability to efectively convey or receive
messages from others. Hence, these individuals heavily depend on the sign language (a gesture-based communication) method,
typically involving hand movements and expressions. To the best of our knowledge, there are currently no comprehensive review
and/or survey articles available that cover the literature on speech disabilities and sign language detection and interpretation via
smartphones utilizing machine learning and/or deep learning approaches. Tis study flls the gap in the literature by analyzing
research publications on speech disabilities, published from 2012 to July 2023. A rigorous search and standard strategy for
formulating the literature along with a well-defned theoretical framework for results and fndings have been used. Te paper has
implications for practitioners and researchers working in accessibilities in general and smart/intelligent gadgets and applications
for speech-disabled people in specifc.

1. Introduction have a speech disability and this number is expected to rise to

1 in 4 by 2050. Te impacts of hearing loss are very serious.
A speech disorder, also known as a speech disability, is For example, people with speech disabilities are unable to
a condition where an individual faces difculty in efectively communicate with others which may lead to social isolation,
communicating verbally with others. One of the primary loneliness, and frustration. Tese conditions signifcantly
challenges for individuals with speech disorders is their impact individuals’ lifestyles and academic performance,
inability to convey messages directly through spoken lan- often resulting in employment challenges. In many de-
guage. Furthermore, some individuals with speech disorders veloping countries, there are a very limited number of
may also experience hearing loss, a prevalent issue world- specialized schools to cater to the needs of students with
wide. Te prevalence of speech disorders and hearing loss is speech disabilities and hearing impairments [1].
steadily on the rise, with an increasing number of individuals Sign language is a way of communication among people
afected by these conditions each day. According to the sufering from speech disorders and/or hearing loss prob-
World Health Organization (WHO), an estimated 430 lems. It is a language for speech-disordered people through
million people, which is 5% of the total world population, which they can communicate with other people and convey
3637, 2024, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1155/2024/1487500, Wiley Online Library on [24/09/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
2 Advances in Human-Computer Interaction

their messages. Sign alphabets rely on static hand poses to interface and keeping low latency on the cloud processing
symbolize individual letters of the alphabet, employing remains a major issue [7]. Smartphones equipped with an
gestures as a form of nonverbal communication. Te pro- increasing number of cameras have prompted researchers to
gression in computer vision has opened doors to the de- explore their potential in vision-based sign language rec-
velopment of sophisticated models capable of recognizing ognition applications. In the vision-based approach,
these signs, interpreting hand confgurations, and seamlessly a smartphone’s camera is employed to capture images or
translating them into both text and voice [2]. For instance, in videos of hand gestures. Subsequently, these frames undergo
a study by Raziq and Latif [3], the authors proposed a ges- processing to recognize the signs and generate text or speech
ture-based approach for Pakistan Sign Language (PSL) output. It is important to note that vision-based approaches
recognition, focusing on training and communication may entail a trade-of in accuracy compared to sensor-based
modules to detect sign language and convert it to text. methods. Tis is among various challenges in image pro-
Tere is no universal sign language in the world, and most cessing, including variations in lighting conditions, sensi-
people rely on region-specifc sign languages. Today, there are tivity to the user’s skin color, and the presence of complex
138–300 varieties of sign language across the world [4]. backgrounds within the image [8].
Moreover, there is a persistent communication gap between Numerous review articles have been written on acces-
hearing-disabled people, because they rely on sign language, sibility for speech disorder problems, regional and global
which is a problem for normal people due to their less un- sign languages, sensors-based approaches, and gesture-
derstanding of sign language. Typically, sign language recog- based recognition systems. Te following few paragraphs
nition through gadgets entails a two-step process: frst, the summarize and discuss the contributions in terms of survey
detection of hand gestures within the image, followed by their papers or reviews and their contributions along with a dis-
classifcation into the corresponding alphabet. Numerous cussion on the research gap.
methodologies incorporate the use of hand-tracking devices In a study by Ardiansyah, et al. [9], a review of studies
such as Leap Motion and Intel RealSense, accompanied by the has been performed between 2015 and 2020. Tey selected
application of machine learning algorithms like support vector the 22 most relevant studies regarding their research
machines (SVMs) to classify these gestures [5]. Hardware questions. In this study, the most popular method to obtain
devices, such as Microsoft’s kinetic sensors, are capable of data is through a camera. Diferent techniques were com-
constructing a three-dimensional (3D) model of the hand while pared and CNN was the most popular as it was more ac-
tracking hand movements and their orientations [6]. Although curate and used by 11 researchers out of 22. Similarly, a brief
hardware-based techniques can ofer a relatively high level of review of recent trends in sign language recognition by
accuracy, their widespread adoption is impeded by the sig- Nimisha and Jacob [10] discussed the two main approaches,
nifcant initial setup costs. which are the vision-based approach (VBA) and the gesture-
Numerous information and communication technologies based approach (GBA). Te image or vision-based sys-
(ICTs) are used for the detection and translation of diferent tematic literature review (SLR) and their approach com-
sign languages used by speech-disordered people. However, prising feature extraction and classifcation are mainly
some of these technologies are either expensive or socially discussed. Moreover, a comparative analysis of the tech-
unacceptable to many people sufering from speech disabilities. niques and achievements (in terms of accuracy) of nine
Te computer-based techniques were widely used; however, diferent studies on VBA and three studies on GBA is also
the computer is not portable and hence cannot be used by most available in this study.
people on the go. For such, a specialized environment is A review of smart gloves for the conversion of signs to
necessary. Furthermore, it is crucial to employ socially accepted speech for the mute community was proposed [11]. In this
devices to address these challenges. study, there was an absence of comparisons across various
Te ubiquitous presence of smartphones is undeniable. research papers. Te study primarily concentrated on
Tese devices can efciently execute a wide range of machine a single approach, specifcally the glove-based approach for
and deep learning applications. Notable examples include gesture recognition. Similarly, the perspective and evolution
convolutional neural networks (CNNs), K-nearest neighbors of gesture recognition for sign language are presented [12].
(KNN), deep convolutional generative adversarial networks Tey analyzed diferent gesture recognition devices through
(DCGANs), deep neural networks (DNNs), support vector a timeline with important features and achieved recognition
machines (SVMs), recurrent neural networks (RNNs), and rates. Tey concluded that Leap Motion is a good option for
3-D convolutional neural networks. Te smartphone can sign language as it is cheap, easy to use, and accurately
translate a sign language gesture to speech and vice versa in recognizes the hands. Some work on vision-based sign
real time to convey a proper message to other people. Some language recognition systems is also proposed by Sharma
prototypical-level applications also exist; however, they are and Singh [8]. In this study, diferent vision-based methods
either region-specifc or not accurate and hence rarely used. are analyzed along with the datasets used.
Tis problem highlights the need for a universal sign lan- A comprehensive review of wearable sensor-based sign
guage with no geographical boundaries and specifcations. language recognition is discussed by Kudrinko et al. [13].
Te smartphone processor and camera can be used for Tey conducted a review of studies between 1991 and 2019,
the detection of sign language. As mobile hardware tech- focusing on a total of 72 diferent research eforts. Tis
nology is getting more sophisticated over time and moving review paper aimed to discern prevailing trends, best
towards cloud infrastructure, maintaining a user-friendly practices, and existing challenges within the feld. Various
3637, 2024, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1155/2024/1487500, Wiley Online Library on [24/09/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
Advances in Human-Computer Interaction 3

attributes, such as sign language variation, sensor confgu- several researchers for vision-based sign language
ration, classifcation methods, study designs, and perfor- recognition.
mance metrics, were systematically analyzed and compared. A comprehensive survey on sign language recognition
It is important to note that this particular study exclusively using smartphones is presented in the study by Ghanem
examined the sensor-based approach. Additionally, the et al. [7]. In this paper, the authors explored the latest ad-
paper proposed a review specifcally centered around hand vancements in mobile-based sign language recognition.
gestures and sign language recognition techniques [14]. Tey Tey categorized existing solutions into sensor-based and
focused on a comprehensive exploration of the challenges, vision-based approaches, highlighting their respective ad-
diverse approaches, and the application domain of gesture vantages and disadvantages. Te authors’ primary focus was
recognition. Furthermore, they studied the various tech- on feature detection and sign classifcation algorithms.
niques and technologies utilized in sensor-based gesture Similarly, an automatic sign language recognition survey
recognition, providing valuable insights into this area of was done in the study [21]. Tey reviewed the studies
research. published between 2008 and 2017. Te authors discussed the
A technical approach to Chinese Sign Language pro- advancement of sign language recognition. Te authors also
cessing is discussed in the study by Kamal et al. [15]. Tey provided an overview of state-of-the-art building blocks of
provided an overview of Chinese Sign Language Recognition automatic sign language recognition like feature extraction,
(CSLR). Te paper discusses numerous issues related to classifcation, and sign language databases.
Chinese Sign Language. Similarly, another review on A study by Suharjito et al. [22] conducted a review of
system-based sensory gloves for sign language recognition sign language recognition application systems for hearing
and state of the art between 2007 and 2017 was presented by loss or speech-disordered individuals, employing an input-
Ahmed et al. [16]. Tey reviewed the studies published process-output framework. Tey evaluated various sign
between 2007 and 2017. Te authors explored and in- language recognition approaches and identifed the most
vestigated the SLR using the glove sensor approach. Te efective approach. Additionally, the study focused on dif-
articles are divided into four categories that are framework, ferent acquisition methods and classifcation techniques,
review and study, development, and hand gesture types. presenting their respective advantages and disadvantages.
Numerous recommendations put forth by researchers aim to Tis comprehensive analysis ofers valuable insights for
address both current and anticipated challenges, ofering researchers seeking to develop improved sign language
a wealth of opportunities for further research in this feld. recognition systems.
Te study on a review of automatic translation from In summary, this discussion above has encompassed
Arabic to Arabic Sign Language is presented in the study by selected systematic literature reviews (SLRs) and survey
Ayadi et al. [17]. Te authors presented work related to papers covering diverse topics of interest, while also high-
Arabic Sign Language (ArSL). Tey discussed the classical lighting notable contributions in these areas. Certain reviews
machine translation approach (direct, transfer-based, and are specifcally tailored to region-based sign languages, such
interlingua) and the corpus-based approach (memory, ex- as Chinese and American Sign Languages. Meanwhile,
ample, and statistical). Te authors also described the lan- others have become obsolete, ofering minimal relevance to
guage challenges, such as morphology, syntax, and structure. contemporary modern approaches. To address this research
Te study provides an extensive list of important works gap, this paper conducts a comprehensive analysis and re-
related to ArSL machine translation. Additionally, it ofers view of publications focused on sign language detection and
a comprehensive review of feature extraction methods in interpretation techniques, particularly those employing
sign language recognition systems by Suharjito et al. [18]. machine and deep learning approaches. Te review en-
Te review of studies published between 2009 and 2018 was compasses publications from esteemed journals and pres-
analyzed. Te authors reviewed and presented the progress tigious conferences spanning the past decade, ranging from
of feature extraction in sign language recognition. Te au- 2012 to July 2023. Te insights derived from this review hold
thors conclude that there is a considerable improvement in signifcant implications for a wide spectrum of stakeholders,
tracking hand regions by active sensors but still, there is including practitioners, researchers, developers, and in-
room for improvements in vision-based approaches. dustries engaged in accessibility solutions, software, and
A review of gesture recognition focusing on sign lan- hardware development, and the creation of smart devices
guage in a mobile context is presented in the study by Neiva tailored to individuals with speech disorders. Te major
and Zanchettin [19]. A review of studies published between contributions of this paper include
2009 and 2017 is presented. Te total number of papers that
(i) A complete up-to-date analysis of the publications
were analyzed and compared was 43. Te authors covered
published from 2012 to July 2023 through a rigorous
static and dynamic gestures, simple and complex back-
search and standard selection criteria.
grounds, facial and gaze expressions, and the use of special
mobile hardware. Similarly, a review of vision-based (ii) A detailed yet comprehensive discussion on current
American Sign Language (ASL) recognition, its tech- trends in the feld of disabilities specifcally for
niques, and outcomes are discussed in the study by Shiva- speech disorder people.
shankara and Srinath [20]. Te authors presented a review of (iii) A discussion on diferent machine learning ap-
ASL. Te authors highlighted the work and comparison of proaches for smart gadgets (smartphones in
3637, 2024, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1155/2024/1487500, Wiley Online Library on [24/09/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
4 Advances in Human-Computer Interaction

particular) along with sensor-based approaches 2.2. Time Frame and Digital Repositories. Te time for
used in smart gloves. searching the relevant literature is from 2012 to July 2023
(both years included) shown in Table 2. Te use of smart-
Tis paper organized and categorized (in a comprehen-
phones for sign language detection and identifcation has
sive manner) the available literature from diferent per-
evolved over the years due to the widespread adoption of
spectives and points of view discussed in the Materials and
smartphones and their growing role in assisting individuals
Methods section. A compact and concise literature is pre-
with disabilities, including speech disorders, visual im-
sented in respect of sign language recognition. Tis study
pairments, and related challenges. Since then, a reasonable
may help the practitioners to better understand the area,
amount of literature is available and mapped in this paper.
specifcally in mobile-based sign language detection and
We selected IEEE Xplore, ScienceDirect, ACM Digital Li-
recognition systems. It may also help the researchers to be
brary, and Google Scholar for searching the literature. Tese
fully aware of diferent approaches and research progress in
repositories were selected due to the reasons that they
this feld. Tis work comes under the category of accessibility
provide relevant publications, results, and analytics. Aca-
for people sufering from hearing loss or speech disorders.
demic search engines, such as Google Scholar, are also used
Te remainder of the paper is structured as follows.
for meaningful searches and insights.
Section 2 encompasses the “Materials and Methods,” out-
lining the approach used for examining the existing liter-
ature. Section 3, titled “Findings and Discussion,” 2.3. Teoretical Framework and Initial Results. Table 3 shows
investigates the explanation of seven research questions. a list of strings that we have used for searching and mapping
Section 4, labelled “Meta-Analysis,” provides a comprehen- the literature. Te search strings were searched using dif-
sive overview of the paper’s analysis, and it also touches ferent web search engines (discussed above). Te search
upon potential avenues for future research in Section 5 strings tabulated in Table 3 were applied in the selected
“Open Research Questions.” Finally, Section 6 serves as the digital repositories. Te results are recorded in Table 3.
conclusion, and the references are listed at the end of Te publications are categorized as journal papers and
the paper. conferences. Only prestigious conferences, i.e., supported by
ACM, IEEE, or Springer, are considered. Te ratio is shown
2. Materials and Methods in Figure 2.
Similarly, the year-wise frequency of the selected pub-
Tis study presents a systematic literature review (SLR) on lication is shown in Figure 3. We selected papers from 2012
sign language detection and interpretation via smartphone- to July 2023. We have seen a healthy growth of publications
based machine or deep learning approaches. Tis study is on these accessibilities, sign language, and smartphones as
mapped and conducted based on the guidelines presented by tools for speech-disordered people.
Kitchenham et al. [23] and Moher et al. [24]. Te research Table 4 presents the summary (most relevant papers) of
questions are designed to identify the research gap and are the publications along with years, types, and publishers. We
framed in Table 1. selected only well-reputed journals and conferences.

2.1. Search Strategy. Tis section discusses the search 3. Findings and Discussion
strategy for searching and mapping the relevant literature. Tis section is dedicated to addressing the research questions
We used the PRISMA framework for selecting the most raised and discussed in Table 1. Additionally, it provides an
relevant studies. We have adhered to the PRISMA frame- exhaustive review of the selected publications from a pool of
work [24] for structuring our search and selection meth- 163 research papers. It covers a wide range of aspects within
odology, illustrated in Figure 1. Te PRISMA framework is the research on smartphones as assistive devices, the ap-
a widely recognized and established methodology for con- plication of machine and deep learning approaches for
ducting systematic literature reviews. It ofers a set of individuals with speech disorders, the compilation of
guiding principles and a fowchart (refer to Figure 1) that comprehensive datasets utilized in research, region-specifc
aids researchers in adopting a systematic approach to ensure sign languages, and a detailed examination of the evaluation
the reporting quality is accurate, comprehensive, and metrics employed in experiments, each discussed in dedi-
transparent. Tis, in turn, forms the foundation for making cated subsections. Moreover, this section discusses the
well-founded and evidence-based decisions when selecting fndings, research gap, and possible directions for future
relevant literature. Figure 1 illustrates the initial search re- research.
sults, which amounted to 233,860 records. After screening
and removing duplicates, 281 studies were left of which 163
studies were the most relevant and are included for analysis. 3.1. RQ1: What Is the Current Status of Smartphone-Based
Te criteria for inclusion/exclusion of publication are Sign Language? In a study by Ghanem et al. [7], the authors
defned in Table 2. Te literature has been tabulated, ana- discussed in detail a survey of existing techniques used for
lyzed, and mapped based on criteria defned in Table 2. smartphone-based sign languages. Moreover, the authors
Advances in Human-Computer Interaction

Table 1: Research questions.

RQ# Research question Motivation
To study and map the current status of overall sign languages using smartphones as
RQ1 What is the current status of smartphone-based sign language?
a device for detection and interpretation, especially in 2023
How machine learning, deep learning, and lightweight deep learning techniques are To study deep learning and lightweight deep learning techniques used for the
RQ2
used for the detection and interpretation of sign languages? detection and interpretation of sign languages
To specify the diferent datasets, used for detection and interpretation of sign
RQ3 What are the types of datasets used for sign language recognition?
languages
To study and map the most popular approaches to sign language detection and
RQ4 What are the most popular approaches for recognizing sign language?
interpretation
RQ5 Which sign languages are targeted? To study the sign languages which are detected and interpreted
RQ6 What evaluation metrics are used in the experiments? To study what metrics are used in the experiments of the sign languages
To summarize the performance of models in sign language recognition, specifcally
RQ7 Which models have demonstrated better performance for specifc sign languages? highlighting which models have demonstrated better performance for specifc sign
languages
5

Identification
Records identified through google Additional records identified
scholar through other sources
(n = 233860) (n =0)

Records afer duplicates and irrelevant

removed (n = 1827)
Screening

Records screened Records excluded

(n = 374) (n = 60)

Full-text articles assessed Full-text articles

Eligibility

for eligibility excluded, afer reading

(n =314) (n =33)

Studies included afer in

depth analysis and
synthesis
(n =281)
Included

Studies included in
quantitative synthesis
(meta-analysis)
(n = 163)

Figure 1: Te identifcation process of primary studies [24].

Table 2: Inclusion and exclusion criteria.

Te searched string appeared in the title, abstract, or keywords of the study
Inclusion criteria Te publication is written only in the English language
Studies in journals, conferences, and book chapters from 2012 to July 2023
Blogs, keynotes, and weak reference studies, such as Wikipedia, dictionaries, and
Exclusion criteria thesaurus
Duplicate studies, i.e., studies published in more than one publisher’s database

developed an interactive Android mobile application cen- 3.2. RQ2: How Machine Learning, Deep Learning, and
tered around machine learning, aimed at bridging the Lightweight Deep Learning Techniques Are Used for the De-
communication gap between individuals with hearing loss tection and Interpretation of Sign Languages? Over time,
and the general population. In this connection, they in- numerous techniques have been investigated for efcient
troduced the PSL dataset [141]. Te approach used in this recognition of sign and gesture languages. Te majority of
study involved training the data through the SVM model, sign language recognition systems rely on machine
enabling automatic recognition of captured signs using the learning, deep learning, and lightweight deep learning
static symbols stored in the database. Numerous approaches approaches. Table 6 presents a compilation of selected
to machine and deep learning are used in various applica- studies and their respective approaches for detecting sign
tions. Table 5 provides a list of several of these approaches. languages through deep learning methods. Analyzing the
Table 5 shows a range of techniques organized according table, we can see that CNN is the most dominant technique.
to the year of study and evaluation metric. Notably, the CNN Tese techniques are general and not associated with
deep learning model has gained widespread acceptance specifc hardware, such as smartphones. Moreover, most of
among recent researchers for sign language detection and or the studies use hand gestures as input and recognize it via
recognition. Furthermore, the major evaluation metric some devices, such as custom-built gloves. It is also ob-
employed across the studies is “accuracy,” as indicated in served that CNN is still widely used even in recent years. It
Table 5. is important to recognize that any sign recognition system
3637, 2024, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1155/2024/1487500, Wiley Online Library on [24/09/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
Advances in Human-Computer Interaction 7

Table 3: Studies found in the selected repositories.

String Digital repository Studies found Selected
IEEE Xplore 8 5
ScienceDirect 711 7
Sign language detection smartphone ACM Digital Library 26053 16
Google Scholar 17900 37
Total 44672 65
IEEE Xplore 21 13
ScienceDirect 593 7
Sign language recognition using a smartphone ACM Digital Library 62275 22
Google Scholar 17600 35
Total 80489 77
IEEE Xplore 7 4
ScienceDirect 1231 5
Sign language smartphone deep learning ACM 39102 23
Google Scholar 17100 29
Total 57437 61
IEEE Xplore 19 7
ScienceDirect 1233 6
ACM 32921 21
Real-time smartphone sign language
Google Scholar 17100 44
Total 51262 78
Grand total 233860 281
After removing duplicates 163

Conferences VS Journal Papers

123

Conference papers
Journal Papers
Figure 2: Studies published in conferences and journals.

typically involves several key steps. First, input data are 3.3. RQ3: What Are the Types of Datasets Used for Sign
acquired, often through sources such as smartphone Language Recognition? Table 7(a) provides a comprehen-
cameras or sensors. Te subsequent step requires feature sive discussion of the various types of datasets and their
extraction from the acquired input data. Finally, the signs utilization in numerous studies. Furthermore, in
are classifed using algorithms that are well-suited to the Table 7(b), links to publicly available datasets are pro-
extracted features. Te accuracy of the detection and ex- vided. Upon analyzing these tables, it is observed that
traction system signifcantly infuences the quality of most of the studies have developed their custom datasets.
recognition results. Various approaches have been Additionally, it is notable that many of these datasets are
employed in sign recognition systems, including CNN, language-dependent, such as the PSL, American Sign
KNN, ANN, and SVM, among others. Among these Language (ASL), Malaysian Sign Language, Taiwan Sign
techniques, CNN stands out as a leading approach com- Language (TSL), and China Sign Language (CSL), among
pared to the other methods listed in Table 6. Table 6 also others. Table 7 showcases the studies along with their
depicts the studies and their associated information with respective years, datasets used, and remarks for
each study. each study.
3637, 2024, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1155/2024/1487500, Wiley Online Library on [24/09/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
8 Advances in Human-Computer Interaction

Yearwise Paper Frequency

25
22
20 21 21
19
18
17
15 15
Frequency

10 11
9
7
5
2
0 1
2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023
Years
Figure 3: Number of studies published from 2012 to 2023.

Table 4: Summary of the included literature.

Study Year Type Publisher
[25, 26] 2012 Conference ACM
[27] 2013 Conference ACM
[28–33] 2014 Conference IEEE
[34] 2014 Journal Pensee Journal
[35–41] 2015 Conference IEEE
[42] 2015 Conference Elsevier
[43] 2015 Conference ACM
[44–46] 2016 Conference ACM
[47] 2016 Conference British Machine Vision Conference
[48–50] 2016 Conference IEEE
[51] 2016 Journal Elsevier
[52] 2016 Journal International Journal of Electrical and Computer Engineering (IJECE)
[53] 2016 Journal Journal of Information Assurance & Security
[54] 2016 Journal Technology and Health Care
[55] 2017 Conference Heriot-Watt University
[56–59] 2017 Conference ACM
[2, 60–63] 2017 Conference IEEE
[64] 2017 Journal ACM
[65] 2017 Journal Computer Vision and Pattern Recognition
[66] 2017 Journal Far East Journal of Electronics and Communications
[67] 2017 Journal International Journal of Information Technology
[68] 2017 Journal Sensors
[69–71] 2018 Conference ACM
[72–78] 2018 Conference IEEE
[79, 80] 2018 Conference Springer
[81] 2018 Journal Entropy
[82] 2018 Journal Informatics
[83] 2018 Journal Elsevier
International Journal on Recent and Innovation Trends in Computing and
[84] 2018 Journal
Communication
[85] 2018 Journal Springer
[86–99] 2019 Conference ACM
[100] 2019 Conference Association for Computational Linguistics
[101] 2019 Conference Elsevier
[102–114] 2019 Conference IEEE
[115] 2019 Conference Springer
[116] 2019 Journal Journal of Education and Practice
[117] 2019 Journal International Journal of Ambient Computing and Intelligence (IJACI)
[118] 2019 Journal International Journal of Interactive Multimedia and Artifcial Intelligence
3637, 2024, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1155/2024/1487500, Wiley Online Library on [24/09/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
Advances in Human-Computer Interaction 9

Table 4: Continued.
Study Year Type Publisher
[119] 2019 Journal ACM
[120–125] 2020 Conference ACM
[126] 2020 Conference European Language Resources Association (ELRA)
[127, 128] 2020 Conference IEEE
[129, 130] 2020 Conference Springer
[131] 2020 Journal Applied Sciences
[132] 2020 Journal Computer Vision and Pattern Recognition
[133] 2020 Journal Telkomnika
[134] 2020 Journal Springer
[135] 2020 Journal International Journal of Advanced Trends in Computer Science and Engineering
[136] 2020 Journal Elsevier
[137] 2020 Journal ACM
[138] 2020 Journal IEEE
[139] 2021 Conference Atlantis Press
[140] 2021 Conference IEEE
[141] 2021 Journal Elsevier
[142] 2021 Journal Springer
[143] 2022 Journal IEEE
[144–146] 2022 Conference Springer
[147] 2023 Journal Elsevier
[148–150] 2023 Journal Springer
[151] 2023 Journal ACM
[152] 2023 Conference Springer

Numerous publicly available datasets are used by dif- 3.5. RQ5: Which Sign Languages Are Targeted? Diferent
ferent articles. Some of them can be accessed via links shown countries used their regional sign languages for research and
in Table 7(b). Some datasets are custom-made and not contributed to the accessibility domain for speech disorder
publicly available. people. Te American Sign Language is the dominant sign
language in the research as shown in Table 8.

3.4. RQ4: What Are the Most Popular Approaches for Rec-
ognizing Sign Language? Sign language recognition com- 3.6. RQ6: What Evaluation Metrics Are Used in the
monly utilizes sensor-based and vision-based techniques to Experiments? Te systems that use sign language dataset(s)
observe hand motion and posture [7]. Te sensor-based ap- are usually evaluated using standard metrics such as accu-
proach involves the use of sensors, such as those embedded in racy, precision, recall, and F1 score. From the literature,
gloves or smartphones, to track hand movements. Tese most of the systems were evaluated by detecting and
sensors, whether external or internal to the mobile device, interpreting the sign languages, and hence accuracy is the
capture data related to hand gestures. For example, glove-based frequently used metric as shown in Figure 4. Similarly,
approaches utilize multiple sensors within the gloves to precision and recall were also used.
monitor the position and movement of fngers and the palm,
providing coordinates for subsequent processing. Tese devices
may be connected wirelessly via Bluetooth. Te glove contains 3.7. RQ7: Which Models Have Demonstrated Better Perfor-
ten fexors for tracking fnger posture [39]. In the sensor-based mance for Specifc Sign Languages? Numerous machine and
approach, a combination of sensors, including a G-sensor and deep learning models have been employed for detecting and
a Gyroscope sensor, is employed to monitor hand orientation recognizing diverse sets of sign languages. Tis process
and motion. Tese sensors continuously capture signals related encompasses the training and testing of data using specifc
to hand data, which are then wirelessly transmitted to a mobile sign language datasets, which can include data ranging from
device for hand state estimation. Te choice of recognition hand gestures to video frames, as well as data collected from
method depends on the input data and the dataset utilized. In wearable sensors. As previously discussed, gestures are
this particular case, the authors utilized template matching as captured using mobile cameras, while data from wearable
a classifcation method, which encompasses fve dynamic sign sensors are collected through gloves. Table 9 provides an
classes. In the vision-based approach, hand gestures are ob- overview of studies centered on various sign languages,
served through the mobile camera, and a series of processing ofering insights into their respective accomplishments,
steps are applied to identify the signs within the video stream. primarily evaluated in terms of accuracy.
10
Table 5: Techniques of sign language recognition using smartphones.
Study Year Techniques Evaluation metric
[153] 2023 DeepVision transformers Accuracy, precision
[154] 2023 8-Layer CNN Accuracy
[155] 2023 K-nearest neighbors (KNN) Accuracy
[150] 2023 Deep learning (DL) combined with CNN and RNN Accuracy
[147] 2023 DNN Accuracy with [email protected]
[146] 2022 CNN Accuracy
[144] 2022 SVM Accuracy
Inaudible acoustic signal to estimate channel information and capture the sign
[143] 2022 Accuracy
language in real time
[156] 2022 CNN Accuracy
[157] 2022 CNN, DCGAN Accuracy
[141] 2021 SVM Accuracy, precision, recall, F1 score
[158] 2021 CNN Accuracy
[159] 2021 3DCNN Accuracy
[160] 2021 CNN, RNN Accuracy
[137] 2020 ISL parser, Hamburg notation system, signing gesture markup language, 3D avatar BLEU score, accuracy
[138] 2020 CNN Word recognition rate
[127] 2020 Long short-term memory (LSTM) Accuracy
[128] 2020 AutoML, transfer learning Precision, recall, F1 score, accuracy
[129] 2020 MobileNet and ResNet Accuracy
[133] 2020 MobileNet Accuracy
[132] 2020 MobileNet-V3 Accuracy
[120] 2020 Artifcial neural networks (ANNs) Accuracy
[102] 2019 State-of-the-art pose estimation method Accuracy
[110] 2019 CNN Accuracy
[105] 2019 Simple classifcation algorithms from machine learning Accuracy
[103] 2019 SVM Accuracy, precision, recall, F measure
[104] 2019 SVM Accuracy, precision, recall, specifcity, F1 measure
[109] 2019 Elliptical Fourier descriptor and LSTM Training time, testing time, accuracy
AdaBoost, multilayer perceptron, Naı̈ve Bayes, random forest, SVM, dynamic
[119] 2019 Accuracy
feature selection and voting
[91] 2019 CNN, LSTM, and connectionist temporal classifcation (CTC) Accuracy, WER
[101] 2019 MIT invertor Accuracy
[90] 2019 LSTM and CTC Accuracy, WER
[87] 2019 OpenPose, hidden Markov model Accuracy
[115] 2019 Gesture recognition algorithm of talking hands Accuracy
[74] 2018 Flex sensor with Arduino Accuracy
[70] 2018 CNN Accuracy
[84] 2018 CNN Accuracy, recognition time
[82] 2018 Naı̈ve Bayes, multilayer perceptron (MLP) Accuracy, F1 score
[75] 2018 KNN Accuracy, recognition time
[79] 2018 ANN Word matching score (WMS)
[83] 2018 ANN, minimum distance classifer WMS
[60] 2017 Neural network N.A
Advances in Human-Computer Interaction

[57] 2017 Binarized neural network, LSTM Detection ration (DR), reliability ration (RR), WER
[2] 2017 KNN, SVM linear, radial basis function SVM, random forest F measure, ROC, accuracy
[67] 2017 Discrete-time warping Accuracy
[61] 2017 Arduino N.A
[50] 2016 SVM Accuracy
[49] 2016 Backpropagation neural network Accuracy
[45] 2016 Dynamic time warping Recognition time, extensibility, recognition time (accuracy)
[52] 2016 Euclidean, normalized Euclidian, and Mahalanobis distance WMS
Optical character recognition, Microsoft Arabic Toolkit Service (ATKS), named
[51] 2016 Recognition time, usability
entity recognizer (NER)
[41] 2015 Neural networks (NNs) with log-sigmoid, NN with symmetric Elliott, and SVM Accuracy, classifcation time, memory usage, battery consumption
[42] 2015 Microcontroller Accuracy
[39] 2015 Flex sensors, inertial sensors Sensitivity, accuracy
KNN classifcation. Te time needed by the system to recognize a single sign is
[37] 2015 Accuracy
between 6 frames per second (FPS) and 20 FPS.
[40] 2015 Arduino Accuracy, error rate
[28] 2014 Recognition algorithm using histogram of oriented gradients (HOG) Recognition rate, processing time
Principle component analysis (PCA) for feature extraction and Euclidean distance
[33] 2014 Accuracy
for classifcation
[26] 2012 Sign modeling language (SML), animation engine N.A
11

3637, 2024, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1155/2024/1487500, Wiley Online Library on [24/09/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
12
Table 6: Techniques of sign language recognition using deep learning.
Study Year Techniques Evaluation metric
[153] 2023 DeepVision transformers Accuracy, precision
[154] 2023 8-Layer CNN Accuracy
[155] 2023 KNN Accuracy
[161] 2023 Attention-based Bi-LSTM Accuracy
[150] 2023 Deep learning (DL) combined with CNN and RNN Accuracy
[147] 2023 DNN Accuracy with [email protected]
[146] 2022 CNN Accuracy
[144] 2022 SVM Accuracy
Inaudible acoustic signal to estimate channel information and capture the sign
[143] 2022 Accuracy
language in real time
Hybrid convolutional neural network + bidirectional long short-term memory Peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), Fréchet
[162] 2022
(CNN + Bi-LSTM) inception distance (FID), temporal consistency metric (TCM)
[163] 2022 3D convolution net Accuracy
[156] 2022 CNN Accuracy
[157] 2022 CNN, DCGAN Accuracy
[164] 2022 VGG-19 PSNR, SSIM, FID, TCM
[141] 2021 SVM Accuracy, precision, recall, F1 score
[158] 2021 CNN Accuracy
[159] 2021 3DCNN Accuracy
[160] 2021 CNN, RNN Accuracy
[139] 2021 Spyder, TensorFlow, OpenCV, Keras Accuracy
[140] 2021 KNN Accuracy
[142] 2021 2D CNN, SVD, and LSTM Time recognition, accuracy
[125] 2020 3D CNN Siamese network Accuracy
[131] 2020 Conv3D Sentence error rate (SER), accuracy
[122] 2020 ResNet-D model Accuracy, time
[134] 2020 CNN Accuracy, precision, recall, F1 score
[135] 2020 Hidden Markov model (HMM) Accuracy
[123] 2020 CNN-LSTM-HMM Accuracy
[136] 2020 CNN Accuracy
[126] 2020 CNN Accuracy
[130] 2020 Stochastic multistate (SMS) WER
[124] 2020 CNN LSTM Accuracy, precision, recall, F1 measure
[116] 2019 CNN NA
[99] 2019 3D-ResNet, CTC WER
[97] 2019 Visual Geometry Group (VGG)-16, VGG-19 Accuracy
[94] 2019 CNN Accuracy
[93] 2019 Convolutional-based attention module (CBAM)-ResNet Accuracy
[86] 2019 Neural network and QuadroConvPoolNet Accuracy
[95] 2019 MLP, SVM, and CNN Accuracy
[106] 2019 ANN, SVM, HMM Accuracy
[117] 2019 CNN Accuracy
[114] 2019 CNN, LSTM Accuracy
[118] 2019 VGG-19 Recognition rate
Advances in Human-Computer Interaction

[80] 2018 CNN Accuracy

[85] 2018 Adaptive graph matching Accuracy, TWRF, FWRF
[81] 2018 Restricted Boltzmann machine Top-1 accuracy, Top-5 accuracy
[78] 2018 Inception v3 Accuracy
[69] 2018 RNN Accuracy
[76] 2018 LSTM Accuracy
[68] 2017 Dynamic vision sensor, CNN, RNN Accuracy
[64] 2017 3D signing avatar, Blender animation software Accuracy
[58] 2017 Nearest neighbor Accuracy
[63] 2017 CNN Accuracy
[59] 2017 Finite Legendre transform, linear discriminant analysis, KNN Accuracy
[55] 2017 LSTM Accuracy
[65] 2017 CNN Accuracy
[48] 2016 CNN Top-1 accuracy, Top-5 accuracy
[47] 2016 Hybrid-CNN HMM Accuracy
[3] 2016 Correlation classifcation algorithm Accuracy, precision, recall
[44] 2016 CNN Accuracy
[46] 2016 SVM Accuracy
[54] 2016 Maximum a posteriori (MAP) Accuracy
[43] 2015 Leap Motion Technology Accuracy
[36] 2015 CNN Accuracy
[34] 2014 ANN, vision-based Accuracy, MSE
[31] 2014 A skin and motion detector, hand detection using multiple proposals, chains model Accuracy
[29] 2014 KNN, cross-correlation Accuracy
[30] 2014 Deep belief network Precision, recall
[32] 2014 Microcontroller Precision, recall
[27] 2013 K-nearest neighbor Accuracy
Mean, standard deviation, aspect ratio hand cropping algorithm (ARHCA), no
[25] 2012 Multilayer perceptron
ARHCA
13

Table 7: (a) Datasets used in sign language recognition. (b) Links to publicly available dataset.
Study Year Dataset Remarks
(a)
[141] 2021 PSL dataset 37 alphabets
[165] 2021 ISLAN (Indian Sign Language) Collection of 700 sign images, and 24 sign videos
[139] 2021 SIBI dataset 8 static word signs. 19200 total images are included
[140] 2021 Custom made numbers from 1 to 5
(i) RKS-PERSIANSIGN: this dataset comprises 10,000
RGB videos showcasing 100 Persian sign words. Tese
videos are contributed by 10 individuals, including 5
women and 5 men, with 100 video samples available for
each Persian sign word
(ii) First-person: this dataset consists of 100,000 RGB-D
frames depicting 45 diferent hand action categories
performed with 26 distinct objects, capturing various hand
[142] 2021 RKS persiansign, frst-person, ASVID, isoGD
confgurations. Only the RGB sequences from the ASVID
dataset are used in this context
(iii) isoGD: this dataset contains a total of 47,933 RGB and
depth video samples across 249 class labels. For your
reference, only the RGB samples are utilized in this dataset.
It is further divided into three subdatasets, with 35,878
samples designated for training, 5,784 samples for
validation, and 6,271 samples for testing
[137] 2020 HamNoSys database 3000 words
Te dataset generated consists of 51 common word signs
[138] 2020 Chinese Sign Language from which 60 sentences were created. Instances of
sentences are 20400 from 34 volunteers
[127] 2020 Korean Sign Language 17 words used for training
Data augmentation is used to obtain a benchmark dataset
based on Chinese Sign Language (CSL). One dataset is
[128] 2020 China Sign Language
obtained from Kaggle and the other is built from 30-second
video frames
American Sign Language (ASL) and Bengali Sign A dataset is generated which contains 1000 data points for
[120] 2020
Language (BdSL) each of the letters of ASL and BdSL
Tis dataset has 25000 clips over 222 signers and covers
[132] 2020 MS-ASL dataset
1000 most frequently used ASL gestures
Tis dataset has 30 consonants and 6 vowels of BSL
[133] 2020 Bangla Sign Language characters. Te dataset holds 36 × 50 � 1800 images in total
as it has 50 samples for each sign
Te dataset has 301 videos with an average duration of
[129] 2020 German Sign Language
9 minutes
Advances in Human-Computer Interaction

is, 23 English alphabets, 0–10 digits, and 67 commonly used

[134] 2020 Indian Sign Language
words. Tere are 300 images of each instance totaling 35000
images
Te dataset has four unique word signs. Each sign has 50
[135] 2020 Custom made images with diferent positions and light levels. Te total
number of images is 1000
[123] 2020 German Sign Language Public dataset. RWTH-PHOENIX-weather 2014
(i) RKS-PERSIANSIGN:
(1) Contains: 10,000 RGB videos
(2) Content: 100 Persian words
(3) Contributors: 10 individuals
(4) Purpose: likely used for Persian sign language
recognition. Tis dataset provides video samples for
training and evaluating models for recognizing
(ii) Persian sign language gestures
First-person dataset:
(1) Contains: 100,000 RGB-D frames
(2) Content: 45 hand action categories for 26 diferent
RKS-PERSIANSIGN frst-person dataset NYU hand
[136] 2020 objects
pose dataset
(3) Purpose: this dataset seems focused on recognizing
hand actions related to interactions with various objects.
Te RGB-D frames can be used for training and evaluating
models capable of understanding hand-object interactions
(iii) NYU hand pose dataset:
(1) Contains: 81,009 image sequences
(2) Content: 36 joints
(3) Purpose: likely used for hand pose estimation. Tis
dataset provides a large number of image sequences
capturing various hand poses, which can be used to train
and test models for hand pose estimation tasks
15

Table 7: Continued.
Study Year Dataset Remarks
Public dataset
[126] 2020 Flemish Sign Language Te total samples are 18730 from 67 native signers with 100
classes
Te three gestures are feeling uncomfortable, seeing
[102] 2019 Te dataset contains three gestures
a doctor, and taking medicine
(i) Te ASL alphabet dataset contains 87,000 images. Te
sign language and static gesture recognition dataset
ASL alphabet dataset. Sign language and static
[110] 2019 contains 1,687 images
gesture recognition dataset
(ii) Te authors created their dataset from these two
datasets which contain 73,488 images
A total of 10 samples of each alphabet were taken for
[105] 2019 American Sign Language
accuracy
10 alphabets Alif, Ba, Ta, Kha, Dal, Dhad, Tah, Ghayn,
[103] 2019 Arabic Sign Language
Lam, and La. 2000 images used for training
26 letters A to Z
[104] 2019 British Sign Language Training performed on 520 samples (26 classes with 20
samples per class)
Custom made
(i) Word count: the dataset consists of a total of 1,440
infectional words
[109] 2019 Indonesian language infectional words (1) Training data: 954 infectional words
(2) Testing data: 486 infectional words
(ii) Data sources: the data were recorded by three teachers
from Santi Rama school for the hearing impaired in Jakarta
Two datasets: one is word-level (70 ASL words) and the
[91] 2019 ASL dataset
other is sentence-level (100 sentences)
[101] 2019 Arabic Sign Language Only 5 letters were taken for the experiment
5 volunteers to perform 26 alphabet signs with 30
[90] 2019 Custom-made repetitions. Tat is, 26 × 30 × 5 alphabet signs (3,900) in the
dataset
Swedish keyword signing targeted children with
[87] 2019 Swedish Sign Language signs dataset
communicative disorders
[115] 2019 Custom-made 40 signs fve times each totaling 200 for testing
Advances in Human-Computer Interaction

(v) Total pictures: the dataset contains a total of around

21,000 pictures
RWTH-PHOENIX-weather-2014
(i) Training set:
(1) Number of videos: 5,672
(2) Use: typically used to train machine learning or deep
learning models
(ii) Validation set:
[99] 2019 German Sign Language weather forecast program (1) Number of videos: 540
(2) Use: used during the model development process to
fne-tune hyperparameters and assess model performance
(iii) Test set:
(1) Number of videos: 629
(2) Use: reserved for evaluating the fnal model’s
performance and assessing its generalization to unseen data
Custom-made
[97] 2019 Ghanaian Sign Language Te dataset consists of 66000 images in RGB color with 33
classes of static gestures having 24 alphabets and 9 digits
Custom-made
Ten words were selected. A diferent number of videos were
[94] 2019 Korean Sign Language
selected from the Internet for each word. Te total no. of
videos is 421
Te authors selected 100 kinds of sign language words. Te
[93] 2019 CSL training set consists of 2964, the validation set has 1044,
and the testing set has 1005 videos
Custom-made
[119] 2019 ASL
26 alphabets
17

3637, 2024, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1155/2024/1487500, Wiley Online Library on [24/09/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
18
Table 7: Continued.
Study Year Dataset Remarks
ASL dataset: Massey University of researchers
Tis dataset consists of 2425 images from 5 individuals
RSL:
Custom-made
[86] 2019 ASL Russian Sign Language (RSL)
Te data for RSL are collected from fve YouTube videos.
Te total number of gestures in RSL is 33. Only the 26 static
gestures are taken and the rest of the dynamic gestures are
not included in this work
Te static sign language has 24 alphabets. J and Z are
excluded because they are dynamic. Also, it included and
[95] 2019 Custom made
captured from seven native and nonnative signers with
alike lighting
[106] 2019 ASL Tere are 6000 words in the ASL dictionary
Public dataset
Te dataset collected from Kaggle contains pictures of static
[117] 2019 ASL hand motions of ASL with 24 classes. Te database consists
of 47475 pictures from which 33000 (70%) pictures were
used in the training set and 1445 (30%) pictures for testing
LSA64 dataset Public dataset:
Te authors selected 30 gestures and 50 video streams for
[114] 2019 each gesture. After video processing, 90,000 images were
Argentinian Sign Language
created representing the sequence of dynamic gestures. Te
number of images for each category is 3000
A comprehensive collection of American Sign Language
(ASL) gestures representing 24 English letters (excluding
“Y” and “Z”). Tese gestures are captured in the form of
[6, 118] 2019 ASL expressive hand movements, providing a rich resource for
ASL recognition
Tese ASL gestures used Kinect technology with
contributions from 5 diferent individuals
Public dataset
ASLLVD, the American Sign Language lexicon video
dataset, features nearly 10,000 ASL signs by 6 native
[100] 2019 ASL signers. Te dataset focuses on 50 hand-picked ASL signs,
each signed by 6 diferent individuals, totaling 300 videos.
Tese videos include various angles, but our analysis
concentrated on front-view recordings
Custom-made
Te authors collected video data for 25 ASL signs from 100
[96] 2019 ASL
users where each sign was executed three times each. Te
total number of instances was 7500
Advances in Human-Computer Interaction

totals 30 for one alphabet. Tus, the total number of

instances is 780
Te dataset consists of 2500 images for alphabets and
[113] 2019 ISL dynamic words. Te authors augmented this dataset and
produced 5157 images
CSL Te authors have created a database of four tables to store
symbols with important descriptions. Tey have used
[88] 2019
ASL HamNoSys which consists of 200 symbols consisting of
hand shapes, hand orientation, location, and movements
Custom-made
Te study concentrates on static ASL gestures from A to Y,
omitting J and Z due to their dynamic nature. Te dataset
[112] 2019 ASL
comprises 24 gesture images captured with a smartphone
camera. Each gesture is represented by 200 images taken by
two users, accounting for a total of 4800 images
Custom-made
Te authors used Microsoft Kinect to record the video
stream dataset. It consists of 64 isolated vocabularies. Each
[92] 2019 Tai Sign Language (TSL)
word was performed by 8 nonnative TSL signers and each
signer acted 5 times for each word. Tus, there are
64 × 8 × 5 � 2560 video samples in total
Custom-made
[89] 2019 Brazilian Sign Language Authors recorded videos for 26 letters of the alphabet in
Libras with 13 users. Te total number of videos was 338
Alphabets A to Z and numbers 1 to 10 used in this
[74] 2018 Indonesian Sign Language
experiment
[70] 2018 Indonesian Sign Language Alphabets A to Z taken
Te open dataset given at Kaggle called sign language A set of 28 × 28 images representing the standard American
[84] 2018
MNIST Sign Language (ASL) alphabet, excluding J and Z
19

Table 7: Continued.
Study Year Dataset Remarks
22 gestures were taken out of 26 from French Sign
Language. 4 gestures, that is, J, P, Y, and Z, were left out
[82] 2018 French Sign Language because of their nonstatic nature. Each gesture was
performed by 57 participants. Te total dataset contains
1.25 million samples
Digits 0 to 9 and alphabets a to z were taken for the
[75] 2018 Indian Sign Language (ISL)
experiment
Digits 0 to 9 and alphabets a to z were taken for the
[79] 2018 Indian Sign Language (ISL)
experiment
[83] 2018 Custom built. Indian Sign Language 18 signs with each sign by 10 diferent signers recorded
(i) ISL dataset: used SVM for this dataset
Contains 4 signs, that is, A, B, C, and the word “Hello”
(ii) ASL dataset: used KNN for this dataset
Contains 10 ASL fngerspelling alphabets from a to i and k.
Te letter j is not included. Te total number of samples was
5254
Indian Sign Language
(iii) ISL: used CNN for this dataset
American Sign Language
[71] 2018 Te total dataset is 5000 samples for 200 signs done by fve
British Sign Language
Indian Sign Language users
Turkish Sign Language
(iv) Authors used ANN for the following 3 datasets
(v) ASL: consists of letters from A to Z
(vi) British Sign Language: contains alphabets from A to Z
(vii) Turkish Sign Language:
Consists of alphabets from A to Z. Te letters Q, W, and X
are excluded
LSA64 dataset: 10 subjects, 5 repetitions, 64 sign types, 3200
videos
[72] 2018 Argentinian Sign Language
RWTH-PHOENIX-weather database: 50 classes, 1297
training videos, 238 testing videos
Public dataset
[73] 2018 Tere are 900 pictures including 25 samples for each of 36
characters consisting of 26 letters and 10 digits
Custom-made
[77] 2018 ISL 200 sign language words. Each sign is performed by 5
diferent signers
Custom-made
[80] 2018 ISL A dataset of 5000 images and 100 images each for 50 most
commonly used words was created
Custom-made
[85] 2018 ISL
Te dataset consists of 200 words to form sentences
Advances in Human-Computer Interaction

Korean professional signers

Custom-made
Te dataset consists of SIBI words performed by 2 teachers
[76] 2018 SIBI (Sistem Isyarat Bahasa Indonesia) fuent in this language. It consists of 21 root words and 155
infectional words. Each word is recorded 5 times by each
teacher, thus resulting in a total of 1760 signs
Static gestures for the English alphabets from A to Z and
[60] 2017 Custom-made
digits from 0 to 9
Static gestures for the English alphabets from A to Z and
[62] 2017 Custom-made
digits from 0 to 9
Static gestures for the English alphabets from A to Z and
[2] 2017 Custom-made
digits from 0 to 9
Dataset: 1000 samples, 50 Indonesian sign words, 20
[67] 2017 Indonesian Sign Language
samples per sign, 500 for training, 500 for testing
Custom-made
[61] 2017 ISL
26 alphabets from A to Z and 12 basic words
Custom-made
[66] 2017 ISL
18 diferent words were included in the dataset
[56] 2017 Ubicomp.eti.uni 18 diferent words were included in the dataset
[57] 2017 ASL 103 signs
Te dataset has a total of 720 images (30 for every ASL sign
[68] 2017 ASL image). Te dataset consists of alphabets from A to Y. Te
letters J and Z are excluded
Custom-made
[64] 2017 Sinhala Sign Language (SSL) Te dataset consists of 61 SSL fngerspelling signs (words)
and 40 SSL number signs
21

3637, 2024, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1155/2024/1487500, Wiley Online Library on [24/09/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
22
Table 7: Continued.
Study Year Dataset Remarks
Custom-made
[58] 2017 Greek Sign Language 5 participants (2 male, 3 female) learned and performed 15
signs, four times each, totaling 300 evaluation samples
Custom-made
[63] 2017 Korean Sign Language 30 diferent gestures are included in this dataset. Te
training data are 72% and the testing data are 28%
[59] 2017 Tai Sign Language (TSL) Te dataset consists of 10 words. Each word has 10 samples
Public dataset
NGT (Nederlandse Gebarentaal) sign language of the
[55] 2017 Te dataset consists of 40 glosses (words) taken from the
Netherlands
NGT dataset
Custom-made
[65] 2017 ASL Te dataset consists of 25 images from 5 people for each
alphabet and digits 0–9
[50] 2016 ASL 16 alphabets taken for training and testing
[49] 2016 Indonesian Sign Language 24 gestures from A to Y excluding J and Z
Custom-made
[45] 2016 ASL
Te dataset consists of 20 ASL signs
Public dataset:
Tis dataset consists of 588 signs which include 10 numbers
[51] 2016 Arabic Sign Language (ArSL)
from 0 to 9, 28 alphabets, and diferent categories like
family, job, colors, and sports
6 alphabets from A to F with 20 samples for each letter
[3] 2016 Pakistan Sign Language
collected
[52] 2016 Continuous sign language 18 signs with each sign by 10 diferent signers recorded
(i) Danish Sign Language: this dataset consists of 2,149
Danish Sign Language
signs
(ii) New Zealand Sign Language: this dataset consists of
[48] 2016 New Zealand Sign Language
4,155 signs
(iii) RWTH-PHOENIX-weather 2014: this dataset consists
RWTH-PHOENIX-weather 2014
of 65,227 signs
RWTH-PHOENIX-weather 2012
RWTH-PHOENIX-weather multisigner 2014
Tis dataset consists of 65,227 signs
[47] 2016 German Sign Language SIGNUM single signer:
Tis dataset consists of 450 basic signs. Isolated signs are
450 and continuous sentences are 780. Te total number of
images is 5,970,450
American Sign Language image dataset (ASLID) Public datasets
[44] 2016 American Sign Language lexicon video dataset Training set: 808 ASLID images from six signers. Test set:
(ASLLVD) 479 ASLID images from two signers
Advances in Human-Computer Interaction

[37] 2015 Indonesian Sign Language Alphabets A to Z

[40] 2015 ASL Only the letters A to Z are included for testing
[43] 2015 Greek Sign Language Greek Sign Language alphabets
Public dataset
RWTH-PHOENIX-weather corpus:
[36] 2015 German Sign Language (DGS)
Dataset: 2137 sentence segments, 14717 gloss annotations,
189,363 frames
Hand gesture image database
[28] 2014 Custom built Te test dataset was prepared by four persons each of
whom showed 19 signs with three rotation variations
[33] 2014 PSL 300 samples taken from 30 individuals with 10 signs each
Custom-made
[34] 2014 PSL Tis dataset consists of 500 images of 37 alphabets. 426
images were utilized for training and 74 for testing
Te number of one-handed videos and frames is 42 and 902
Te number of two-handed videos and frames is 48 and
1337
Dataset DS2:
Te number of one-handed videos and frames is 42 and
1276
[31] 2014 Dataset DS1 Te number of two-handed videos and frames is 48 and
1945
Dataset DS3:
Te number of one-handed videos and frames is 42 and
1197
Te number of two-handed videos and frames is 48 and
1735
23

3637, 2024, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1155/2024/1487500, Wiley Online Library on [24/09/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
24
Table 7: Continued.
Study Year Dataset Remarks
Custom-made
[29] 2014 PSL Te dataset consists of 37 alphabets. 6 samples are recorded
for each alphabet
Custom-made dataset
[30] 2014 ASL Dataset: 24 static letters signed by 5 individuals, 60,000
images
Te sign-to-letter translation by using a hand glove,
[32] 2014 ArSL
microcontroller, and display unit
Custom-made
[27] 2013 Tai Sign Language (TSL) Te dataset consists of 42 TSL alphabets. Several videos are
taken for each alphabet
Custom-built
[26] 2012 A word is an input to the smartphone which is converted to
video animation
Custom-made
Te dataset consists of two sets. One is the vowel set which
[25] 2012 Brazilian Sign Language (Libras)
is A, E, I, O, and U. Te other set has the set which has B, C,
F, L, and V
Name Link (access date 25-August-2023)
(b)
PSL https://data.mendeley.com/datasets/y9svrbh27n/1
https://guiggh.github.io/publications/frst-person-
First-person
hands/
Purdue https://engineering.purdue.edu/RVL/Database/
RVL-SLLL ASL/asl-database-front.htm
Corpus NGT https://www.ru.nl/en/cls/research
http://www.cbsr.ia.ac.cn/users/jwan/database/
isoGD
isogd.html
https://www.phonetik.uni-muenchen.de/
SIGNUM
forschung/Bas/SIGNUM/
WLASL https://dxli94.github.io/WLASL/
http://vlm1.uta.edu/~srujana/ASLID/ASL_Image_
ASLID
Dataset.html
German Sign https://www-i6.informatik.rwth-aachen.de/
Language ~koller/RWTH-PHOENIX/
Danish Sign
https://www.tegnsprog.dk/
Language
ArSL https://menasy.com/
Advances in Human-Computer Interaction

Table 7: Continued.
Study Year Dataset Remarks
How2Sign https://how2sign.github.io/
GSL dataset https://vcl.iti.gr/dataset/gsl/
https://chalearnlap.cvc.uab.cat/dataset/40/
AUTSL
description/
LSA64 https://facundoq.github.io/datasets/lsa64/
Ubicomp https://ubicomp.eti.uni-siegen.de/home/datasets/
ASL fnger https://www.kaggle.com/datasets/mrgeislinger/
spelling asl-rgb-depth-fngerspelling-spelling-it-out
Sign language https://www.kaggle.com/datasets/datamunge/
MNIST sign-language-mnist
Indian Sign https://data.mendeley.com/datasets/rc349j45m5/1
Language doi: 10.17632/rc349j45m5.1
25

Table 8: Sign languages targeted.

Sign language Study
[30, 40, 44, 45, 50, 57, 65, 68, 71, 78, 81, 84, 86,
American Sign Language
88, 91, 96, 98, 100, 105, 106, 110, 112, 117–120, 132]
Arabic Sign Language [51, 101, 103]
Argentinian Sign Language [114]
Bangla Sign Language [120, 133]
Brazilian Sign Language [25, 89]
British Sign Language [71, 104]
China Sign Language [88, 93, 128, 138]
Croatian Sign Language [131]
Danish Sign Language [48]
Flemish Sign Language [126]
French Sign Language [82]
German Sign Language [36, 47, 99, 123, 129]
Ghanaian Sign Language [97]
Greek Sign Language [43, 46, 58]
Hong Kong Sign Language [122]
Indian Sign Language [61, 66, 71, 75, 77, 80, 83, 85, 107, 113, 134]
Indonesian Sign Language [37, 49, 67, 70, 74, 76, 108, 139]
Korean Sign Language [54, 63, 69, 94, 127]
Malaysian Sign Language [42]
New Zealand Sign Language [48]
Pakistan Sign Language [3, 29, 33, 34, 116, 141]
Persian Sign Language
Russian Sign Language [86]
Sign language of Netherlands [55]
Sinhala Sign Language (SSL) [64]
South African Sign Language [41]
Tai Sign Language [27, 59, 92]
Turkish Sign Language [71]

Te contribution of publishers has been analyzed based

on the selected publications. While it is evident that each
publisher has made substantial contributions to research in
the feld of accessibility for individuals with speech disor-
ders, it is noteworthy that a majority of the selected papers in
this paper were published in IEEE journals and conferences,
as illustrated in Figure 6.
Moreover, the most highly cited paper among the se-
lected publications has been identifed. Te paper with the
highest number of citations was authored by Cheok et al. in
2019, titled “A review of hand gesture and sign language
Figure 4: Evaluation metrics by frequency used in diferent recognition techniques [14].” As of the latest available data, it
research. has accumulated 456 citations, as illustrated in Figure 7.
Similarly, the analysis of the selected literature for this
paper has been conducted with a focus on country-wise
4. Meta-Analysis contributions. In terms of country-wise contributions, India
stands out as a signifcant contributor to publications related
Tis section ofers a multilayered examination of the col- to speech disorders, as depicted in Figure 8. Te
lected literature, exploring various dimensions, including United States follows as the second most prominent con-
publisher contributions, contributions by country, and ci- tributing country.
tation analysis. Numerous approaches have been thoroughly
tested and validated on specifc sign languages, as extensively 5. Open Research Questions
discussed earlier. For instance, Figure 5 provides a com-
parative analysis of various studies on American Sign Tis section explores the potential open research questions
Language (ASL) along with the achieved accuracy levels. It is and challenges that currently exist. While the advancing
important to note that the accuracy of these approaches and hardware and software capabilities of smartphones are no
models is contingent upon the complexity and variability of longer a computational constraint, the multifaceted nature
signs within a specifc sign language. of sign languages, each with its diverse set of gestures,
3637, 2024, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1155/2024/1487500, Wiley Online Library on [24/09/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
Advances in Human-Computer Interaction 27

Table 9: Models and their evaluation performance on specifc sign languages.

Study Year Model Sign language Results/performance
[154] 2023 8-layer CNN ISL 99.34% accuracy
[156] 2022 CNN ISL 70.0% accuracy
Top-1 (95.99%) accuracy
[160] 2021 CNN and RNN ISL
Top-3 (99.46%) accuracy
[158] 2021 CNN ASL 87.5% accuracy
[143] 2022 Built-in speakers and microphones, inaudible acoustic signal ASL 97.2% accuracy at word-level
[166] 2021 AutoML ASL 100% accuracy
[159] 2021 3DCNN KSL 91.0% accuracy
[51] 2016 Cloud computing-based approach ArSL 77%–84% for short sentences
[103] 2019 SVM ArSL 92.5% accuracy
[49] 2016 Backpropagation neural network Indonesian SL 91.66% accuracy
[76] 2018 2-Layer LSTM Indonesian SL 95.15% accuracy
[70] 2018 CNN Indonesian SL 100% accuracy

Comparative analysis of various studies on ASL and their achievement in

terms of accuracy (%)
97.2 100
100
87.5
90
80
70
Accurcay Achived

60
50
40
30
20
10
0 0 0 0 0 0
0
[158] [143] [161]
Studies on ASL
Figure 5: Comparative analysis of various studies in terms of accuracy.

Publisher’s Contributions

0 10 20 30 40 50 60 70
Number of Papers per Publishers

Other Elsevier
Springer IEEE
MDPI ACM
Figure 6: Number of studies based on publisher’s contributions.
3637, 2024, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1155/2024/1487500, Wiley Online Library on [24/09/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
28 Advances in Human-Computer Interaction

Top 10 most cited papers

Using Deep Convolutional Networks for Gesture

128
Recognition in American Sign Language
A review on systems-based sensory gloves for sign
188
language recognition state of the art between 2007 and 2017
A New Data Glove Approach for Malaysian Sign
130
Language Detection
Deep Learning of Mouth Shapes for Sign Language 103
Sign Language Recognition, Generation, and
298
Translation: An Interdisciplinary Perspective
Image-Based and Sensor-Based Approaches to
126
Arabic Sign Language Recognition
Deep Sign Hybrid CNN-HMM for Continuous Sign
201
Language Recognition
A review of hand gesture and sign language recognition techniques 456
Deep Hand How to Train a CNN on 1 Million Hand Images
307
When Your Data Is Continuous and Weakly Labelled
American sign language recognition with the kinect 432

0 50 100 150 200 250 300 350 400 450 500

Figure 7: Top ten most cited papers.

Top 10 country wise contributions

Greece 4
Brazil 4
Korea 5
Germany 5
Malaysia 7
Pakistan 9
China 10
Indonesia 11
USA 15
India 29
0 5 10 15 20 25 30 35
Figure 8: Number of papers country-wise.

continues to present signifcant challenges. Moreover, the and robustness of sign language detection and interpretation
challenges also include social acceptability and pervasiveness on smartphones to ensure reliable and real-time communi-
at low cost. Besides, the reliance on sign language(s) and its cation for users?” Tis is because it involves real-time image
translation for individuals sufering from speech disorders processing and source constraints, such as processing and
has unique challenges that need proper investigation, for storage [148]. Delays in processing with false positive re-
example, compatibility issues, multilingual translation, ed- sponses may further increase frustration for speech-disabled
ucation level, real-time gesture generation, and translation. people. While smartphones are portable, the input of ges-
Te following subsection provides an in-depth elaboration tures on smartphones may require specifc tools or the
of the most salient issues and challenges identifed in the presence of an individual to operate the smartphone’s
existing literature. camera for individuals with disabilities. Without these
provisions, there is a risk of improper gesture input and
consequently an increased chance of errors.
5.1. Accuracy, Robustness, and Real-Time Detection. Te
accuracy of real-time translation of sign language is chal-
lenging due to various factors, such as light conditions, 5.2. Multilingual Support. Every region of the world has its
power consumption, social acceptability, and privacy con- own sign language for its speech-disabled people. Tis makes
straints. Te question is “How can we improve the accuracy it difcult to translate one sign language to another and hence
3637, 2024, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1155/2024/1487500, Wiley Online Library on [24/09/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
Advances in Human-Computer Interaction 29

the scope becomes narrow [148]. Te question of “What academia and practitioners can focus on one or more of these
techniques can be developed to support multiple sign languages questions to contribute to the development of innovative,
on smartphones, accommodating diverse user needs?” still low-cost, socially acceptable, and efective solutions.
exists. Furthermore, there is a pressing need to establish
a universal standard for sign language. Such a standardized 6. Conclusion
language could facilitate the development of universal smart
devices, ultimately leading to a reduction in the overall cost of Te detection and interpretation of sign language for people
equipment designed for these purposes. with speech disorders, utilizing cost-efective of-the-shelf
devices, particularly smartphones, has gained substantial
attention within the research and academic communities.
5.3. Gesture Recognition. As mentioned, the sign languages Using a smartphone for accessibility solutions is not an
are detected via sensor (hardware approach) or by vision exception due to its growing capabilities in terms of pro-
approach. Te sensor approaches, i.e., gloves or other cessing, mobility, storage capacity, and social acceptability.
wearable devices, are not socially acceptable and hence rarely Tis paper presented a systematic literature review (SLR) on
used by speech-disordered people. In the vision-based ap- sign language detection and interpretation using pervasive
proach, we have image processing, which itself requires lots and ubiquitous computing devices, such as smartphones.
of energy, power, and storage [167]. Te question “How can Te objective is to comprehensively analyze the progress
machine learning algorithms be optimized to recognize a wide achieved thus far in the machine and deep learning ap-
range of sign language gestures and expressions accurately?” is proaches using smartphones. Moreover, to analyze the
yet to be answered. One reason may be that machine and approaches employed in enhancing accessibility for in-
deep learning algorithms are resource-intensive, and hence dividuals with speech disorders, it is important to gather
little attention is given to smartphones. Terefore, existing insights regarding the recent machine and deep learning
machine and deep learning algorithms require proper op- approaches, available datasets, evaluation metrics, and
timization for smartphones. current research and emerging trends. In this connection,
this paper is intended to provide valuable insights for re-
5.4. Data Privacy and Security. Privacy is everyone’s right and searchers and practitioners engaged in accessibility initia-
also for people with special needs including the visually im- tives, particularly in the domain of speech disorders. Tis
paired [168, 169] and people sufering from speech disorders. study highlighted the most valuable literature published
Te sign language talking patterns are vulnerable due to pro- from 2012 to July 2023. Moreover, it highlighted a detailed
cessing by a machine [170]. Moreover, the sign language talking yet comprehensive literature, datasets, and numerous ma-
in public may lead to privacy breaches. Terefore, the following chine and deep learning approaches used on smartphones.
question arises: “What measures can be implemented to ensure Te paper specifcally focuses on the detection and in-
the privacy and security of sign language data transmitted and terpretation of sign languages via smartphones. Tis study
processed on smartphones?” Tis question needs proper at- suggests that the development of a universal sign language
tention. Te messages in digital form have numerous security could greatly beneft both practitioners and developers in
issues, such as chat leakages and hacking, among others. As this feld since it may mitigate the overhead costs associated
a case study, some attempts have been made by Michigan with learning, detecting, and translating multiple sign lan-
State University (https://msutoday.msu.edu/news/2019/new- guages. Moreover, the focus should be on socially acceptable
technology-breaks-through-sign-language-barriers) to address devices instead of expensive or complex wearable devices.
numerous pressing issues. However, more work is needed in Tis review paper may serve as a valuable contribution to the
this domain to ensure that sign language interpretation is risk- existing body of knowledge and is expected to ofer
free. Proper encryption/decryption by the machine (used for a roadmap for future research in the domain of accessibility,
translation) could also improve privacy issues. specifcally for speech-disabled individuals. Future work can
be carried out in diferent areas, such as real-time accurate
translation by smartphones, preserving privacy during
5.5. Low-Light and Noisy Environments. Image processing in translation, and accurate gesture recognition in low-light
low light generates false positives, which directly afect the conditions.
performance and results [171, 172]. Te question “How can
sign language detection systems on smartphones perform Data Availability
efectively in low-light conditions and noisy environments?”
still exists. Moreover, due to battery constraints, smart- Te collected data (in an Excel sheet) will be provided upon
phones have limited battery life, which tends to deplete request. Most of the basic statistics regarding the systematic
rapidly during image processing activities under low-light literature review are discussed within the paper.
conditions. Te machine and deep learning application(s)
may further contribute to battery depletion. Disclosure
Tese research questions include various aspects of sign
language(s) detection on smartphones and ofer opportunities Tis study was conducted at the Department of Computer
to advance this feld to better serve the needs of individuals Science, City University of Science and Information Tech-
with hearing and speech disability problems. Researchers/ nology, Peshawar, Pakistan.
3637, 2024, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1155/2024/1487500, Wiley Online Library on [24/09/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
30 Advances in Human-Computer Interaction

Conflicts of Interest IEEE Reviews in Biomedical Engineering, vol. 14, pp. 82–97,
2021.
Te authors declare that there are no conficts of interest [14] M. J. Cheok, Z. Omar, M. H. Jaward, and Cybernetics, “A
regarding the publication of this paper. review of hand gesture and sign language recognition
techniques,” International Journal of Machine Learning and
Cybernetics, vol. 10, no. 1, pp. 131–153, 2019.
References [15] S. M. Kamal, Y. Chen, S. Li, X. Shi, and J. Zheng, “Technical
approaches to Chinese sign language processing: a review,”
[1] W. H. O., “Hearing loss,” 2021, https://www.who.int/health- IEEE Access, vol. 7, pp. 96926–96935, 2019.
topics/hearing-loss. [16] M. A. Ahmed, B. B. Zaidan, A. A. Zaidan, M. M. Salih, and
[2] Z. A. Memon, M. U. Ahmed, S. T. Hussain, Z. A. Baig, and M. M. B. Lakulu, “A review on systems-based sensory gloves
U. Aziz, “Real time translator for sign languages,” in Pro- for sign language recognition state of the art between 2007
ceedings of the 2017 International Conference on Frontiers of and 2017,” Sensors, vol. 18, no. 7, p. 2208, 2018.
Information Technology (FIT), pp. 144–148, Bremen, Ger- [17] K. Ayadi, Y. O. ElHadj, and A. Ferchichi, “Automatic
many, June 2017. translation from Arabic to Arabic Sign Language: a review,”
[3] N. Raziq and S. Latif, “Pakistan sign language recognition in Proceedings of the 2018 JCCO Joint International Con-
and translation system using leap motion device,” in In- ference on ICT in Education and Training, International
ternational Conference on P2P, Parallel, Grid, Cloud and Conference on Computing in Arabic, and International
Internet Computing, pp. 895–902, Springer, Cham, Swit- Conference on Geocomputing (JCCO: TICET-ICCA-GECO),
zerland, 2016. pp. 1–5, Hammamet, Tunisia, November 2018.
[4] AI-Media, “Sign Language alphabets from around the [18] W. Suharjito, G. Kusuma, and A. Zahra, “Feature Extraction
world,” 2021, https://www.ai-media.tv/sign-language- methods in sign language recognition system: a literature
alphabets-from-around-the-world/. review,” in Proceedings of the 1st 2018 Indonesian association
[5] R. Sreemathy, M. Turuk, S. Chaudhary, K. Lavate, A. Ushire, for pattern recognition international conference (INAPR),
and S. Khurana, “Continuous word level sign language pp. 11–15, Jakarta, Indonesia, January 2019.
recognition using an expert system based on machine [19] D. H. Neiva and C. Zanchettin, “Gesture recognition: a re-
learning,” International Journal of Cognitive Computing in view focusing on sign language in a mobile context,” Expert
Engineering, vol. 4, pp. 170–178, 2023. Systems with Applications, vol. 103, pp. 159–183, 2018.
[6] Z. Zafrulla, H. Brashear, T. Starner, H. Hamilton, and [20] S. Shivashankara and S. Srinath, “A review on vision based
P. Presti, “American sign language recognition with the American sign language recognition, its techniques, and
kinect,” in Proceedings of the 13th international conference on outcomes,” in Proceedings of the 2017 7th International
multimodal interfaces, pp. 279–286, Boston, MA, USA, June Conference on Communication Systems and Network Tech-
2011. nologies (CSNT), pp. 293–299, Nagpur, India, November
[7] S. Ghanem, C. Conly, and V. Athitsos, “A survey on sign 2017.
language recognition using smartphones,” in Proceedings of [21] A. Er-Rady, R. Faizi, R. O. H. Tami, and H. Housni,
the 10th International Conference on PErvasive Technologies “Automatic sign language recognition: a survey,” in Pro-
Related to Assistive Environments, pp. 171–176, Island of ceedings of the 2017 International Conference on Advanced
Rhodes, Greece, June 2017. Technologies for Signal and Image Processing (ATSIP),
[8] S. Sharma and S. Singh, “Vision-based sign language rec- pp. 1–7, Fez, Morocco, May 2017.
ognition system: a Comprehensive Review,” in Proceedings of [22] S. Suharjito, R. Anderson, F. Wiryana, M. C. Ariesta, and
the 2020 International Conference on Inventive Computation G. P. Kusuma, “Sign language recognition application sys-
Technologies (ICICT), pp. 140–144, Coimbatore, India, tems for deaf-mute people: a review based on input-process-
February 2020. output,” Procedia Computer Science, vol. 116, pp. 441–448,
[9] A. Ardiansyah, B. Hitoyoshi, M. Halim, N. Hanafah, and 2017.
A. J. P. C. S. Wibisurya, “Systematic literature review,” [23] B. Kitchenham, O. Pearl Brereton, D. Budgen, M. Turner,
American Sign Language Translator, vol. 179, pp. 541–549, J. Bailey, and S. J. I. Linkman, “Systematic literature reviews
2021. in software engineering–a systematic literature review,”
[10] K. Nimisha and A. Jacob, “A brief review of the recent trends Information and Software Technology, vol. 51, no. 1, pp. 7–15,
in Sign Language Recognition,” in Proceedings of the 2020 2009.
International Conference on Communication and Signal [24] D. Moher, A. Liberati, J. Tetzlaf, D. G. Altman, and
Processing (ICCSP), pp. 186–190, Chennai, India, July 2020. P. Group, “Preferred reporting items for systematic reviews
[11] K. Sohelrana, S. F. Ahmed, S. Sameer, and O. Ashok, “A and meta-analyses: the PRISMA statement,” Annals of In-
review on smart gloves to convert sign to speech for mute ternal Medicine, vol. 151, no. 4, pp. 264–269, 2009.
community,” in Proceedings of the 2020 8th International [25] M. dos Santos Anjo, E. B. Pizzolato, and S. Feuerstack, “A
Conference on Reliability, Infocom Technologies and Opti- real-time system to recognize static gestures of Brazilian sign
mization (Trends and Future Directions)(ICRITO), language (libras) alphabet using Kinect,” in Proceedings of the
pp. 1262–1264, Noida, India, June 2020. 11th Brazilian Symposium on Human Factors in Computing
[12] J. Galván-Ruiz, C. M. Travieso-González, A. Tejera-Fett- Systems, pp. 259–268, Cuiaba, Brazil, November 2012.
milch, A. Pinan-Roescher, L. Esteban-Hernández, and [26] M. Boulares and M. Jemni, “Mobile sign language translation
L. Domı́nguez-Quintana, “Perspective and evolution of system for deaf community,” in Proceedings of the in-
gesture recognition for sign language: a review,” Sensors, ternational cross-disciplinary conference on web accessibility,
vol. 20, no. 12, p. 3571, 2020. pp. 1–4, Lyon, France, April 2012.
[13] K. Kudrinko, E. Flavin, X. Zhu, and Q. Li, “Wearable sensor- [27] T. Suksil and T. H. Chalidabhongse, “Hand detection and
based Sign Language Recognition: a comprehensive review,” feature extraction for static Tai Sign Language recognition,”
3637, 2024, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1155/2024/1487500, Wiley Online Library on [24/09/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
Advances in Human-Computer Interaction 31

in Proceedings of the 7th International Conference on [41] M. Seymour and M. Tšoeu, “A mobile application for South
Ubiquitous Information Management and Communication, African sign language (SASL) recognition,” AFRICON,
pp. 1–6, Kota Kinabalu, Malaysia, June 2013. vol. 2015, pp. 1–5, 2015.
[28] P. Lukas, O. Yuji, M. Yoshihiko, and I. Hiroshi, “A HOG- [42] A. Z. Shukor, M. F. Miskon, M. H. Jamaluddin, F. B. Ali@
based hand gesture recognition system on a mobile device,” Ibrahim, M. F. Asyraf, and M. B. B. Bahar, “A new data glove
in Proceedings of the Image Processing (ICIP) IEEE In- approach for Malaysian sign language detection,” Procedia
ternational Conference on IEEE, Paris, France, October 2014. Computer Science, vol. 76, pp. 60–67, 2015.
[29] M. Sami, H. Ahmed, A. Wahid, U. Siraj, F. Ahmed, and [43] D. Kotsidou, C. Angelis, S. Dragoumanos, and
S. Shahid, “Pose recognition using cross correlation for static A. Kakarountas, “Computer assisted gesture recognition for
images of Urdu sign language,” in Proceedings of the 2014 the Greek sign language/fngerspelling,” in Proceedings of the
International Conference on Robotics and Emerging Allied 19th Panhellenic Conference on Informatics, pp. 241-242,
Technologies in Engineering (iCREATE), pp. 200–204, Athens, Greece, October 2015.
Islamabad, Pakistan, April 2014. [44] S. Gattupalli, A. Ghaderi, and V. Athitsos, “Evaluation of
[30] L. Rioux-Maldague and P. Giguere, “Sign language fnger- deep learning based pose estimation for sign language rec-
spelling classifcation from depth and color images using ognition,” in Proceedings of the 9th ACM international
a deep belief network,” in Proceedings 2014 Canadian conference on pervasive technologies related to assistive en-
Conference on Computer and Robot Vision, pp. 92–97, vironments, pp. 1–7, New York, NY, USA, February 2016.
Montreal, Canada, May 2014. [45] P. Paudyal, A. Banerjee, and S. K. Gupta, “Sceptre: a per-
[31] Z. Zhang, C. Conly, and V. Athitsos, “Hand detection on sign vasive, non-invasive, and programmable gesture recognition
language videos,” in Proceedings of the 7th International technology,” in Proceedings of the 21st International Con-
Conference on PErvasive Technologies Related to Assistive ference on Intelligent User Interfaces, pp. 282–293, New York,
Environments, pp. 1–5, Cuiaba, Brazil, November 2014. NY, USA, June 2016.
[32] H. El Hayek, J. Nacouzi, A. Kassem, M. Hamad, and S. El- [46] M. Simos and N. Nikolaidis, “Greek sign language alphabet
Murr, “Sign to letter translator system using a hand glove,” in recognition using the leap motion device,” in Proceedings of
Proceedingsa of the Te Tird International Conference on e- the 9th Hellenic Conference on Artifcial Intelligence, pp. 1–4,
Technologies and Networks for Development (ICeND2014), Athens, Greece, August 2016.
pp. 146–150, Beirut, Lebanon, December 2014. [47] O. Koller, O. Zargaran, H. Ney, and R. Bowden, “Deep sign:
[33] K. Kanwal, S. Abdullah, Y. B. Ahmed, Y. Saher, and
hybrid CNN-HMM for continuous sign language recogni-
A. R. Jafri, “Assistive glove for Pakistani Sign Language
tion,” in Proceedings of the British Machine Vision Conference
translation,” in Proceedings of the 17th IEEE International
2016, New York, NY, USA, December 2016.
Multi Topic Conference, pp. 173–176, Karachi, Pakistan,
[48] O. Koller, H. Ney, and R. Bowden, “Deep hand: how to train
December 2014.
a cnn on 1 million hand images when your data is continuous
[34] N. S. Khan, A. Shahzada, S. Ata, A. Abid, M. S. Farooq, and
and weakly labelled,” in Proceedings of the IEEE conference on
M. T. Mushtaq, “A vision based approach for Pakistan sign
computer vision and pattern recognition, pp. 3793–3802, Las
language alphabets recognition,” Pensee Journal, vol. 76,
Vegas, NV, USA, June 2016.
no. 3, 2014.
[49] R. Hartanto and A. Kartikasari, “Android based real-time
[35] H. Elleuch, A. Wali, A. Samet, and A. M. Alimi, “A static
hand gesture recognition system for real time mobile device static Indonesian sign language recognition system pro-
monitoring,” in Proceedings of the 2015 15th international totype,” in Proceedings of the 2016 8th International Con-
conference on intelligent systems design and applications ference on Information Technology and Electrical Engineering
(ISDA), pp. 195–200, Marrakech, Morocco, December 2015. (ICITEE), pp. 1–6, Yogyakarta, Indonesia, October 2016.
[36] O. Koller, H. Ney, and R. Bowden, “Deep learning of mouth [50] C. M. Jin, Z. Omar, and M. H. Jaward, “A mobile application
shapes for sign language,” in Proceedings of the IEEE In- of American sign language translation via image processing
ternational Conference on Computer Vision Workshops, algorithms,” in Proceedings of the 2016 IEEE Region 10
pp. 85–91, Santiago, Chile, December 2015. Symposium (TENSYMP), pp. 104–109, Bali, Indonesia, May
[37] R. Y. Hakkun and A. Baharuddin, “Sign language learning 2016.
based on android for deaf and speech impaired people,” in [51] M. M. El-Gayyar, A. S. Ibrahim, and M. Wahed, “Translation
Proceedings of the 2015 International Electronics Symposium from Arabic speech to Arabic Sign Language based on cloud
(IES), pp. 114–117, Surabaya, Indonesia, September 2015. computing,” Egyptian Informatics Journal, vol. 17, no. 3,
[38] H. Lahiani, M. Elleuch, and M. Kherallah, “Real time hand pp. 295–303, 2016.
gesture recognition system for android devices,” in Pro- [52] G. Ananth Rao and P. Kishore, “Sign Language recognition
ceedings of the 2015 15th International Conference on In- system simulated for video captured with smart phone front
telligent Systems Design and Applications (ISDA), camera,” International Journal of Electrical and Computer
pp. 591–596, Marrakech, Morocco, December 2015. Engineering, vol. 6, no. 5, p. 2176, 2016.
[39] L.-J. Kau, W.-L. Su, P.-J. Yu, and S.-J. Wei, “A real-time [53] H. Lahiani, M. Elleuch, and M. Kherallah, “Real time static
portable sign language translation system,” in Proceedings of hand gesture recognition system for mobile devices,” Journal
the 2015 IEEE 58th International Midwest Symposium on of Information Assurance Security, vol. 11, 2016.
Circuits and Systems (MWSCAS), pp. 1–4, Fort Collins, CO, [54] K.-W. Kim, M.-S. Lee, B.-R. Soon, M.-H. Ryu, and J.-N. Kim,
USA, August 2015. “Recognition of sign language with an inertial sensor-based
[40] M. Elmahgiubi, M. Ennajar, N. Drawil, and M. S. Elbuni, data glove,” Technology and Health Care, vol. 24, no. s1,
“Sign language translator and gesture recognition,” in Pro- pp. S223–S230, 2015.
ceedings of the 2015 Global Summit on Computer & In- [55] B. Mocialov, G. Turner, K. Lohan, and H. Hastie, “Towards
formation Technology (GSCIT), pp. 1–6, Sousse, Tunisia, June continuous sign language recognition with deep learning,” in
2015. Proceedings of the Workshop on the Creating Meaning With
3637, 2024, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1155/2024/1487500, Wiley Online Library on [24/09/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
32 Advances in Human-Computer Interaction

Robot Assistants: Te Gap Left by Smart Devices, Las Vegas, [69] S.-K. Ko, J. G. Son, and H. Jung, “Sign language recognition
NV, USA, August 2017. with recurrent neural network using human keypoint de-
[56] C. K. Mummadi, F. P. P. Leo, K. D. Verma, S. Kasireddy, tection,” in Proceedings of the 2018 Conference on Research in
P. M. Scholl, and K. Van Laerhoven, “Real-time embedded Adaptive and Convergent Systems, pp. 326–328, Honolulu,
recognition of sign language alphabet fngerspelling in an Hawaii, October 2018.
IMU-based glove,” in Proceedings of the 4th international [70] P. Yugopuspito, I. M. Murwantara, and J. Sean, “Mobile sign
Workshop on Sensor-based Activity Recognition and In- language recognition for bahasa Indonesia using convolu-
teraction, pp. 1–6, Rostock, Germany, September 2017. tional neural network,” in Proceedings of the 16th In-
[57] Q. Dai, J. Hou, P. Yang, X. Li, F. Wang, and X. Zhang, “Te ternational Conference on Advances in Mobile Computing
sound of silence: end-to-end sign language recognition using and Multimedia, pp. 84–91, Yogyakarta, Indonesia, No-
smartwatch,” in Proceedings of the 23rd Annual International vember 2018.
Conference on Mobile Computing and Networking, pp. 462– [71] R. A. A. R. Agha, M. N. Sefer, and P. Fattah, “A compre-
464, Snowbird, UT, USA, October 2017. hensive study on sign languages recognition systems using
[58] N. Gkigkelos and C. Goumopoulos, “Greek sign language (SVM, KNN, CNN and ANN),” in Proceedings of the First
vocabulary recognition using Kinect,” in Proceedings of the International Conference on Data Science, pp. 1–6, Madrid,
21st Pan-Hellenic Conference on Informatics, pp. 1–6, Larissa, Spain, October 2018.
Greece, September 2017. [72] D. Konstantinidis, K. Dimitropoulos, and P. Daras, “A deep
[59] R. Jitcharoenpory, P. Senechakr, M. Dahlan, A. Suchato, learning approach for analyzing video and skeletal features in
E. Chuangsuwanich, and P. Punyabukkana, “Recognizing sign language recognition,” in Proceedings of the 2018 IEEE
words in Tai Sign Language using fex sensors and gyro- International Conference on Imaging Systems and Techniques
scopes,” i-CREATe, vol. 4, 2017. (IST), pp. 1–6, Krakow, Poland, October 2018.
[60] P. Loke, J. Paranjpe, S. Bhabal, and K. Kanere, “Indian sign [73] M. Taskiran, M. Killioglu, and N. Kahraman, “A real-time
language converter system using an android app,” in Pro- system for recognition of American sign language by using
ceedings of the 2017 International conference of Electronics, deep learning,” in Proceedings of the 2018 41st International
Communication and Aerospace Technology (ICECA), Conference on Telecommunications and Signal Processing
pp. 436–439, Coimbatore, India, April 2017. (TSP), pp. 1–5, Athens, Greece, July 2018.
[61] S. Y. Heera, M. K. Murthy, V. Sravanti, and S. Salvi, “Talking [74] E. S. Haq, D. Suwardiyanto, and M. Huda, “Indonesian sign
language recognition application for two-way communica-
hands—an Indian sign language to speech translating
tion deaf-mute people,” in Proceedings of the 2018 3rd In-
gloves,” in Proceedings of the 2017 International conference
ternational Conference on Information Technology,
on innovative mechanisms for industry applications (ICI-
Information System and Electrical Engineering (ICITISEE),
MIA), pp. 746–751, Bengaluru, India, February 2017.
pp. 313–318, Yogyakarta, Indonesia, November 2018.
[62] S. Devi and S. Deb, “Low cost tangible glove for translating
[75] K. Shenoy, T. Dastane, V. Rao, and D. Vyavaharkar, “Real-
sign gestures to speech and text in Hindi language,” in
time Indian sign language (ISL) recognition,” in Proceedings
Proceedings of the 2017 3rd International Conference on
of the 2018 9th International Conference on Computing,
Computational Intelligence & Communication Technology
Communication and Networking Technologies (ICCCNT),
(CICT), pp. 1–5, Ghaziabad, India, February 2017. pp. 1–9, Bengaluru, India, July 2018.
[63] S. Shin, Y. Baek, J. Lee, Y. Eun, and S. H. Son, “Korean sign [76] K. Halim and E. Rakun, “Sign language system for Bahasa
language recognition using EMG and IMU sensors based on Indonesia (Known as SIBI) recognizer using TensorFlow and
group-dependent NN models,” in Proceedings of the 2017 long short-term memory,” in Proceedings of the 2018 In-
IEEE Symposium Series on Computational Intelligence ternational Conference on Advanced Computer Science and
(SSCI), pp. 1–7, Honolulu, HI, USA, November 2017. Information Systems (ICACSIS), pp. 403–407, Yogyakarta,
[64] M. Punchimudiyanse and R. G. N. Meegama, “Animation of Indonesia, October 2018.
fngerspelled words and number signs of the Sinhala Sign [77] G. A. Rao, K. Syamala, P. Kishore, and A. Sastry, “Deep
Language,” ACM Transactions on Asian and Low-Resource convolutional neural networks for sign language recogni-
Language Information Processing, vol. 16, no. 4, pp. 1–26, tion,” in Proceedings of the 2018 Conference on Signal Pro-
2017. cessing And Communication Engineering Systems (SPACES),
[65] V. Bheda and D. Radpour, “Using deep convolutional net- pp. 194–197, Vijayawada, India, January 2018.
works for gesture recognition in american sign language,” [78] A. Das, S. Gawde, K. Suratwala, and D. Kalbande, “Sign
2017, https://arxiv.org/abs/1710.06836. language recognition using deep learning on custom pro-
[66] G. A. Rao, P. Kishore, D. A. Kumar, A. Sastry, and Com- cessed static gesture images,” in Proceedings of the 2018
munications, “Neural network classifer for continuous sign International Conference on Smart City and Emerging
language recognition with selfe video,” Far East Journal of Technology (ICSCET), pp. 1–6, Mumbai, India, January 2018.
Electronics and Communications, vol. 17, no. 1, pp. 49–71, [79] G. A. Rao, P. Kishore, A. Sastry, D. A. Kumar, and
2017. E. K. Kumar, “Selfe continuous sign language recognition
[67] M. Iqbal, E. Supriyati, T. Listiyorini, and O. Source, “SIBI with neural network classifer,” in 2nd International Con-
blue: developing Indonesian Sign Language Recognition ference on Micro-electronics, Electromagnetics and Telecom-
system based on the mobile communication platform,” In- munications, pp. 31–40, Springer, Singapore, 2018.
ternational Journal of Information Technology and Computer [80] A. Dudhal, H. Mathkar, A. Jain, O. Kadam, and M. Shirole,
Science, vol. 1, 2017. “Hybrid sift feature extraction approach for indian sign
[68] M. Rivera-Acosta, S. Ortega-Cisneros, J. Rivera, and language recognition system based on cnn,” in International
F. Sandoval-Ibarra, “American sign language alphabet rec- Conference on ISMAC in Computational Vision and Bio-
ognition using a neuromorphic sensor and an artifcial Engineering, pp. 727–738, Springer, Cham, Switzerland,
neural network,” Sensors, vol. 17, no. 10, p. 2176, 2017. 2018.
3637, 2024, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1155/2024/1487500, Wiley Online Library on [24/09/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
Advances in Human-Computer Interaction 33

[81] R. Rastgoo, K. Kiani, and S. Escalera, “Multi-modal deep Soft Computing, pp. 192–196, Da Lat, Viet Nam, January
hand sign language recognition in still images using re- 2019.
stricted Boltzmann machine,” Entropy, vol. 20, no. 11, p. 809, [96] P. Paudyal, J. Lee, A. Kamzin, M. Soudki, A. Banerjee, and
2018. S. K. Gupta, “Learn2Sign: explainable AI for Sign Language
[82] C. K. Mummadi, F. P. P. Leo, K. D. Verma et al., “Real-time learning,” 2019, https://ceur-ws.org/Vol-2327/IUI19WS-
and embedded detection of hand gestures with an IMU- ExSS2019-13.pdf.
based glove,” Informatics, vol. 5, no. 2, p. 28, 2018. [97] L. K. Odartey, Y. Huang, E. E. Asantewaa, and
[83] G. A. Rao and P. J. A. S. E. J. Kishore, “Selfe video based P. R. Agbedanu, “Ghanaian Sign Language Recognition
continuous Indian sign language recognition system,” Ain using deep learning,” in Proceedings of the 2019 the In-
Shams Engineering Journal, vol. 9, no. 4, pp. 1929–1939, 2018. ternational Conference on Pattern Recognition and Artifcial
[84] D. J. A. P. A. Rathi, “Optimization of transfer learning for Intelligence, pp. 81–86, Wenzhou, China, August 2019.
Sign Language Recognition targeting mobile platform,” 2018, [98] J. Schioppo, Z. Meyer, D. Fabiano, and S. Canavan, “Sign
https://arxiv.org/abs/1805.06618. Language recognition: learning American Sign Language in
[85] D. A. Kumar, A. Sastry, P. Kishore, and E. K. J. M. T. Kumar, a virtual environment,” in Proceedings of the Extended Ab-
“Indian sign language recognition using graph matching on stracts of the 2019 CHI Conference on Human Factors in
3D motion captured signs,” Multimedia Tools and Appli- Computing Systems, pp. 1–6, Glasgow, UK, May 2019.
cations, vol. 77, no. 24, pp. 32063–32091, 2018. [99] X. Pei, D. Guo, and Y. Zhao, “Continuous Sign Language
[86] I. Makarov, N. Veldyaykin, M. Chertkov, and A. Pokoev, Recognition based on pseudo-supervised learning,” in Pro-
“American and Russian sign language dactyl recognition,” in ceedings of the 2nd Workshop on Multimedia for Accessible
Proceedings of the 12th ACM International Conference on Human Computer Interfaces, pp. 33–39, Nice, France, Oc-
PErvasive Technologies Related to Assistive Environments, tober 2019.
pp. 204–210, Rhodes, Greece, June 2019. [100] J. Kim and P. Neill-Brown, “Improving American Sign
[87] K. Stefanov and M. Bono, “Towards digitally-mediated Sign Language Recognition with synthetic data,” Proceedings of
Language communication,” in Proceedings of the 7th In- Machine Translation Summit, vol. 1, pp. 151–161, 2019.
ternational Conference on Human-Agent Interaction, [101] N. Salem, S. Alharbi, R. Khezendar, and H. Alshami, “Real-
pp. 286–288, Kyoto, Japan, September 2019. time glove and android application for visual and audible
[88] Z. Kang, “Spoken Language to Sign Language translation Arabic sign language translation,” Procedia Computer Sci-
system based on HamNoSys,” in Proceedings of the 2019 ence, vol. 163, pp. 450–459, 2019.
International Symposium on Signal Processing Systems, [102] Y.-J. Ku, M.-J. Chen, and C.-T. King, “A virtual Sign Lan-
pp. 159–164, Beijing, China, September 2019. guage translator on smartphones,” in Proceedings of the 2019
[89] D. F. Lima, A. S. S. Neto, E. N. Santos, T. M. U. Araujo, and Seventh International Symposium on Computing and Net-
T. G. D. Rêgo, “Using convolutional neural networks for working Workshops (CANDARW), pp. 445–449, Nagasaki,
fngerspelling sign recognition in brazilian sign language,” in Japan, November 2019.
Proceedings of the 25th Brazillian Symposium on Multimedia [103] A. M. Zakariya and R. Jindal, “Arabic Sign Language Rec-
and the Web, pp. 109–115, Rio de Janeiro, Brazil, October ognition system on smartphone,” in Proceedings of the
2019. 2019 10th International Conference on Computing, Com-
[90] J. Hou, X.-Y. Li, P. Zhu, Z. Wang, Y. Wang, and J. Qian, munication and Networking Technologies (ICCCNT), pp. 1–5,
“Signspeaker: a real-time, high-precision smartwatch-based Kanpur, India, July 2019.
sign language translator,” in Proceedings of the Te 25th [104] M. Quinn and J. I. Olszewska, “British sign language rec-
Annual International Conference on Mobile Computing and ognition in the wild based on multi-class SVM,” in Pro-
Networking, pp. 1–15, Los Cabos, Mexico, August 2019. ceedings of the 2019 Federated Conference on Computer
[91] Q. Zhang, D. Wang, R. Zhao, and Y. Yu, “MyoSign: enabling Science and Information Systems (FedCSIS), pp. 81–86,
end-to-end sign language recognition with wearables,” in Leipzig, Germany, September 2019.
Proceedings of the 24th International Conference on In- [105] S. B. Rizwan, M. S. Z. Khan, and M. Imran, “American sign
telligent User Interfaces, pp. 650–660, New York, NY, USA, language translation via smart wearable glove technology,” in
August 2019. Proceedings of the 2019 International Symposium on Recent
[92] N. Sripairojthikoon and J. Harnsomburana, “Tai Sign Advances in Electrical Engineering (RAEE), pp. 1–6, Islam-
Language Recognition using 3D convolutional neural net- abad, Pakistan, August 2019.
works,” in Proceedings of the 2019 7th International Con- [106] R. Fatmi, S. Rashad, and R. Integlia, “Comparing ANN,
ference on Computer and Communications Management, SVM, and HMM based machine learning methods for
pp. 186–189, Bangkok, Tailand, July 2019. American sign language recognition using wearable motion
[93] H. Chao, W. Fenhua, and Z. Ran, “Sign language recognition sensors,” in Proceedings of the 2019 IEEE 9th Annual
based on CBAM-ResNet,” in Proceedings of the 2019 In- Computing and Communication Workshop and Conference
ternational Conference on Artifcial Intelligence and Ad- (CCWC), pp. 0290–0297, Las Vegas, NV, USA, January 2019.
vanced Manufacturing, pp. 1–6, Dublin, Ireland, October [107] E. Abraham, A. Nayak, and A. Iqbal, “Real-time translation
2019. of Indian Sign Language using LSTM,” in Proceedings of the
[94] H. Shin, W. J. Kim, and K.-A. Jang, “Korean sign language 2019 Global Conference for Advancement in Technology
recognition based on image and convolution neural net- (GCAT), pp. 1–5, Bangalore, India, October 2019.
work,” in Proceedings of the 2nd International Conference on [108] N. F. P. Setyono and E. Rakun, “Recognizing word gesture in
Image and Graphics Processing, pp. 52–55, Singapore, Feb- sign system for Indonesian Language (SIBI) sentences using
ruary 2019. DeepCNN and BiLSTM,” in Proceedings of the 2019 In-
[95] S. Fayyaz and Y. Ayaz, “CNN and traditional classifers ternational Conference on Advanced Computer Science and
performance for Sign Language Recognition,” in Proceedings information Systems (ICACSIS), pp. 199–204, Bali, Indonesia,
of the 3rd International Conference on Machine Learning and October 2019.
3637, 2024, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1155/2024/1487500, Wiley Online Library on [24/09/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
34 Advances in Human-Computer Interaction

[109] M. H. N. Fauzan, E. Rakun, and D. Hardianto, “Feature [121] S. Krishna, A. R. Jindal, and D. Jayagopi, “Virtual Indian Sign
extraction from smartphone images by using elliptical Language interpreter,” in Proceedings of the 2020 4th In-
fourier descriptor, centroid and area for recognizing Indo- ternational Conference on Vision, Image and Signal Pro-
nesian Sign Language SIBI (sistem isyarat bahasa Indo- cessing, pp. 1–5, Bangkok, Tailand, March 2020.
nesia),” in Proceedings of the 2019 2nd International [122] Z. Zhou, Y. Neo, K.-S. Lui, V. W. Tam, E. Y. Lam, and
Conference on Intelligent Autonomous Systems (ICoIAS), N. Wong, “A portable Hong Kong Sign Language translation
pp. 8–14, Singapore, August 2019. platform with deep learning and jetson nano,” in Proceedings
[110] P. Battistoni, M. Di Gregorio, M. Sebillo, and G. Vitiello, “AI of the Te 22nd International ACM SIGACCESS Conference
at the edge for sign language learning support,” in Pro- on Computers and Accessibility, pp. 1–4, Greece, October
ceedings of the 2019 IEEE International Conference on Hu- 2020.
manized Computing and Communication (HCC), pp. 16–23, [123] D. Bragg, O. Koller, N. Caselli, and W. Ties, “Exploring
Laguna Hills, CA, USA, September 2019. collection of sign language datasets: privacy, participation,
[111] K. Kawaguchi, H. Nishimura, Z. Wang, H. Tanaka, and and model performance,” in Proceedings of the Te 22nd
E. Ohta, “Basic investigation of sign language motion clas- International ACM SIGACCESS Conference on Computers
sifcation by feature extraction using pre-trained network and Accessibility, pp. 1–14, Greece, August 2020.
models,” in Proceedings of the 2019 IEEE Pacifc Rim Con- [124] T. Agrawal and S. Urolagin, “2-way Arabic Sign Language
ference on Communications, Computers and Signal Pro- translator using CNNLSTM architecture and NLP,” in
cessing (PACRIM), pp. 1–4, Victoria, Canada, August 2019. Proceedings of the 2020 2nd International Conference on Big
[112] L. Y. Bin, G. Y. Huann, and L. K. Yun, “Study of con- Data Engineering and Technology, pp. 96–101, Singapore,
volutional neural network in recognizing static American China, January 2020.
Sign Language,” in Proceedings of the 2019 IEEE In- [125] H. Jirathampradub, C. Nukoolkit, K. Suriyathumrongkul,
ternational Conference on Signal and Image Processing Ap- and B. Watanapa, “A 3D-CNN siamese network for motion
plications (ICSIPA), pp. 41–45, Kuala Lumpur, Malaysia, gesture Sign Language alphabets recognition,” in Proceedings
September 2019. of the 11th International Conference on Advances in In-
[113] C. Sruthi and A. Lijiya, “Signet: a deep learning based indian
formation Technology, pp. 1–6, New York, NY, USA, July
sign language recognition system,” in Proceedings of the 2019
2020.
International Conference on Communication and Signal [126] M. De Coster, M. Van Herreweghe, and J. Dambre, “Sign
Processing (ICCSP), pp. 0596–0600, Chennai, India, April
language recognition with transformer networks,” in Pro-
2019.
ceedings of the 12th International Conference on Language
[114] R. Siriak, I. Skarga-Bandurova, and Y. Boltov, “Deep con-
Resources and Evaluation, New York, NY, USA, January
volutional network with long short-term memory layers for
2020.
dynamic gesture recognition,” in Proceedings of the 2019 10th
[127] H. Park, J.-S. Lee, and J. Ko, “Achieving real-time Sign
IEEE International Conference on Intelligent Data Acquisi-
Language translation using a smartphone’s true depth im-
tion and Advanced Computing Systems: Technology and
ages,” in Proceedings of the 2020 International Conference on
Applications (IDAACS), pp. 158–162, Metz, France, Sep-
COMmunication Systems & NETworkS (COMSNETS),
tember 2019.
[115] F. Pezzuoli, D. Corona, M. L. Corradini, and A. Cristofaro, pp. 622–625, Bengaluru, India, January 2020.
“Development of a wearable device for sign language [128] Y. Ding, S. Huang, and R. Peng, “Data augmentation and
translation,” in Human Friendly Robotics, pp. 115–126, deep learning modeling methods on edge-device-based Sign
Springer, Berlin, Germany, 2019. Language Recognition,” in Proceedings of the 2020 5th In-
[116] M. Naseem, S. Sarfraz, A. Abbas, and A. Haider, “Developing ternational Conference on Information Science, Computer
a prototype to translate Pakistan Sign Language into text and Technology and Transportation (ISCTT), pp. 490–497, She-
speech while using convolutional neural networking,” nyang, China, November 2020.
Journal of Education and Practice, vol. 15, 2019. [129] A. Moryossef, I. Tsochantaridis, R. Aharoni, S. Ebling, and
[117] R. Ahuja, D. Jain, D. Sachdeva, A. Garg, C. Rajput, and S. Narayanan, “Real-time sign language detection using
Intelligence, “Convolutional neural network based american human pose estimation,” in European Conference on Com-
sign language static hand gesture recognition,” International puter Vision, pp. 237–248, Springer, Berlin, Germany, 2020.
Journal of Ambient Computing and Intelligence, vol. 10, no. 3, [130] Z. Niu and B. Mak, “Stochastic fne-grained labeling of
pp. 60–73, 2019. multi-state sign glosses for continuous Sign Language
[118] M. Khari, A. K. Garg, R. Gonzalez-Crespo, and E. Verdú, Recognition,” in European Conference on Computer Vision,
“Gesture recognition of RGB and RGB-D static images using pp. 172–186, Springer, Berlin, Germany, 2020.
convolutional neural networks,” International Journal of [131] L. Kraljević, M. Russo, M. Pauković, and M. Šarić, “A dy-
Interactive Multimedia and Artifcial Intelligence, vol. 5, namic gesture recognition interface for smart home control
no. 7, p. 22, 2019. based on Croatian Sign Language,” Applied Sciences, vol. 10,
[119] P. Paudyal, J. Lee, A. Banerjee, and S. K. Gupta, “A com- no. 7, p. 2300, 2020.
parison of techniques for sign language alphabet recognition [132] E. Izutov, “ASL recognition with metric-learning based
using armband wearables,” ACM Transactions on Interactive lightweight network,” 2020, https://arxiv.org/abs/2004.
Intelligent Systems, vol. 9, no. 2-3, pp. 1–26, 2019. 05054.
[120] N. Saquib and A. Rahman, “Application of machine learning [133] T. M. Angona, A. S. Shaon, K. T. R. Niloy et al., “Automated
techniques for real-time sign language detection using Bangla sign language translation system for alphabets by
wearable sensors,” in Proceedings of the 11th ACM Multi- means of MobileNet,” Telkomnika (Telecommunication
media Systems Conference, pp. 178–189, Istanbul, Turkey, Computing Electronics and Control), vol. 18, no. 3,
May 2020. pp. 1292–1301, 2020.
3637, 2024, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1155/2024/1487500, Wiley Online Library on [24/09/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
Advances in Human-Computer Interaction 35

[134] A. Wadhawan and P. Kumar, “Deep learning-based sign recommendations and future directions for app assessment,”
language recognition system for static signs,” Neural Com- Universal Access in the Information Society, vol. 20, pp. 1–16,
puting & Applications, vol. 32, no. 12, pp. 7957–7968, 2020. 2023.
[135] B. M. Saleh, R. I. Al-Beshr, and M. U. Tariq, “D-talk: sign [149] D. David, A. Alamoodi, O. Albahri et al., “Sign language
Language Recognition system for people with disability mobile apps: a systematic review of current app evaluation
using machine learning and image processing,” International progress and solution framework,” Evolving Systems,
Journal of Advanced Trends in Computer Science and Engi- vol. 2023, pp. 1–18, 2023.
neering, vol. 9, no. 4, pp. 4374–4382, 2020. [150] M. M. Balaha, S. El-Kady, H. M. Balaha et al., “A vision-based
[136] R. Rastgoo, K. Kiani, and S. Escalera, “Hand sign language deep learning approach for independent-users Arabic sign
recognition using multi-view hand skeleton,” Expert Systems language interpretation,” Multimedia Tools and Applications,
with Applications, vol. 150, Article ID 113336, 2020. vol. 82, no. 5, pp. 6807–6826, 2023.
[137] P. Kumar, P. Kumar, and S. Kaur, “Sign Language generation [151] R. Kumar Attar, V. Goyal, and L. Goyal, “State of the art of
system based on Indian Sign Language grammar,” ACM automation in Sign Language: a systematic review,” ACM
Transactions on Asian and Low-Resource Language In- Transactions on Asian and Low-Resource Language In-
formation Processing, vol. 19, no. 4, pp. 1–26, 2020. formation Processing, vol. 22, no. 4, pp. 1–80, 2023.
[138] Z. Wang, T. Zhao, J. Ma et al., “Hear Sign Language: a real- [152] R. J. Eunice and D. J. Hemanth, “Deep learning and Sign
time end-to-end Sign Language recognition system,” IEEE Language models based enhanced accessibility of e-
Transactions on Mobile Computing, vol. 21, p. 1, 2020. governance services for speech and hearing-impaired,” in
[139] D. Kuswardhana, I. Rachmawati, G. N. Amani, H. Fauzi, and Proceedings of the Electronic Governance with Emerging
A. H. S. Budi, “Construction of SIBI datasets for Sign Technologies: First International Conference, EGETC 2022,
Language Recognition using a webcam,” Proceedings of the pp. 12–24, Tampico, Mexico, September 2023.
6th UPI International Conference on TVET 2020 (TVET [153] D. R. Kothadiya, C. M. Bhatt, T. Saba, A. Rehman, and
2020), vol. 520, pp. 186–189, 2021. S. A. Bahaj, “SIGNFORMER: DeepVision transformer for
[140] K. Amrutha and P. Prabu, “ML based Sign Language Rec- Sign Language Recognition,” IEEE Access, vol. 11,
ognition system,” in Proceeedings of the 2021 International pp. 4730–4739, 2023.
Conference on Innovative Trends in Information Technology [154] R. Sreemathy, M. Turuk, J. Jagdale, A. Agarwal, and
(ICITIIT), pp. 1–6, Kottayam, India, February 2021. V. Kumar, “Indian Sign Language interpretation using
[141] A. Imran, A. Razzaq, I. A. Baig, A. Hussain, S. Shahid, and convolutional neural networks,” in Proceedings of the
2023 10th International Conference on Signal Processing and
T.-U. J. D. I. B. Rehman, “Dataset of Pakistan Sign Language
Integrated Networks (SPIN), pp. 789–794, Noida, India,
and automatic recognition of hand confguration of Urdu
March 2023.
alphabet through machine learning,” Data in Brief, vol. 36,
[155] N. K. Pandey, A. Dwivedi, M. Sharma, A. Bansal, and
Article ID 107021, 2021.
A. K. Mishra, “An improved Sign Language translation
[142] R. Rastgoo, K. Kiani, and S. Escalera, “Real-time isolated
approach using KNN in deep learning environment,” in
hand sign language recognition using deep networks and
Proceedings of the 2023 International Conference on Dis-
SVD,” Journal of Ambient Intelligence and Humanized
ruptive Technologies (ICDT), pp. 473–477, Greater Noida,
Computing, vol. 13, pp. 591–611, 2021.
India, May 2023.
[143] Y. Wang, F. Li, Y. Xie, C. Duan, and Y. Wang, “HearASL:
[156] A. Singh, A. Wadhawan, M. Rakhra, U. Mittal, A. Al Ahdal,
your smartphone can hear American Sign Language,” IEEE and S. K. Jha, “Indian Sign Language Recognition system for
Internet of Tings Journal, vol. 10, pp. 8839–8852, 2023. dynamic signs,” in Proceedings of the 2022 10th International
[144] S. C. Ke, A. K. Mahamad, S. Saon, U. Fadlilah, and Conference on Reliability, Infocom Technologies and Opti-
B. Handaga, “Malaysian Sign Language translator for mobile mization (Trends and Future Directions)(ICRITO), pp. 1–6,
application,” in Proceedings of the 11th International Con- Noida, India, October 2022.
ference on Robotics, Vision, Signal Processing and Power [157] H. Wang, J. Zhang, Y. Li, and L. Wang, “SignGest: sign
Applications: Enhancing Research and Innovation through Language Recognition using acoustic signals on smart-
the Fourth Industrial Revolution, pp. 909–914, Springer, phones,” in Proceedings of the 2022 IEEE 20th International
Singapore, 2022. Conference on Embedded and Ubiquitous Computing (EUC),
[145] S. Gadge, K. Kharde, R. Jadhav, S. Bhere, and I. Dokare, pp. 60–65, Wuhan, China, December 2022.
“Recognition of Indian Sign Language characters using [158] S. Chavan, X. Yu, and J. Saniie, “Convolutional neural
convolutional neural network,” Proceedings of 3rd In- network hand gesture recognition for American sign lan-
ternational Conference on Artifcial Intelligence: Advances guage,” in Proceedings of the 2021 IEEE International Con-
and Applications: ICAIAA, vol. 2023, pp. 163–176, 2022. ference on Electro Information Technology (EIT), pp. 188–192,
[146] A. F. Shokoori, M. Shinwari, J. A. Popal, and J. Meena, “Sign Mt. Pleasant, MI, USA, May 2021.
Language recognition and translation into pashto language [159] H. Park, Y. Lee, and J. Ko, “Enabling real-time sign language
alphabets,” in Proceedings of the 2022 6th International translation on mobile platforms with on-board depth
Conference on Computing Methodologies and Communica- cameras,” Proceedings of the ACM on Interactive, Mobile,
tion (ICCMC), pp. 1401–1405, Erode, India, March 2022. Wearable and Ubiquitous Technologies, vol. 5, no. 2, pp. 1–30,
[147] S. Siddique, S. Islam, E. E. Neon, T. Sabbir, I. T. Naheen, and 2021.
R. Khan, “Deep learning-based bangla Sign Language de- [160] V. Gomathi and Dr Gomathi V, “Indian Sign Language
tection with an edge device,” Intelligent Systems with Ap- Recognition through hybrid ConvNet-LSTM networks,”
plications, vol. 18, Article ID 200224, 2023. EMITTER International Journal of Engineering Technology,
[148] D. David, A. Alamoodi, O. Albahri et al., “Landscape of sign vol. 9, no. 1, pp. 182–203, 2021.
language research based on smartphone apps: coherent lit- [161] E. Rajalakshmi, R. Elakkiya, V. Subramaniyaswamy et al.,
erature analysis, motivations, open challenges, “Multi-semantic discriminative feature learning for sign
3637, 2024, 1, Downloaded from https://onlinelibrary.wiley.com/doi/10.1155/2024/1487500, Wiley Online Library on [24/09/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
36 Advances in Human-Computer Interaction

gesture recognition using hybrid deep neural architecture,”

IEEE Access, vol. 11, pp. 2226–2238, 2023.
[162] B. Natarajan, E. Rajalakshmi, R. Elakkiya et al., “Develop-
ment of an end-to-end deep learning framework for sign
language recognition, translation, and video generation,”
IEEE Access, vol. 10, pp. 104358–104374, 2022.
[163] E. Rajalakshmi, R. Elakkiya, A. L. Prikhodko et al., “Static
and dynamic isolated Indian and Russian sign language
recognition with spatial and temporal feature detection using
hybrid neural network,” ACM Transactions on Asian and
Low-Resource Language Information Processing, vol. 22,
pp. 1–23, 2022.
[164] B. Natarajan and R. Elakkiya, “Dynamic GAN for high-
quality sign language video generation from skeletal poses
using generative adversarial networks,” Soft Computing,
vol. 26, no. 23, pp. 13153–13175, 2022.
[165] R. Elakkiya, “Dataset,” 2021, https://data.mendeley.com/
datasets/rc349j45m5/1.
[166] A. A. Nerlekar, “Sign Language recognition using smart-
phones,” Tesis, California State University, Northridge, CA,
USA, 2021.
[167] O. M. Sincan, J. Junior, C. Jacques, S. Escalera, and
H. Y. Keles, “Chalearn LAP large scale signer independent
isolated sign language recognition challenge: design, results
and future research,” in Proceedings of the IEEE/CVF Con-
ference on Computer Vision and Pattern Recognition,
pp. 3472–3481, Nashville, TN, USA, June 2021.
[168] A. Khan, S. Khusro, and I. Alam, “BlindSense: an
accessibility-inclusive universal user interface for blind
people,” Engineering, Technology & Applied Science Research,
vol. 8, no. 2, pp. 2775–2784, 2018.
[169] B. Niazi, S. Khusro, A. Khan, and I. Alam, “A touch sensitive
keypad layout for improved usability of smartphones for the
blind and visually impaired persons,” Artifcial Intelligence
Perspectives in Intelligent Systems, vol. 1, pp. 427–436, 2016.
[170] D. Bragg, O. Koller, N. Caselli, and W. Ties, “Exploring
collection of sign language datasets: privacy, participation,
and model performance,” in Proceedings of the 22nd In-
ternational ACM SIGACCESS Conference on Computers and
Accessibility, pp. 1–14, Greece, October 2020.
[171] K. Amrutha and P. Prabu, “Evaluating the pertinence of pose
estimation model for sign language translation,” In-
ternational Journal of Computational Intelligence and Ap-
plications, vol. 22, no. 01, Article ID 2341009, 2023.
[172] S. Sen, S. Narang, and P. Gouthaman, “Real-time Sign
Language Recognition system,” in Proceedings of the 2023
Advanced Computing and Communication Technologies for
High Performance Applications (ACCTHPA), pp. 1–6, Tri-
vandrum, India, August 2023.

PRISMA 2020 Flow Diagram For New Systematic Reviews Which Included Searches of Databases, Registers and Other Sources
No ratings yet
PRISMA 2020 Flow Diagram For New Systematic Reviews Which Included Searches of Databases, Registers and Other Sources
1 page
ADEYANJU 2021 Machine Learning (VOR)
No ratings yet
ADEYANJU 2021 Machine Learning (VOR)
37 pages
Development of A Sign Language Recognition System Using Machine Learning
No ratings yet
Development of A Sign Language Recognition System Using Machine Learning
8 pages
m_DEVELOPINGSIGNLANGUAGERECOGNITIONMODELFORAFAANOROMOOWORDSUSINGADEEPLEARNINGAPPROACH
No ratings yet
m_DEVELOPINGSIGNLANGUAGERECOGNITIONMODELFORAFAANOROMOOWORDSUSINGADEEPLEARNINGAPPROACH
13 pages
Animated Sign Language For People With Speaking and Hearing Disability Using Dee
No ratings yet
Animated Sign Language For People With Speaking and Hearing Disability Using Dee
5 pages
10_Wearable_Sensor-Based_Sign_Language_Recognition_A_Comprehensive_Review
No ratings yet
10_Wearable_Sensor-Based_Sign_Language_Recognition_A_Comprehensive_Review
16 pages
7th Sem Report Sign Language Recognition
No ratings yet
7th Sem Report Sign Language Recognition
15 pages
Retrieve
No ratings yet
Retrieve
21 pages
PFX-48420843 (1)
No ratings yet
PFX-48420843 (1)
6 pages
Sign recognition research paper
No ratings yet
Sign recognition research paper
16 pages
Sign Language Detection and Recognizatio
No ratings yet
Sign Language Detection and Recognizatio
7 pages
Recent Advances on Deep Learning for Sign Language
No ratings yet
Recent Advances on Deep Learning for Sign Language
52 pages
Sign Language Detection Using Machine Learning
No ratings yet
Sign Language Detection Using Machine Learning
6 pages
IJRAR23B3375
No ratings yet
IJRAR23B3375
5 pages
Sem 8th 2nd - Merged
No ratings yet
Sem 8th 2nd - Merged
33 pages
Sign Language Recogntion Report
No ratings yet
Sign Language Recogntion Report
29 pages
The Use of TensorFlow Action Recognition As The Main Component in Making A Sign Language Translator Speaker For Speech-Impaired People
No ratings yet
The Use of TensorFlow Action Recognition As The Main Component in Making A Sign Language Translator Speaker For Speech-Impaired People
8 pages
Mathematics 11 03729
No ratings yet
Mathematics 11 03729
20 pages
Mohammed Maqdoom Jahagirdarp2Yo
No ratings yet
Mohammed Maqdoom Jahagirdarp2Yo
9 pages
Research Paper3
No ratings yet
Research Paper3
8 pages
Paper 114-Harnessing AI To Generate Indian Sign Language From Natural Speech
No ratings yet
Paper 114-Harnessing AI To Generate Indian Sign Language From Natural Speech
10 pages
Research Paper
No ratings yet
Research Paper
7 pages
IJCRT2402561
No ratings yet
IJCRT2402561
9 pages
Sign Language Detection
No ratings yet
Sign Language Detection
13 pages
Sign Language Recog Paper-1
No ratings yet
Sign Language Recog Paper-1
19 pages
Methodologies For Sign Language Recognition A Survey
No ratings yet
Methodologies For Sign Language Recognition A Survey
4 pages
Referencia N°06
No ratings yet
Referencia N°06
52 pages
pppp1
No ratings yet
pppp1
6 pages
PAPER 1
No ratings yet
PAPER 1
5 pages
Final Minor Report
No ratings yet
Final Minor Report
24 pages
Cognitive Access Design
From Everand
Cognitive Access Design
Christopher Miller
No ratings yet
Statistical Semantics: Fundamentals and Applications
From Everand
Statistical Semantics: Fundamentals and Applications
Fouad Sabry
No ratings yet
REPORT - FINAL - Praga
No ratings yet
REPORT - FINAL - Praga
29 pages
Sign4all_A_Low-Cost_Application_for_Deaf_People_Communication
No ratings yet
Sign4all_A_Low-Cost_Application_for_Deaf_People_Communication
11 pages
Sign Language Recognition System With Speech Output
No ratings yet
Sign Language Recognition System With Speech Output
5 pages
BIt On
No ratings yet
BIt On
12 pages
Language Identification: Fundamentals and Applications
From Everand
Language Identification: Fundamentals and Applications
Fouad Sabry
No ratings yet
Speech Aid Technologies
From Everand
Speech Aid Technologies
Cassian Pereira
No ratings yet
Hand Gesture Recognition using Machine Learning and Computer Vision
No ratings yet
Hand Gesture Recognition using Machine Learning and Computer Vision
38 pages
2021a1r002
No ratings yet
2021a1r002
51 pages
Improving Language Acquisition in Sensory Deficit
No ratings yet
Improving Language Acquisition in Sensory Deficit
6 pages
Abstract
No ratings yet
Abstract
1 page
Assistive Tech Tools
From Everand
Assistive Tech Tools
Cassian Pereira
No ratings yet
On Sign Language Detection
No ratings yet
On Sign Language Detection
25 pages
Sign Language Recognition Reseach Paper-1
No ratings yet
Sign Language Recognition Reseach Paper-1
8 pages
Alkeshcpp
No ratings yet
Alkeshcpp
37 pages
Real-Time Conversion for Sign-to-Text and Text-to-Speech Communication using Machine Learning
No ratings yet
Real-Time Conversion for Sign-to-Text and Text-to-Speech Communication using Machine Learning
8 pages
Bachelor of Technology IN Artificial Intelligence and Machine Learning
No ratings yet
Bachelor of Technology IN Artificial Intelligence and Machine Learning
14 pages
synopsis a baja
No ratings yet
synopsis a baja
10 pages
The Virtualsign Channel For The Communication Between Deaf and Hearing Users
No ratings yet
The Virtualsign Channel For The Communication Between Deaf and Hearing Users
8 pages
Survey Sign Language Production 2023
No ratings yet
Survey Sign Language Production 2023
23 pages
Sign Language Recognition
No ratings yet
Sign Language Recognition
9 pages
Deep Learning For Sign Language Recognition
No ratings yet
Deep Learning For Sign Language Recognition
4 pages
Hand Gesture Recognition For Deaf and Blind People
No ratings yet
Hand Gesture Recognition For Deaf and Blind People
4 pages
Natural Language Understanding: Fundamentals and Applications
From Everand
Natural Language Understanding: Fundamentals and Applications
Fouad Sabry
No ratings yet
Visualizing Language: CNNs For Sign Language Recognition
No ratings yet
Visualizing Language: CNNs For Sign Language Recognition
6 pages
Sign Speak: Recogninzing Sign Language With Machine Learning
No ratings yet
Sign Speak: Recogninzing Sign Language With Machine Learning
12 pages
Recognizing Sign Language Using Machine Learning and Deep Learning Models
No ratings yet
Recognizing Sign Language Using Machine Learning and Deep Learning Models
11 pages
Sign Language Recognition
No ratings yet
Sign Language Recognition
12 pages
Sign Language Recognition System Using Machine Learning
No ratings yet
Sign Language Recognition System Using Machine Learning
6 pages
Voice to Hand Gesture Recognition App for People with Hearing Disabilities
No ratings yet
Voice to Hand Gesture Recognition App for People with Hearing Disabilities
6 pages
Nair 2021 - A Systematic Review of Digital Storytelling in Improving Speaking Skills
No ratings yet
Nair 2021 - A Systematic Review of Digital Storytelling in Improving Speaking Skills
15 pages
CLA Suplemento
No ratings yet
CLA Suplemento
40 pages
Ptaceketal2023 MetaanlysisMAC
No ratings yet
Ptaceketal2023 MetaanlysisMAC
20 pages
Business Incubators and Sustainability: A Literature Review
No ratings yet
Business Incubators and Sustainability: A Literature Review
12 pages
Sustainability 16 01125
No ratings yet
Sustainability 16 01125
15 pages
(20635303 - Journal of Behavioral Addictions) Fear of Missing Out (FoMO) and Internet Use - A Comprehensive Systematic Review and Meta-Analysis
No ratings yet
(20635303 - Journal of Behavioral Addictions) Fear of Missing Out (FoMO) and Internet Use - A Comprehensive Systematic Review and Meta-Analysis
22 pages
How Generative AI Influences Students Self-Regula
No ratings yet
How Generative AI Influences Students Self-Regula
15 pages
2011 CJA Guide For Authors
No ratings yet
2011 CJA Guide For Authors
29 pages
The Role of Digital Health in Supporting Cancer Patients
No ratings yet
The Role of Digital Health in Supporting Cancer Patients
19 pages
1-IsRS Technical Guidelines For Stereotactic
No ratings yet
1-IsRS Technical Guidelines For Stereotactic
12 pages
The impact of continuous professional development on teaching quality: a systematic review
No ratings yet
The impact of continuous professional development on teaching quality: a systematic review
10 pages
The Impact of Study and Learning Strategies On Post-Secondary Student Academic Achievement - A Mixed-Methods Systematic Review
No ratings yet
The Impact of Study and Learning Strategies On Post-Secondary Student Academic Achievement - A Mixed-Methods Systematic Review
80 pages
4 Castellanos Et El 2020 Mangrove Research Colombia Temporal Trends Geographical Coverage
No ratings yet
4 Castellanos Et El 2020 Mangrove Research Colombia Temporal Trends Geographical Coverage
34 pages
Jun 2021
No ratings yet
Jun 2021
11 pages
Responsible Artificial Intelligence in Human Resources Management A Review of The Empirical Literature
No ratings yet
Responsible Artificial Intelligence in Human Resources Management A Review of The Empirical Literature
16 pages
Effectiveness of Semi-Occluded Vocal Tract Exercises (Sovtes) in Patients With Dysphonia: A Systematic Review and Meta-Analysis
No ratings yet
Effectiveness of Semi-Occluded Vocal Tract Exercises (Sovtes) in Patients With Dysphonia: A Systematic Review and Meta-Analysis
19 pages
Diagnostics 13 00429 v2
No ratings yet
Diagnostics 13 00429 v2
18 pages
language-learning-strategies-used-by-esl-students-in-enhancing-english-proficiency-a-systematic-review-2013-2022
No ratings yet
language-learning-strategies-used-by-esl-students-in-enhancing-english-proficiency-a-systematic-review-2013-2022
26 pages
Applsci 13 07405
No ratings yet
Applsci 13 07405
17 pages
CiBNP Research Course Syllabus
No ratings yet
CiBNP Research Course Syllabus
12 pages
Advances in Human-Computer Interaction - 2024 - Alam - Exploring Sign Language Detection On Smartphones A Systematic
No ratings yet
Advances in Human-Computer Interaction - 2024 - Alam - Exploring Sign Language Detection On Smartphones A Systematic
36 pages
The School Environment and Student Health
No ratings yet
The School Environment and Student Health
11 pages
Food Marketing Influences Childrens Attitudes Pre
No ratings yet
Food Marketing Influences Childrens Attitudes Pre
14 pages
retrieve (12) (1)
No ratings yet
retrieve (12) (1)
26 pages
Trends in Green Chemistry Research Between 2012 and 2022: Current Trends and Research Agenda
No ratings yet
Trends in Green Chemistry Research Between 2012 and 2022: Current Trends and Research Agenda
20 pages
Optimization of Municipal Solid Waste Collection System- Systematic Review With Bibliometric Literature Analysis
No ratings yet
Optimization of Municipal Solid Waste Collection System- Systematic Review With Bibliometric Literature Analysis
12 pages
A Systematic Review On Contractual Challenges in Construction Industry During A Pandemic Situation (COVID 19)
No ratings yet
A Systematic Review On Contractual Challenges in Construction Industry During A Pandemic Situation (COVID 19)
11 pages
Artificial Intelligence AI Learning Tools in K-12
No ratings yet
Artificial Intelligence AI Learning Tools in K-12
39 pages
Curriculum Assessment Practices That Incorporate Learning Outcomes in Higher Education - A Systematic Literature Review
No ratings yet
Curriculum Assessment Practices That Incorporate Learning Outcomes in Higher Education - A Systematic Literature Review
18 pages

Advances in Human-Computer Interaction - 2024 - Alam - Exploring Sign Language Detection On Smartphones A Systematic

Uploaded by

Advances in Human-Computer Interaction - 2024 - Alam - Exploring Sign Language Detection On Smartphones A Systematic

Uploaded by

Hindawi

Advances in Human-Computer Interaction

Iftikhar Alam ,1 Abdul Hameed,2 and Riaz Ahmad Ziar 3

Correspondence should be addressed to Riaz Ahmad Ziar; [email protected]

Academic Editor: Christos Troussas

1. Introduction have a speech disability and this number is expected to rise to

Table 1: Research questions.

Records afer duplicates and irrelevant

Records screened Records excluded

Full-text articles assessed Full-text articles

for eligibility excluded, afer reading

Studies included afer in

Figure 1: Te identifcation process of primary studies [24].

Table 2: Inclusion and exclusion criteria.

Table 3: Studies found in the selected repositories.

Conferences VS Journal Papers

Yearwise Paper Frequency

Table 4: Summary of the included literature.

[80] 2018 CNN Accuracy

is, 23 English alphabets, 0–10 digits, and 67 commonly used

(v) Total pictures: the dataset contains a total of around

totals 30 for one alphabet. Tus, the total number of

Korean professional signers

[37] 2015 Indonesian Sign Language Alphabets A to Z

Table 8: Sign languages targeted.

Te contribution of publishers has been analyzed based

Table 9: Models and their evaluation performance on specifc sign languages.

Comparative analysis of various studies on ASL and their achievement in

Top 10 most cited papers

Using Deep Convolutional Networks for Gesture

0 50 100 150 200 250 300 350 400 450 500

Figure 7: Top ten most cited papers.

Top 10 country wise contributions

gesture recognition using hybrid deep neural architecture,”

You might also like