IRJET- Hand Gesture based Recognition using CNN Methodology

International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056
Volume: 07 Issue: 01 | Jan 2020 www.irjet.net p-ISSN: 2395-0072
© 2020, IRJET | Impact Factor value: 7.34 | ISO 9001:2008 Certified Journal | Page 567
HAND GESTURE BASED RECOGNITION USING CNN METHODOLOGY
Shailesh bachani1, Shubham dixit2, Rohin chadha3
1,2,3Student, Department of Computer Engineering, NBN Sinhgad School of Engineering, Ambegaon, Pune,
Maharastra, India, 411041
---------------------------------------------------------------------***----------------------------------------------------------------------
Abstract - Sign Language is an intriguing themeandlikeActionRecognition.Particularlyalongsidetheincredibleadvancement
of Deep Learning. Video-based Sign Language Recognition is our worry since we need to perceivea signbytheshapeaswell as
by the activity the underwriter does. The issue is communication through signing is extremely mind boggling and differ. The
variety of communication via gestures is makingtheframework harderto perceiveeveryoneofthewordsprecisely.Numerous
specialists have been exploring Sign Language Recognition for quite a while. Such huge numbers of techniques had been
utilized to discover which one is the best strategy. On account of closeness between Sign Language Recognition and Action
Recognition.
Key Words: American Sign Language, Sign Language, Convolutional Model, Natural Language.
1. INTRODUCTION
Hard of hearing is a descriptor word with importance incapable to hear. At times hard of hearing happened to certain people
groups and make them limited to do correspondence with one another. Readily, there is a strategy calledSignLanguagewhich
is an excellent method to cause them to have the option to speak with others. Be that as it may, the issue isn't all peoplegroups
are comprehended or willing to find out about Sign Language. With this examination, we trust we can manufacture a superior
correspondence with hard of hearing people groups. In view of the multifaceted nature and assorted varietyofSign Language.
It makes this theme much increasingly harder. For instance, each nation has its very own Sign Language and standard. Like
America and Argentina Sign Language is totally unique thing. And furthermore some of the time Sign Language has some
comparable sign with various significance. Which is, it made Deep Learning harder to remember them. And afterward we
likewise ought to consider of contrasts between individuals who do the Sign Language. Presently, you can envision how hard
Deep Learning ought to learn numerous things about this point to get a decentexactnessandgoodtobeexecutedasa gadgetto
translate the Sign Language. We found that a few specialists had been doing admirably in this subject.Like,J.Huangattempted
to make Sign Language Recognition System utilizing his proposed 3D Convolutional Neural Network Model and L. Pigou
attempted his proposed Convolutional Neural Network with video information and some preprocessing. Gesture based
communication is mind boggling and jumpers. Notwithstanding utilizing distinctive dataset of a similar strategy could have a
noteworthy effect of result? That is the reason we need to attempttoexecutethei3dcommencementmodel intoSignLanguage
and break down the outcome. What's more, improve this theme to be better.
1.1 Scope:
1.1. Use and awareness of computer interface through ASL interpretation.
2. Education and training will be easier through ASL interpretation/visualization for deaf and dumb people.
3. Serving the mankind by use of technology.
4. Social aspect like humanity can increase in individual mind by involving physically impaired people in our day to day life.
5. Blind people can also use the same system by extending it for voice interface.
1.2 Objective:
The sign language recognition system feasible for muted persons because of them can be communicate in deafvia thissystem.
The system can capture hand gesture and navigate the words as in text format it will use for muted persons for seen these
words and understanding the sentences.
It is also usable for blind peoples because they need to communicate with normal persons (using speech recognition).
2. LITERATURE SURVEY
2.1“An Approach for Minimizing the Time Taken for Translating Sign Language to Simple Sentence in English”
Aradhana Kar, Pinaki Sankar

Sign Language is the language of deaf. There are different types of sign languages spread all over the world. American Sign
Language (ASL) is one of the sign languages. ASL is used by deaf Americans. We had created a system that translates sign
language videos to simple sentence in English.
2.2“Deep Convolutional Neural Networks for Sign Language Recognition”
G.Anantha Rao, Guntur (DT)
Extraction of complex head and hand movements along with their constantly changing shapes forrecognitionofsignlanguage
is considered a difficult problem in computer vision
2.3“American Sign Language Recognition using Deep Learning and Computer Vision”
Kshitij Bantupalli, Ying Xie
Speech impairment is a disability which affects an individual’s ability to communicate using speech and hearing. People who
are affected by this use other media of communication such as sign language.
2.4 “Recent Developments in Sign Language Recognition Systems”
M.F. Tolba, A.S.Elons
Automated translation systems for sign languages are important in a world that is showing a continuously increasing interest
in removing barriers faced by physically challenged individuals in communicating and contributing to the society and the
workforce
2.5 “Interactive Systems for Sign Language Learning”
Iurii Krak, ii Kryvonos
In the article the problems of communication of deaf people uses sign language are considering. An analysis of sign language
information transfer which includes humanhands,body, fingersmovements,changeofmimicryand emotionsonhumanfaceis
brought.
2.6 “Moment Based Sign Language Recognition For Indian Languages”
Umang Patel, Aarti G. Ambekar
Communication plays a major role in day to day life. But it is very difficultfornormal peopletocommunicate withdeaf,dumb&
blind people & vice versa.
2.7 “Hand Sign Language Recognition for Bangla Alphabet using Support Vector Machine”
Md Azher Uddin, Shayhan Ameen Chowdhury
The sign language considered as the main language for deaf and dumb people. So, a translatorisneededwhena normal person
wants to talk with a deaf or dumb person. In this paper, we present a framework for recognizing Bangla Sign Language (BSL)
using Support Vector Machine.
2.8 “Sign Language Learning System with Image Sampling and Convolutional Neural Network”
Yangho Ji, Sunmok Kim, Ki-Baek Lee
This paper proposes a novel sign language learning system based on 2D image sampling and concatenating to solve the
problems of conventional sign recognition. The system constructs the training data by samplingandconcatenatingfroma sign
language demonstration video at a certain sampling rate.
2.9 “Machine Learning Techniques for Indian Sign Language Recognition”
Kusumika Krori Dutta, Sunny Arokia Swamy Bellary

Sign language is the only medium through which especially abled people can connect to rest of the world through different
hand gestures. With the advances in machine learning techniques, Hand gesture recognition (HGR) became a very important
research topic.
2.10“Gesture Recognition Using Kinect for Sign Language Translation”
Harsh Vardhan Verma, Eshan Aggarwal, Satish Chandra
Sign Language is a widely used method of communicationamongthecommunityofdeaf-mute people.Itcontains someseriesof
body gestures, which enables a person to interact without the need of spoken words. Although the use of sign languageisvery
popular among the deaf-mute people but the other communities don’t even try to learn it, this creates a gulfofcommunication
and hence becomes a cause of the isolation of physically impaired people.
2.2 EXISTING SYSTEM APPROACH
The system converts the ASL video to simple sentences in English. These simple sentences were converted into audio form.
The searching technique used in video processing module gives the matched folder. Based on the rightmatch,theSignWriting
Image File is retrieved and stored in a folder. This folder served as the input to Natural Language Generation Module.
Fig 1:System Architecture
2.2.1 Disadvantages:
1. It could not accurate.
2. Less stability provides.
3. Time consuming for capturing images.
4. It is not flexible for blind person.
3. CONCLUSION
The sign language recognition system feasible for muted persons because of them can be communicate in deafvia thissystem.
The system can capture hand gesture and navigate the words as in text format it will use for muted persons for seen these
words and understanding the sentences. It is also usable for blind peoples because they need to communicate with normal
persons (using speech recognition).
REFERENCES
[1] Deaf Cambridge Dictionary. (2018). Retrieved from Cambridge Dictionary:
https://dictionary.cambridge.org/dictionary/english/deaf
[2] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009). ImageNet: A Large-Scale Hierarchical Image Database.
Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on (pp. 248-255). IEEE. Miami, FL, USA .

[3] Escalera, S., Baró, X., Gonzàlez, J., Bautista, M., Madadi, M., Reyes, M., . . . Guyon, I. (2014). ChaLearn Looking at People
Challenge 2014: Dataset and Results. Workshop at the European ConferenceonComputerVision(pp.459-473).Springer,.
Cham.
[4] Feichtenhofer, C., Pinz, A., & Wildes, R. P. (2016). Spatiotemporal Residual Networks for Video Action Recognition.
Advances in neural information processing systems, (pp. 3468-3476)
[5] Huang, J., Zhou, W., & Li, H. (2015). Sign Language Recognition using 3Dconvolutional neural networks.IEEEInternational
Conference on Multimedia and Expo (ICME) (pp. 1-6). Turin: IEEE.
[6] Jaoa Carriera, A. Z. (2018). Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset. Computer Vision and
Pattern
[7] Kay, W., Carriera, J., Simonyan, K., Zhang, B., Hillier, C., Vijayanarasimhan, S., . . . Zisserman, A. (2017). The Kinetics Human
Action Video Dataset. Computer Vision and Pattern Recognition, arXiv:1705.06950v1, 1-22.
[8] Feichtenhofer, C., Pinz, A., & Zisserman, A. (2016). Convolutional two-stream network fusion for video action recognition.
Compoter Vision and Pattern Recognition (pp. 1933-1941). Las Vegas Valley: IEEE
[9] Going Deeper with Convolutions,Szegedy et al. CVPR 2015, IEEE Explore.Independently Recurrent Neural Network
(IndRNN): Building A Longer and Deeper RNN, Shuai Li, Wanqing Li, Chris Cook, Ce Zhu, Yanbo Gao, CVPR 2018
[10] Independently Recurrent Neural Network (IndRNN): BuildingALongerandDeeperRNN,ShuaiLi,WanqingLi,ChrisCook,
Ce Zhu, Yanbo Gao, CVPR 2018

IRJET- Hand Gesture based Recognition using CNN Methodology

More Related Content

What's hot (20)

Similar to IRJET- Hand Gesture based Recognition using CNN Methodology (20)

More from IRJET Journal (20)

Recently uploaded (20)

IRJET- Hand Gesture based Recognition using CNN Methodology