Rahman2020 Article IoTBasedSmartAssistantForBlind
Rahman2020 Article IoTBasedSmartAssistantForBlind
net/publication/344219052
IoT Based Smart Assistant for Blind Person and Smart Home Using the Bengali
Language
CITATIONS READS
14 6,032
6 authors, including:
All content following this page was uploaded by Mohammad Motiur Rahman on 12 September 2020.
ORIGINAL RESEARCH
IoT Based Smart Assistant for Blind Person and Smart Home Using
the Bengali Language
Md. Wahidur Rahman1 · Rahabul Islam1 · Md. Mahmodul Hasan1 · Shisir Mia1 ·
Mohammad Motiur Rahman1
Abstract
Bengali is the seventh most spoken language, and 300 million people around the world speak in Bengali. The smart home
is responsible for the digitization of modern society. This research reflects an Internet of Things (IoT) based smart solution,
especially for the people who have a visual impairment along with the smart home. The proposed methodology presents
a developed android application which will act as a smart assistant. The smart assistant is responsible for communicating
with a blind person, browsing thoughts, and sharing ideas using the Bengali language. The proposed solution also focuses
on the embodiment of IoT based blind stick. This article further assures a structural foundation of a smart home where blind
persons and regular users will be able to control all the home appliances using Bengali voice command. This paper also
illustrates experimental data analysis, such as transmission time calculation for GSM/GPRS and Bluetooth module, empiri-
cal coverage area calculation, a system usability scale (SUS), and data analysis of blind stick. We have found a SUS score
of 84%. However, the proposed system will be beneficial and handy.
Keywords Smart assistant · Internet of Things (IoT) · Blind stick · Automation · Voice command
SN Computer Science
Vol.:(0123456789)
300 Page 2 of 13 SN Computer Science (2020) 1:300
and regular activities. Sometimes family members or author- project was enriched with several types of sensors, includ-
ized persons are failing to manage all those things required ing Ultrasonic sensor, MQ2 gas sensor, Vibrator sensor, a
for a blind person. Thus, blind persons need an intelligent GPS module that helps to find a blind person location, and
assistant who will ensure their safety as well as an associa- an ESP8266 Wi-Fi module which ensures IoT based applica-
tive partner. tion. The authors used Arduino as a micro-controller. The
On the contrary, the term “Smart Home” become very authors also use a cloud server called “ThingSpeak cloud” to
popular in the modern world. Modern age people are willing meet IoT interactions. In short, the authors drew a very low-
to use smart technology in their regular activities. Thus, the power and cost-effective IoT based project to ensure full sup-
business market [4] of the smart home is forecast to catch port for the blind person. In paper [8], the authors focused
an amount of 53 billion USD. Identically, this is a massive on the development of a smart assistant, especially for blind
sum of money in a particular sector of business. Though it people using voice command. This low-cost system was to
has different types of aspects in business, we use this system build to ascertain safety as well as ensuring real-time virtual
as a solution, especially for a blind person. assistant. In this project, a speech recognizer received voice
Our motivation is to develop a system where blind people command from a blind person, and python interpreted soft-
can talk to an intelligent assistant, having the ability to con- ware converted them into corresponding instructions. These
trol all the electronics components using voice command, instructions ensured users to send or receive an email, access
ensuring his movement, and daily activities. Admittedly, this daily online news, weather forecasting, etc. The authors of
image turned into a real face. Thus, we propose our system the paper [9] had proposed an IoT based smart security sys-
with the following features: tem for a blind person. The proposed scheme utilized a rasp-
berry and camera module responsible for detecting a human
• A smart assistant to communicate with a blind person face from a live video streaming. The exposed face was com-
using the Bengali language. pared with the images stored in the database. The proposed
• An IoT based blind stick as a conduit of a blind person model was able to alert the authorized person of the blind
due to ensure the right movement. person about the identification through the mechanism of
• A smart home for a blind person as well as regular users IoT using an android application. In a study of papers [5–9],
to make a smart way to use home appliances. the authors had only proposed the architectural design of IoT
based blind assistant but no home automation terminology
This article is organized into six sections. “Related Work” had been presented. The authors of the paper [10] proposed
describes the background of related contributions. “Pro- a solution to develop home automation strategies and also
posed Methodology and System Illustration” presents the assure security in a smart home. In this project, the authors
overall methodology and system diagram of our proposed tried to find a path to support the audio visualization sys-
system; “Results and Discussion” provides results and ana- tem which would enable to track out the authorized user
lyzes the results with discussion; “Conclusion” describes the and restrict unauthorized users. This paper also provided
conclusion of this research. “Future Work” depicts a short an efficient handling process of particular home appliances.
description of future work. Home automation was enriched by telephone dialing and
internet connection. A software application installed in PC
acted as a Home assistant, which ensured real-time monitor-
Related Work ing in the smart home on daily activities. The collected data
are stored in the pc for future investigation. Moreover, this
This section presents the background study of previous work paper reflected on low power consumption and made this
related to our proposed system. Many eminent contributors research more nature-friendly. The aim of paper [11] was
had placed a noteworthy sign in IoT and smart assistant on to develop a smart assistant who was capable of interact-
the blind person. The authors of the paper [5] aimed at the ing with household appliances wirelessly. This low cost and
development of a smart stick that was capable of detecting IoT based system were very user-friendly. The core part of
an obstacle and provided navigation. This proposed sys- this project was Raspberry pi. Raspberry pi collected data
tem was able to detect obstacles within 3 m. Authors of from the users along with sensors and interprets those col-
this paper tried to build up an IoT based system using GPS lected required instructions to operate a fan, TV, washing
and GSM modules. When smart stick tracked out danger, it machine, and several types of home appliances. The authors
would send corresponding information to authorized person- also provided additional features called “Five Finger detec-
nel to ensure safety for blind persons. GPS module assured tion,” which would enable users to pause or play musical
the updated location to keep track of the blind person. In songs. The authors of the paper [12] worked on the embodi-
paper [6, 7], the authors worked on a project to develop ment of an intelligent system. The aim was to develop a
an Electrical Travelling Aid (ETA) named BliDar. This project which could be controlled through voice command
SN Computer Science
SN Computer Science (2020) 1:300 Page 3 of 13 300
as well as a remote controller. The authors tried to combine command, communication with the outside world, interrup-
several types of features, including sensor control, facial tion in user’s activity and electric agenda. The authors of
recognition, and speech pattern recognition, which assured papers [10–15] had focused on the development of home
improved intelligent systems. The authors used Raspberry automation using smart assistant for a blind person and
pi as the microcontroller to operate respective home appli- regular users with the evolution of the user’s expectation.
ances as well as to build the required intelligent system. The But the authors did not propose the development of a blind
authors of the paper [13] presented a comparative study of stick with virtual assistant using the mechanism of IoT. The
six open-source schemes for interactive home automation. proposed method implements the structural design of IoT
The authors claimed that visually disabled persons had faced based blind stick and home automation through the voice
difficulties while accessing those applications. Thus, the command.
authors proposed a significant adjustment to overcome these
issues for helping visually impaired persons. The paper [14]
provides a survey on 42 visually impaired persons’ habits Proposed Methodology and System
and expectations about home automation terminology. The Illustration
authors also discussed the flexibility and usability of home
automation in a blind person’s daily routines. The authors In this section, we discuss the methodology of our proposed
also traced some instructions for designers and program- system, along with significant illustrations. We have drawn
mers about these facts. In paper [15], the authors presented a this proposed system for blind persons and regular users. We
report on the evaluation of counting the fear and acceptance have classified our proposed solution in three interconnected
of home automation technology. They have performed their components. Firstly, we mention the working principles of
evolution on eight healthy people about four matrices: voice our intelligent assistant. Secondly, we discuss interactions
SN Computer Science
300 Page 4 of 13 SN Computer Science (2020) 1:300
between the application and IoT based blind stick. Finally, system. We have provided the name of the smart Assistant to
we hit upon the embodiment of IoT enabled smart home. mark unique. Since this whole project is based on the Ben-
gali language, we have provided a Bengali name in response
Working Principle of Smart Assistant to making it more familiar with general users.
A blind person, as well as regular users, can be able to
The core part of our proposed system is a smart assistant communicate with our developed intelligent Assistant. To
who plays a significant role in maintaining the overall
SN Computer Science
SN Computer Science (2020) 1:300 Page 5 of 13 300
SN Computer Science
300 Page 6 of 13 SN Computer Science (2020) 1:300
the connection. If any connection is established, the users stick. Here, we have placed three ultrasonic sensors in
will select the respective paired devices from the Bluetooth front, right and left direction [16–18]. An ultrasonic sen-
paired list. After that, the scheme will be able to read data sor measures distance within 3 m by using sound reflec-
from the sensors placed inside the blind stick and send it tion methodology and echoes. Thus these three sensors
towards the smart assistant. In this way, the blind stick is will alarm the users about the obstacles around him.
handled and ruled by the smart assistant. When a blind person is close to an obstacle, the stick
Figure 3 shows a flow chart of how we can connect the immediately alarms the user by ringing the buzzer. More-
smart home with our proposed smart assistant. In this fig- over, our smart assistant connected with the blind stick
ure, the system first initializes the mobile application as will continuously alert the user in the Bengali language
well as the smart home. After that, the users need to press such as “Samne badha!” A light detector sensor have been
on the button placed on the smart assistant to connect to placed to inform users about day/night. We have used a
the smart home. Then the system will check the Bluetooth GPS/GPRS module in response to ensure IoT based devel-
connection between the app and the smart home. A succes- opment. This module continuously tracks out the blind
sive attempt to establish a Bluetooth connection will result person’s location and alarms their relatives. If the blind
in selecting the required devices from the Bluetooth paired person falls in danger, smart assistant will send an email
list. Thus, the users will be able to control the correspond- with the tracked location to the authorized person. Again,
ing smart home using our proposed smart assistant using GPRS module will take immediate action by performing
the Bengali Language. If the connection is unavailable, the a voice call along with an SMS to the emergency contact.
system will allow the users to recheck the Bluetooth con- Figure 4 shows the corresponding block diagram of our
nection and settings. Then, the users have to pair with the developed blind stick.
required devices. After that, the users will be able to man-
age the smart home. Implementation and Working Principle of Smart
Home
Working Procedure of Smart Blind Stick
This section is going to discuss the depiction of IoT based
In this part, we discuss about the embodiment of blind Smart home [19, 20]. We use home automation to make
stick. In our developed system, we have taken several this system more reliable. Due to operate smart home, the
types of sensors to make the system more reliable and users have to ensure Bluetooth connectivity. A blind person
effective. Figure 4 shows structural development of blind
SN Computer Science
SN Computer Science (2020) 1:300 Page 7 of 13 300
easily controls all the electronic components such as an do these, the users need to provide voice command towards
electric lamp or blub, fan, TV, freeze, washing machine the application in Bengali [21].
through voice commanding the smart assistant in the Ben- Figure 6 shows a flow chart of how the Bengali voice
gali language. command ensures the respective automated system. Herein,
To automate our system, we have placed several types of the system is provided with the Bengali voice command as
sensors including a Gas sensor which responsible for detect- input. After receiving the voice command from the users,
ing any types combustible gases such as LPG, a temperature the Google speech recognizer will respond. Then the system
sensor which is liable for measuring the current temperature converts the upcoming voice into text and compare it with
in the atmosphere, a light sensor which is responsible for the text inside the smart assistant. If the system finds a valid
detecting light and provides more on automation and finally, match, the smart assistant will execute the command and
a motion sensor which detects any types of presence in the perform the corresponding tasks associated with that com-
home. Figure 5 shows the corresponding block diagram and mand. A negative response in tracking valid text will result
working principles of our developed smart home. In this in showing a text notification “Invalid voice command”.
figure, we can identify how smart assistant can control this Then, the system will avail a voice notification in Bengali
automation process. The blind person can easily track out to grant the correct command and continue the procedures
the current status of the home through a smart assistant. To sequentially.
SN Computer Science
300 Page 8 of 13 SN Computer Science (2020) 1:300
Table 1 Transmission delay calculation of the communication Figure 8a shows our experimental data about the Bluetooth
devices region as well as a mobile coverage area in response to assist
Number of test GSM/GPRS module Blue- a blind person through the smart assistant. We have found
(milliseconds) tooth a circular region with a radius of 30 m. Thus, if a blind per-
module son remains in this region, Bluetooth communication will
(milli-
seconds) have occurred. In case of failure of this region, the mobile
connection will be automatically activated using the GPRS
1 1200 55 module. After that, the smart assistant will track blind per-
2 1300 42 son by obeying the principles of IoT [21–24]. This whole
3 1000 52 system is also tested on 220 V AC supply to check its cover-
4 1200 50 age area and capability.
5 1400 58 Thirdly, we have redacted a System Usability Scale
6 1200 55 (SUS) among 15 randomly selected persons about our
7 1200 52 developed Smart assistant as well as the whole system.
8 1300 45 To accomplish this goal, we had covered their eyes by a
9 950 49 piece of cloth; thus, they can act as blind persons. This
10 1200 52 testing result shows whether our smart assistant is suitable
11 1050 60 for interactive IoT based home automation. After walking
12 1100 48 100–150 m, we have found an appeasement result from
13 1300 46 the blind stick and our smart assistant. In case of failure
14 1000 62 of Bluetooth connection, GPS/GPRS module was used.
15 1200 50 SUS result shows that 28% of people strongly endorse,
Average 1173.3 51.73 56% people endorse. That is total of 84% people endorse
in our developed system. People remain 8% neutral, 6%
people disagree, and the rest of 2% strongly disagree in
Results and Discussion our solution. The corresponding result is shown in Fig. 8b.
Fourthly, we have performed an experiment on the blind
This research has been enriched with some particular types stick to check its capability in the real world system. We
of experiments on several topics. First of all, we have experi- have used three ultrasonic sensors in the left, right, and
mented with the transmission time of Bluetooth and GPS/ forward direction in our developed blind stick. Table 2
GPRS module. To accomplish this goal, we have captured 15 shows the corresponding results of our respective experi-
samples to examine transmission. Admittedly, these experi- ment, according to Eq. 1. In this equation:
ments are only to track out how fast we can transmit com- v∗t
mand within a short region and in a large geographical area. s= , (1)
2
Table 1 shows the results of separate experiments. We have
performed these experiments on both blind stick and smart where, v is the speed of sound = 340m∕s , t is the time
home. We found that Bluetooth data transmission is faster required to travel and receive, s is the corresponding
than GPS/GPRS data transmission. This table explains how distance.
transmission time is varied from one another. Through this Time taken in the pulse is the travel time of ultrasonic
table, we can easily track out the difference in data transmis- signals, which we have needed only half of this. Therefore,
sion between them. We have found an average data transmis- time is taken as time/2.
sion time of 51.73 ms for the Bluetooth module using the Here, Speed of sound at sea level = 343 m/s or
application. On the other hand, we have uncovered an aver- 34,300 cm/s.
age time of 1173.3 ms required for data transmission through Thus, Distance = 17,150 * Time (unit cm).
the GPRS module. This experiment indicates which module Finally, we have ensured the structural development
is more effective for IoT-based solutions. Figure 7 shows model of our proposed solution. In the development pro-
how transmission time is scattered from the mean value. The cess, we first developed an IoT based smart blind stick
figure also illustrates the region of each transmission delay along with the android application. After completing this
according to the average value. process, we have moved forward to attach a smart home
Secondly, we have tried to figure out the range of Blue- with our proposed model. Figure 9 shows the developed
tooth in our developed scheme. This experiment con- blind stick, the android application as a smart assistant and
firms connectivity with the region from a smart assistant. the smart home, respectively.
SN Computer Science
SN Computer Science (2020) 1:300 Page 9 of 13 300
Conclusion the uniqueness and novelty of this research. Using the Ben-
gali language, users can communicate with a smart assis-
We have introduced a smart solution for blind people as well tant. The Bengali language is also responsible for control-
as regular users. The users can easily afford this solution in ling all the home appliances. We have operated all the home
their regular activities. Bengali voice command also assures appliances in the 220 V AC supply. Again, the development
SN Computer Science
300 Page 10 of 13 SN Computer Science (2020) 1:300
Future Work
SN Computer Science
SN Computer Science (2020) 1:300 Page 11 of 13 300
SN Computer Science
300 Page 12 of 13 SN Computer Science (2020) 1:300
Compliance with Ethical Standards 11. Chayapathy V, Anitha GS, Sharath B. IOT based home automation
by using personal assistant. In: 2017 International Conference on
Smart Technologies for Smart Nation (SmartTechCon), Banga-
Conflict of Interest The authors declare that they have no conflict of
lore; 2017.
interest
12. Al-Amin M, Aman SZ. Design of an Intelligent Home Assistant.
In: 2016 7th International Conference on Intelligent Systems,
Modelling and Simulation (ISMS), Bangkok; 2016. pp. 69–71.
References https://doi.org/10.1109/ISMS.2016.34.
13. de Oliveira GAA, de Bettio RW, Freire AP. Accessibility of the
smart home for users with visual disabilities: an evaluation of
1. The SEMTECH website. Available: https://www.semtech.com/
open source mobile applications for home automation. In: Pro-
lora/what-is-lora. [Last accessed on 24 Aug 2020]
ceedings of the 15th Brazilian Symposium on Human Factors in
2. The CODEMITH DEV website. Available: https://codesmithd
Computing Systems. 2016. pp. 1–10
ev.com/17-statistics-that-explain-the-rise-of-voice-assist ants/.
14. Leporini B, Buzzi M. Home automation for an independent living:
[Last accessed on 24 Aug 2020]
investigating the needs of visually impaired people. In: Proceed-
3. The DhakaTribune website. Available: https://www.dhakatribu
ings of the Internet of Accessible Things. 2018. pp. 1–9.
ne.com/bangladesh/2018/10/12/experts-750-000-people-suffe
15. Portet F, Vacher M, Golanski C, Roux C, Meillon B. Design
r-from-blindn ess-in-bangla desh. [Last accessed on 24 Aug
and evaluation of a smart home voice interface for the elderly:
2020]
acceptability and objection aspects. Pers Ubiquit Comput.
4. The STATISTA website. Available: https://www.statista.com/
2013;17(1):127–44.
topics/2430/smart-homes/. [Last accessed on 24 Aug 2020]
16. John LM. An IoT based human assistant robot for blind per-
5. Radhika R, Pai PG, Rakshitha S, Srinath R. Implementation of
son. Tathapi with ISSN 2320-0693 is an UGC CARE Journal.
smart stick for obstacle detection and navigation. Int J Latest
2020;19(10):236–50.
Res Eng Technol. 2016;2(5):45–50.
17. Wahidur Rahman Md, Islam R, Harun-Ar-Rashid Md. IoT based
6. Saquib Z, Murari V, Bhargav SN. BlinDar: An invisible eye for
blind person’s stick. Int J Comput Appl. 2018;182(16):19–211.
the blind people making life easy for the blind with Internet of
18. Kumar M, Shimi SL. Voice recognition based home automation
Things (IoT). In: 2017 2nd IEEE International Conference on
system for paralyzed people. Int J Adv Res Electro Commun Eng
Recent Trends in Electronics, Information and Communication
(IJARECE). 2015;4(10):2508–15.
Technology (RTEICT). 2017. pp. 71–75. IEEE.
19. Khattar S, Sachdeva A, Kumar R, Gupta R. Smart home with
7. Sathya Narayanan E, Nithin BP, Vidhyasagar P. IoT based smart
virtual assistant using raspberry Pi. In: 2019 9th International
walking cane for typhlotic with voice assistance. In 2016 Online
Conference on Cloud Computing, Data Science and Engineering
International Conference on Green Engineering and Technolo-
(Confluence). 2019. pp. 576–579. IEEE.
gies (IC-GET). 2016. pp. 1–6. IEEE.
20. AlShu’eili H, Gupta GS, Mukhopadhyay S. Voice recognition
8. Bose VP, Malpthak A, Bansal U , Harsola A. Digital assistant
based wireless home automation system. In: 2011 4th Interna-
for the blind. In: 2017 2nd International Conference for Conver-
tional Conference on Mechatronics (ICOM). 2011. pp. 1–6. IEEE.
gence in Technology (I2CT), Mumbai; 2017. pp. 1250–1253
21. Kirankumar T, Bhavani B. A Sustainable automated system for
9. Fernandes R, Rodrigues AP, Sudeepa KB. IoT based smart secu-
elderly people using voice recognition and touch screen technol-
rity for the blind. In: 2018 Fourteenth International Conference
ogy. Int J Sci Res (IJSR). 2013;2(8):265–7.
on Information Processing (ICINPRO). 2018. pp. 1–4. IEEE.
22. Demir E, Köseoğlu E, Sokullu R, Şeker B. Smart home assistant
10. Balasubramanian K, Cellatoglu A. Improvements in home
for ambient assisted living of elderly people with dementia.
automation strategies for designing apparatus for efficient
Procedia Comput Sci. 2017;113:609–14.
smart home. IEEE Trans Consum Electron. November
2008;54(4):1681–7.
SN Computer Science
SN Computer Science (2020) 1:300 Page 13 of 13 300
23. Gunawan TS, Yaldi IRH, Kartiwi M, Mansor H. Performance 26. Abu MA, Nordin SF, Suboh MZ, Yid MSM, Ramli AF. Design
evaluation of smart home system using internet of things. Int J and development of home security systems based on internet of
Elect Comput Eng. 2018;8(1):400. things via favoriot platform. Int J Appl Eng Res. 2018;13(2):1253.
24. Khan M, Silva BN, Han K. Internet of things based energy aware
smart home control system. IEEE Access. 2016;4:7556–66. Publisher’s Note Springer Nature remains neutral with regard to
25. Rahman W, Rashid HA, Islam R, Rahman MM. Embodiment of jurisdictional claims in published maps and institutional affiliations.
IoT based smart home security system. In: 2018 International
Journal for Research in Applied Science and Engineering Tech-
nology (IJRASET). 2017. pp. 05–14. ijraset
SN Computer Science