Enhancing_Smart_Mirror_with_Human_Health_Care_Perspective_towards_Intelligence_in_IoT_Based_Environment
Enhancing_Smart_Mirror_with_Human_Health_Care_Perspective_towards_Intelligence_in_IoT_Based_Environment
Authorized licensed use limited to: TEXAS INSTRUMENTS VIRTUAL LIBRARY. Downloaded on April 16,2022 at 09:51:57 UTC from IEEE Xplore. Restrictions apply.
2020 2nd International Conference on Power, Energy, Control and Transmission Systems
the mirror can show how a particular item of clothing house into a Smart House. It allows user to control the
would look on the user. Also, it can show which house, using a tablet or a Smartphone or an
clothes fit user and which do not fit anymore. All this application, merely by logging in and clicking on the
can be told by keeping a track of gain/loss in the device. The house is controlled in the sense, that all
weight of the user. its appliances are controlled by the user and this gives
[4] This paper gives emphasis to stimulating and an advantage of saving electricity by switching on
encouraging people who seem sad to the mirror. The only those appliances which are required by the user.
mood of a user is detected through facial recognition [12] This research paper presents an idea of building a
and if the person seems sad, this mirror is an angel for Smart Mirror with Augmented Reality, to help user
him/her. Based on the mood, it displays things on the decide if a specific kind of make-up suits the user or
mirror. Say if the person is sad or confused or just not not. It is just like the project, where the user could
in a good mood, it might just display the pictures of decide what clothes to wear using the mirror. This
his/her loved ones, which is bound to make anyone at mirror helps the person see how they would look with
least come out of a bad mood. a specific kind of make-up, whether it would suit
[5] This research work is all about building a voice- them or not, without putting it on.
based assistant. It comes in line with project and [13] This paper is like the previous one, except that it
products like Alexa, Google Home etc. However, it offers more. It provides the user with different angles
brings the Smart Mirror and such projects together, by of the face, while putting on make-up. It also helps in
integrating them, therefore, giving the best of all the zooming in a part of the face of the user, thereby
worlds. It was created to implement the benefits of a giving easier access to the user. It also provides a
Smart Mirror and a Smart Speaker into one product, voting system, where people online can vote whether,
so that both can be used together. a combination of make-up looks good or not on a
[6] This is an attempt to build a simple Smart Mirror, user.
with gesture recognition. It refers the existing projects [14] The goal of the mirror in this project is to help them
like Home Mirror and Magic Mirror. The aim was to wake-up early, engage them in sports activities and
fabricate a simulation and a prototype of a Smart hence, motivate them and put them in a good mood
Mirror project that could implement gesture for the remaining day. The mirror categorizes people
recognition. into two types - sportspersons and morning persons
[7] This research work talks about using the eye and hence based on their category, motivates them
movement to track focus of a user. There are two accordingly. It detects the moods and feelings of the
zones in this draft of the project, one which is focused user through their gestures and then acts accordingly.
on the eye focused area and the other which is not [15] The Smart Mirror in this project is about developing
focused upon by the eye. The one which is not the Magic Mirror by enhancing its already existing
focused upon the eye is reflected off of the mirror widgets and adding more functionalities to it, to make
surface, to give a better perspective of the things we it all the more attractive. It utilizes of the modules
see. developed by developers around the globe available
[8] This research work presents a Smart Mirror which is for the Magic Mirror to build a mirror that's even
integrated with the Smart Home system. It lets the better.
user control the appliances of his/her home
environment using a mirror. There is also a facial III. SYSTEMANALYSIS
recognition-based authentication system, and after
logging in, the feed is personalized for the user and 3.1SYSTEM ARCHITECTURE
the user can adjust the widgets based on his choice. The image below shows the layout of the
[9] This paper presents a basic Smart Mirror, which lets prototype presents, Fig 1 :‘SMART MIRROR WITH HUMAN
user be updated about the outside world, by HEALTHCAREPERSPECTIVE’.
displaying news information and images. There are
two modes in this mirror, the power off mode and the
power on mode. In power off mode, the mirror acts as
a reflective surface, however, in the power on mode,
it becomes a Smart Mirror.
[10] This project in the paper talks about motivating users
to remain healthy and fit. It focuses on preventing
cardio-metabolic diseases. The mirror acts as a
counsellor, motivating users to be self-aware of their
health conditions and hence, helping them take
actions based on it. It inspires people to do something 3.2EXISTING SYSTEM
on their own and keep them in a healthy lifestyle. It Various models of Smart Mirror projects have
detects the symptoms of cardio-metabolic diseases been developed in the past and some of the most popular
through their faces, and then warns them and inspires projects (DIY) are Home Mirror and Magic Mirror. Most
them to do something about it, by improving their of the Smart Mirrors are either focused on improving the
lifestyle. basic functionalities or just focused on one particular
[11] This project introduces the idea of that of Smart aspect based on the user's choice say Smart Home
House. A Smart House is project, that converts a functionalities or increasing security through Smart
Authorized licensed use limited to: TEXAS INSTRUMENTS VIRTUAL LIBRARY. Downloaded on April 16,2022 at 09:51:57 UTC from IEEE Xplore. Restrictions apply.
2020 2nd International Conference on Power, Energy, Control and Transmission Systems
Mirror.
There are more of Smart Mirror projects than
their Smart Mirror products, because it is still an
upcoming technology and product in the market. Some of
the projects related to Smart Mirror are Smart Speakers
like Alexa, Google Home etc. They are next level projects
and our project can be further enhanced in future to add
functionalities of Smart Speakers.
Magic Mirror is one of the most accepted and widely used
Smart Mirror project.
Fig 3: Proposed Block Diagram with enhanced Features
It has countless third-party modules available IMPLEMENTATION
online, which can be edited and added. A mirror is a Smart Mirror is coded in Python programming
toolthat never goes out of fashion and Smart Mirror has language. The GUI of python acts as a front-face of the
taken advantage of just that, to make us more productive mirror, which is displayed on the mirror. The back-end of
and time-efficient. Some of the Smart Mirror products the mirror is the logic which is also coded in Python. The
that have been deployed and became popular too are- basic flow of the project starts with the server, which
Smart Mirror built by a Japanese company Seraku. fetches data from the APIs used.
Their Smart Mirror is powered by an Android tablet. The server consists of the http requests which gets data in
The unique thing about this Smart Mirror is that, it is the json
built for water conservation and that is achieved format. Json format is the neither converted to text
through the functionality of Water Flow. format or into HTML format to makes its result usable.
Cyber secure Mirror built in Hong Next the data is worked upon either using calculations or
Kong. It can be controlled through a smart phone app just normally displaying it using various fields in the GUI.
or a cable TV remote. Although these Smart Mirrors, The user is registered using facial recognition. The user
have become popular in the market, there is still less logs in using speech and facial recognition.
demand for them, because of their cost price and As soon as the user logs in, user's emotion is displayed,
upcoming technology. It will take a while to become based on its facial attributes. After logging in, the data
used to the new technology. feed personalized for the logged in user is displayed and it
gets updated based on the timer set in the co
4.1REQUIREMENTS
4.1.1 Software Requirements
• Backend of the project is made suing Python which
hosts a simple webpage and opens a web browser
directed to that page
• Website is built using Python GUIPySide2
• Database is used to store information about users
• Dataset is used for emotion recognition and facial
recognition
• Google Speech Recognition is used for speech
identification
Authorized licensed use limited to: TEXAS INSTRUMENTS VIRTUAL LIBRARY. Downloaded on April 16,2022 at 09:51:57 UTC from IEEE Xplore. Restrictions apply.
2020 2nd International Conference on Power, Energy, Control and Transmission Systems
overcomes almost all limitations of other algorithms. lot in their scales, rotation etc. hence, it makes them all
It uses histograms of images and then histograms invariant tin scale and puts them all in the same scale
are compared for facial recognition. Histograms are space. Maxima and minima are calculated using the
created using data vectors of the snapshots of the faces Laplacian of Gaussians.
captured. 4.2.5 SURF
It is enhancement of SIFT algorithm. It speeds it
up. It also uses LOC along with Hessian Matrix as shown
in (Figure). Hessian matrix is a matrix depiction of the
image snapshots of the users used for facial recognition.
Authorized licensed use limited to: TEXAS INSTRUMENTS VIRTUAL LIBRARY. Downloaded on April 16,2022 at 09:51:57 UTC from IEEE Xplore. Restrictions apply.
2020 2nd International Conference on Power, Energy, Control and Transmission Systems
5.2 Clock
Clock is used to display the current time. Current
time is time of the computer the code works upon. Hence,
it depends upon the locale. By default, it is in the 12
hoursformat,butitcanbechangedto24hoursformat too.
Along with the time, it also shows the date and day. Fig 9 : Fitbit in Smart Mirror
Time is updated every millisecond using the
timer function in the code. this will displays the Clock 5.5 Calendar
module in the Smart Mirror Project. CalendarwidgetofPySide2isdisplayedonthemirror
. It is displayed so that the user can see the whole month
at a glance whenever he/she wants.
It helps the user decide when he/she is free to
schedule events or other meets or other tasks.
Authorized licensed use limited to: TEXAS INSTRUMENTS VIRTUAL LIBRARY. Downloaded on April 16,2022 at 09:51:57 UTC from IEEE Xplore. Restrictions apply.
2020 2nd International Conference on Power, Energy, Control and Transmission Systems
5.10 AQI
AQI tells the quality of air in the area you are in.
The area is identified using the logical address of the
computer using ipstack.com.
Based on the AQI value, the message is
Fig 10: Symptoms-Causes Glossary in the Smart Mirror. displayed whether it is safe to go outside or not.
Authorized licensed use limited to: TEXAS INSTRUMENTS VIRTUAL LIBRARY. Downloaded on April 16,2022 at 09:51:57 UTC from IEEE Xplore. Restrictions apply.
2020 2nd International Conference on Power, Energy, Control and Transmission Systems
a lot of languages, and availability of a Python IDLE, to education.In 2012 IEEE Virtual Reality Workshops
simply run the Python code. (VRW) (pp. 115-116).IEEE.
We have used an LED screen to display the [3] Rempel, D., 2018. Smart Home Mirror.
information which can be easily connected to the [4] Olowolayemo, A., Alenazi, S. and Seri, F.A.S.,
Raspberry Pi by means of an HDMI cable. 2018, February. Mirror that Talks: A Self-
Therefore, making the whole setup very compact Motivating Personal Vision Assistant. In
and adjustable even in a small space. There are a total of Proceedings of the 2018 International Conference
ten functionalities/ modules in this project. Functionalities on Image and Graphics Processing (pp. 157-161).
added in this mirror can be bundled into two categories - ACM.
basic functionalities and healthcare functionalities. [5] Assudani, M., Kazi, A.S., Sherke,
Under Basic functionalities comes four features - P.O.,Dwivedi,S.V.andShaikh,Z.S.,2018,January.“H
Weather, Clock, To-Do List and a Calendar Widget. ermione1.0”- A voice Based Home Assistant
Under Health Care functionalities comes six System. In National Conference on Advances in
features - AQI, Medicine Reminders, Doctor Engineering and Applied Science(NCAEAS).
Appointments, Health Tips, Fitbit Data sync and [6] Hossain, M.A., Atrey, P.K. and El Saddik, A.,
Symptoms-causes glossary. 2007. Smart mirror for ambient homeenvironment.
VII. CONCLUSION [7] Meine, R.K., Hewlett Packard Development Co
The main goal of a Smart Mirror is to integrate LP, 2003. System and method for displaying
technology with a normal mirror, to make everyday tasks information on a mirror. U.S.
more productive and less time consuming. After Patent6,560,027.Colantonio, S., Coppini, G.,
requirements gathering phase came the design phase, in Germanese, D.,
which it was decided how the GUI would look. Giorgi,D.,Magrini,M.,Marraccini,P.,Martinelli,M.,
The requirement of the phase starting with the Morales, M.A., Pascali, M.A., Raccichini,
coding of the GUI and integrating all the modules, that G.andRighi, M., 2015. A smart mirror to promote a
have been coded separately, into a single GUI code. This healthy lifestyle. Biosystems Engineering, 138,
phase of integrating all the modules showed a lot of tough pp.33-43.
times but it worked out in the end. Smart Mirror based on [8] Helal, S., Mann, W., El-Zabadani, H., King, J.,
health care perspective has considered as a good deal of Kaddoura, Y. and Jansen, E., 2005. The gator tech
betterment in the future, but as for a prototype, it gives an smart house: A programmable pervasive space.
idea about the healthcare perspective of a Smart Mirror. Computer, (3),pp.50-60.
[9] Rahman, A.M., Tran, T.T., Hossain, S.A. and El
Saddik, A., 2010, October. Augmented rendering
VIII. FUTURE ENHANCEMENT of makeup features in a smart interactive mirror
Some of the functionalities that can be added in the system for decision support in cosmetic products
future in the implementation are - selection.In 2010 IEEE/ACM 14th International
• Adding sensors to the project, like blood pressure or Symposium on Distributed Simulation and Real
heart rate sensor, to automatically sense the values of Time Applications (pp. 203-206).IEEE.
these devices and use it to intelligently tell the user [10] Iwabuchi, E., Nakagawa, M. and Siio, I., 2009,
what should be done based on the results obtained. July. Smart makeup mirror: computer-augmented
• Having a speech recognition scheme or an mirror to aid makeup application. In International
arrangement, that interacts with user like a chat box, Conference on Human-Computer Interaction (pp.
for searching about diseases, symptoms, causes or 495-503). Springer, Berlin,Heidelberg.
anything that the user wants to look for online and [11] Besserer, D., Bäurle, J., Nikic, A., Honold, F.,
display the result on the smart mirror screen. Schüssel, F. and Weber, M., 2016, November.
• Know how much time has been spent by the user in Fitmirror: a smart mirror for positive affect in
front of the mirror, therefore, giving him/her an idea everydayusermorningroutines.InProceedingsofthe
about the daily time usage of the Smart Mirror. Workshop on Multimodal Analyses
• Automatically turn the screen off, if nobody is there enablingArtificial Agents in Human-Machine
in front of the mirror, thereby saving power and Interaction (pp. 48-55). ACM.
electricity. This can be achieved through motion [12] Tang, C.Y. and Pai, N.C., Cal-Comp Electronics
detection. and Communications Co Ltd, 2017.
Smartmirror.U.S. Patent Application 29/566,396.
[16] Vision with Python. Packt PublishingLtd.
IX. REFERENCES [17] Barga, R., Fontama, V., Tok, W.H. and Cabrera-
Cordon, L., 2015. Predictive analytics with
[1] GeorgiyBrussenskiy, Christopher Chiarella, Vishal Microsoft Azure machine learning. Berkely,
Nagda, “Smart Mirror – An Interactive touch- free CA:Apress.
mirror that maximizes time-efficiency and [18] K Meenakshi, R Sunder, A Kumar, N Sharma, “An
productivity”, 2014 intelligent smart tutor system based on emotion
[2] Blum, T., Kleeberger, V., Bichlmeier, C. and analysis and recommendation engine” , 2017
Navab, N., 2012, March. mirracle: An augmented International Conference on IoT and Application
reality magic mirror system for anatomy (ICIOT), 1-4
Authorized licensed use limited to: TEXAS INSTRUMENTS VIRTUAL LIBRARY. Downloaded on April 16,2022 at 09:51:57 UTC from IEEE Xplore. Restrictions apply.
2020 2nd International Conference on Power, Energy, Control and Transmission Systems
Authorized licensed use limited to: TEXAS INSTRUMENTS VIRTUAL LIBRARY. Downloaded on April 16,2022 at 09:51:57 UTC from IEEE Xplore. Restrictions apply.