Proceedings of the International Conference on Artificial Intelligence and Smart Systems (ICAIS-2021)

IEEE Xplore Part Number: CFP21OAB-ART; ISBN: 978-1-7281-9537-7

Artificial Intelligence based Smart Door with

Face Mask Detection
Baluprithviraj.K.N Bharathi.K.R Chendhuran.S
Department of EIE Department of EIE Department of EIE
Kongu Engineering College
Kongu Engineering College
Erode, Tamil Nadu, India
Kongu Engineering College
2021 International Conference on Artificial Intelligence and Smart Systems (ICAIS) | 978-1-7281-9537-7/20/$31.00 ©2021 IEEE | DOI: 10.1109/ICAIS50930.2021.9395807

Erode, Tamil Nadu, India Erode, Tamil Nadu, India

[email protected] m [email protected] m
[email protected]
Lokeshwaran.P
Department of EIE
Kongu Engineering College
Erode, Tamil Nadu, India
[email protected]

Abstract—In this pandemic situation, health plays an The problem is related to general object detection to detect
important role in everyone’s life. Most of the people are not the classes of objects and face detection is to detect a
aware of preventing themselves and their surroundings from particular class of objects. Nowadays in public places like
this pandemic. Face mask is essential to prevent ourselves and malls, temples, hospitals, wearing face mask is mandatory so
others. S o, people are in need to wear face mask regularly. that our smart device will be very useful in those public
People who visit home won’t wear mask due to their places. This smart device can be inserted in the entry point of
unawareness which may affect people. People may not know if those places so that people can prevent themselves from this
someone visits their home when they are not there. AI based pandemic. The Fig.1 and 2 depicts the working of the Face
smart device (Raspberry pi with AI model with camera) is
proposed in this project which identifies whether a person is
Mask Detection Platform.
wearing face mask and gives us an alert message (via mobile
app). This device is integrated with a mobile app. Mobile app
identifies if someone enters home when people are not
physically present in their home. This smart device
automatically opens the door only if people wear face mask.
This device works both day and night. It can be used in
multiple places like malls, shops, hospitals and temples

Keywords—Face mask, AI model, Automatic opening and

closing of door, Mobile app

I. INT RODUCT ION

In this paper the idea is in this pandemic situation, face mask is
the essential thing that everyone should wear. Due to unavoidable
situations, people may visit home and people will be unaware of
wearing face mask sometimes. This may affect people and so the
Fig.1. Detection of face mask: with mask
solution for this problem is designing a smart device which will
detect face mask. It will also act as smart door and if the person
comes home wearing a face mask, it will open the door and if not,
it won’t open the door. It is integrated with a mobile app so that
owner will receive notification, if someone visits home. In this
pandemic situation, checking of temperature plays a vital role in
public places so that this smart device can also be integrated with
the temperature measurement so that it would be more benefited to
the society

II. FACE MASK DETECTION

Wearing a face mask has become a mandatory rule in
almost all the country because of the COVID 19 Pandemic.
It is been an effective tool in controlling the spread of the Fig.2. Detection of face mask: without mask
COVID 19 virus. So wearing a face mask has created a
normal situation in the pandemic. However regulating and
monitoring large group of people wearing a mask is a tedious III. FACE RECOGNITION SYSTEM
process. Also it led to many hazels among people in public Facial recognition refers to the optimal way to find and
places like Super market, Mall, Church etc. It is really a recognize the facial features using any technology like
challenging job to recognize a face mask on any monitoring Artificial Intelligence with the help of any hardware device
system. Face mask detection is to detect whether a person to capture the images like Video Cameras. Generally in
wearing a mask or not and to detect the location of the face. facial recognition software the facial features from the live

978-1-7281-9537-7/21/$31.00 ©2021 IEEE 543

Authorized licensed use limited to: Cornell University Library. Downloaded on May 22,2021 at 23:36:45 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of the International Conference on Artificial Intelligence and Smart Systems (ICAIS-2021)
IEEE Xplore Part Number: CFP21OAB-ART; ISBN: 978-1-7281-9537-7

image or video will be mapped using biometric details. The circuit face segmentation and detection using semantic
result is achieved by comparing these images with segmentation on any arbitrary. To perform the unmanned
predefined dataset containing the desired images. Also semantic segmentation of human face. This method reduces
Several countries initiative and the demand for surveillance the problem of erroneous predictions and find applications in
systems increased the need for facial recognition. So in the latest tasks such as facial part detection is also the task[8].
trends of market, facial recognition formed to raise rapidly
and the value can be expected to reach a value of 9.06 billion
USD by 2024[1]. The mask detection system is presented to detect a type
of mask and masks of different shapes from the video
It is always good to adapt and design according to the
market demands. Because of the pandemic, there is a huge streams for following the rules that applied. Deep Learning
algorithm and PyTorch Library is used for detection from the
demand for optimal facial recognition systems. The AI
powered facial recognition system uses the biometric data of images/video streams. The proposed system is developed to
detect the people with masks or without masks. The
each individual to obtain the facial features to classify the
images accordingly. It also can alert if someone not wearing MobileNETV2 classifier training process is used as the first
phase of the model done by using the PyTorch framework of
the mask. It can be done using mobile apps, software etc. [2].
deep learning and implementation formed with the OPENCV
TensorFlow is used in the Convolutional Neural Network of python [9]. The Multi-Task Cascaded Convolutional
(CNN) model which includes Keras library and OpenCV to Neural Network (MTCNN) approach is used for the
identify the face mask to protect themselves. For building occluded faced detection problem. The embedded model in
this model, face mask dataset is used and the value consist of the Google FaceNet is performed by the facial features
about 1,376 images with 690 images containing people with extraction. CNN is successively applied for detection and
face masks and without face masks available with recognition problems. The Support Vector Machine (SVM)
686 images. The generated images are required to build classification task has been characterized [10]. The FaceNet
a CNN model using TensorFlow to identify a face mask from trained model tested on the datasets to show the better face
the webcam of PC. recognition rates. Receiver Operating Characteristics curve is
a visualization technique is used to displays the definite
IV LITERATURE REVIEW presentation of a classification model. FaceNet model is a
Deep learning method is proposed to distinguis h facial baseline for a deep network and included with 22 deep
recognition and the person is recognized whether the person convolutional network layers. This model is the
is wearing a facemask or without a facemask. It is characteristics of identification, face recognition,
implemented by using a Raspberry Pi, OpenCV,Tensor verification, detection of images, prediction of real images
Flow and Python programming languages. An alarm system and clustering of neural networks [14].
consists with the real time facemask recognition by way of
Convolutional Neural Networks which need for noticing of V ATTRIBUTES
persons wearing a facemask or without the facemask. The To detect to a face, we need its complete facial features.
validation accuracy was achieved with high performance by So it becomes difficult to detect a masked face since it gives
the trained model of the CNN [4]. A new facemask-wearing incomplete facial features [15]. However modern
condition identification method was developed by technologies have come to rescue us. With the help of
combining of classification networks (SRCNet), image Artificial intelligence and Machine Learning. We can
super-resolution and measures a three category classification develop different models for different applications such as
problem based on 2D facial images. Image pre-processing, classification, processing etc., with accurate results. However
facial detection cropping, facemask wearing condition it requires several attributes such as Face orientation, Eye
identification are the major steps in the proposed algorithm. location, Mask location, Face location bounded by shapes
identification of facemask-wearing conditions with the High such as a square:
accuracy was achieved by the proposed SRCNet. To x Shape ie., square is annotated by the Location of
improve the performance on low-quality images and the Faces, annotated by a shape
proposed method achieves higher accuracy [5]. x Face Alignment, includes left-front front, left, , right,
right-front.
The proposed system is designed to identify either a x Eye centres which mask as the Location of eyes
person is wearing a mask or without mask and reporting the x Location of Masks with necessity details are
corresponding authority in a smart city network. Deep annotated by a shape i.e., rectangles.
learning plays an amazing role in object detection which is x Different type of Mask presents in the market: i.e,
trained on the dataset that consists of images of people from man-made masks or without logo, face mask covered
various sources. Labeled image data are trained and testing by hand, etc.
the dataset where the facial images with mask and the person x Sealing Degree, the function of face for recognition
without the are detected. The system is designed with the consist into four regions - eyes, nose, chin and
with a specific components capture the person GPS location mouth.
of the CCTV camera and The corresponding authority cwn
detetct the image with meticulous time is sent via SMS to the
corresponding authority [6]. The proposed method is VI CONNECTING AI WITH DATABASE
presented to generate the accurate face segmentation masks
from arbitrary image size. The proposed model also shows
the recognizing non-frontal faces. Multiple facial marks in a Once the device detects number of people entering and
single frame can be detected[7]. For any arbitrary, The detect whether those people wearing face mask in accurate

978-1-7281-9537-7/21/$31.00 ©2021 IEEE 544

Authorized licensed use limited to: Cornell University Library. Downloaded on May 22,2021 at 23:36:45 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of the International Conference on Artificial Intelligence and Smart Systems (ICAIS-2021)
IEEE Xplore Part Number: CFP21OAB-ART; ISBN: 978-1-7281-9537-7

way and if someone is detected with no face mask then alert

notification will be sent [16].
Detection of face mask is completely based on python
and linking with database so also based on python. By
adding firebase library and sending a json data of number of
people present, number of people taken responsible to wear
mask or number of people without wearing the mask [17].
This value will reflect in app and real time database needs
library files to be imported in the device and create a project
in firebase database which is shown in fig.3

Fig.5. Simulation website

A. Connecting database with simulation software

Create a web app in firebase to interface with the simulation
software. After creating a web app in firebase, it will
generate a code snippet as in fig.6.

Fig.3. Connecting AI with firebase

VII SETTING UP DATABASE
To start a new project, create an account in firebase
database and open a new project shown in fig. 4.

Fig. 6. Code for connecting simulation websie with firebase

By the code shown in fig 7, the data from the firebase

can be fetched and according to the variations in the data,
the door will automatically open and close. This particular
code is responsible for simulation. Since we have added
firebase analytics, we can get the usage of database.

Fig.4. Firebase
Open up the project which is created and produce a
realtime database in build and make security and testing
mood to read and write value in database [18]. After creating
a database a link and security key will be generated which
will be using in this python code.
VII SIMULATION
To get a virtual door it needs a simulation software there
are no simulation software available to simulate a door
using firebase and so by using html css created a new
simulation software i.e. a webpage which is local hosted in a
device shown in fig.5.
Fig.7 Interfacing AI with firebase and website

978-1-7281-9537-7/21/$31.00 ©2021 IEEE 545

Authorized licensed use limited to: Cornell University Library. Downloaded on May 22,2021 at 23:36:45 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of the International Conference on Artificial Intelligence and Smart Systems (ICAIS-2021)
IEEE Xplore Part Number: CFP21OAB-ART; ISBN: 978-1-7281-9537-7

VIII MOBILE APPLICATION

Mobile application helps the owner to know whether
someone enters their home when they are not available in the
home. It also acts as a security system. When someone enters
home, smart device will capture the image and immediately
sends to the owner via mobile app so that the owner can
allow the known person inside.

A. Create app using MIT App Inventor

The MIT App Inventor which is an intuitive, visual
programming environment is shown in fig. 8 that allows
everyone even children to build fully functional apps for Fig.10 Front panel of mobile app

C. Back End
The database values which are continuously updated by
the AI model device which is used to give alert notification
and monitor the activities shown in fig. 11. The back end is
completely coded with google's blocky language. which is
simple and easy to develop from the particular code and data
can be fetched continuously from the database.

Fig 8. MIT App Inventor

Mobile phones and tablets. The people innovative to MIT

App Inventor with the responsible with create invention take
a generate the first app up and time required for running is
less than 30 minutes.

B. Front End
In the designer page, there are multiple options like
labels, textbox, buttons shown in 9. In the front panel
number of people entering, detects the number of people
taken responsibility to wear masks and specific number of Fig.11. Interfacing AI model with mobile app
people not wearing masks are displayed using labels in the
app inventor where the count value is taken from the
firebase database shown in fig.10. IX EXISTING METHOD
The World is now facing unprecedented pandemic like
this because COVID 19 is having high spreading and
reproductive rates than any other germ which caused
pandemic before. Until people find an efficient vaccine, the
only way to curb the spread of corona virus by wearing the
mask. However, the process of observing and identifying the
huge number of groups people is tough. The existing
technology is available for detecting the face mask but our
idea differs from the existing method is that the smart device
will be inserted in the door and the mobile is designed so that
owner will get notification and it also acts as a smart door
and if the person wore face mask, the door will open
automatically and if not the door won’t open.

Fig.9. Designing App

978-1-7281-9537-7/21/$31.00 ©2021 IEEE 546

Authorized licensed use limited to: Cornell University Library. Downloaded on May 22,2021 at 23:36:45 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of the International Conference on Artificial Intelligence and Smart Systems (ICAIS-2021)
IEEE Xplore Part Number: CFP21OAB-ART; ISBN: 978-1-7281-9537-7

X. PROPOSED METHOD opens and closes the door and it is integrated with a mobile
The proposed system detects whether a individual app which send notifications to owner. The software
developed in the project is about detecting the face with
is wearing of a face mask and follows the safety
mask or without mask which identifies a person is wearing
measures or not using a detector which employs SSD face mask and gives us alert (via mobile app). This device is
for face detection and a neural network to detect integrated with a mobile app. Mobile app let us to know if
presence of a face mask. If that detects a person with someone enters home when people are not physically
face mask the door will open and close and there will present in their home. This smart device automatically
be an alert from the mobile app if people enters opens the door only if people wear face mask. This device
without mask. From fig, 12, the block diagram for works both day and night. It can be used in multiple places
smart door with mask detection consists of Firebase, like malls, shops, hospitals, temples and other public places.
mobile app, push notification, device and AI model, The merits of face mask detection is that people can prevent
simulation of door, html5, css , bootstrap. Convolution themselves from this pandemic and it also acts as a smart
door .The demerits of this smart device is that during night
operation is an element-wise matrix multiplication
time the image won’t be clearly visible .
operation.Convolutional layers take the three-dimensional
input matrix we mentioned before and they pass a filter (also
known as convolutional kernel) considered in the image, it
formed to creating in a small window of pixels at a time (i.e REFERENCES
3x3 pixels) and this window until the whole image has been
scanned. The dot product is used The convolutional
[1] Vinitha, Velantina, “COVID-19 Facemask Detection
operation calculates the dot product generation of the pixel with Deep learning and Computer Vision”, IRJET,
values in the current filter window including with the vo.7, issue 8, pp.3127-3132, 2020
weights defined in the filter. The output of this operation is [2] Meenpal, “Facial Mask Detection using Semantic
the final convoluted image. Segmentation”, International Conference on
Computing, Communications and Security, 2019.
Convolutional Neural Networks (CNN) are neural networks [3] Sammy V. Militante, Nanette V. Dionisio, “Real-Time
Facemask Recongition with Alarm System using Deep
most commonly used to analyze images. Learning”, 11th IEEE Control and System Graduate
A CNN receives an image as an input in the form of a 3D Research Colloquium (ICSGRC 2020), pp.106-110,
matrix. The first two dimensions corresponds to the width 2020
and height of the image in pixels while the third one [4] Bosheng Qin, Dongxiao Li, “Identifying Facemask-
Wearing Condition Using Image Super-Resolution with
corresponds to the RGB values of each pixel. Classification Network to Prevent COVID-19”,
CNNs consist of the following sequential modules (each one Sensors, MDPI, pp.1-23, 2020
may contain more than one layer) [5] Mohammad Marufur Rahman, Md. Motaleb Hossen
Manik, Md.Milon Islam, Saifuddin Mahmud, Jong-
1. Convolution Hoon Kim, “An Automated System to Limit COVID-19
2. Activation function Using Facial Mask Detection in Smart City Network”,
IEEE International IOT, Electronics and Mechatronics
3. Pooling Conference (IEMTRONICS), 2020.
4. Fully connected layers [6] Toshanlal Meenpal, Ashutosh Balakrishnan, Amit
5. Output layer Verma, “Facial Mask Detection using Semantic
Segmentation”, 4th International Conference on
Computing, Communications and Security (ICCS),
pp.1-5, 2019
[7] Sneha Sen, Harish Patidar, “Face Mask Detection
System for COVID_19 Pandemic Precautions using
Deep Learning Method”, Journal of Emerging
Technologies and Innoative Research (JETIR), vol. 7,
issue 10, pp.16-21
[8] Md.Sabbir Ejaz and Md.Rabiul Islam, “Masked Face
Recognition using Convolutional Neural Network”,
International Conference on Sustainable Technologies
for Industry 4.0(STI), 2019
[9] Geraldine B. Mangmang,”Face Mask Usage Detection
Using Inception Network”, JARDCS, Vol.12, issue,7,
pp:1660-1667, 2020.
[10] Anubhav Kumar Singh, “Analysis & Study of Face
Recognition and Detection”, IRJET, Vol.07, issue 04
pp. 3289-3293, 2020
Fig.12. Block diagram for smart door [11] Karthik A Patil, Niteen Vittalkar, “Smart Door Locking
System using IoT”, IRJET, Vol.07 issue: 05, pp. 30890-
30894, 2020
[12] Nikita Raut, Varun Dhuldhoya, “A Review on Various
X1.RESULT AND DISCUSSION Techniques for Face Detection”, IRJET, Vol.05, issue:
10 pp. 108-110, 2018
This smart device automatically opens the door [13] Khaoula Karimi, “Secure Smart Door Lock System
only if people wear face mask and notify people who does based on Arduino and Smartphone App”, JARDCS,
not wear mask. It acts as smart door which automatically Vol.12 issue 0, pp.407-414, 2020

978-1-7281-9537-7/21/$31.00 ©2021 IEEE 547

Authorized licensed use limited to: Cornell University Library. Downloaded on May 22,2021 at 23:36:45 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of the International Conference on Artificial Intelligence and Smart Systems (ICAIS-2021)
IEEE Xplore Part Number: CFP21OAB-ART; ISBN: 978-1-7281-9537-7

[14] Gunawan, Gani, Rahman, Kartiwi, “Development of

Face Recognition on Raspberry Pi for Security
Enhancement of Smart Home System”, IJEEI, Vol. 5,
issue 4, pp. 317-325, 2017.
[15] H.Archana, A.Arunjagadeshwaran, R.Kamatchi,
K.N.Baluprithviraj, “Disinfectants Spraying BOT in
Silkworm Rearing House”,IJIRT, vol.5, issue 9, pp.150-
155, 2019
[16] Baluprithviraj.K.N, Idhikash.S, KirubaharanS.R,
Ashwin.V, Guru Prasad.R.S, “Automatic Penalty of
Vehicles for Violation of Traffic Rules using IoT”,
International Journal of Recent Technology and Engine
ering, vol.8, issue 6, pp.415-420, 2020
[17] Mengbo, “Multi-view face detection using frontal face
detector”, IEEJ Transactions on Electrical and
Electronic Engineering, Vol.13 issue 07, pp. 1011-
1019, 2018
[18] Viraktamath S.V,“Face Detection and Tracking using
OpenCV”, The SIJ Transactions on Computer
Networks & Communication Engineering, Vol.04
Issue: 03 pp. 01-06, 2016

978-1-7281-9537-7/21/$31.00 ©2021 IEEE 548

Authorized licensed use limited to: Cornell University Library. Downloaded on May 22,2021 at 23:36:45 UTC from IEEE Xplore. Restrictions apply.