1 Esp32 2021
1 Esp32 2021
28th 2021
Abstract—The rapid growth of technology in the modern some of the systems implemented with the help of
society has raised many questions on the terms like security and raspberry pi are quite expensive and complex to build [2].
privacy. Due to the evolution in the technology and The above said methods are not easy to maintain and can
industrialization the terms like security and privacy has consume lot of electrical energy. The devices can also be
become imperative for a common person. Authentication is a worn out easily.
key factor which helps for the identification of authorized
people and helps in eradicating fraudulent activities, robberies, The proposed system using facial recognition helps in
and many other social crimes. Most of the crimes are due to the preventing the limitations of the existing methods. The
vulnerabilities in the door locking systems which can be easily
proposed method consists of a camera sensor esp32 for
accessible by the outsiders. Though there are solutions like
continuously monitoring the face of a person. The faces of
smart doorbells and video streaming, which have limitations
like heavy cost, complex and have loopholes in the security
the authorized persons are stored in the SD card of the
issues. To diminish the limitations and to enhance the security esp32 cam [3, 4]. Whenever a person stands in front of the
Smart door unlock systems using face recognition is proposed. door the esp32 cam recognizes the face of the person by
The proposed system consists of a camera sensor popularly comparing the authorized faces of authorized persons in
known as esp32-cam for storing the pictures of persons and for the database. The “AI-Thinker” algorithm is used in
live streaming. The proposed system recognizes the face of the detecting and recognizing the face of a person. If the
person standing in front of the door with the help AI-Thinker person is an authorized person then the door gets unlocked
in the esp32-cam. The face of the person is compared with the automatically [5]. If the person is an unauthorized person
faces of the authorized persons which are stored in the SD card then the door gets locked automatically. The proposed
of esp32-cam.If the person is an authorized person then the system is of low cost and easy to maintain. It can help the
door gets unlocked which can be achieved with the hardware disabled people for accessing the doors with ease. The
component solenoid lock. If the person is an unauthorized proposed system can prevent intruders from entering in to
person then the door will be locked. The proposed system helps the house. It helps in providing security to common people
in adapting from traditional mechanical lock methods to with affordable price. It can overcome the main
enhanced security methods. It also helps in case of losing keys disadvantage of traditional lock systems which are
and helpful for disabled persons with easier access. restricted to number of keys.
Keywords— Authentication, esp32-cam, solenoid lock,
The proposed system consists of Arduino idle software
security, facial recognition, AI-Thinker.
for writing the code. It consists of a relay module and ftdi
board as hardware components. It can help in eradicating
I. INTRODUCTION social crimes and theft activities. The rest of the paper
deals with the related work, proposed methodology, results
Security plays a vital role in day to day life. and analysis, conclusion and references.
Authorization helps humans for getting access to some
authorized information. Security not only helps in
preserving confidentiality but also helps in preserving the II. LITERATURE SURVEY
lives of human beings. Many criminal activities are takes
Jahnavi S,et.al proposed smart anti-theft locking
place in the society due to lack of security, this sheds light
system[1] that is trained with the features of the authorized
on the security to consider it as a notable issue. So there is
persons and is stored in the database. The process begins
a need to enhance the level of security for maintaining
with capturing the image of a person using a raspberry pi
integrity [1]. Authorization can be done by any means like
camera. It is followed by face detection done using the
finger prints, face recognition, password, pin configuration
Viola-Jones algorithm. This algorithm provides greater
etc. Out of these finger prints and facial recognition can be
accuracy in the detection of the object. The feature
a herculean task for an intruder to exploit. There are many
extraction and face detection are done using the Local
methods implemented for accessing the doors
Binary Pattern (LBP) algorithm. This algorithm highlights
automatically which helps in maintaining security. But
the facial characteristics of the original image by creating
some of the proposed methods like accessing of doors with
an intermediate image. The Principal Component Analysis
the help of doorbells failed in preserving security, and
(PCA) algorithm is used to extract the local features of the
Authorized licensed use limited to: PES University Bengaluru. Downloaded on January 18,2023 at 11:58:31 UTC from IEEE Xplore. Restrictions apply.
intermediate image. Each face image in the test set is If the person’s face is recognized as well as the fingerprint
compared with the images in the training set. If there is a is matched through the android app, the door is unlocked.
match the access is provided to the authorized person. If The main limitations in this work are i) This project
not, the access to the door is denied and an alarm is raised involves more complexity in maintaining the security ii) It
alerting the person. The main pitfall in this work is the is expensive as it uses more electronic components.
algorithm used for detecting the facial points trains the
model a bit slow. Abdul Basit, et .al came up with an idea of Wearable
Device Used for Smart Doorbell in Home Automation
Farhat Lamia Barsha, et.al comes with an idea of an System [8] which are used for unlocking the doors by
Automated Door Accessing System for visually impaired Biometric Authentication, it introduces the idea of storing
people [2]. This system will be attached to the door to the Biometrics of the visitors in a local database and
detect visitors through face recognition and inform users recognizes them later .When a Visitor comes and rang the
whether the guest is known or unknown while users can bell, it captures all the necessary information like Face
automatically open the door through voice command using image, Iris image, and Voice of the visitor. It keeps records
a desktop application. This device can identify any quite of all the visitors that visit the house. Also, it alerts about
suspicious activities like an individual roaming ahead of the any motion outside the house and enables interaction with
door for an extended time and it also generates an alert in the visitor from anywhere. It can be controlled by Smart
the audio form in any kind of situation. The detection will Ring, which is essential for monitoring and controlling the
be conducted on the captured image to identify whether the device. The main pitfalls in this project are i) This system is
person in that image is carrying any dangerous weapon. If complex to built ii) It is also expensive.
an intruder wants to break in, a message will be
automatically sent to the user relatives. Outside the door, III. PROPOSED METHODOLOGY
sonar is placed which can detect the presence of a person. A
Speaker is used, which gives voice alert and it is connected The main goal of the project is to access the door without
to a USB port of the raspberry pi cam. A Webcam is used human intervention. This can be done with the help of
to detect the face. The system uses a detector circuit that's esp32 cam [6]. The proposed system uses facial and
built with Arduino to detect risky metal instruments. The detection and recognition for allowing the access to the
main drawbacks in this system are i) It will not show the authorized persons. This can be achieved by collecting and
face of the unauthorized person ii) Complex to build and storing the pictures of the authorized persons in the
maintain. database, where the espcam can have access to the
stored pictures in the database through web server.
Adarsh V Patil, et .al proposed remotely accessible
smart door lock [5] can be completely monitored and When a person appears at the door, the ESP32 CAM
controlled from a remote location using an android captures the person and checks whether the person has
application on a smart phone. This can be carried out as access or not, by mapping with the pictures stored in micro
when a visitor presses a calling bell/push button at the door, SD card of ESP32 CAM. ESP32 CAM is used to take the
a Raspberry Pi with HD camera gets powered up and photos of the authorized persons and are stored in the
notifies the owner with a message and streams the visitor’s micro SD card. Smart ESP32 CAM-AI thinker helps the
live identity on the owner’s mobile over the Internet. The process of recognizing the face of authorized person. If the
owner can easily open/close the door remotely by entering a person is detected as authorized then the door gets
password that was pre-set. In addition, he can also send an unlocked, if not then the door remains locked. This can be
audio message to the visitor by entering the corresponding done by using solenoid lock as hardware lock [7,8]. All the
text in his smart phone app. This system contains the hardware components used, are connected with jumper
components Switch, Video Camera, Raspberry Pi 3, Servo wires. FTDI board helps in connecting the hardware
Motor, Raspbian OS, Motion Package, and Blynk Software. components with the software components. Arduino idle is
This project equips the owner with remote surveillance and the software required to write components. for facial
access to his home in real-time along with a way to leave detection and recognition. the excess electricity and helps
his responses with the visitor. The main limitations in this in powering the whole circuit when required.
work are i) This project works only with the mobile ii)A The proposed model architecture as shown in Fig1 is
quite expensive one. used in solving real time problems and crimes like thefts,
robbery, looting private data etc. The proposed model is
Manasi Jeste, et.al [7] proposed work makes use of face also useful for the age old and disabled people who face lot
recognition and fingerprint as two-point authentication. of hurdles while opening the traditional lock systems. The
Face recognition is used as the first pin authentication project helps in achieving confidentiality which is required
whereas fingerprint is a second authentication pin. The while solving the important missions by authorities.
proposed method can be implemented by using a hardware
lock. In the software part, face recognition and fingerprint
algorithms are trained using machine learning algorithms.
For face recognition, the Local Binary Patterns Histograms
(LBPH) algorithm is implemented. This algorithm
describes the original image in a better way by highlighting
the facial characteristics. The fingerprint recognition is
developed using a mobile phone app. The app uses a
fingerprint database from the mobile of the user which
already exists in the mobile or can create its own database.
269
Authorized licensed use limited to: PES University Bengaluru. Downloaded on January 18,2023 at 11:58:31 UTC from IEEE Xplore. Restrictions apply.
12DC Recognize (authorized images)
Power
Bread board Show (authorized images)
9v Voltage
Start
Regulator LED1 LED2 5V
GND
IO2
IO3
7805
IO4 ESP32
ESP32 CAM
CAM
5V DC Facial
Recognition
Relay Comparing
module with database
Solenoid ON OFF
lock
Person
arrives
yes
Fig 1.Proposed Architecture
Door is Authorized
A. Facial Recognition unlocked
270
Authorized licensed use limited to: PES University Bengaluru. Downloaded on January 18,2023 at 11:58:31 UTC from IEEE Xplore. Restrictions apply.
Lock (door) implementation for working whereas conventional lock
needs information from software applications in order to
C. Extension and Verification respond. All these factors performance analysis is shown
This module mainly focuses on testing the proposed in Fig 3.
system. If an authorized person wants to grant access to other
persons, then the pictures of other persons are captured with
the help of esp cam. The pictures of those persons are stored
in to the SD card. The person face is recognized when he
appears for the next time. The process is as follows:
271
Authorized licensed use limited to: PES University Bengaluru. Downloaded on January 18,2023 at 11:58:31 UTC from IEEE Xplore. Restrictions apply.
The proposed system when compared with already Authentic Yes Yes Yes Yes Yes Yes
existing applications by considering the list of features as ation
shown in Table 1 depicts the proposed system strengths Web Yes No No No No Yes
and also supports various features under effective price Applicatio
and size which are lacked in existing conventional lock n
systems. No No Yes No No Yes
Automati
b. Applications c lock
The main aim of this work is for locking and unlocking CONCLUSION
doors but it also applies to other areas like fingerprints,
security pins and other gesture recognitions. This system can The proposed system mainly focuses on providing
be implemented in real-time applications like smart security to the persons with affordable prices. The
attendance systems, secured places like Banks, confidential proposed system consists of hardware components like
office rooms etc. The reachability of this project is high as it solenoid lock, relay module, ftdi board, jumper wires,
is developed at a low cost. esp32 cam which are easy to maintain. The proposed
system unlocks the door with simple facial recognition
c. Attacks and Threats which is a boon for age old and disabled persons. The
Different types of attacks are possible on IoT applications proposed system is not complex and it is easy to access.
due to the vulnerabilities in the design and development of The proposed system helps in preventing unauthorized
those applications. Cyber attacks on IoT devices have accesses. The proposed system can monitor 24/7 with the
increased nowadays. With the reference to the literature help of esp cam. The proposed system uses key points and
survey, here we have taken necessary measures to prevent facial points for recognition so that error rate is
some of the attacks which can be possible to do on the diminished. Excess electricity can also be managed with
proposed system. Few attacks are Authorized attack, System the help of relay module. The proposed system is
attack, Cyber attack. applicable to many sectors like banks, offices, server
rooms etc. It can safeguard all the private information of a
i. Authorized Attack person. The proposed system helps in minimizing human
intervention. The proposed system is feasible since it can
Accessing the system and malfunctioning by an be extended for further authentication with less time
unauthorized user leads to this attack. In this proposed latency. This model can also be upgraded with ease of
system, the lock doesn't open when an unauthorized person effort. This helps in unlocking the door whenever a person
tries to access the door. When an intruder breaks the cam and loses his key and it can help in solving the problem of
tries to enter the house, still it will not unlock as the face is memorizing of keys.
not recognized.
ii. System attack REFERENCES
System attacks are also a threat to the application. In this [1] Jahnavi S, C.Nandhini, “Smart Anti-Theft Door Locking
proposed system, all the users’ data and their facial images System”, IEEE 1st International Conference on Advanced
Technologies in Intelligent Control Environment Computing &
are stored in the system only through a particular IP address. Communication Engineering (ICATIECE), March2019.
The IP address is hidden and can only be accessed by the [2] Farhat Lamia Barsha,ZarinTasneem,SanzidaMojib,Masuda
Owner/ creator. Manipulation of data and any alterations to Afrin., “An IoTbased Automated Door Accessing System for
the data can be done only through the web, which has a Visually Impaired People”, 5th IEEE international Women in
specified URL associated with it. Engineering (WIE) Conference on Electrical and Computer
Engineering (IEEE WIECON-ECE),2019.
iii. cyber attack [3] B. G., R. Ch, K. M., L. B. A. and M. C. V., "Revelation of
Geospatial Information using Augmented Reality," 2021 Sixth
Cyber attacks mainly occur through the internet. International Conference on Wireless Communications, Signal
Through this intruders enter the applications and take control Processing and Networking (WiSPNET), pp. 303-308,2021.
of them. The proposed system cannot be accessed through [4] Koneru V., Shaik I., Rupa C., Krishnaveni M., Rachana R.,
the internet. Breaking the system is not possible compared to “Remotely Controlled Surveillance Vehicle with Live Video
Streaming”, In: Chaki N., Pejas J., Devarakonda N.,
major systems which require an internet connection. This RaoKovvur R.M. (eds) Proceedings of International
can be achieved by the personal internal connection of our Conference on Computational Intelligence and Data
system to the user-accessible devices. Engineering. Lecture Notes on Data Engineering and
Communications Technologies, vol 56.
Table 1. Comparison with the Existing Systems Springer,Singapore,2021.
Methods Kenici Ivo Gurlove Zhaon Propose [5] Adarsh V Patil, SreevarshaPrakash, Akshay S,
NgoTung Mahadevaswamy, ChandanbPatgar, Sharath Kumar A J,
Features Yoned Slugan Singh anMu d
Son [9] a [11] ovic [14] [15] System “RemoteyAccessibe Smart Lock Security System with
[12] Essential Features”, IEEE , International Conference on
Wi-Fi/ Yes/No Yes/Ye Yes/No Yes/No Yes/Ye Yes/Yes Wireless Communications Signal Processing and Networking
s s
(WiSPNET), 21-23 March20.
Bluetooth
[6] S. Irfan, C. Rupa, K. Vinay, M. K. Veni and R. Rachana,
Camera Yes No Yes No No Yes
"Smart Virtual Circuit based Secure Vehicle Operating
Support System," 2020 2nd International Conference on Innovative
Power Medium Mediu High Mediu Mediu Low Mechanisms for Industry Applications (ICIMIA), pp.386-
m m m
390,2020.
Consumpt
[7] ManasiJeste, PareshGokhale, Shrawani Tare, YutikaChougule,
ion
“A. Chaudhari .Two-point security system for doors/lockers
272
Authorized licensed use limited to: PES University Bengaluru. Downloaded on January 18,2023 at 11:58:31 UTC from IEEE Xplore. Restrictions apply.
using Machine learning and Internet Of Things”,.IEEE Fourth
International Conference on Inventive Systems and Control
(ICISC 2020) September2020.
[8] Abdul Basit, KomalSaxena,AjayRana, “A Wearable Device
Used for Smart Doorbell in Home Automation System”, .IEEE,
International Women in Engineering (WIE) Conference on
Electrical and Computer Engineering (WIECON-ECE), 26-27
Dec.2020.
[9] NgoTungSon, LePhuongChi, Phan Truong Lam,Tran Van
Dinh. “Combination of Facial Recognition and Interaction with
Academic Portal in Automatic Attendance System”, .ACM, 8th
International Conference on Software and Computer , February
2019.
[10] Rupa C., Srivastava G., Gadekallu T.R., Maddikunta P.K.R.,
Bhattacharya S, “A Blockchain Based Cloud Integrated IoT
Architecture Using a Hybrid Design” In: Gao H., Wang X.,
Iqbal M., Yin Y., Yin J., Gu N. (eds) Collaborative Computing:
Networking, Applications and Worksharing.
CollaborateCom2020. Lecture Notes of the Institute for
Computer Sciences, Social Informatics and
Telecommunications Engineering, vol 350.Springer,2021.
[11] Kenichi Yoneda, Gary, M. Weiss, ThaierHayajneh,
“Smartphone and Smart watch-Based Biometrics using
Activities of daily living”, IEEE Access (Journal),
September2019.
[12] Ivo Sluganovic, Marc Roeschlin, B. Rasmussen, Ivan
Martinovic, “Analysis of Reflexive Eye Movements for Fast
Replay-Resistant Biometric Authentication”, ACM,
Transactions on Privacy and Security (journal), January2019.
[13] Hussain S.S., Rupa C., Avadhani P.S., Reddy E.S. “An
Advanced Authentication System Using Rotational
Cryptographic Algorithm”. In: Satapathy S., Avadhani P.,
Udgata S., Lakshminarayana S. (eds) ICT and Critical
Infrastructure: Proceedings of the 48th Annual Convention of
Computer Society of India- VolII. Advances in Intelligent
Systems and Computing, vol249.Springer,2014.
[14] Gurlove Singh, Amit Kumar Goel, “Face Detection and
Recognition System using Digital Image Processing”, IEEE,
2nd International Conference on Innovative Mechanisms for
Industry Applications(ICIMIA), April2020.
[15] Zhaonan Mu, Wenting Li, Chao Lou, “ Investigation and
Application of Smart Door Locks based on Bluetooth Control
Technology”, IEEE, Asia-Pacific Conference on Processing,
Electronics and Computers(IPEC),April 2020.
[16] RupaCh, GautamSrivastava, Thippa Reddy Gadekallu, Praveen
Kumar Reddy Maddikunta, Sweta Bhattacharya, “Security and
privacy of UAV data using blockchain technology, Journal of
Information Security and Applications,Volume55,2020.
[17] Ravi Gatti and Dr. Shivashankar, “Extension of Modified Joint
Uplink and Downlink Scheduling algorithm for LTE-A in
Femtocell Networks”, International Journal of Sensors, Wireless
Communications and Control, Bentham Science Publisher, ISSN:
1868-5137 (print), ISSN: 1868-5145 (online), Vol. 10 , No. 5,
DOI: 10.2174/2210327909666191126112740, 2020.
[18] Rajendra Prasad P, Dr. Shivashankar, “Multitier Energy System
Review on Secure Intrusion Detection System in MANETs”, 2017
2nd IEEE International Conference on Recent Trends in
Electronics Information & Communication Technology, ISBN:
978-1-5090-3704-9, 2017.
[19] Arjun Kumar GB, Dr. Shivashankar, “Simulation Research on
Integrated Hybrid PV- Wind Driven Generator Supplying AC/DC
Micro-grid”, International Journal of Engineering and Technology,
Engg Journals Publications, vol. 11, no.4, pp. 822-835, doi:
10.21817/ijet/2019 /v11i4/191104039, 2019
[20] K. N. SunilKumar and Shivashankar, "A review on security and
privacy issues in wireless sensor networks," 2017 2nd IEEE
International Conference on Recent Trends in Electronics,
Information & Communication Technology (RTEICT), 2017, pp.
1979-1984, doi: 10.1109/RTEICT.2017.8256945.
[21] R. Gatti and Dr. Shivashankar, "Study of different resource
allocation scheduling policy in advanced LTE with carrier
aggregation", 2017 2nd IEEE International Conference on Recent
Trends in Electronics, Information & Communication Technology,
pp. 2257-2261, DOI: 10.1109/RTEICT.2017.8257002, 2017.
273
Authorized licensed use limited to: PES University Bengaluru. Downloaded on January 18,2023 at 11:58:31 UTC from IEEE Xplore. Restrictions apply.