Face Detection and Recognition for Automatic

Attendance System

Onur Sanli and Bahar Ilgen(&)

Department of Computer Engineering, Istanbul Kültür University,

Istanbul, Turkey
[email protected], [email protected]

Abstract. Human face recognition is an important part of biometric verifica-

tion. The methods for utilizing physical properties, such as human face have
seen a great change since the emergence of image processing techniques.
Human face recognition is widely used for verification purposes, especially if
individuals attend to lectures. There is a lot of time lost in classical attendance
confirmations. In order to solve this time loss, an Attendance System with Face
Recognition has been developed which automatically tracks the attendance
status of the students. The Attendance System with Face Recognition performs
daily activities of the attendance analysis which is an important aspect of face
recognition task. By doing this in an automated manner, it saves time and effort
in classrooms and meetings. In the scope of the proposed system, a camera
attached to the front of the classroom continuously captures the images of the
students, detects the faces in the images, compares them with the database, and
thus the participation of the student is determined. Haar filtered AdaBoost is
used to detect the real-time human face. Principal Component Analysis
(PCA) and Local Binary Pattern Histograms (LBPH) algorithms have been used
to identify the faces detected. The paired face is then used to mark course
attendance. By using the Attendance System with Facial Recognition, the effi-
ciency of lecture times’ utilization will be improved. Additionally, it will be
possible to eliminate mistakes on attendance sheets.

Keywords: Face detection
Face recognition

Principal Component Analysis (PCA)
Local Binary Pattern Histogram (LBPH)
Viola-Jones

1 Introduction

Automatic facial analysis, including facial recognition and facial expression recognition,
has become a very active topic in computer vision area. Face authentication/verification
is utilized in several tasks including civil and security applications, electronic transac-
tions. It is also useful in the areas such as interactive games and movies, image searching
and wearable systems.
Biometric face recognition is basically applied in three main areas: (1) continuous
participation systems and employee management, (2) visitor management systems, and
(3) minimal authorization systems and access control systems. In the scope of the

© Springer Nature Switzerland AG 2019

K. Arai et al. (Eds.): IntelliSys 2018, AISC 868, pp. 237–245, 2019.
https://doi.org/10.1007/978-3-030-01054-6_17
238 O. Sanli and B. Ilgen

research area, holistic methods, such as Principal Component Analysis (PCA) and
Local Binary Pattern Histograms (LBPH) have been utilized frequently.
Face recognition task in classrooms or meeting rooms is one of the most popular
applications in the research area. Traditionally, the attendance information of the stu-
dents is supplied manually using the attendance sheet issued by the members of the
class, which is a time-consuming activity. It is also very difficult to verify the atten-
dance information of a single student without considering his/her respond in a large
classroom environment with distributed branches. The facial recognition shows how to
use an effective attendance system to automatically record the presence of a registered
person in a designated place. The proposed system also maintains a daily record for
each individual with a universal system time.
Although it is straightforward for human perception, automatic facial recognition
requires considering several challenges. These difficulties result from variations
between the appearances of the same face and similar appearances of the different
faces. The former is known as intrapersonal variations. This includes the variations of
the same face caused by the issues such as head pose, lighting and face expression.
There may be additional difficulties such as hair styling and aging which can also
reduce the accuracy of the system. The latter difficulty is also known as interclass
similarity. It usually results from similar faces of relatives, twins or any people.
An image can be expressed in mathematics as a multidimensional matrix and a
matrix value. It can be treated as a vector with image size and direction. It is known as
an image vector. If x represents a p x q image vector, then x is the matrix of the image
vector. Thus, the image matrix can be represented as x = {x1, x2,…, xn}T, where T is
the transpose of the matrix x. It is also known that the certain cases such as definition of
glass in an image matrix might be very difficult and it requires additional approaches
that can handle these limitations. The algorithm proposed in this paper successfully
overcomes these limitations. The rest of the paper is organized as follows. Section 2
summarizes previous work on the research area. In Sect. 3, the proposed methodology
is explained. Sections 4 and 5 present our experimental results, conclusion and the
future work respectively.

2 Related Work

There are several efforts in the face recognition area which provides support to
numerous applications. The automated attendance system in [1] involves automating
attendance follow-up with face recognition and perception techniques. The purpose of
the system is to keep track of the students’ attendance in the classes and to ensure that
the faculties have access to the information of the students who are kept in these
registers. In the study, Deep Reinforcement Learning approach with Convolutional
Neural Networks (CNNs) was used to compare user identities for mobile payments,
focusing on a face recognition process under uncertain facial features. The OpenCV [2]
library was used in system development. PCA algorithm is also utilized for learning
and training phase of face images. The images of the users can be taken with the phone
camera. The central database can be accessed in the proposed mobile-based system.
 Face Detection and Recognition for Automatic Attendance System 239

The results showed that the facial image of CNN schemas could increase sensitivity to
recognition compared to average sensitivity.
In the scope of the research carried out by Symbiosis International University [3],
Viola & Jones algorithm is used for face detection while PCA and LDA algorithms are
used for face recognition. Face detection and recognition were performed with a
camera module connected to Raspberry Pi. The result of the study indicates that the
face recognition-based system was more efficient than the biometric and Radio Fre-
quency Identification (RFID) based systems. In [4], OKAO vision library provided by
OMRON Corporation for face recognition and detection is utilized. Two types of
cameras have been used in the system. One is a sensor camcorder on the ceiling to
acquire the views of seats which students are sitting on. The other is a camera in front
of the seats to capture images of the faces of the students. In the scope of the study, it
has been observed that the continuous observation has increased the estimation per-
formance for attendance.
Face images can be considered as the concatenated form of micro images. In [5]
face is divided into small cells to extract LBP histogram for efficient and concatenated
face representation. Both shape and texture information are utilized for the represen-
tation. Nearest neighbor classifier is used in the experiments. The results indicate that
the proposed simple and efficient method provides fast feature extraction. Furthermore,
there is a modified LBPH algorithm [6] as an alternative to the LBPH algorithm used
for face recognition. The LBPH algorithm has low recognition rate under illumination
variance, expression variation, and attitude deviation. In order to solve this problem, a
modified neighborhood gray-median (MLBPH) based LBPH algorithm has been pro-
posed. The results have shown that the MLBPH algorithm has a higher recognition rate
than the LBPH algorithm.
Another work is the face recognition system [7], which is made up of smart glasses
that emerge with the advancement in today’s technology. The aim of the study is to
develop an effective face recognition system on smart glasses. The achieved results of
the proposed efficient system were accurate and satisfactory.
In a face recognition system, the traditional method of data size reduction is the
rearrangement of image vectors in the face. This causes the loss of structural properties
by the data itself. As a result, the sensitivity of recognition cannot be high. The efforts
in the area [8] aim to develop a data reduction method based on Multilinear Dis-
criminant Subspace Projection (MDSP). As a result of the experiments, it is concluded
that MDSP has better results than conventional size reduction algorithms in terms of
classification accuracy.
Furthermore, there is a Neural Aggregation Network (NAN) study [9] for video
face recognition. This work was carried out using a set of video or images with more
than one hundred instances of a person. To achieve this, a fixed-size feature repre-
sentation for face recognition was created. Experiments on IJB-A, YouTube Face, and
Celebrity-1000 video face recognition tests have shown in the end that it performs
better than pure aggregation methods and provides the best correctness. An automated
attendance management system has been also developed to overcome the difficulties
[10] such as pose, illumination, rotation and other factors. The system integrates several
techniques such as integral images, AdaBoost and Haar-like features.
240 O. Sanli and B. Ilgen

3 Methodology

In the scope of this study, we use a two-stage mechanism in the system we have
developed. It consists of an initial face detection that is followed by face recognition.
Haar classifier and Viola-Jones algorithm are utilized for the face detection. PCA and
LBPH algorithms have been used in the face recognition phase.

3.1 Viola-Jones Algorithm

The Haar cascade classifier can be found in the OpenCV library [11]. Paul Viola and
Micheal Jones [12] used Haar-like features shown in Fig. 1 for object detection. It is
also known as the Viola and Jones object detection framework (Viola and Jones object
detection structure). In the most basic sense, objects that are desired to be found by a
certain algorithm are first introduced to the computer. Then the images or video frames
in which similar shapes exist are scanned to find that object. There is a need for positive
pictures that contain the object sought for training as well as the negative pictures that
the object does not exist. In classifier training, objects in positive pictures are scanned
as frame sets in certain sizes. The sum of the pixel values of the black region in the
frame and the darkness values in the white region are summed to generate specific
target values. Edge, line and center surround features are shown in Fig. 1.

Fig. 1. Edge, line and center surround features.

These classifiers including features, are called weak classifiers. An object will have
many of these weak classifiers, and there is an object sought with high accuracy in the
point where these weak classifiers are gathered.

3.2 Principal Component Analysis (PCA)

PCA is one of the most frequently used method for data compression, dimensionality
reduction and data decorrelation. The objective of the PCA [13] is to obtain a small
number of mutually independent data from a large number of data with linear rela-
tionships. When Principal Component Analysis is applied in face recognition problem
the steps below are followed.
 Face Detection and Recognition for Automatic Attendance System 241

• Centre and Standardize calculations

Let fa1 ; a2 ; a3 ; . . . an g be our data set for training. Data set average

1 Xn
Avg ¼ a
i¼1 n
ð1Þ
n

• Calculate the covariance matrix

Each item in the training data set is different from Yi = ai-Avg vector. Covariance
matrix Cov:

1 Xn
Cov ¼ Yi:YiT ð2Þ
n i¼1

• Find the eigenvectors of covariance matrix and transform the data to be in terms of
the components
Pick n important eigenvectors of Cov and calculate the k differences of weight
vectors Wik for each element in the data set from 1 to n.
Xn Xn
wik ¼ k¼1
eT :ðai
i¼1 k
 AvgÞ ð3Þ

3.3 Local Binary Pattern Histograms (LBPH)

The LBP [14] is a simple and efficient texture measurement operator which utilizes
thresholding the neighborhood of pixel for pixel labelling. The LBP operator creates a
tag for each pixel of the view. It consists of one or more zeros. These labels are formed
by comparing the pixels adjacent to the center pixel N  N. In general, LBPP,R can be
defined by three different circular neighborhoods. P represents the neighboring number
and R represents the sampling Radius as it is shown “Fig. 2” [15, 16].

Fig. 2. Various circular LBPP,R operators

242 O. Sanli and B. Ilgen

The operator LBP8.1 is used in this study. That is, neighborhood analysis is done
using 3  3 matrices.
Xp1   p
LBP8;1 ðxc Þ ¼ p¼0
u x p  x c 2 ð4Þ

1; y  0
uð yÞ ¼ ð5Þ
0; y\0

In (5), y is the difference between the center pixel and the neighboring pixel. xc is
the center pixel produced by the LBP tag, xp is the neighbor of center pixel, and u(y) is
the bit produced by the LBP operator. An example is shown in “Fig. 3” [15].

Fig. 3. LBP8,1 operator application

4 Experimental Results

In the scope of this study, all system requirements were applied and successfully
completed. Depending on the test results applied to the system, face recognition and
detection can be used to keep an entry log. Additionally, a person who is not registered
to the system can also be added to the system by introducing his/her own face. As a first
attempt, a certain number of gray scale face samples have been gathered. Once the
registration has been made, the attendance log can be kept for the person. We used our
own face database on the system and using these faces we have realized training
processes with LBPH and PCA algorithms. While comparing the efficiencies of the
PCA and LBPH algorithms, a threshold value for each algorithm was determined in
experiments with examples. Based on these thresholds, experiments were conducted
according to the number of face samples found. Reliability rates based on the number
of these facial specimens were examined. We evaluated certain number of samples and
noticed that the reliability ratio that emerged after the identification of one person’s face
did not exceed the threshold values we have seen. While these ratios show less fluc-
tuations in the EigenFace algorithm that we applied with PCA, we found more devi-
ations in the LBPH algorithm. Finally, the EigenFace algorithm we implemented with
PCA was a little bit more efficient than LBPH, but the LBPH algorithm gave the
opposite result when the PCA algorithm caused the program to visibly slow down
during the processing of the data and polling acquisitions. As a result of the com-
parison, PCA gave more reliable ratios with lower performance. The performance was
better for the LBPH algorithm when the PCA was well below the reliability ratios.
A sample interface of our automatic attendance system is shown in Fig. 4. When
the face matching is performed, the face of the student is circled in the green box. Then
the student ID and name are displayed by the system. The course attendance of the
 Face Detection and Recognition for Automatic Attendance System 243

Fig. 4. Sample screen of automatic face detection and recognition system.

matching student is marked in the database and information about the day and time is
saved immediately. During the face recognition process, if the student does not match
with a sample in the face database, an additional option of “adding as new student” is
selected. Then the face samples are taken as soon as the add button is pressed. After
receiving face samples, the system starts to train these face samples and records the
information in the training set file.
Table 1 shows the test cases that have been successfully completed in the scope of
this study.

Table 1. Automatic attendance system test cases

Objective Description Result
Verify login details – Login to the system using – User should login to the
username and password with system
test data
Verify the lectures after – Login to system – User logins to the system
login – Be redirected to course selection and displays lectures
page
Verify the attendance – Login to system – User logins to the system
lists after lecture – Select course from the table and displays attendance
selection list
Verify the detecting and – Login to system using password – User logins to the system
recognizing face and test data – User displays lectures
– Start course selection – Redirecting attendance
– Select classifier page
– Start video capture – User opens camera
– Select new student and enter – Detecting and recognizing
related info face
– Adding new student to the
system
244 O. Sanli and B. Ilgen

5 Conclusions and Future Work

This study shows that the proposed approach provides better solution compared to
traditional attendance tracking methods, biometric fingerprint and RFID systems.
Algorithms such as PCA and Viola-Jones for biometric face recognition and face
detection were used efficiently in the system. Our experiments have been conducted
using face images of 10 students. The reliability results of face recognition system were
obtained between the ranges 75% and the 95% during the experiments. Although the
developed system has been designed for schools and classrooms, it can also be adapted
and used in work places and security systems. It is also important to bear in mind that
better face recognition results may be yielded in terms of LBPH performance on such
systems after some improvements on the utilized algorithms. As a future work, the
system can be further improved by introducing additional features in recognition phase
such as body, object and motion detection.

References
1. Wang, P., Lin, W.H., Chao, K.M., Lo, C.C.: A face-recognition approach using deep
reinforcement learning approach for user authentication. In 2017 IEEE 14th International
Conference on e-Business Engineering (ICEBE), pp. 183–188. IEEE, November 2017
2. Bradski, G., Kaehler, A.: OpenCV. Dr. Dobb’s J. Softw. Tools 3 (2000)
3. Patil, A., Shukla, M.: Implementation of classroom attendance system based on face
recognition in class. Int. J. Adv. Eng. Technol. 7(3), 974 (2014)
4. Kawaguchi, Y., Shoji, T., Weijane, L.I.N., Kakusho, K., Minoh, M.: Face recognition-based
lecture attendance system. In: The 3rd AEARU Workshop on Network Education, pp. 70–75
(2005)
5. Ahonen, T., Hadid, A., Pietikainen, M.: Face description with local binary patterns:
application to face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 28(12), 2037–2041
(2006)
6. Zhao, X., Wei, C.: A real-time face recognition system based on the improved LBPH
algorithm. In: 2017 IEEE 2nd International Conference on Signal and Image Processing
(ICSIP), pp. 72–76. IEEE, August 2017
7. Xu, W., Shen, Y., Bergmann, N., Hu, W.: Sensor-assisted multi-view face recognition
system on smart glass. IEEE Trans. Mob. Comput. 17(1), 197–210 (2018)
8. Mei, M., Huang, J., Xiong, W.: A discriminant subspace learning based face recognition
method. IEEE Access (2017)
9. Yang, J., Ren, P., Chen, D., Wen, F., Li, H., Hua, G.: Neural aggregation network for video
face recognition. arXiv preprint (2017)
10. Shirodkar, M., Sinha, V., Jain, U., Nemade, B.: Automated attendance management system
using face recognition. Int. J. Comput. Appl. (2015). (0975–8887) International Conference
and Workshop on Emerging Trends in Technology (ICWET 2015)
11. Bradski, G., Kaehler, A.: Learning OpenCV: Computer Vision with the OpenCV Library.
O’Reilly Media, Inc., Sebastopol (2008)
12. Soni, L.N., Datar, A., Datar, S.: Implementation of Viola-Jones Algorithm based approach
for human face detection. Int. J. Curr. Eng. Technol. 7, 1819–1823 (2017)
 Face Detection and Recognition for Automatic Attendance System 245

13. Wimmer, H., Powell, L.: Principle component analysis for feature reduction and data
preprocessing in data science. In: Proceedings of the Conference on Information Systems
Applied Research ISSN, vol. 2167, p. 1508 (2016)
14. Shan, C.: Learning local binary patterns for gender classification on real-world face images.
Pattern Recogn. Lett. 33(4), 431–437 (2012)
15. Tuncer, T., Engin, A.V.C.I.: Yerel İkili Örüntü Tabanli Veri Gizleme Algoritmasi: LBP-
LSB. Türkiye Bilişim Vakfı Bilgisayar Bilimleri ve Mühendisliği Dergisi 10(1), 48–53
16. Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant
texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell.
24(7), 971–987 (2002)