International Conference on Electrical Engineering and Information & Communication Technology (ICEEICT) 2014

Face Recognition System using PCA-ANN

Technique with Feature Fusion Method

Rizoan Toufiq Md. Rabiul Islam

Department of Computer Science & Engineering Department of Computer Science & Engineering
Rajshahi University of Engineering & Technology Rajshahi University of Engineering & Technology
Rajshahi-6204, Bangladesh Rajshahi-6204, Bangladesh
[email protected] [email protected]

Abstract— Biometric technology plays a vital role for developed, which can be used to detect faces by using skin
providing the security which is imperative part in secure system. color from DCT coefficient of Cb and Cr feature vectors [10].
Human face recognition is a potential method of biometric In recent, back-propagation algorithm also has been introduced
authentication. This paper presents a process of face recognition with facial expression detection and hand writing recognition
system using principle component analysis with Back- system [11][12][13].
propagation neural network where features of face image has
been combined by applying face detection and edge detection In this paper, we focus the face recognition system where
technique. In this system, the performance has been analyzed the feature has been extracted and fused, this feature dimension
based on the proposed feature fusion technique. At first, the has been reduced by using PCA technique, and finally we used
fussed feature has been extracted and the dimension of the the back-propagation classifier to classify the feature vectors.
feature vector has been reduced using Principal Component In the paper, we analysis this system based on different
Analysis method. The reduced vector has been classified by Back- biometric characteristic. In section II, we describe the proposed
propagation neural network based classifier. In recognition stage, feature fusion based face recognition system. Feature
several steps are required. Finally, we analyzed the performance extraction technique has been described in section III. About
of the system for different size of the train database. The back-propagation algorithm will be described in section IV.
performance analysis shows that the efficiency has been After that experimental result will be described in section V.
enhanced when the feature extraction operation performed Finally conclusion has been given in section VI.
successfully. The performance of the system has been reached
more than 92% for the adverse conditions.
II. PROPOSED CLASSIFIER FUSION TECHNIQUE FOR THE
Keywords— facet detection, edge detection, priciple component FACE RECOGNITION SYSTEM
analysis, back-propagation algorithm, feature fusion, false
The issues of the design and implementation of the face
rejection rate.
recognition system can be subdivided into two stages. These
are train the faces and test the faces. In the first part, the system
I. INTRODUCTION has been learned different amount of faces from the database,
Face Recognition system is a pattern recognition technique and then face detection technique name Viola-Jones object
which is developed based on some mathematical model and detection has been implemented on the train facial database.
has two vital steps i.e. feature extraction and classification. We cropped the facial portion of the database image. Then we
Image processing technique has been applied to extract feature used the edge detection technique to detect the shape of the
from face database and some classification techniques are train facial image. Finally we fused the two characteristic of the
applied to recognize the unknown facial template. In recent facial images into a feature vector. On this feature vector,
year, different type of research has been performed to develop principal component analysis technique has been applied to
the face recognition system. The dimension of the face image reduce the dimension. At last, this reduced feature has been
is the vital issue to resist the performance of recognition used as the input of the back-propagation algorithm. Finally we
system. Principal component analysis has been used to reduce classified the facial database by using the BPNN algorithm and
the dimension of the face database and extracted the feature the weighed matrix for the network has been saved to use in
face that is used in face recognition technique recognition stage. In the second stage, an unknown facial
[1][2][3][4][5][6]. A recent work, the dimension of the facial image has been given as input in the system to perform the
image has been reduced using principal component analysis recognition act. We applied the same technique to extract the
technique and the neural network has been used to face reduced facial feature from the unknown face image and the
recognition [7][8]. Back-propagation algorithm has been train network takes a decision based on the network
proposed for face detection technique, which offers great knowledge, which facial image of the database is closest one
Àexibility in contrast to the knowledge-based approaches [9]. with the unknown facial image. We have pre-defined a
The skin-based color features extracted from two dimensional threshold value to reject the unknown facial image which is
Discrete Cosine Transfer (DCT) and neural networks has been measures on the deviation from the known facial image and the

978-1-4799-4819-2/14/$31.00 ©2014 IEEE

train database facial image. The system has been shown in the represented as “Integral Image” which provides a quick
following block diagram, environment to detect a face. AdaBoost learning algorithm
(Freund and Schapire, 1995) has been used to select a small
Train Facial Test Face number of critical visual features from a very large set of
Database Image potential features. Finally a method has been provided for
combining classi¿ers in a “cascade” which allows background
regions of the image to be quickly discarded [19][20]. Then we
cropped the facial region from the original facial image. Edge
values and facial portion have merged in a feature vector. This
Viola-Jones vector has been reduced by PCA technique.
Face Edge
Face
detection The purpose of PCA is to reduce the large dimensionality
Detection
of the data space (observed variables) to the smaller intrinsic
dimensionality of feature space (independent variables), which
are needed to describe the data economically. It is known that
the independent variable is subject to just as much deviation or
error as the dependent variable [1].
Feature An image of size NxN can be represented as a point in a N2
Fusion vector dimensional space. Let, we have a face image I(x,y), be two
dimensional N by array of (8 bit) intensity value. M training
images are represented by I1, I2 ,…., IM and each image Ii is
represented as a vector īi. Now the average face vector is
Dimension calculated using the following equation:
Reduction 1 M
using PCA
Ψ= ¦īi
M i =1 (1)

Here, Ψ is the average face vector. Now the deviation is

calculated from the mean (average) face vector for each
image. The equation will be as following:
Train a
Φi = Γi −Ψ (2)
network by
BPN method A = [Φ1,Φ2....ΦM ] (3)

Here, Φi is the deviation vector for i-th image. A is the set

Take Decision Based on of deviation vector of M images. We can calculate the
Knowledge Gathered from covariance using A vector as
Train Database and Test Image
C = AAT (4)

Fig. 1. Block diagram of developed feature fusion based face recognition

Here C is an N2xN2 matrix and A is an N2xM matrix.
system Instead of the matrix AAT, consider the matrix ATA. Remember
A is an N2xM matrix, thus ATA is a MxM matrix. If we find the
Eigenvectors of this matrix, it would return M Eigenvectors,
III. FEATURE EXTARCTION AND DIMENSION REDUCTION each of dimensions Mx1, let’s call these Eigenvectors υi .The
TECHNIQUE best M Eigenvectors can be found using below equation:
In developed system, first we detect the edge of the facial
image. Canny edge detector operator has been used which μ i = Aυ i (5)
developed based on two criteria i.e. low error rate and
localization of edge points. In this technique, a numerical Each face in the training set (minus the mean),ĭi can be
optimization form has been proposed that is a direct tradeoff represented as a linear combination of Eigenvectors ȝi i.e.
between the direction and localization of noise. Using Canny's K
design, we also show that a solution to his precise formulation Φi = ¦ω j μ j (6)
of detection and localization for an infinite extent filter leads j =1
to an optimal operator in one dimension, which can be
efficiently implemented by two recursive filters moving in These weights can be calculated as:
opposite directions. This canny edge detection technique has
been developed based on different parameters i.e Regularized ω j = μTjΦi (7)
Laplacian Zero Crossings, scale multiplication etc
[14][15][16][17][18]. We have used viola-jones face detection
algorithm to detect a face on image in run time. Here image is
 Each normalized training image is represented in this basis
1 M 2
as a vector. Ep = ¦ δ pk (17)
2 k =1
Ω i = [ ω1 ,ω 2 ,...,ω k ] ′ (8) After finishing the training stage, we used the trained
weight value for an unknown pattern feature vector to
Features extraction process provides us with feature calculate the output. Then we calculated the distance between
matrixes containing the recognizable features of the face the output for the train pattern and the test pattern. The closest
images. [2], [3], [4], [5], [6]. pattern from database has been selected [21] [22][23].

V. EXPERIMENTAL RESULTS AND PERFORMANCE

IV. BACK-PROPAGATION ALGORITHM ANALYSIS
Here the feature vector calculated by PCA technique has In this work, main challenge is to extract the feature from
been used as input in the back-propagation training algorithm. the face database. We fussed the feature of the facial image
We get Ω i = [ ω1 ,ω 2 ,...,ω k ] ′ input vector. Now we can where the localization of the face has been given more priority
calculate the j-th hidden unite as following and then the edge of the face. Sometimes face detection system
N in run time has been failed, in this case, only edge is used as
net hpj = ¦ w hjiω pi h
+θj (9) the feature of the image. The efficiency of the face recognition
system highly depended on the feature fusion system. We will
i =1
discuss the experimental result in the following sub section.
where w hji is the weight on the connection from the ith
h
input unit, and θ j is the bias term. The "h" superscript refers A. Feature Extraction from the Facial Image
to quantities on the hidden layer. Now we calculate the output In this work, we first detect the edge of the image and save
from the hidden layer as the edge value of the original image. Then we implemented the
h Viola-Jones face detection algorithm on the original image and
i pj h
= f j ( net pj ) (10) take the portion of the facial area. The following algorithm has
h been implemented on feature extraction technique.
Assume that the function f be differentiable and written
j
Algorithm Feature_Extract {
h'
as f and move to the output layer to calculate the net-input
j
Img ĸRead a image;
values to each unit. Edge_img ĸ Canny_edge(Img);
L
net opk = ¦ wkjo i pj + θ k0 (11)
For i = 1: rows {
j =1 For j = 1:cols {
Now we can calculate the outputs:
If Edge_img(i,j) = 0 then
o o o
pk = f k ( net pk ) (12) Res_img(i,j) = Img(i,j);
end
where the "o" superscript refers to quantities on the output end
o
layer. Assume that the function f k be differentiable and end
o' detect_face_position ĸ Viola-Jones(Img):
written as f k . Now we can calculate the error terms for the
Feature_Img ĸ take the gray value according to
output units and error terms for the hidden units detect_face_position into Res_img.
o
δ pk = (y pk − o pk ) f ko' ( net 0pk ) (13)
end
From the above algorithm, we find the normal feature from
δ pjh = f j ( net pj )¦ δ pk wkj
h' h o o
(14) the original facial image. The feature has been shown in the
following figure.
Notice that the error terms on the hidden units are
calculated before the connection weights to the output-layer
units have been updated.
wkjo ( t + 1 ) o
= wkj ( t ) + ηδ pk i pj (15)
o

The factor Ș is called the learning-rate parameter. Update

weights on the hidden layer can be done by the following
equation, (a) (b)
whji ( t + 1 ) = whji ( t ) + ηδ pj
n
x pi (16)
Fig. 2. (a) Original Facial Image (b) Featured Facial Image
The order of the weight updates on an individual layer is
not important. The error term is calculated by
 Sometimes, Viola-Jones algorithm has been failed to detect
the facial region in run time. This caused the problem and the
feature face will be as the following figure.

(a) (b)
Fig. 6. (a) Unknown Facial Image (b) Selected Database Facial Image

D. Face Recognition Rate and Performance Analysis

(a) (b) In this work, BioID face database has been used to analysis
Fig. 3. (a) Original Facial Image (b) Featured Facial Image
the performance of the recognition system. The BioID Face
Database has been created for the purpose of face detection
research where real world condition has been emphasized. The
B. Dimension Reduction of the Featured Facial Image dataset consists of 1521 gray level images with a resolution of
In this stage, we reduced the dimension of the featured 384x286 pixels. Each one shows the frontal view of a face of
facial image by using principle component analysis. The one out of 23 different test persons [24]. We used only 10
weighted value has been used to classify the facial image of the person facial images and each person has 10 images for train
different person. Five featured eigen images has been shown in and test purpose.
the following figure.
TABLE I. PERFORMANCE OF THE FACE RECOGNITION SYSTEM
Num. of Test Learning Performance
Num. of Train Image Error rate
Images
0.0100 80
Fig. 4. Five Eigen Images 9.89 x 10-6 76
20 50 9.67 x 10-6 72
C. Train and Recognition Model of Fcae Pattern 9.85 x 10-6 82
9.90 x 10-6 74
In this stage, the featured vector has been classified by 9.82 x 10-6 70
back-propagation algorithm. The error rate has been pre- 9.91 x 10-6 92
defined to learn the system i.e. e-05. The learning processes 30 50 0.003 82
have been shown in the following figure. 9.55 x 10-6 90
9.94 x 10-6 80
9.82 x 10-6 84
9.68 x 10-6 80
40 50 9.95 x 10-6 76
0.009 72
9.80 x 10-6 84
9.55 x 10^-6 86
9.93 x 10^-6 94
50 50 0.01 80
9.76 x 10^-6 88
9.84 x 10^-6 92

In the above table, we have seen that the performance of the

recognition system is 94%. Here the performance highly
Fig. 5. Train the Facial Database (Epoch v/s M.S.E)
influence by the feature extraction technique. Many times, we
Here, we trained the database where 50 images have been have poor learning rate but the feature extraction technique
stored, there were 10 people and each has 5 different facial effect the performance of the system and gave a good result.
images. In recognition stage, this trained network has been E. False rejection rate of the system
used to recognize the unknown facial template. The sample Here the false rejection rate has been give for the threshold
resultant figure has been shown in the following figure. value. If the value is exceed the unknown pattern has been
rejected. We trained 50 facial images where each person has 5
individual images. We tested 50 unknown facial images.
 TABLE II. PERFORMANCE OF THE FACE RECOGNITION SYSTEM [5] Siddharth Gautam, “Face recognition using eigen faces and
dimensionality reduction by PCA,” International Journal of Emerging
Threshold Value Rejection Rate Research in Management &Technology, ISSN: 2278-9359, Vol. 2, Issue
5, pp. 21-28, May 2013.
0.048 12
0.038 15 [6] Ritu Upadhayay and Rakesh Kumar Yadav, “Kernel principle
component analysis in face recognition system: a survey,” International
0.028 19
Journal of Advanced Research in Computer Science and Software
0.018 21
Engineering, Vol. 3, Issue 6, pp. 348-353, June 2013.
0.008 23
[7] P.Latha, Dr. L. Ganesan and Dr. S. Annadurai, “Face Recognition using
Here the false rejection graph has been shown in the Neural Networks”, Signal Processing: An International Journal (SPIJ)
Vol. 3, Issue 5, pp. 153-160.
following figure.
[8] Santaji Ghorpade, Jayshree Ghorpade and Shamla Mantri, "International
Journal of Computer Science & Information Technology (IJCSIT)", Vol
2, No 6, December 2010.
[9] Kotropoulos Bassiou Kosmidis, C. Kotropoulos, N. Bassiou, T.
Kosmidis, and I. Pitas, “Frontal Face Detection using Support Vector
Machines and Back-propagation Neural Network”, Conference of Proc.
Int. Conf. Image Processing, pp. 1026-1029, 2001.
[10] Mohamed, A.S.S.; Ying Weng; Ipson, S.S.; Jianmin Jiang, "Face
detection based on skin color in image by neural networks," Intelligent
and Advanced Systems, 2007. ICIAS 2007. International Conference on
, vol., no., pp.779,783, 25-28 Nov. 2007.
[11] Le Cun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W.
Hubbard, and L. D. Jackel, “Backpropagation Applied to Handwritten
Zip Code Recognition”, Journal Neural Computation, Vol. 1, Issue 4,
Fig. 7. False Rejection Rate v.s Threshold Value Winter,1989, pp. 541-551, MIT Press Cambridge, MA, USA.
[12] Nazil Perveen, Shubhrata Gupta, Kesari Verma, "Facial Expression
In this case, we trained 50 images of 10 person. We finally Recognition System using Statistical Feature and Neural Network",
set the threshold value to reject a facial image. When we kept International Journal of Computer Applications (0975 – 888), Volume
48– No.18, June 2012.
the threshold value low, the performance of the system has
[13] Christine L. Lisetti and David E. Rumelhart, “Facial Expression
been better. Recognition Using a Neural Network”, Proceedings of the Eleventh
International Florida Artificial Intelligence Research Society
VI. CONCLUSION Conference, pp. 328 – 332, 1998.
We analyzed on the feature extraction technique and the [14] Canny, J., "A Computational Approach To Edge Detection", IEEE
performance of the face recognition system where the Trans. Pattern Analysis and Machine Intelligence, 8(6):679–698, 1986.
performance of the system vastly depended on the accuracy of [15] R. Deriche, "Using Canny's criteria to derive a recursively implemented
the face detection and edge detection technique. The Viola- optimal edge detector", Int. J. Computer Vision, Vol. 1, pp. 167–187,
April 1987.
Jones face detection technique has been applied here which
performance make effect on this recognition system. In this [16] Lindeberg, Tony "Edge detection and ridge detection with automatic
scale selection", International Journal of Computer Vision, 30, 2, pp
work, the vital issues is that if we fussed the feature of facial 117—154, 1998. (Includes the differential approach to non-maximum
images by applying different feature extraction techniques then suppression.)
the performance of the recognition system will be enhanced [17] Kimmel, Ron and Bruckstein, Alfred M. "On regularized Laplacian
based on the performance of feature extraction techniques. In zero crossings and other optimal edge integrators", International Journal
critical background image can be used to get the high of Computer Vision, 53(3):225-243, 2003. (Includes the geometric
performance of the recognition system because here we only variational interpretation for the Haralick-Canny edge detector.)
used the edge of the facial image i.e., shape of the face and [18] Paul Ba, Lei Zhang and Xiaolin Wu, "Canny Edge Detection
Enhancement by Scale Multiplication", IEEE Transactions on Pattern
detect the facial portion only. Then we merge these features. Analysis and Machine Intelligence, Vol. 27, No. 9, September 2005.
We focus the region of interest of the facial image. The [19] Paul Viola , Michael Jones, "Robust Real-time Object
performance of the system is 94% for some case. Detection",Second International Workshop on Statistical and
Computational Theories of Vision–Modeling, Learning, Computing, and
Sampling, Vancouver,Canada, July 13, 2001.
REFERENCES
[20] 2. Paul Viola , Michael Jones, "Robust Real-Time Face Detection",
[1] Pearson K., “On lines and planes of closest fit to systems of points in International Journal of Computer Vision 57(2), 137–154, 2004.
space,” Philosophical magazine,2, pp. 559-572, 1901.
[21] S, Rajasekaran and G. A. Vijayalakshmi pal, “Neural networks, fuzzy
[2] Matthew Turk and Alex Pentland, “Eigenfaces for recognition,” Journal logic, and genetic algorithms synthesis and applications,” PHI Learning
of Cognitive Neuroscience, Vol. 3, No. 1, pp. 71–86 , 1991. Pvt. Ltd, ISBN- 81-203-2186-3, 2004.
[3] Peter N. Belhumeur, Joao P. Hespanha, and David J. Kriegman, [22] R. Beale and T Jackson, Neural Computing: An Introduction, Adam
“Eigenfaces vs. Fisherfaces: recognition using class specific linear Hilger, Bristol, Philadelphia and New York, 1990.
projection,” IEEE Transactions on Pattern Analysis and Machine
[23] James A. Freeman, David M. Skapura, “Neural Networks, Algorithms,
Intelligence, Vol. 19, No. 7, July 1997.
applications and Programming Techniques”, Addison-Wesley Pub (Sd),
[4] Ivanna K. Timotius, Iwan Setyawan and Andreas A. Febrianto, “Face June, 1991.
recognition between two person using kernel principal component
[24] Collected from the web documentation :
analysis and support vector machines,” International Journal on
http://www.bioid.com/index.php?q=downloads/software/bioid-face-
Electrical Engineering and Informatics, Vol. 2, No. 1, 2010.
database.htm.