2017 IEEE 2nd International Conference on Signal and Image Processing

A Real-time Face Recognition System Based on the Improved LBPH Algorithm

XueMei Zhao, ChengBing Wei*

School of Electronic Information
Qingdao University
Qingdao, China
e-mail: [email protected]; [email protected]

Abstract—The Local Binary Pattern Histogram(LBPH) gray value of the pixel is replaced by the median value of its
algorithm is a simple solution on face recognition problem, neighborhood sampling valueˈand then creating the face
which can recognize both front face and side face. However, database of our own, extracting LBP feature and MLBP
the recognition rate of LBPH algorithm under the conditions feature, and being trained in the face database. After that, it
of illumination diversification, expression variation and is extracted the feature of testing images, established block
attitude deflection is decreased. To solve this problem, a
histogram, and compared with the database, and then printed
modified LBPH algorithm based on pixel neighborhood gray
the corresponding name in face recognition frame.
median(MLBPH) is proposed. The gray value of the pixel is
replaced by the median value of its neighborhood sampling
Start
value, and then the feature value is extracted by the sub blocks
and the statistical histogram is established to form the MLBPH Load Face Database
feature dictionary, which is used to recognize the human face
identity compared with test image. Experiments are carried on N
Is Camera Open?
FERET standard face database and the creation of new face
database, and the results show that MLBPH algorithm is Y
superior to LBPH algorithm in recognition rate.
Read an Image

Keyword-face recognition; local binary pattern histogram;

neighborhood median; statistical histogram; face database Face Detection

I. INTRODUCTION Cut and Save Face as Gray from the Image

Face recognition technology is an important research

project in the field of computer vision and pattern Face Recognition

recognition, it can identify the identities and other

information according to the visual features of face image, Read next Image
having a very broad prospects for development. It is widely
used in authentication, criminal investigation, video
surveillance, robot intelligence and medical science and so End

on. It has wide application value and commercial value. As a

Figure 1. System flow chart.
biological feature, facial features have the characteristics of
good, direct and convenient compared with other biological II. OVERALL DESIGN
features. Therefore, face recognition is more acceptable for
users. Face recognition system includes four main parts:
Over the years, many scholars have developed a variety information acquisition module, feature extraction module,
of kinds of face recognition algorithms, including Sparse classification module and training classifier database module.
Coding(SC) algorithm[ 1 ], Local Binary Pattern (LBP) The image information collected by the information
algorithm[ 2 ], Histograms of Oriented Gradients(HOG) acquisition module will be used as a test sample for analysis.
algorithm[3], Deep Convolution Network algorithm[4], Linear In the feature extraction module, a series of salient features
Discriminant Analysis (LDA) algorithm[ 5 ], Gabor feature which can represent human identity information are
algorithm[6], but also in the unceasing development. OpenCV extracted and analyzed. In the classification module, the
is an open source computer vision library that has three classifier trained by database is used to classify the test
built-in face recognition algorithms, Eigenfaces[ 7 ], samples to determine the identity information of the samples.
Fisherfaces[8] and Local Binary Pattern Histogram (LBPH)[9]. The system flow chart is shown in Fig. 1.
Compared with the other two algorithms, the LBPH A. Face Detection
algorithm can not only recognize the front face, but also
recognize the side face, which is more flexible. In order to OpenCV provides a Haar cascade classifier, which can be
enhance the robustness of illumination changes, expression used for face detection. The Haar cascade classifier uses the
change and attitude deflection, a LBPH algorithm based on AdaBoost algorithm to detect multiple facial organs
pixel neighborhood gray median(MLBPH) is proposed. The including the eye, nose, and mouth. First, it reads the image
to be detected and converts it into gray image, then loads

978-1-5386-0969-9/17/$31.00 ©2017 IEEE 

Haar cascade classifier to judge whether it contains human gp is the gray value of P neighborhoods of the pixel C,
face. If so, it continues to examine the eyes, nose, and mouth the radius of which is R. gc is the gray value of the pixel
and draw a rectangular frame on the corresponding organ. If C(xc,yc).
not, it continues to test the next picture. The flow chart of the This algorithm makes the LBP operator no longer limited
detection process is shown in Fig. 2. to fixed radius and neighborhood, and can meet the needs of
more different size and texture features.
N For each pixel of an image, it computes its LBP
Load Haar eigenvalues. Then these eigenvalues can form a LBP feature
Read an Convert it to Face Whether a Face
Image Gray Image
cascade
classifier
Detection Exit or Not spectrum. The LBPH algorithm uses the histogram of LBP
characteristic spectrum as the feature vector for classification.
Y It divides a picture into several sub regions, then extracts
LBP feature from each pixel of sub region, establishing a
Save the
Image in
Draw a
Rectangular
Y Whether Other Detect Other statistical histogram of LBP characteristic spectrum in each
Organs Exit or Not Facial Organs
Rectangle frame on Face sub region, so that each sub region can using a statistical
N
histogram to describe the whole picture by a number of
statistical histogram components. The advantage is to reduce
Figure 2. Face detection flow chart. the error that the image is not fully aligned in a certain range.
In order to enhance the robustness of illumination
B. Feature Extraction changes, expression change and attitude deflection, inspired
The LBP operator is used to describe the contrast by median filtering, a LBPH algorithm based on pixel
information of a pixel to its neighborhood pixels. The neighborhood gray median(MLBPH) is proposed. Compared
original LBP operator is defined in the window of 3*3. with the LBPH algorithm, the improvement of the MLBPH
Using the center pixel value as the threshold of window, it algorithm is that when the LBP eigenvalues are calculated,
compares with the gray value of the adjacent 8 pixels. If the the central pixels are replaced by the median values of the
neighborhood pixel value is greater than or equal to the sampled values of their neighborhood sampling points. The
center pixel value, the value of pixel position is marked as 1, definition of the median is shown in formula 3. The MLBP
otherwise marked as 0. The function is defined as shown in eigenvalue can be described in Fig. 5.
formula 1.It can be described in Fig.3.
­1 x t 0 ­ x 1 n / 2 , n is odd number
 s ( x) ® (1)  ® (3)
¯0 x  0 ¯ xn / 2  xn / 21 / 2 , n is even number

     
7KUHVKROG   %LQDU\
  
'HFLPDO
     

Figure 3. Original LBP operator. Figure 5. MLBP operator.

In this way, the 8 points in the 3*3 neighborhood are The rest steps and parameters of the MLBPH algorithm
compared to produce 8-bit binary numbers. Converting it to are the same as those of the LBPH algorithm. The flow chart
decimal numbers, the LBP values of the central pixel points of the MLBPH algorithm is shown in Fig. 6.
of the window are obtained, which is used to reflect the
texture features of the region. The current LBPH algorithm
uses an improved circular LBP operator. It can be
represented by Fig.4. and formula 2.  Label 
 
Form
Divide Extract Construct
All MLBP
Image MLBP Statistical
Training Feature
Into Feature Histogram
Image Value
Blocks Value
Dictionary

Calculating
Divide Extract
Figure 4. Circular LBP operator. Test Image MLBP
Construct
Statistical
MLBPH
Feature of
Similarity
Degree,
Recognition
Image Into Feature End
Histogram Sub Blocks Judging
Blocks Value
Category
P 1
LBP
P
R
¦ s( g
P 0
P  g C )2 P (2)
Figure 6. MLBPH algorithm flow chart.


C. Face Database
Experiments were performed on the FERET face
database and the created face database.
The FERET database contains multiple subsets .Select
one of the subsets for the experiment. The subset contains
200 different people, each with 7 different face images,
including light, facial expressions, and gesture changes.
Some of the human face images are shown in Fig. 7.

Figure 9. Face recognition in dim light.

As can be seen from TABLE I and TABLE II, the

MLBPH algorithm can more effectively highlight the local
Figure 7. Some images of FERET face database.
features of illumination changes, thus improving the
Created face database stores our own face images. It is recognition rate.
created on the basis of face detection. Make different facial B. Expression Change Experiment
expressions and postures to scene and detect faces. The
Select the different expression images of each person as
saved pictures are stored in a same folder to form the created
test-images, the other images are used as the train-images, so
face database. Some of the human face images are shown in
there are 1200 images in training-library. Repeat the
Fig. 8.
experiment 10 times, then take the average. The results are
shown in TABLE III.

TABLE III. DIFFERENT RECOGNITION RATE OF FERET DATABASE

Algorithm Correct Times Wrong Times Recognition rate

Figure 8. Some images of created face database.
LBPH 1176 24 98.00%
TABLE I. DIFFERENT RECOGNITION RATE OF FERET DATABASE
Recognition MLBPH 1186 14 98.83%
Algorithm Correct Times Wrong Times
rate
LBPH 1122 78 93.50%
Read images with different expression, read 500 frames
MLBPH 1167 33 97.25%
at a time, repeat 10 times, and take the average. The results
are shown in TABLE IV. The recognition results are shown
III. EXPERIMENTS AND DISCUSS in figure 10.
In the experiment, each person in the face database has TABLE IV. DIFFERENT RECOGNITION RATE OF CREATED DATABASE
different labels. First, train the face database, then extract the
LBP texture features and MLBP texture features of each test Algorithm Correct Times Wrong Times Recognition rate
image. Finally, classify and recognize the face information.
LBPH 422 78 84.4%
A. Illumination Change Experiment
MLBPH 441 59 88.2%
Select the weak illumination images of each person as
test-images, the other images are used as the train-images, so
there are 1200 images in training-library. Repeat the
experiment 10 times, then take the average. The results are
shown in Table I.
Read images in the dim light, read 500 frames at a time,
repeat 10 times, and take the average. The results are shown
in Table II. The recognition results are shown in figure 9.

TABLE II. DIFFERENT RECOGNITION RATE OF CREATED DATABASE

Algorithm Correct Times Wrong Times Recognition rate

LBPH 146 354 29.2%

MLBPH 187 313 37.4% Figure 10. Face recognition of different expression.


 As can be seen from TABLE III and TABLE IV, the TABLE VII. DIFFERENT RECOGNITION RATE OF CREATED Database
MLBPH algorithm can more effectively highlight the local
Algorithm 10% 30% 60%
features of expression changes, thus improving the
recognition rate. LBPH 97.4% 76.2% 29.4%

C. Attitude Deflection Experiment MLBPH 99.2% 82.6% 36.2%

Select the different attitude images of each person as 
test-images, the other images are used as the train-images, so
there are 1000 images in training-library. Repeat the
experiment 10 times, then take the average. The results are
shown in Table IV.

TABLE V. DIFFERENT RECOGNITION RATE OF FERET DATABASE

Algorithm Recognition
Correct Times Wrong Times
rate
LBPH 886 114 88.60%

MLBPH 926 74 92.60%

Figure 12. 10% recognition rate of created database.
In the experiment, rotating the head around the camera, it
can be found that the deflection angle is about 30 degrees at
both left and right. Read 500 frames at a time, repeat 10
times, and take the average. The results are shown in Table
VI. The recognition results are shown in figure 11.

TABLE VI. DIFFERENT RECOGNITION RATE OF CREATED DATABASE


Algorithm Left 30° Front Right 30°

LBPH 50.2% 76.2% 43.4% 

Figure 13. 30% recognition rate of created database.
MLBPH 55.0% 82.6% 48.6%

Figure 11. Face recognition of different attitude.


As can be seen from TABLE V and TABLE VI, The Figure 14. 60% recognition rate of created database.
front face recognition rate is higher than the side recognition
rate. Meanwhile, the recognition rate of MLBPH algorithm is As can be seen from TABLE VII, the smaller the
higher than that of LBPH algorithm under the same angle. proportion of the face lens, the higher the recognition rate.
That is, in the distance without affecting recognition, the
D. Experiment of Face Proportion Change of Lens farther away from the lens, the higher the face recognition
When the distance between the faces with camera is rate is. This is because the camera has some nonlinear
different, the face proportion of lens is also different, which distortion. The nearer the face distance lens is, the more
results in different recognition rates. Experiments were serious the distortion is, which leads to the decrease of
conducted with 3 proportions, as shown in Fig.12. Fig.13. recognition rate. However, in the same distance, the
Fig. 14. Each proportion reads 500 frames and repeat 10 recognition rate of MLBPH algorithm is higher than that of
times, calculates the average of the 10 identification rates, LBPH algorithm. The feasibility of the improved algorithm
and results are shown in Table VII. is proved once again.


 IV. CONCLUSION AND PROSPECT [2] J. CHAO W L, DING J J, LIU J Z. Facial expression recognition
based on improved local binary pattern and class-regularized locality
The recognition rate of local binary pattern histogram preserving projection. Signal Processing, 2015, 117:1-10.
(LBPH) algorithm can be reduced under illumination, [3] J. HU Liqiao, QIU Runhe. Face recognition based on adaptive
expression change and attitude deflection. To solve this weighted HOG. Computer Enigeering and Applications, 2017, 53(3):
problem, a LBPH algorithm based on neighborhood gray 164-168.
median(MLBPH) is proposed. Use the median of the [4] J. Hai qiang LONG,Tai zhe TAN. Computer Simulation, 2017, 34(1):
neighborhood sampling values to instead of intermediate 322-325.
values, thereby reducing the effects of extraction conditions
[5] J. Yu yan JIANG, Ping LI, Qing WANG. Labeled LDA model based
on the characteristic value of illumination. Experiments are on shared background topic. Acta Electronica Sinica, 2015, 2013, (9):
carried out on FERET standard face database and created 1794-1799.
face database. And the results show that MLBPH algorithm [6] J. WU Qi, WANG Tang-hong, LI Zhan-li. Imporved face recognition
is superior to LBPH algorithm in recognition rate. However, algorithm based on Gabor feature and collaborative representation.
the problem of lens distortion still has not been solved, which Computer Engineering and Design, 2016, 37(10): 2769-2774.
will affect the face recognition rate. This is the direction of [7] J. Turk M, Pentland A. Eigenfaces for recognition. Journal of
our next work. cognitiveneuroscience, 1991, 3(1): 71-86.

REFERENCES [8] J. P. Bellhumeur, J.Hespanha, D. Knegman. Eigenfaces vs

fisherfaces:Recognition using class specific linear projection.IEEE
[1] J. Olshausan B A, Field D J.Emergence of simple-cell receptive field Transactions onPattern Analysis and MachineIntelligence. Special
properties by learning a sparse code for natural images. Nature, 1996, Issue on FaceRecognition, 1997, 17 (1) :711-720.
381(6583):607-609.
[9] M. Joe Minichino, Joseph Howse. Learning OpenCV 3 Computer
Vision with Python. 2016: 16-82.