FACE RECOGNITION USING LBPF

A Mini Project Report submitted to

JNTU Hyderabad in partial fulfilment
of the requirements for the award of the degree

BACHELOR OF TECHNOLOGY
In
ELECTRONICS AND COMMUNICATION ENGINEERING

Submitted by

ARE DIVYA 20RG1A0406

GULLAPALLI SOWJANYA 20RG1A0426
KOTTAPALLY MANASA 20RG1A0429
VETTINI RACHANA 20RG1A0455
Under the Guidance of
Mrs.K.MANASA
B.Tech, M.TECH.

Associate Professor

DEPARTMENT OF ELECTRONICS & COMMUNICATION ENGINEERING

MALLA REDDY COLLEGE OF ENGINEERING FOR WOMEN
(Approved by AICTE New Delhi and Affiliated to JNTUH)
(An ISO 9001: 2015 Certified Institution)
(B.Tech Programs(CSE,ECE) Accredited by NBA)
Maisammaguda,Medchal (M), Hyderabad-500100, T.S
SEPTEMBER 2023
DEPARTMENT OF ELECTRONICS & COMMUNICATION ENGINEERING
MALLA REDDY COLLEGE OF ENGINEERING FOR WOMEN
(Approved by AICTE New Delhi and Affiliated to JNTUH)
(An ISO 9001: 2015 Certified Institution)
(B.Tech Programs(CSE,ECE) Accredited by NBA)
Maisammaguda, Medchal (M), Hyderabad-500100, T. S.
SEPTEMBER 2023

CERTIFICATE

This is to certify that the Mini project entitled “FACE RECOGNITION

USING LBPF” has been submitted by Are Divya (20RG1A0406),

Gullapalli Sowjanya (20RG1A0426), Kottapally Manasa

(20RG1A0429), Vettini Rachana (20RG1A0455) in partial
fulfillment of the requirements for the award of BACHELOR OF
TECHNOLOGY in ELECTRONICS & COMMUNICATION ENGINEERING. This
record of bonafide work was carried out by them under my guidance and
supervision. The result embodied in this mini-project report has not been
submitted to any other University or Institute for the award of any
degree.

Mrs.K.Manasa Dr. Selvamani Indrajith

Associate Professor Head of the Department
Project Guide

External Examiner
 ACKNOWLEDGEMENT
The Mini Project work carried out by our team in the Department of
Electronics and Communication Engineering, Malla Reddy College of
Engineering for Women, Hyderabad. This work is original and has not
been submitted in part or full for any degree or diploma of any other
university.
We wish to acknowledge our sincere thanks to our project guide
Mrs.K.Manasa Associate Professor of Electronics & Communication
Engineering Department for formulating the problem, analysis, guidance and
her continuous supervision during the project work.

We acknowledge our sincere thanks to Dr. Vaka Murali Mohan,

Director, MRCEW. Dr.Kanaka Durga Returi, Principal and Professor of
Computer Science & Engineering Department. Dr.Selvamani Indrajith,
Professor and Head of the Electronics & Communication Engineering
Department, for their kind cooperation in making this Mini Project work a
success.

We extend our gratitude to Sri. Ch. Malla Reddy, Founder-Chairman

and Sri. Ch. Mahender Reddy, Secretary, MRCEW and
Sri. P. Praveen Reddy Director, MRGI for their kind cooperation in providing
the infrastructure for successful completion of our Mini project work.

We convey our special thanks to the Entire Teaching faculty and non-
teaching staff members of the Electronics & Communication Engineering
Department for their support in making this project work a success.

ARE DIVYA (20RG1A0406)

GULLAPALLI SOWJANYA (20RG1A0426)
KOTTAPALLY MANASA (20RG1A0429)
VETTINI RACHANA (20RG1A0455)

i
 INDEX

Chapter Page No
ABSTRACT iv
LIST OF FIGURES v
LIST OF TABLES vi
1.INTRODUCTION 1
1.1 Face Detection 4
1.2 Face Recognition 5
1.3 Local Binary Patterns Operator 6
1.4 Face Description with LBP 7
1.5 The Paradigm of the Face Recognition 8
1.6 Local Binary Patterns 11
1.7 Principles of Local Binary Patterns 12
1.8 Uniform of Local Binary Patterns 14
1.9 Face Recognition Using Binary Patterns 14
1.10 Feature Vectors 15
1.11 Comparing the Feature Vectors 15
2.LITERATURE SURVEY 25
3.EXISTING SYSTEM 31
4.PROPOSED SYSTEM 32
5.SYSTEM REQUIREMENTS 36
5.1 Software Required 36
5.2 Hardware Required 36
6.RESULTS AND DISCUSSIONS 37
7.CONCLUTION & FUTURE SCOPE 42
7.1 Conclusion 42
7.2 Future Scope 42
8.APPENDIX 43
8.1 Technologies Used 43
8.1.1 Python 43
8.2 How does Machine Learning Work 43
8.3 Library Files 44

ii
 8.3.1 Pandas 44
8.3.2 Matplotlib 44
8.3.3 Seaborn 45
8.3.4 Sklearn 45
8.3.5 Keras 46
9.BIBLOGRAPHY 53

iii
 ABSTRACT
A real time face recognition system is capable of identifying or verifying a
person from a video frame. To recognize the face in a frame, first you need to
detect whether the face is present in the frame. If it is present, mark it as a
region of interest (ROI), extract the ROI and process it for facial recognition.
Facial recognition is done using LBP features. The application prepared has 2
main steps, one is face registration and second is face recognition. Face
registration capture the person face and train classifier while face recognition
recognise face using trained classifier.
Keypoints: Face Recognition, Local Binary pattern, Face detection, ROI,
Graphical User Interface.

iv
 LIST OF FIGURES

FIG.NO FIGURE NAME PAGE NO

Principle of an Identification Process with Face

1.1 11
Recognition
1.2 A Pre-Processed Image Divided into 64 Regions 11

1.3 The Original LBP Operator 12

1.4 Different Texture Primitives Detected by the LBP 12

1.5 Different Texture Primitives Detected by the LBP 14

Flow Chat of the LBP process Flow Chat of the

4.1 34
Proposed System
4.2 Flow Diagram for Whole System 35

6.1 Different Input Facial Image 37

6.2 Facial Image from the Database 37

6.3 Facial Expression from Webcam 39

6.4 Facial Capturing 40

6.5 Facial Expression Detection 40

6.6 Facial Recognition python and Open CV from 41

Webcam

v
 LIST OF TABLES

TABLE NO TABLE NAME PAGE NO

6.1 Recognition Rate of the Research 38

vi
 CHAPTER 1: INTRODUCTION
The face of a human being conveys a lot of information about identity
and emotional state of the person. Face recognition is an interesting and
challenging problem, and impacts important applications in many areas such
as identification for law enforcement, authentication for banking and security
system access, and personal identification among others. In our research
work mainly consists of three parts, namely face representation, feature
extraction and classification. Face representation represents how to model a
face and determines the successive algorithms of detection and recognition
images. Facial expression is one of the most powerful, natural and immediate
means for human beings to communicate their emotions and intensions.
Face recognition is an interesting and challenging problem, and impacts
important applications in many areas such as identification for law
enforcement, authentication for banking and security system access, and also
personal identification among others . The face plays a major role in our social
intercourse in conveying identity and emotion. The human ability to recognize
faces is remarkable. Modern Civilization heavily depends on person
authentication for several purposes. Face recognition has always a major
focus of research because of its non-invasive nature and because it is peoples
primary method of person identification. A face recognition system would
allow user to be identified by simply walking past a surveillance camera.
Human beings often recognize one another by unique facial
characteristics. One of the latest biometric technologies, automatic facial
recognition, is based on this phenomenon. Facial recognition is the most
successful form of human surveillance. Facial recognition technology, is being
used to improve human efficiency when recognizing faces, is one of the fastest
growing fields in the biometric industry. Interest in facial recognition is being
fueled by the availability and low cost of video hardware, the ever-increasing
number of video cameras being placed in the workspace, and the noninvasive
aspect of facial recognition systems.
In this area, the variations of light, gesture, expression, age or the
imaging conditions in the face image have become a key challenge. For this
extracting local structure based on texture and structural characteristics is

1
important. In Local Binary Pattern (LBP) which is frequently used for texture
classification, for each pixel, a binary code is produced by thresholding its
value with the value of the centre pixel. A histogram is created to collect the
occurrences of different binary patterns.
For face recognition, the face area is first divided into small regions from
which Local Binary Pattern (LBP) histograms are extracted and concatenated
into a single, spatially enhanced feature histogram efficiently representing the
face image. Efficiency & the simplicity of the proposed method allows for very
fast feature extraction. The textures of the facial regions are locally encoded
by the LBP patterns while the whole shape of the face is recovered by the
construction of the face feature histogram. The idea behind using the LBP
features in face recognition is that the face images can be seen as composition
of micro-patterns which are invariant with respect to monotonic grey scale
transformations. Combining these micro-patterns, a global description of the
face image is obtained.
The research of face recognition technique based on local structure is
to be characterized in the field of face classification in recent years. And the
variations of light, gesture, expression, age or the imaging conditions have
become a key challenge. But as an invariability feature of face image, texture
and structural characteristics are not sensitive to illumination variations.
Therefore, extracting local structure based on texture and structural
characteristics is important. Local Binary Pattern (LBP) operator has been
proved to be a theoretically simple yet very effective multiresolution statistical
texture descriptor in terms of the characteristics of the local structure.
It can measure the texture information of local area of gray image. In
the past ten years, the operator has been widely used in texture classification,
image retrieval and other areas such as facial image analysis. Because of the
direct and simple calculation, insensitivity to the light and rotation, capability
for capturing image detail, the operator can extract the patterns of local region
which are more favourable. However, LBP has two shortcomings: one of them
is the incompetent of capturing global patterns which are important for
recognition, the other is that it is sensitive to significant noise because the
operator only compares the gray among pixels.

2
 The consequence of these disadvantages is that LBP can’t extract the
important structures from important area of face image completely. The
traditional solution is to block the facial image, and then connect the
histogram of every block’s LBP as the total histogram feature. Although this
method eliminates the noise, it neglects the global features. Zhang et al
utilized image’s block as a pixel to operate LBP , improved the performance of
face recognition by extracting multi-scale features, and decreased the
characteristic dimension, but the correct rate is 80% while classifying due to
partition.
Ma [5] et al combined Gabor filters and LBP as texture detectors,
although it’s effective, the process is time consuming due to the large feature
dimensions. In this paper, we propose a novel representation, called
Threshold LBP (TLBP), to overcome the incompetent of capturing global
patterns. In TLBP, a majority of patterns are extracted by setting a threshold
to reduce the influence of local structures. After merging the patterns
extracted by original LBP as face patterns, several advantages are presented.
First, it encodes not only microstructures but also macrostructures of image
patterns. Second, it is more robust than LBP and can reduce the impact of
local evident noises while capture the global patterns. Therefore, TLBP can
improve the result of face classification. The results show that it is feasible to
realize the face classification by the features of mixing extracted by the TLBP
and LBP, and the correctness rate of the recognition is higher.
Face recognition is the ability to establish a subject’s identity based on
facial characteristics. Automated face recognition requires various techniques
from different research fields, including computer vision, image processing,
pattern recognition, and machine learning. In a typical face recognition
system, face images from a number of subjects are enrolled into the system
as gallery data, and the face image of a test subject (probe image) is matched
to the gallery data using a one-to-one or one-tomany scheme. The one-to-one
and one-to-many matchings are called verification and identification,
respectively. Face recognition is one of the fundamental methods used by
human beings to interact with each other.

3
 Attempts to match faces using a pair of photographs dates back to 1871
in a British court. Techniques for automatic face recognition have been
developed over the past three decades for the purpose of automatic person
recognition with still and video images. Face recognition has a wide range of
applications, including law enforcement, civil applications, and surveillance
systems. Face recognition applications have also been extended to smart
home systems where the recognition of the human face and expression is used
for better interactive communications between human and machines.
Fig. 1.1 shows some biometric applications using the face. The face has
several advantages that makes it one of the most preferred biometric traits.
First, the face biometric is easy to capture even at a long distance. Second,
the face conveys not only the identity but C.
Recent efforts have focused on using 3D models, video input, and
different features (e.g., skin texture) to overcome the performance bottleneck
in 2D still face recognition. This chapter begins with a survey of face
recognition in 2D, 3D, and video domains and presents the challenges in face
recognition problems. We also introduce problems in face recognition due to
subject aging. The relevance of facial marks or micro features (e.g., scars,
birthmarks) to face recognition is also presented.
1.1 Face Detection:
The first problem that needs to be addressed in face recognition is face
detection. Some of the well-known face detection approaches can be
categorized as: i) color based, ii) template based, and iii) feature based. Color
based approaches learn the statistical model of skin color and use it to
segment face candidates in an image. Template based approaches use
templates that represent the general face appearance, and use cross
correlation based methods to find face candidates.
State-of-the-art face detection methods are based on local features and
machine learning based binary classification (e.g., face versus non-face)
methods, following the seminal work by Viola et al. The face detector proposed
by Viola et al. has been widely used in various studies involving face
recognition because of its real-time capability, high accuracy, and availability

4
in the Open Computer Vision Library (OpenCV). Fig. 1.2 shows an example
face detection result using the method in.
1.2 Face Recognition:
In a typical face recognition scenario, face images from a number of
subjects are enrolled into the system as gallery data, and the face image of a
test subject (probe image) is matched to gallery data using a one-to-one or
one-to-many scheme. There are three different modalities that are used in face
recognition applications: 2D, 3D, and video. We will review the face
recognition problems in these domains in the following sections.
AUTOMATIC face analysis which includes, e.g., face detection, face
recognition, and facial expression recognition has become a very active topic
in computer vision research. A key issue in face analysis is finding efficient
descriptors for face appearance. Different holistic methods such as Principal
Component Analysis (PCA), Linear Discriminant Analysis (LDA), and the more
recent 2D PCA have been studied widely but lately also local descriptors have
gained attention due to their robustness to challenges such as pose and
illumination changes.
This paper presents a novel descriptor based on local binary pattern
texture features extracted from local facial regions. One of the first face
descriptors based on information extracted from local regions is the eigen
features method proposed by Pentland et al. This is a hybrid approach in
which the features are obtained by performing PCA to local face regions
independently. In Local Feature Analysis, kernels of local spatial support are
used to extract information about local facial components. Elastic Bunch
Graph Matching (EBGM) describes faces using Gabor filter responses in
certain facial landmarks and a graph describing the spatial relations of these
landmarks.
The validity of the component-based approach is also attested by the
study conducted by Heisele et al. in which a component-based face
recognition system clearly outperformed global approaches on a test database
containing faces rotated in depth. Using local photometric features for object
recognition in the more general context has become a widely accepted
approach. In that setting, the typical approach is to detect interest points or

5
interest regions in images, perform normalization with respect to affine
transformations, and describe the normalized interest regions using local
descriptors.
This bag-of-keypoints approach is not suited for face description as such
since it does not retain information on the spatial setting of the detected local
regions but it does bear certain similarities to local feature-based face
description. Finding good descriptors for the appearance of local facial regions
is an open issue. Ideally, these descriptors should be easy to compute and
have high extra-class variance (i.e., between different persons in the case of
face recognition) and low intraclass variance, which means that the descriptor
should be robust with respect to aging of the subjects, alternating illumination
and other factors.
The texture analysis community has developed a variety of different
descriptors for the appearance of image patches. However, face recognition
problem has not been associated to that progress in texture analysis field as
it has not been investigated from such point of view. Recently, we investigated
the representation of face images by means of local binary pattern features,
yielding in outstanding results that were published in the ECCV 2004
conference. After this, several research groups have adopted our approach. In
this paper, we provide a more detailed analysis of the proposed
representation, present additional results and discuss further extensions.
1.3 Local Binary Patterns Operator:
The LBP operator was originally designed for texture description. The
operator assigns a label to every pixel of an image by thresholding the 3 3-
neighborhood of each pixel with the center pixel value and considering the
result as a binary number. Then, the histogram of the labels can be used as
a texture descriptor. See Fig.1.1 for an illustration of the basic LBP operator.
To be able to deal with textures at different scales, the LBP operator was later
extended to use neighborhoods of different sizes.
Defining the local neighborhood as a set of sampling points evenly spaced
on a circle centered at the pixel to be labeled allows any radius and number
of sampling points. Bilinear interpolation is used when a sampling point does
not fall in the center of a pixel. In the following, the notation ðP;RÞ will be

6
used for pixel neighborhoods which means P sampling points on a circle of
radius of R. See Fig.1.2 for an example of circular neighborhoods. Another
extension to the original operator is the definition of so-called uniform
patterns.
A local binary pattern is called uniform if the binary pattern contains at
most two bitwise transitions from 0 to 1 or vice versa when the bit pattern is
considered circular. For example, the patterns 00000000 (0 transitions),
01110000 (2 transitions) and 11001111 (2 transitions) are uniform whereas
the patterns 11001001 (4 transitions) and 01010011 (6 transitions) are not.
In the computation of the LBP histogram, uniform patterns are used so that
the histogram has a separate bin for every uniform pattern and all no uniform
patterns are assigned to a single bin. Ojala et al. noticed that in their
experiments with texture images, uniform patterns account for a bit less than
90 percent of all patterns when using the (8,1) neighborhood and for around
70 percent in the (16,2) neighborhood. We have found that 90.6 percent of the
patterns in the (8,1) neighborhood and 85.2 percent of the patterns in the
(8,2) neighborhood are uniform in case of pre-processed FERET facial images
1.4 Face Description with LBP:
In this work, the LBP method presented in the previous section is used
for face description. The procedure consists of using the texture descriptor to
build several local descriptions of the face and combining them into a global
description. Instead of striving for aholistic description, this approach was
motivated by two reasons: the local feature-based or hybrid approaches to
face recognition have been gaining interest lately which is understandable
given the limitations of the holistic representations. These local feature-based
and hybrid methods seem to be more robust against variations in pose or
illumination than holistic methods.
Another reason for selecting the local feature-based approach is that
trying to build a holistic description of a face using texture methods is not
reasonable since texture descriptors tend to average over the image area. This
is a desirable property for ordinary textures, because texture description
should usually be invariant to translation or even rotation of the texture and,
especially, for small repetitive textures, the small-scale relationships

7
determine the appearance of the texture and, thus, the large-scale relations
do not contain useful information. For faces, however, the situation is
different: retaining the information about spatial relations is important.
This reasoning leads to the basic methodology of this work. The facial
image is divided into local regions and texture descriptors are extracted from
each region independently. The descriptors are then concatenated to form a
global description of the face. See Fig. 1.3 for an example of a facial image
divided into rectangular regions. The basic histogram can be extended into a
spatially enhanced histogram which encodes both the appearance and the
spatial relations of facial regions.
As the m facial regions R0; R1; ... Rm1 have been determined, a
histogram is computed independently within each of the m regions. The
resulting m histograms are combined yielding the spatially enhanced
histogram. The spatially enhanced histogram has size m n, where n is the
length of a single LBP histogram. In the spatially enhanced histogram, we
effectively have a description of the face on three different levels of locality:
The LBP labels for the histogram contain information about the patterns on a
pixel-level, the labels are summed over a small region to produce information
on a regional level, and the regional histograms are concatenated to build a
global description of the face.
It should be noted that when using the histogram-based methods,
despite the examples in Fig. 3, the regions R0; R1; ... Rm1 do not need to be
rectangular. Neither do they need to be of the same size nor do shape and
they not necessarily have to cover the whole image. For example, they could
be circular regions located at the fiducial points like in the EBGM method. It
is also possible to have partially overlapping regions. If recognition of faces
rotated in depth is considered, it may be useful to follow the procedure of
Heisele et al. And automatically detect each region in the image instead of first
detecting the face and then using a fixed division into regions.
1.5 The Paradigm of the Face Recognition:
Despite of the fact that at this moment already numerous of commercial
face recognition systems are in use, this way of identification continues to be
an interesting topic for researchers. This is due to the fact that the current

8
systems perform well under relatively simple and controlled environments,
but perform much worse when variations in different factors are present, such
as pose, viewpoint, facial expressions, time (when the pictures are made) and
illumination (lightening changes).
The goal in this research area is to minimize the influence of these
factors and create robust face recognition system. A model for face recognition
The process of person identification by using face recognition can be split into
three main phases .These are face representation, feature extraction and
classification. Face representation is the first task, that is, how to model a
face. The way to represent a face determines the successive algorithms of
detection and identification.
For the entry-level recognition (that is, to determine whether or not the
given image represents a face), the image is transformed (scaled and rotated)
till it has the same ‘position’ as the images from the database. In the feature
extraction phase, the most useful and unique features (properties) of the face
image are extracted. With these obtained features, the face image is compared
with the images from the database. This is done in the classification phase.
The output of the classification part is the identity of a face image from the
database with the highest matching score, thus with the smallest differences
compared to the input face image. Also a threshold value can be used to
determine if the differences are small enough. After all, it could be that a
certain face is not in the database at all.
Facial skin Face recognition is an active research area, and they can be
utilized in wide range applications such as surveillance and security. Face is
a sophisticated multidimensional structure and needs a good computing
technique for recognition. Face recognition system can be utilized in two
modes: Verification and Identification. Facial feature extraction includes
localizing probably the most characteristics features of the face image like
eyes, nose and mouth regions. There are two forms of solution to extract
features from the facial images and they are: Geometric feature considers the
deviation fit, location, space between two eyes and period of the nose.
The looks feature presents the looks variations of the face image, such
as wrinkles and furrows. The looks feature needs to be extracted on either the

9
whole face image or specific regions in a facial image. The face is certainly one
of the main features of the human beings and usually used as identification.
Recognizing different human faces is not an arduous task for humans, but it
is quite hard to the system to acknowledge the human faces. LBP generate
the binary code that describes local texture pattern by normalizing intensity
values in neighborhood. The eyes and nose region is extracted from the LBP
face image and then LBP histograms are drawn for each pixel of the image.
PCA is just a way of identifying patterns in data and expressing the info to
highlight their similarities and differences.
Image Acquisition:
This is actually the first faltering step where we acquire image using any
digital device Images employed for facial expression recognition are static
images or image sequences. 2-D monochrome (grey-scale) facial image
sequences are the most popular form of pictures employed for automatic
expression recognition.
Pre-processing:
Image pre-processing takes the form of signal conditioning (such as noise
removal, and normalization from the variation of pixel position or brightness),
along with segmentation, location, or tracking of the face or its parts.
Feature Extraction:
Feature extraction converts pixel data in to a higher-level representation
of shape, motion, color, texture, and spatial configuration of the or its
components.
Classification:
Expression categorization is conducted by a classifier, which frequently
contains types of pattern distribution, coupled to a choice procedure. A wide
range of classifiers, covering parametric along with non-parametric
techniques, has been placed on the automatic expression recognition
problem.
Post-processing:
Post-processing aims to boost recognition accuracy, by required domain
knowledge to correct classification errors or by coupling together several
degrees of a classification hierarchy.

10
 Input Face Feature Image
image representation extraction Database

Identify of
Face Classification

Fig 1.1: Principle of an identification process with face recognition

1.6 Local Binary Patterns:
There exist several methods for extracting the most useful features from
(pre-processed) face images to perform face recognition. One of these feature
extraction methods is the Local Binary Pattern (LBP) method. This relative
new approach was introduced in 1996 by Ojala et al. With LBP it is possible
to describe the texture and shape of a digital image. This is done by dividing
an image into several small regions from which the features are extract.

Fig 1.2 : A pre-processed image divided into 64 region

These features consist of binary patterns that describe the surroundings
of pixels in the regions. The obtained features from the regions are
concatenated into a single feature histogram, which forms a representation of
the image. Images can then be compared by measuring the similarity
(distance) between their histograms. According to several studies [2, 3, 4] face
recognition using the LBP method provides very good results, both in terms
of speed and discrimination performance. Because of the way the texture and

11
shape of images is described, the method seems to be quite robust against
face images with different facial expressions, different lightening conditions,
image rotation and aging of persons.
1.7. Principles of Local Binary Patterns:
The original LBP operator was introduced by Ojala et al. [15]. This
operator works with the eight neighbors of a pixel, using the value of this
center pixel as a threshold. If a neighbor pixel has a higher gray value than
the center pixel (or the the same gray value) than a one is assigned to that
pixel, else it gets a zero. The LBP code for the center pixel is then produced by
concatenating the eight ones or zeros to a binary code (figure 1.3).

Fig 1.3: The Original LBP Operator

Later the LBP operator was extended to use neighborhoods of different
sizes. In this case a circle is made with radius R from the center pixel. P
sampling points on the edge of this circle are taken and compared with the
value of the center pixel. To get the values of all sampling points in the
neighborhood for any radius and any number of pixels, (bilinear) interpolation
is necessary. For neighborhoods the notation (P, R) is used. Figure 1.4
illustrates three neighbor-sets for different values of P and R.

Fig 1.4: Different texture primitives detected by the LBP

12
 If the coordinates of the center pixel are (xc, yc) then the coordinates of
his P neighbors (xp, yp) on the edge of the circle with radius R can be
calculated with the sinus and cosines:

If the gray value of the center pixel is gc and the gray values of his
neighbors are gp, with p = 0, ..., P − 1, than the texture T in the local
neighborhood of pixel (xc, yc) can be defined as:

Once these values of the points are obtained is it also possible do

describe the texture in another way. This is done by subtracting the value of
the center pixel from the values of the points on the circle. On this way the
local texture is represented as a joint distribution of the value of the center
pixel and the differences:

Since t(gc) describes the overall luminance of an image, which is unrelated

to the local image texture, it does not provide useful information for texture
analysis. Therefore, much of the information about the textural
characteristics in the original joint distribution (Eq. 3) is preserved in the joint
difference distribution (Ojala et al. 2001)

Although invariant against gray scale shifts, the differences are affected
by scaling. To achieve invariance with respect to any monotonic
transformation of the gray scale, only the signs of the differences are
considered. This means that in the case a point on the circle has a higher gray
value than the center pixel (or the same value), a one is assigned to that point,
and else it gets a zero:

Where

13
In the last step to produce the LBP for pixel (xc, yc) a binomial weight 2p is
assigned to each sign s(gp − gc). These binomial weights are summed:

The Local Binary Pattern characterizes the local image texture around (xc, yc).
The original LBP operator in figure 1.3 is very similar to this operator with P
= 8 and R = 1, thus LBP8,1. The main difference between these operators is
that in LBP8,1 the pixels first need to be interpolated to get the values of the
points on the circle
1.8 Uniform of Local Binary Patterns:
A Local Binary Pattern is called uniform if it contains at most two
bitwise transitions from 0 to 1 or vice versa. In a matter of fact this means
that a uniform pattern has no transitions or two transitions. Only one
transition is not possible, since the binary string needs to be considered
circular. The two patterns with zero transitions, with for example eight bits,
are 00000000 and 11111111. Examples of uniform patterns with eight bits
and two transitions are 00011100 and 11100001. For patterns with two
transitions are P(P −1) combinations possible.

Fig 1.5 : Different texture primitives detected by the LBP

Using only uniform Local Binary Patterns has two important benefits. The first
one is that it saves memory.
1.9 Face Recognition using Local Binary Patterns:
We explained how the LBP-method can be applied on images (of faces)
to extract features which can be used to get a measure for the similarity

14
between these images. The main idea is that for every pixel of an image the
LBP-code is calculated. The occurrence of each possible pattern in the image
is kept up. The histogram of these patterns, also called labels, forms a feature
vector, and is thus a representation for the texture of the image. These
histograms can then be used to measure the similarity between the images,
by calculating the distance between the histograms.
1.10 Feature Vectors:
Once the Local Binary Pattern for every pixel is calculated, the feature
vector of the image can be constructed [10]. For an efficient representation of
the face, first the image is divided into K2 regions. In figure 1.7 a face image
is divided into 82 = 64 regions. For every region a histogram with all possible
labels is constructed. This means that every bin in a histogram represents a
pattern and contains the number of its appearance in the region. The feature
vector is then constructed by concatenating the regional histograms to one
big histogram.
1.11 Comparing the Feature Vectors:
To compare two face images, a sample (S) and a model (M), the
difference between the feature vectors has to measure. This can be done with
several possible dissimilarity measures for histograms:

Histogram Intersection:

Log-likelihood Statistic:

Chi square statistic (x2 ):

In these equations Si,j and Mi,j are the sizes of bin i from region j (number
of appearance of pattern L(i) in region j). Because some regions of the face
images (for example the regions with the eyes) could contain more useful

15
information than others, a weight can be set for each region based on the
importance of the information it contains. According to article the x2 performs
slightly better than histogram intersection and the log-likelihood statistic.
The face area is first divided into small regions from which Local Binary
Patterns (LBP), histograms are extracted and concatenated into a single
feature vector. This feature vector forms an efficient representation of the face
and is used to measure similarities between images. Facial expression is one
of the most powerful, natural and immediate means for human beings to
communicate their emotions and intensions. Automatic facial expression
analysis is an interesting and challenging problem, and impacts important
applications in many areas such as human–computer interaction and data-
driven animation.
Due to its wide range of applications, automatic facial expression
recognition has attracted much attention in recent years [1–4]. Though much
progress has been made [5–24], recognizing facial expression with a high
accuracy remains difficult due to the subtlety, complexity and variability of
facial expressions. Deriving an effective facial representation from original face
images is a vital step for successful facial expression recognition. There are
two common approaches to extract facial features: geometric feature-based
methods and appearance-based methods. Geometric features present the
shape and locations of facial components, which are extracted to form a
feature vector that represents the face geometry.
Recently Valstar et al. Have demonstrated that geometric feature-based
methods provide similar or better performance than appearance-based
approaches in Action Unit recognition. However, the geometric feature-based
methods usually require accurate and reliable facial feature detection and
tracking, which is difficult to accommodate in manyituations. With
appearance-based methods, image filters, such as Gabor wavelets, are applied
to either the whole-face or specific face-regions to extract the appearance
changes of the face. Due to their superior performance, the major works on
appearance-based methods have focused on using Gabor-wavelet
representations.

16
 However, it is both time and memory intensive to convolve face images
with a bank of Gabor filters to extract multi-scale and multi-orientational
coefficients. In this work, we empirically study facial representation based on
Local Binary Pattern (LBP) features for person-independent facial expression
recognition. LBP features were proposed originally for texture analysis, and
recently have been introduced to represent faces in facial images analysis [29–
31]. The most important properties of LBP features are their tolerance against
illumination changes and their computational simplicity.
We examine different machine learning methods, including template
matching, programming technique Support Vector Machine (SVM), Linear
Discriminant Analysis (LDA) and the linear, to perform facial expression
recognition using LBP features. Our study demonstrates that, compared to
Gabor wavelets, LBP features can be derived very fast in a single scan through
the raw image and lie in low-dimensional feature space, while still retaining
discriminative facial information in a compact representation. We further
formulate Boosted-LBP by learning the most discriminative LBP features with
AdaBoost, and the recognition performance of different classifiers are
improved by using the Boosted-LBP features.
We also evaluate the generalization ability of LBP features across
different databases. One limitation of the existing facial expression recognition
methods is that they attempt to recognize facial expressions from data
collected in a highly controlled environment given high resolution frontal
faces. However, in real-world applications such as smart meeting and visual
surveillance, the input face images are often at low resolutions. Obviously low-
resolution images in realworld environments make real-life expression
recognition much more difficult. Recently Tian et al. made a first attempt to
recognize facial expressions at low resolutions.
In Tian studied the effects of different image resolutions for each step of
automatic facial expression recognition. In this work, we investigate LBP
features for low-resolution facial expression recognition. Experiments on
different image resolutions show that LBP features perform stably and
robustly over a useful range of low resolutions of face images. The encouraging
performance on real-world compressed video sequences illustrated their

17
promising applications in real-world environments. This paper is an extended
version of our previous work described. The main contributions of this paper
are summarized as follows: We empirically evaluate LBP features for person-
independent facial expression recognition.
Different machine learning methods are exploited to classify expressions
on several databases. LBP features were previously used for facial expression
classification, and more recently, following our work, Liao et al. presented an
extended LBP operator to extract features for facial expression recognition.
However, these existing works were conducted on a very small database
(JAFFE) using an individual classifier. In contrast, here we comprehensively
study LBP features for facial expression recognition with different classifiers
on much larger databases.
We investigate LBP features for low-resolution facial expression
recognition, a critical problem but seldom addressed in the existing work. We
not only perform evaluation on different image resolutions, but also conduct
experiments in real-world compressed video sequences. Compared to the
previous work, LBP features provide just as good or better performance, so
are very promising for real-world applications. We formulate Boosted-LBP by
learning the most discriminative LBP histograms with AdaBoost for each
expression, and the recognition performance of different classifiers are
improved by using the Boosted-LBP features.
We also evaluate the generalization ability of LBP features cross different
databases. The remainder of this paper is structured as follows. Automatic
facial expression recognition has attracted much attention from behavioral
scientists since the work of Darwin in 1872. Suwa et al. made the first attempt
to automatically analyze facial expressions from image sequences in 1978.
Much progress has been made in the last decade, and a thorough survey of
the exiting work can be found. Here we briefly review some previous work in
order to put our work in context.
Automatic facial expression recognition involves two vital aspects: facial
representation and classifier design. Facial representation is to derive a set of
features from original face images to effectively represent faces. The optimal
features should minimize within-class variations of expressions while

18
maximize between class variations. If inadequate features are used, even the
best classifier could fail to achieve accurate recognition. In some existing work
optical flow analysis has been used to model muscles activities or estimate
the displacements of feature points. However, flow estimates are easily
disturbed by the nonrigid motion and varying lighting, and are sensitive to the
inaccuracy of image registration and motion discontinuities.
Facial geometry analysis has been widely exploited in facial
representation, where shapes and locations of facial components are extracted
to represent the face geometry. For example, Zhang et al. used the geometric
positions of 34 fiducial points as facial features to represent facial images. In
image sequences, the facial movements can be qualified by measuring the
geometrical displacement of facial feature points between the current frame
and the initial frame. Valstar et al. presented AU detection by classifying
features calculated from tracked fiducial facial points.
Their method detects a similar amount of AUs with similar or higher
recognition rates than those reported. So they argued that the facial
representation based on tracked facial points is well suited for facial
expression analysis. Recently they further presented a fully automatic AU
detection system that can automatically localize facial points in the first frame
and recognize AU temporal segments using a subset of most informative
spatiotemporal features selected by AdaBoost. However, the geometric
feature-based representation commonly requires accurate and reliable facial
feature detection and tracking, which is difficult to accommodate in many
situations.
In motions of facial features are measured by simultaneously using an
active Infra-Red illumination and Kalman filtering to deal with large variations
of head motion. Another kind of method to represent faces is to model the
appearance changes of faces. Holistic spatial analyses including Principal
Component Analysis (PCA), Linear Discriminant Analysis (LDA) , Independent
Component Analysis (ICA) and Gabor wavelet analysis have been applied to
either the whole-face or specific face regions to extract the facial appearance
changes.

19
 Donato et al. explored different techniques to represent face images for
facial action recognition, which include PCA, ICA, Local Feature Analysis
(LFA), LDA and local schemes such as Gabor-wavelet representation and local
principal components. Best performances were obtained using Gabor-wavelet
representation and ICA. Due to their superior performance, Gabor-wavelet
representations have been widely adopted in face image analysis. However,
the computation of Gabor-wavelet representations is both time and memory
intensive, for example, the Gabor-wavelet representation derived from each
48 48 face image has the high dimensionality of Oð105 Þ. Recently Local
Binary Patterns have been introduced as effective appearance features for
facial image analysis.
We compared LBP features with Gabor features for facial expression
recognition, and studied their performance over a range of image resolutions.
In we further presented facial expression manifold learning in the LBP feature
space. More recently, Liao et al. introduced20 an improved LBP operator to
extract features in both intensity and gradient maps for facial expression
recognition, and also tested their methods on facial images of reduced
resolutions.
However, their experiment was carried out on a very small database (213
images from 10 subjects). In this work, we comprehensively study LBP
features for facial expression recognition on several databases. 2.2. Facial
expression recognition Different techniques have been proposed to classify
facial expressions, such as Neural Network, Support Vector Machine (SVM),
Bayesian Network (BN) and rule-based classifiers. In Lyons et al.’ work, the
principle components of the feature vectors from training images were
analyzed by LDA to form discriminant vectors, and facial image classification
was performed by projecting the input vector of a testing image along the
discriminant vectors.
Cohen et al. compared different Bayes classifiers, and Gaussian Tree-
Augmented-Naive (TAN) Bayes classifiers performed best. Bartlett et al.
performed systematic comparison of different techniques including AdaBoost,
SVM and LDA for facial expression recognition, and best results were obtained
by selecting a subset of Gabor filters using AdaBoost and then training SVM

20
on the outputs of the selected filters. Pantic and Rothkrantz adopted rule-
based reasoning to recognize action units and their combination. To exploit
the temporal behaviors of facial expressions, different techniques were
presented for facial expression recognition in image sequences.
There have been several attempts to track and recognize facial
expressions over time based on optical flow analysis. Tian et al. presented a
Neural Network based approach to recognize facial action units in image
sequences. Hidden Markov Models (HMMs) have been widely used to model
the temporal behaviors of facial expressions from image sequences. Cohen et
al. proposed a multi-level HMM classifier, which allows not only to perform
expression classification on a video segment, bust also to automatically
segment a long video sequence to the different expressions segments without
resorting to heuristic methods of segmentation.
But HMMs cannot deal with dependencies in observation. Dynamic
Bayesian Networks (DBNs) recently were exploited for sequence-based
expression recognition. Kaliouby and Robinson proposed a system for
inferring complex mental states from videos of facial expressions and head
gestures, where a multi-level DBN classifier was used to model complex
mental states as a number of interacting facial and head displays. Zhang and
Ji explored the use of multisensory information fusion technique with DBNs
for modeling and understanding the temporal behaviors of facial expressions
in image sequences. Chang et al. proposed a probabilistic video-based facial
expression recognition method based on manifolds.
Lee and Elgammal recently introduced a framework to learn
decomposable generative models for dynamic appearance of facial expressions
where facial motion is constrained to one dimensional closed manifolds. The
learned model can generate different dynamic facial appearances for different
people and for different expressions, so enabling simultaneous recognition of
faces and facial expressions.
Data Facial expressions can be described at different levels. A widely
used description is Facial Action Coding System (FACS), which is a human-
observer-based system developed to capture subtle changes in facial
expressions. With FACS, facial expressions are decomposed into one or more

21
Action Units (AUs). AU recognition or detection has attracted much attention
recently.
Meanwhile, psychophysical studies indicate that basic emotions have
corresponding universal facial expressions across all cultures. This is
reflected by most current facial expression recognition systems that attempt
to recognize a set of prototypic emotional expressions including disgust, fear,
joy, surprise, sadness and anger. Therefore, in this work, we also focus on
prototypic expression recognition.
We consider both 6-class prototypic expression recognition and 7-class
expression recognition by including the neutral expression. We mainly
conducted experiments on the Cohn–Kanade database, one of the most
comprehensive database in the current facial-expression-research
community. The database consists of 100 university students aged from 18
to 30 years, of which 65% were female, 15% were African-American and 3%
were Asian or Latino. Subjects were instructed to perform a series of 23 facial
displays, six of which were based on description of prototypic emotions.
Image sequences from neutral to target display were digitized into 640
490 pixel arrays with 8-bit precision for grayscale values. Fig. 1.1 shows
some sample images from the Cohn–Kanade database. For our experiments,
we selected 320 image sequences from the database. The only selection
criterion was that a sequence could be labeled as one of the six basic
emotions. The sequences come from 96 subjects, with 1–6 emotions per
subject. For each sequence, the neutral face and three peak frames were
used for prototypic expression recognition, resulting in 1280 images (108
Anger, 120 Disgust, 99 Fear, 282 Joy, 126 Sadness, 225 Surprise and 320
Neutral).
To evaluate the generalization performance to novel subjects, we adopted
a 10-fold cross-validation testing scheme in our experiments. More precisely,
we partitioned the dataset randomly into ten groups of roughly equal numbers
of subjects. Nine groups were used as the training data to train classifiers,
while the remaining group was used as the test data. The above process was
repeated ten times for each group in turn to be omitted from the training
process. We reported the average recognition results on the test sets.

22
 Following Tian, we normalized the faces to a fixed distance between the
two eyes. We manually labeled the eyes location, to evaluate LBP features in
the condition of no face registration errors. Automatic face registration can be
achieved by face detection and eye localization, which will be addressed in our
future work. Facial images of 110 150 pixels were cropped from original
frames based on the two eyes location. No further registration such as
alignment of mouth was performed in our algorithms. As the faces in the
database are frontal view, we did not consider head pose changes. For realistic
sequences with head pose variation, head pose estimation can be adopted to
detect front or near front view.
Illumination changes exist in the database, but there was no attempt
made to remove illumination changes in our experiments, due to LBP’s gray-
scale invariance. Fig.1.2 shows an example of the original face image and the
cropped image. Local Binary Patterns (LBP) The original LBP operator was
introduced by Ojala et al. and was proved a powerful means of texture
description.
The operator labels the pixels of an image by thresholding a 3 3
neighborhood of each pixel with the center value and considering the results
as a binary number (see Fig. 1.3 for an illustration), and the 256-bin
histogram of the LBP labels computed over a region is used as a texture
descriptor. The derived binary numbers (called Local Binary Patterns or LBP
codes) codify local primitives including different types of curved edges, spots,
flat areas, etc (as shown in Fig. 4), so each LBP code can be regarded as a
micro-texton. The limitation of the basic LBP operator is its small
neighborhood which can not capture dominant features with large scale.
Automatic face recognition (AFR) consists in person identification from
digital images using a computer. This field has been intensively studied during
the past a few decades and its importance is constantly growing particularly
due to the nowadays security issues. It has been proved that local binary
patterns (LBP) are an efficient image descriptor for several tasks in computer
vision field including automatic face recognition. It considers a very small local
neighbourhood of a pixel to compute the feature vector values. The individual

23
values are then computed using the differences between intensity values of
the central and surrounding pixels.
In this paper, we propose a novel image descriptor called Enhanced local
binary patterns (E-LPB). This method improves the original LBP operator by
considering larger central area and larger neighbourhood to compute the
feature vector values. These properties keep more information about the
image structure and can compensate some noise, image variance issues and
the differences between train / test images. This method of computation of
the LBP operator considering more points has, to the best of our knowledge,
never been done before and it is thus the main contribution of this paper.
The proposed method is evaluated on two standard corpora, UFI and
FERET face datasets. UFI dataset is chosen to show the results in real
conditions where the images are noisy, vary in the pose and are illuminated
differently. FERET corpus is used in order to show the results of one training
sample issue. In this case, we have only one image for training. Therefore it is
not possible to improve the results by training step as presented for instance
in and we focus thus rather on the descriptor itself.

24
 CHAPTER 2: LITERATURE SURVEY
Unsang Park [1] Automatic face recognition has been extensively studied over
the past decades in various domains (e.g., 2D, 3D, and video) resulting in a
dramatic improvement. However, face recognition performance severely
degrades under pose, lighting and expression variations, occlusion, and aging.
Pose and lighting variations along with low image resolutions are major
sources of degradation of face recognition performance in surveillance video.
We propose a video-based face recognition framework using 3D face modelling
and Pan-Tilt-Zoom (PTZ) cameras to overcome the pose/lighting variations
and low resolution problems. We propose a 3D aging modeling technique and
show how it can be used to compensate for age variations to improve face
recognition performance. The aging modeling technique adapts view invariant
3D face models to the given 2D face aging database. We also propose an
automatic facial mark detection method and a fusion scheme that combines
the facial mark matching with a commercial face recognition matcher.
The proposed approach can be used i) as an indexing scheme for a face image
retrieval system and ii) to augment global facial features to improve the
recognition performance. Experimental results show i) high recognition
accuracy (>99%) on a large scale video data (>200 subjects), ii) ∼10%
improvement in recognition accuracy using the proposed aging model, and iii)
∼0.94% improvement in the recognition accuracy by utilizing facial marks.
T. Ahonen, A. Hadid and M. Pietikainen[2] This paper presents a novel and
efficient facial image representation based on local binary pattern (LBP)
texture features. The face image is divided into several regions from which the
LBP feature distributions are extracted and concatenated into an enhanced
feature vector to be used as a face descriptor. The performance of the proposed
method is assessed in the face recognition problem under different challenges.
Other applications and several extensions are also discussed.
T. Ahonen, A. Hadid, M. Pietikainen and T. M aenpaa[3]In this paper, we
propose a novel descriptor for face recognition on grayscale images, depth
images and 2D+depth images. It is a compact and effective descriptor
computed from the magnitude and the direction difference. It can be
concatenated with conventional descriptors such as well-known Local Binary
25
Pattern (LBP) and Weber Local Binary Pattern (WLBP), to ehance their
discrimination capability.
To evaluate the performance of our descriptor, we conducted extensive
experiments on three types of images using four different databases. The
experimental results demonstrate the robustness and superiority of our
approach, and the performances of our new descriptor surpass that without
magnitude and direction difference. At the end, we further compare our
descriptor with Convolution Neural Network (CNN) to show the compactness
and effectiveness of the proposed approach
R. Gottumukkal and V.K. Asari[4] A face recognition algorithm based on
modular PCA approach is presented in this paper. The proposed algorithm
when compared with conventional PCA algorithm has an improved recognition
rate for face images with large variations in lighting direction and facial
expression. In the proposed technique, the face images are divided into
smaller sub-images and the PCA approach is applied to each of these sub-
images.
Since some of the local facial features of an individual do not vary even when
the pose, lighting direction and facial expression vary, we expect the proposed
method to be able to cope with these variations. The accuracy of the
conventional PCA method and modular PCA method are evaluated under the
conditions of varying expression, illumination and pose using standard face
databases.
T. Ojala, M. Pietikainen and D. Harwood[5] Face recognition is the
simplest person identification method. Face has been has been chosen
as modality for person identification owning to its simplicity in
implementation as well as being a non-invasive method in achieving
results. Besides its popularity, face recognition still faces issues on its
accuracy.
It has been observed that while using principal component analysis on
varying ambient illumination, recognition accuracy reduces whereas
using local binary pattern gives 100% accuracy. For classificat ion,
support vector machine is adopted as classifier. The system utilises Yale
database and ORL database for experimental results.

26
M. Turk and A. Pentland[6] The performance of a local feature based system,
using Gabor-filters, and a global template matching based system, using a
combination of PCA (Principal Component Analysis) and LDA (Linear
Discriminant Analysis), was correlated with human performance on a
recognition task involving 32 face images. Both systems showed qualitative
similarities to human performance in that all but one of the calculated
correlation coefficients were very or moderately high.
The Gabor filter model seemed to capture human performance better than the
PCA-LDA model since the coefficients for this model were higher for all
examined conditions. Analysis of additional systems based on only PCA, only
LDA, and ICA (Independent Component Analysis) is currently in progress.
M. Kirby and L. Sirovich[7] Face recognition is one of the most challenging
biometric modalities for personal identification. This is due to a number of
factors, including the complexity and variability of the signal captured by a
face device. Several issues incur in the management of a face template as
user’s identity.
Data dimensionality reduction, compactness of the representation,
uniqueness of the template and ageing effects, are just but a few of the issues
to be addressed. In this paper we present the current state of the art in face
recognition technology and how this related to the proper management of a
user’s identity. Some real cases are presented and some conclusions are
drawn
W. Zhao, R. Chellappa, P. J. Phillips, and A. Rosenfeld[8] Attentiveness,
identification and recognition of a human face in a crowd play a starring role
in the perspective of visual surveillance. Human vision system is not
attending, identifying and recognizing all the faces with the same perception.
It is biased towards some faces in the crowd due to their higher relative visual
saliency and segment wise perception with respect to the surroundings faces.
Using different computer vision-based techniques enormous researches have
been carried out on attention, recognition and identification of the human face
in context of different applications.
This paper proposes a novel technique to explain and analyse how the
attention, identification and recognition of a face in the crowd guided through

27
segmentation and their relative visual saliency. The proposed method is
stretched out the solution, using the concept of segmentation and relative
visual saliency which is evaluated on the intensity values and respective
spatial distance of the faces.
S. Z. Li and A. K. Jain (eds.), Springer-Verlag, Secaucus, NJ [9] The face is
the frontal portion of the human head, extending from the forehead to the
chin and includes the mouth, nose, cheeks, and eyes. Being the foremost part
in one’s interactions with the outer world, the face houses most of the
fundamental sensory organs necessary for perceiving the world around,
namely, eyes for seeing, nose for smelling, mouth for tasting, and ears for
hearing. The face is considered to be the most commonly used biometric trait
by humans; we recognize each other and, in many cases, establish our
identities based on faces. Hence, it has become a standard practice to
incorporate face photographs in various tokens of authentication such as ID
cards, passports, and driver’s licenses.
W. Zhao and R. Chellappa[10] Sensitivity to variations in illumination is a
fundamental and challenging problem in face recognition. In this paper, we
describe a new method based on symmetric shape-from-shading (SFS) to
develop a face recognition system that is robust to changes in illumination.
The basic idea of this approach is to use the symmetric SFS algorithm as a
tool to obtain a prototype image which is illumination-normalized. Applying
traditional SFS algorithms to real images of complex objects (in terms of their
shape and albedo variations) such as faces is very challenging. It is shown
that the symmetric SFS algorithm has a unique point-wise solution. In
practice, given a single real face image with complex shape and varying
albedo, even the symmetric SFS algorithm cannot guarantee the recovery of
accurate and complete shape information.
For the particular problem of face recognition, we utilize the fact that all faces
share a similar shape making the direct computation of the prototype image
from a given face image feasible. The symmetry property has been used to
develop a new model-based source-from-shading algorithm which is more
accurate than existing algorithms. Using the symmetric property, we are also

28
able to gain insight into how changes in illumination eigen-subspace based
face recognition systems.
T. Chen, Y. Wotao, S. Z. Xiang, D. Comaniciu[11] In this paper, we present
the logarithmic total variation (LTV) model for face recognition under varying
illumination, including natural lighting conditions, where we rarely know the
strength, direction, or number of light sources. The proposed LTV model has
the ability to factorize a single face image and obtain the illumination invariant
facial structure, which is then used for face recognition. Our model is inspired
by the SQI model but has better edge-preserving ability and simpler parameter
selection. The merit of this model is that neither does it require any lighting
assumption nor does it need any training. The LTV model reaches very high
recognition rates in the tests using both Yale and CMU PIE face databases as
well as a face database containing 765 subjects under outdoor lighting
conditions
M. Grudin[12] This paper introduces an efficient method for the recognition of
human faces in 2D Digital images using a feature extraction technique that
combines the global and local information in frontal view of facial images. The
proposed feature extraction includes human face localization derived from the
shape information. Efficient parameters are defined to eliminate irrelevant
data while Pseudo Zernike Moments (PZM) with a new moment orders
selection method is introduced as face features. The proposed method while
yields better recognition rate, also reduces the classifier complexity.
This paper also examines application of various feature domains as face
features using the face localization method. These include Principle
Component Analysis (PCA) and Discrete Cosine Transform (DCT). The Radial
Basis Function (RBF) neural network has been used as the classifier and we
have shown that the proposed feature extraction method requires an RBF
neural network classifier with a simpler structure and faster training phase
that is less sensitive to select training and testing images. Simulation results
on the Olivetti Research Laboratory (ORL) database and comparison with other
techniques indicate the effectiveness of the proposed method.

29
P. A. Viola and M. J. Jones[13] This paper describes a face detection
framework that is capable of processing images extremely rapidly while
achieving high detection rates. There are three key contributions. The first is
the introduction of a new image representation called the “Integral Image”
which allows the features used by our detector to be computed very quickly.
The second is a simple and efficient classifier which is built using the
AdaBoost learning algorithm (Freund and Schapire, 1995) to select a small
number of critical visual features from a very large set of potential features.
The third contribution is a method for combining classifiers in a “cascade”
which allows background regions of the image to be quickly discarded while
spending more computation on promising face-like regions. A set of
experiments in the domain of face detection is presented. The system yields
face detection performance comparable to the best previous systems (Sung
and Poggio, 1998; Rowley et al., 1998; Schneiderman and Kanade, 2000; Roth
et al., 2000). Implemented on a conventional desktop, face detection proceeds
at 15 frames per second.
T. Ojala, M. Pietik¨ainen and D. Harwood[14] This chapter presents a
Feature Local Binary Patterns (FLBP) method that encodes both local and
feature information, where the feature pixels may be broadly defined by, for
example, the edge pixels, the intensity peaks or valleys in an image, or new
feature information derived from the local binary patterns or LBP. FLBP thus
is expected to perform better than LBP for texture description and pattern
recognition. For a given pixel and its nearest feature pixel, a distance vector
is first formed by pointing from the given pixel to the feature pixel.
A True Center (TC), which is the center pixel of a neighborhood, is then located
on the distance vector by a TC parameter. A Virtual Center (VC), which
replaces the center pixel of the neighborhood, is specified on the distance
vector by a VC parameter. FLBP is then defined by comparing the neighbors
of the true center with the virtual center. Note that when both the TC and VC
parameters are zero, FLBP degenerates to LBP, which indicates that LBP is a
special case of FLBP. Other special cases of FLBP include FLBP1 when the VC
parameter is zero and FLBP2 when the TC parameter is zero.

30
 CHAPTER 3: EXISTING SYSTEM
The goal of face detection is to detect and locate faces in the image, to
extract human face to use in other areas. Nowadays, there are many different
algorithms to accomplish face detection or recognition, such as Fisher faces,
Eigenfaces, Scale-invariant Feature Transform and Speed-Up Robust
Features (SURF). In this section, LBPF-based face detection algorithm is
introduced. LBPF algorithm is the combination of Local Binary Patterns (LBP).
LBP is an easy but powerful way to extract and label the pixels of an image.
Using the LBPF, we can easily represent face images with just a
straightforward vector.
Among other biometric methods, face recognition is only one of the ways
to identify any individual subject. Face recognition identifies any one by
comparing the physical characteristics of the item. There are two face
recognition modes, still images and live video. The first step in face recognition
is face detection. Therefore, to perform facial recognition, the system must
position the face earlier in the input image or video stream. This step is called
face acquisition or detection in this section, the main face recognition methods
are described. Such as neural networks, geometric feature matching, feature-
based methods and graph matching. The following techniques were studied
in the context of facial illustrations. The geometric feature matching method
depends on the calculation of a set of geometric features of the facial image.
The general specification is defined by a vector showing the position and size
of the main facial feature, for example, the eyebrows, nose, mouth and facial
contours.

31
 CHAPTER 4: PROPOSED SYSTEM
Face recognition is not a simple problem since an unknown face image
seen in the extraction phase is usually different from the face image seen in
the classification phase. Although local binary features has been extracted
from the face image for face recognition that there are several face image uses
in the database that compared with the input face image. The face image
depends on viewing lighting and environmental conditions. In addition the
face image changes according to the expressions. In the research work, which
is flexible and efficient, should be solved the problems.
To implement the face recognition in this research work, we proposed the
Local Binary patterns methodology. Local Binary Pattern works on local
features that uses LBP operator which summarizes the local special structure
of a face image [11]. LBP is defined as an orders set of binary comparisons of
pixels intensities between the center pixels and its eight surrounding pixels.
The Face Recognition Algorithm Input: Training Image set. Output: Feature
extracted from face image and compared with centre pixel and recognition
with unknown face image.
1. Initialize temp = 0
2. FOR each image I in the training image set
3. Initialize the pattern histogram, H = 0
4. FOR each center pixel tc 𝜖 I
5. Compute the pattern label of tc, LBP (1)
6. Increase the corresponding bin by 1.
7. END FOR
8. Find the highest LBP feature for each face image and combined into single
vector.
9. Compare with test face image.
10. If it match it most similar face in database then successfully recognized.
Algorithm We start by summarizing the main common steps of the
algorithms used in this work. Then we describe each step in detail. The
proposed face recognition process consists of four main parts:

32
1.Preprocessing:
We begin by applying the Tan and Triggs’ illumination normalization
algorithm to compensate for illumination variation in the face image. No
further preprocessing, such as for example face alignment, is performed in
the preprocessing.
2.LBP operator application:
In the 2nd stage LBP are computed for each pixel, making a fine scale
textural description of the image.
3.Local feature extraction process:
Local features are produced by computing histograms of LBP over local
image regions.
4.Classification:
Each face image in test set is classified by comparing it against the face
images in the training set. The comparison is performed using the local
features obtained in the previous step in the algorithm.
The first two steps are shared by all the algorithms. The algorithms we
explore in this work vary in how they perform the last two steps. Face
recognition is not a simple problem since an unknown face image seen in the
extraction phase is usually different from the face image seen in the been
extracted from the face image for face recognition that there are several face
image uses in the database that compared with the input face image. The face
image depends on viewing lighting and environmental conditions. In addition
the face image changes according to the expressions. In the research work,
which is flexible and efficient, should be solved the desired problems. [12-15]
To implement the face recognition in this research work, we proposed the
Local Binary patterns methodology. Local Binary Pattern works on local
features that uses LBP operator which summarizes the local special structure
of a face image. LBP is defined as an orders set of binary comparisons of pixels
intensities between the center pixels and its eight surrounding pixels in the
image

33
 Capture a
face image

The face image is divided

into several blocks

Histogram calculated Yes

for each block

Block LBP histograms

are concentrating into

Face
Imag
e

No
Facial recognition are
represented by LBP

Fig 4.1: Flowchart of the LBP Process Flowchart of the Proposed

System

34
 Face image

LBP

Feature
Extraction

Training Set

LBP
Input
classificatio
image
n

Face Identify the

Recognition Face image

Fig. 4.2 .Flow diagram for whole system

35
 CHAPTER 5: SYSTEM REQUIREMENTS

5.1 SOFTWARE REQUIRED:

 Operating system : Windows XP/7.
 Coding Language : Python
 Tool : Keras/NumPy /Pandas/TensorFlow

5.2 HARDWARE REQUIRED:

 System : Pentium IV 2.4 GHz
 Hard Disk : 40 GHz
 Floppy Drive : 1.44 Mb
 Monitor : 15 VGA Colour
 Ram : 512 MbRam

36
 CHAPTER 6: RESULT AND DISCUSSIONS
This implementation is used to test the performance of the LBP-method on
different kind of face images. Several parameters, like the LBP operator (P and
R), non-weighted or weighted regions and the dividing of the regions, are
varied to see the influence of these parameters on the performance. For this
experiment we have collected lots of face images, some of them are collected
from photographs taken with a Canon Power shot A610 camera and some are
taken from A4Tech webcams. And also collected face images from the face
database.In the proposed algorithm, different types of face images have been
recognized. Based on algorithm, the face image of an unknown identity is
compared with face images of known individuals from a large database. In the
figure 6.1 we can see the input facial images used for input for face recognition
are given below:

Fig 6.1: Different Input Facial Images

And also in the figure 6.2 we can see the facial images that are stored
in the database which compared with the input facial images. If the input face
images are found or the more similarities face images are matched in the
database then we say the face image is successfully recognized.

Fig 6.2: Facial Image from the Database

37
In the experiment we can train the face images in the database. That the facial
images are successfully trained shown in the window mode in the below:

Based on the algorithm the input face images are compared with database
facial images for identification. The face recognition results are shown in
below in window mode:

The following table shows overall face recognition rate:

Table 6.1:Recognition rate of the Research

In the table 6.1 the recognition rate is above 100%. We recognize the
face images from the database face images by comparing between input face

38
image and database image. From the experimental result, it is seen that the
research satisfies all the requirements to recognize the face images.
Proposed algorithm was demonstrated successfully using MATLAB.
Experimental Results obtained using the local image face database shows that
the percentage of the LBP face recognition varies from 65.281%to 100 % with
overall average of about76.96%. The face recognition rate is about99%. It is
found that the face recognition efficiency of the traditional face recognition
approaches, CA and LDA, decreases while facing the face variations on pose,
illumination and expression. Compared with PCA and LDA, the LBPalgorithm
is more robust to face conditions, and gets the better recognition performance.
In addition, all the algorithms perform better on expression variation than on
illumination condition, which means illumination variation is more
challenging for face recognition.
Facial Expression Detection using Webcam
This project we have designed two separate application such as Person
Registration and Facial Expression Detection.
1) Person Registration: using this application we can add person details
such as person ID, person Name and his face
2) Facial Expression Detection: using this application we can predict facial
expression and then we can predict the person in the image with
expressions.
To run project first double, click on ‘run_register_person.bat’file to get be on
the screen

Fig 6.3: Facial Expression from webcam

39
In above screen enter person ID and person name and then click on ‘Register
Person’ button to capture person face and to get below screen.

Fig 6.4: Face Capturing

In above screen person face captured and now click on ‘Train Model’ button
to add this person to training model so while dectecting expression person
name can be identified.In the above screen person registered as Kumar.

Fig 6.5: Face Expression Detection

40
Fig 6.6: Facial Recognition Python and Open CV from Webcam

41
 CHAPTER 7: CONCLUSION & FUTURE SCOPE
7.1 CONCLUSION:
The Proposed system has been done to the performance of a face
recognition system by making use of feature extraction with Local Binary
Patterns [12]. It mainly consists of three parts, namely face representation,
feature extraction and classification. Face representation represents how to
model a face and determines the successive algorithms of detection and
recognition. The most useful and unique features of the face image are
extracted in the feature extraction phase. In the classification the face image
is compared with the images from the database. This method represents the
local feature of the face and matches it with the most similar face image in
database. The accuracy of the system is above 100% by the Local Binary
Patterns algorithm.
7.2 FUTURE SCOPE:
It is obvious that the result of this face recognition system is good but
there is scope for future improvement. Due to time constraints we were not
able to implement some objectives that should have made the research work
a better proposition. The main improvement will pursue the performances,
recognizes the real-time face recognition [13]. I would like to improve my code
for face image recognition as well as clean up the code in order to improve
performance. Many difficulties has been faced when recognized face images
from database such as pose and lighting variations, expression variations, age
variations, and facial occlusions. In future to improve the pose correction,
quality based frame selection, aging correction, and mark based matching
techniques can be combined to build a unified system for video based face
recognition.

42
 CHAPTER 8: APPENDIX
8.1 Technologies Used
8.1.1 Python
Python is one of just a few remarkable programming languages that can
claim to be both simple and contemporary. You'll be astonished to learn how
common it is to concentrate on the answer to the problem in terms of linguistic
structure (for example, the architecture of the framework you're writing). The
Python programming language is a beautiful and simple to learn computer
language. It has a useful extended records structure as well as an easy-to-
understand but effective approach to thing-focused programming
organisation. Python is a fantastic language for content and short application
development at all levels in a variety of fields, thanks to its excellent phrase
structure and dynamic creation, as well as its readable nature.
Python is a high-level programming language that may be used to
programme in a variety of ways. Python has a format rationale that
concentrates on code intelligibility and employs essential whitespace. It was
initially published in 1991 and turned into planned by Guido van Rossum. It
creates systems with clear collaboration programming on both small and large
scales. In July 2018, Van Rossum quit as the language organization's founder
after 30 years. Python's heads have a unique type of gadget and explicit
memory. It includes a large and growingly well-known library that supports a
wide range of programming models, including object-oriented, essential,
utilitarian, and procedural. Python mediators are provided for various
operating designs. The standard Python execution, CPython, is open source
programming that follows an association-based development approach, as do
all of Python's other projects. Python and CPython are governed by the Python
Software Foundation, a non-profit organization.
8.2 How does Machine Learning work?
Machine data collection is a sort of reality analysis that automates the
improvement of evaluation procedures. This is a branch of engineering insight
that is based on the potential that robots can perceive from given measures,
become aware of instances, and make decisions with little or no human

43
involvement. Because of advancements in virtual innovation, framework
learning today is nothing like device learning in the past.
The frequency of patter, as well as the prospect that computers can catch
without being adjusted to do specified tasks, prompted manufactured insight
agents to investigate whether machines might obtain knowledge from
datasets. Because frameworks may operate freely while being subjected to
clean measures, the issue of gadget dominance is no longer a concern.
They create dependable, repetitive judgments and impacts using previous
estimates as a reference. It's not a perfect innovation right now, but it's filling
in the gaps. Modules used in the project
8.3 LIBRARY FILES
8.3.1 Pandas
Pandas is now the most used data cleaning and inspection machine. It
stresses information exploration, purification, change, and visualization.
Pandas is an open-source Python library that uses its acceptable measuring
designs to provide endowments in contemporary execution real-factor
overseeing and evaluation. Python is frequently used for document processing
and planning. When it came to records evaluation, it had almost no duties.
Pandas were able to deal with the situation. With Pandas, we may also
complete 5 standard phases in the control and evaluation of bits of ability,
paying special attention to the start of actual load, set up, control, form, and
autonomous.
Python with Pandas has been used in a variety of areas, including
bookkeeping, financial concerns, data analysis, and research, among many
others.
8.3.2 Matplotlib
Matplotlib is a graphical charting, data representation, and pass-stage
tool for Python and its mathematical extension NumPy. In comparison to
MATLAB, it provides a useful open-source alternative in this regard. Designers
may also use matplotlib APIs (Application Programming Interfaces) to include
charts in graphical user interfaces. The majority of Matplotlib utilities are
included within the pyplot sub module, and are commonly imported under
the plt assumed name:

44
 8.3.3 Seaborn
Seaborn is a Python module for creating quantifiable images.It expands
on the zenith of matplotlib and incorporates cautiously pandas measurements
structures. Seaborn assists you with finding and comprehending your
information. Seaborn is a data representation library built on top of matplotlib
and firmly incorporated with pandas information structures in Python.
Perception is critical a piece of Seaborn that works within investigation and
aptitude of insights.
8.3.4 sklearn
In Python, Scikit-examine is the most useful machine for examining the
bundle. The sklearn package provides for gadget learning and factual showing
incorporates capabilities such as class, relapse, grouping, and dimensionality
reduction, to name a few. Scikit-learn provides a Python interface that enables
various oversaw and unaided data collection on computations. It's covered by
a liberal smoothed-out BSD licence and distributed under a variety of Linux
distributions, making it suitable for all academic and contemporary uses.
Python is a cutting-edge programming language with a proven translator for
remarkable reason programming.
Python has a design approach that emphasizes on script clarity,
particularly the usage of monster whitespace, which was originally
disseminated in 1991 and converted into the plan by Guido van Rossum.
Python has a fantastic organizing system, as well as a particularly designed
memory board.
It has a large and comprehensive vital library and supports a variety of
programming styles, including article-oriented, dreadful, utilitarian, and
procedural. Python is a language that has been decoded. At runtime, the
translator interacts with Python. You don't have to join your application before
you begin it. This is exactly the same as PERL and PHP. Python is a very
sophisticated programming language. You may even record your artworks by
connecting with the mediator while lying down at a Python order. Python, too,
recognizes that improvements must be implemented quickly.
A part of this is clean and fast code, as well as going right off the stage to
enormous forms that avoid repetitive code. The word "feasibility" is also linked

45
to the word "possibility." This may be a completely pointless examination, but
it demonstrates how an awesome arrangement code you should check,
examine, or even suspect might help you locate issues or adjust ways. This
vertical style, the ease with which a product programme designer of any
vernacular may learn core Python skills, and the massive infinite library are
all essential to any other area where Python reigns supreme. Every one of its
frameworks became simple to build up, save several times, and then be
altered and renewed by individuals who didn't have a Python genuine history
all without breaking.
8.3.5 Keras
Keras is a Python-based deep learning API that runs on top of
TensorFlow, a machine learning framework. It was designed with the goal of
allowing for fast experimentation. It is essential to be able to move from
concept to outcome as quickly as possible in order to do the best assessments.
Keras is a simple character that isn't currently blind. Keras is easy to research
and utilize as a result of this. As a Keras user, you're more useful, allowing
you to pursue more ideas and do so faster than your opponents, which helps
you win device dominating competitions. This adaptability does not come at
the sacrifice of usability: because Keras integrates deeply with low-degree
TensorFlow capacity, it enables you to create particularly hackable work
methods in which every piece of ability may be remade.
SOURCE CODE:
import tkinter
import PIL.Image, PIL.ImageTk
from tkinter import simpledialog
import time
from tkinter import messagebox
import os
from tkinter import *
import random
from datetime import date
from PIL import Image
import numpy as np

46
import cv2
class App:
global classifier
global labels
global X_train
global Y_train
global text
global img_canvas
global cascPath
global faceCascade
global tf1, tf2
global capture_img
global student_details
global recognizer
def __init__(self, window, window_title, video_source=0):
global cart
global text
cart = []
self.window = window
self.window.title("Face Recognition based on Local Binary Pattern
Features")
self.window.geometry("1300x1200")
self.video_source = video_source
self.vid = MyVideoCapture(self.video_source)
self.canvas = tkinter.Canvas(window, width = self.vid.width, height =
self.vid.height)
self.canvas.pack()
self.font1 = ('times', 13, 'bold')

self.l2 = Label(window, text='PERSON NAME')

self.l2.config(font=self.font1)
self.l2.place(x=110,y=550)
self.tf2 = Entry

47
(window,width=60)
self.tf2.config(font=self.font1)
self.tf2.place(x=250,y=550)
self.btn_snapshot=tkinter.Button(window, text="Register Person",
command=self.capturePerson)
self.btn_snapshot.place(x=50,y=600)
self.btn_snapshot.config(font=self.font1)
self.btn_train=tkinter.Button(window, text="Train Model",
command=self.trainmodel)
self.btn_train.place(x=250,y=600)
self.btn_train.config(font=self.font1)
self.btn_train=tkinter.Button(window, text="Recognize",
command=self.recognize)
self.btn_train.place(x=450,y=600)
self.btn_train.config(font=self.font1)

self.text=Text(window,height=35,width=65)
scroll=Scrollbar(self.text)
self.text.configure(yscrollcommand=scroll.set)
self.text.place(x=1000,y=90)
self.text.config(font=self.font1)
self.cascPath = "model/haarcascade_frontalface_default.xml"
self.faceCascade = cv2.CascadeClassifier(self.cascPath)
self.recognizer = cv2.face_LBPHFaceRecognizer.create()
self.student_details = []
self.delay = 15
self.update()
self.window.config(bg='turquoise')
self.window.mainloop()
def getImagesAndLabels(self):
names = []
ids = []
faces = []

48
 imagePaths = [os.path.join("person_details",f) for f in
os.listdir("person_details")]
for imagePath in imagePaths:
pilImage = Image.open(imagePath).convert('L')
imageNp = np.array(pilImage,'uint8')
fname = os.path.basename(imagePath)
arr = fname.split("_")
Id = int(arr[1])
names.append(arr[0])
faces.append(imageNp)
ids.append(Id)
return faces,ids,names
def getName(self, Id,names,ids):
name = "Unable to predict name"
for i in range(len(ids)):
if ids[i] == Id:
name = names[i]
break
return name
def recognize(self):
faces,ids,names = self.getImagesAndLabels()
self.recognizer.read("model/Trainner.yml")
ret, frame = self.vid.get_frame()
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
faces = self.faceCascade.detectMultiScale(gray, 1.3,5)
option = 0
for (x, y, w, h) in faces:
face_component = gray[y:y+h, x:x+w]
option = 1
if option == 1:
Id, conf = self.recognizer.predict(face_component)
if(conf < 50):
name = self.getName(Id,names,ids)

49
 messagebox.showinfo("Person is video recognized as : "+name,
"Person is video recognized as : "+name)
else:
messagebox.showinfo("Unable to recognized person in video",
"Unable to recognized person in video")
else:
messagebox.showinfo("Face not detecting in video", "Face not
detecting in video")
def capturePerson(self):
global capture_img
option = 0
ret, frame = self.vid.get_frame()
img = frame
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
faces = self.faceCascade.detectMultiScale(gray,1.3,5)
print("Found {0} faces!".format(len(faces)))
for (x, y, w, h) in faces:
cv2.rectangle(frame, (x, y), (x+w, y+h), (0, 255, 0), 2)
img = frame[y:y + h, x:x + w]
img = cv2.resize(img,(200,200))
option = 1
if option == 1:
self.capture_img = img
cv2.imshow("Capture Face",img)
cv2.waitKey(0)
else:
messagebox.showinfo("Face or person not detected. Please try
again","Face or person not detected. Please try again")
def getImagesAndLabels(self):
names = []
ids = []
faces = []

50
 imagePaths = [os.path.join("person_details",f) for f in
os.listdir("person_details")]
for imagePath in imagePaths:
pilImage = Image.open(imagePath).convert('L')
imageNp = np.array(pilImage,'uint8')
fname = os.path.basename(imagePath)
arr = fname.split("_")
Id = int(arr[1])
print(str(imagePath)+" "+str(Id))
faces.append(imageNp)
ids.append(Id)
names.append(arr[0])
return faces,ids,names
def trainmodel(self):
if self.capture_img is not None:
pid = self.tf1.get()
name = self.tf2.get()

cv2.imwrite("person_details/"+name+"_"+pid+"_0.png",self.capture_img)

cv2.imwrite("person_details/"+name+"_"+pid+"_1.png",self.capture_img)

cv2.imwrite("person_details/"+name+"_"+pid+"_2.png",self.capture_img)

cv2.imwrite("person_details/"+name+"_"+pid+"_3.png",self.capture_img)
faces,Id,names = self.getImagesAndLabels()
self.recognizer.train(faces, np.array(Id))
self.recognizer.save("model/Trainner.yml")
messagebox.showinfo("Training task completed", "Training task
completed")
else:
messagebox.showinfo("Face or person not detected","Face or person
not detected")

51
 def update(self):
ret, frame = self.vid.get_frame()
if ret:
self.photo = PIL.ImageTk.PhotoImage(image =
PIL.Image.fromarray(frame))
self.canvas.create_image(0, 0, image = self.photo, anchor =
tkinter.NW)
self.window.after(self.delay, self.update
class MyVideoCapture:
def __init__(self, video_source=0):
self.vid = cv2.VideoCapture(video_source)
if not self.vid.isOpened():
raise ValueError("Unable to open video source", video_source)
self.width = self.vid.get(cv2.CAP_PROP_FRAME_WIDTH)
self.height = self.vid.get(cv2.CAP_PROP_FRAME_HEIGHT)
self.pid = 0
def get_frame(self):
if self.vid.isOpened():
ret, frame = self.vid.read()
if ret:
return (ret, cv2.cvtColor(frame, cv2.COLOR_BGR2RGB))
else:
return (ret, None)
else:
return (ret, None)
def __del__(self):
if self.vid.isOpened():
self.vid.release()
App(tkinter.Tk(), "Tkinter and OpenCV")

52
 CHAPTER 9: BIBLIOGRAPHY
1. Unsang Park, “Face Recognition: face in video, age invariance, and facial
marks” Michigan State University, 2009.
2. T. Ahonen, A. Hadid and M. Pietikainen, “Face description with Local
Binary Patterns”, Application to Face Recognition. Machine Vision Group,
University of Oulu, Finland, 2006.
3. T. Ahonen, A. Hadid, M. Pietikainen and T. M aenpaa. “Face recognition
based on the appearance of local regions”, In Proceedings of the 17th
International Conference on Pattern Recognition, 2004.
4. R. Gottumukkal and V.K. Asari, “An Improved Face Recognition Technique
Based on Modular PCA Approach” Pattern Recognition Letters, vol. 25, pp.
429- 436, Mar. 2004.
5. T. Ojala, M. Pietikainen and D. Harwood, “A comparative study of texture
measures with classification based on feature distributions” Pattern
Recognition vol. 29, 1996.
6. M. Turk and A. Pentland, “Eigenfaces for recognition”, Cognitive
Neuroscience, 3:72{86, 1991.
7. M. Kirby and L. Sirovich, “Application of the Karhunen-Loeve procedure
for the characterization of human faces” IEEE Transactions on Pattern
Analysis and Machine Intelligence, 12(1):103{108, 1990.
8. W. Zhao, R. Chellappa, P. J. Phillips, and A. Rosenfeld,“ Face recognition:
A literature survey” ACM Computing Surveys (CSUR), 35(4):399{458, 2003.
9. S. Z. Li and A. K. Jain (eds.), “Handbook of Face Recognition” Springer-
Verlag, Secaucus, NJ, 2005.
10. W. Zhao and R. Chellappa “Robust face recognition using symmetric of
shape from-shading” Technical Report, Center for Automation Resaerch
University of Maryland, 1999.
11 . T. Chen, Y. Wotao, S. Z. Xiang, D. Comaniciu, and T. S. Huang, “Total
variation models for variable lighting face recognition”. of the IEEE
Transactions on the Pattern Analysis and Machine Intelligence, 28(9):
1519{1524, 2006.
12 . M.Grudin, “On internal representation in the face recognition
systems”. Pattern Recognition, 33(7):1161 1177, 2000.

53
13. P. A. Viola and M. J. Jones, “Robust real-time face detection”.
International Journal of Computer Vision, 57(2):137{154, 2004.
14. http://fei.edu.br/~cet/facedatabase.html
15. T. Ojala, M. Pietik¨ainen and D. Harwood, “A comparative study
of texture measures with classification based on feature of the
distributions” Pattern Recognition vol. 29, 1996.

54