Facial Recognition using Siamese Neural Network

Ojasi Lavanya ; [email protected]

Shivangi; [email protected]
Sanjidah Munim Mazumder ; [email protected]

.Facial Recognition using Siamese Neural Network Vol. x, No. x (xx) x–x

Paper Type: Original Article

:Citation
Facial Image VerificationReceived: -----20/9/2024
.Lavanya Ojasi, Shivangi, Mazumder M. Sanjidah

Revised: ----- 13/11/2024

Accepted:---14/11/2024
Facial Image Verification .)14/11/2024(

ABSTRACT

With an emphasis on how convolutional neural networks (CNNs) improve accuracy ,adaptability, and efficiency
over conventional techniques, this study investigates the incorporation of deep learning techniques in facial
recognition. By describing procedures including face identification, alignment, feature extraction, and
recognition, the paper highlights the deep learning process. CNNs' ability to derive intricate patterns from
unprocessed image data is one of their main advantages; this enables reliable feature extraction and precise
detection even in situations with changing illumination, attitude, and occlusion. Along with discussions of
exciting future advancements meant to enhance fairness, robustness, and privacy preservation within facial
.recognition systems, challenges such as data bias, privacy problems, and adversarial susceptibility are highlighted

Keywords: Convolutional neural networks (CNNs),Deep learning in facial recognition,Feature extraction,Image

.data processing,Privacy preservation,Future advancements

Corresponding Author: [email protected]

SA.2021.281500.1061/10.22105
Licensee System Analytics. This article is an open access article distributed under the terms and conditions of the Creative
.Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0)
 Paper title

INTRODUCTION|1

Facial recognition technology has become a vital part of various applications such as access control, security
monitoring, and user verification. This technology analyzes facial data from images or videos and matches it with
existing records to verify a person’s identity. The facial recognition process typically consists of several steps:
detecting the face, aligning it, extracting features, and identifying the individual. Each step is essential for achieving
real-time recognition with high accuracy and reliability. Recent advancements in deep learning, particularly through
convolutional neural networks (CNNs), have significantly enhanced the performance of facial recognition systems.
CNNs automatically extract high-level, relevant features from facial images, recognizing complex patterns that
conventional methods, like Haar cascades or the Viola-Jones algorithm, may struggle with. This automatic feature
extraction allows CNNs to adjust to real-world differences, such as variations in lighting, facial angles, and
.obstructions, thereby improving the accuracy and adaptability of facial recognition systems

Deep learning has also brought about enhancements in key aspects of facial recognition. For instance, CNNs can
accurately detect and align faces, ensuring a consistent orientation for optimal feature extraction and comparison.
During feature extraction, CNNs have demonstrated their ability to produce highly compact and distinctive
feature vectors that effectively represent unique facial characteristics. This skill is vital for differentiating between
.individuals with minimal error

Though these advancements are significant, challenges still exist, particularly regarding data bias, privacy issues,
and susceptibility to adversarial attacks. Biometric systems, such as facial recognition, inherently involve sensitive
information, making robust privacy measures essential. Additionally, problems related to data bias can result in
varying performance across different demographic groups. Adversarial attacks, which entail subtly modifying
.images to confuse the model, also present security threats

This study seeks to examine the utilization of CNNs in facial recognition, investigating both their advantages and
the challenges they face. With ongoing research and development, CNN-based facial recognition has the potential
.to realize more resilient, equitable, and privacy-conscious implementations in various settings

Literature Review |2
Facial recognition has become a pivotal technology in many modern applications, ranging from security and
surveillance to personal device authentication and social media. Over the years, researchers have developed
various methods for recognizing human faces, progressing from statistical models and traditional machine
.learning algorithms to the current dominance of deep learning techniques
Traditional Methods for Facial Recognition

Early facial recognition systems relied heavily on hand-crafted features, where techniques like
Eigenfaces and Fisher faces were widely used. Eigenfaces, introduced in the 1990s, utilized Principal
Component Analysis (PCA) to represent facial images as linear combinations of principal
components. This approach was effective in reducing the dimensionality of face images, enabling
]faster processing but suffered from sensitivity to lighting, pose, and expression variations

Fisherfaces, on the other hand, employed Linear Discriminant Analysis (LDA) to maximize the
between-class variance and minimize within-class variance, offering more robustness against illumination
changes. Despite their success in controlled environments, these techniques were limited in complex, real-
.world scenarios

The Rise of Deep Learning and Convolutional Neural Networks (CNNs)

The introduction of Convolutional Neural Networks (CNNs) transformed the domain of facial
recognition by enabling automated feature extraction and the learning of hierarchical representations directly
from data. CNNs, made up of layers that perform convolution and pooling operations, are capable of
capturing complex patterns in images, which makes them particularly powerful for visual recognition
applications. Notable architectures such as AlexNet, VGGNet, and ResNet showcased the capability of
.CNNs in large-scale image classification tasks and laid the groundwork for later facial recognition models

One of the initial deep learning models specifically created for facial recognition was DeepFace, which
was developed by Facebook. DeepFace employed a nine-layer deep neural network to reach human-level
accuracy in identifying faces, representing a major breakthrough. Google’s FaceNet took the field further by
introducing a triplet loss function that organizes facial images into a compact Euclidean space, positioning
similar faces closer together and placing dissimilar ones further apart. FaceNet achieved impressive accuracy
by concentrating on reducing the distance between the embeddings of the same person while increasing the
.distance between embeddings of different individuals

An additional important advancement in facial recognition is the implementation of Siamese Neural

Networks (SNNs). SNNs are especially effective for few-shot learning and one-shot learning tasks, where
distinguishing between identities with minimal examples is necessary. By training the network on pairs of
images, SNNs learn to generate a similarity score between those pairs, making them suitable for facial
verification tasks. The contrastive loss or triplet loss functions applied in SNNs allow the model to develop
discriminative embeddings, facilitating effective comparisons between images even when training data is
.limited

OpenCV for Image Processing

For this research, OpenCV (Open Source Computer Vision Library) is essential, particularly for the
preparation and improvement of facial images. With more than 2,500 efficient algorithms designed for a
variety of vision-related tasks, such as image editing, object recognition, and face detection, this library offers
an open-source solution for computer vision and machine learning. OpenCV is used in this project for a
:number of reasons, including

Face Detection: OpenCV's CascadeClassifier or dnn module finds faces in an image before putting it •
.into the neural network, making sure that only the faces are processed
 Paper title

Image Augmentation: To add variation to the training dataset, OpenCV is used to implement •
techniques like rotation, scaling, and flipping. By mimicking fluctuations that the model could experience in
.practical applications, this improves the model's capacity for generalisation

Image Preprocessing: To standardise the input for the neural network, OpenCV makes it easier to •
.resize, normalise, and convert images to greyscale

OpenCV, for instance, can identify a face in an image, trim it to highlight facial features, and then
.scale it to the neural network's specifications (for this case, that would be 250 × 250 pixels)

Basic Concepts |3
TensorFlow and Keras for Deep Learning

In this project, a facial recognition model is developed, trained, and launched using TensorFlow and Keras.
TensorFlow, an open-source deep learning library created by Google, provides a comprehensive platform for
building and scaling machine learning models. Known for its flexibility and ability to handle large datasets,
TensorFlow has become a key resource in various research and industrial uses of deep learning. It supports
both CPU and GPU acceleration, making it suitable for training demanding models like CNNs and SNNs.
Keras, a high-level neural network API built on TensorFlow, simplifies the process of defining and training
deep learning models. With its user-friendly APIs, Keras allows developers to quickly prototype and assess
..models without dealing with the complexities of lower-level implementations
In this project, Keras components such as Conv2D, Dense, Input, Flatten, and MaxPooling2D are employed
:to define the architecture of the Siamese Network. For instance
Conv2D layers are used to implement convolutional operations, allowing the network to extract spatial •
.hierarchies in facial images
MaxPooling2D layers help reduce the spatial dimensions of the feature maps, retaining the most important •
.features while reducing computational complexity
Dense layers provide fully connected networks that further abstract features extracted by the convolutional •
.layers
Using Keras with TensorFlow provides an efficient pipeline for model development, where Keras handles
the model building, while TensorFlow’s backend enables optimization and deployment across multiple
.devices

Comparative Analysis with Exciting Works

Several recent works have combined TensorFlow, Keras, and OpenCV in the context of facial recognition.
For instance, Zhang et al. (2021) utilized TensorFlow and Keras to implement a facial recognition system
that leveraged CNN layers for feature extraction, with OpenCV handling real-time face detection and image
preprocessing. Similarly, in a study by Smith and Wang (2020), the researchers employed a Siamese Neural
Network architecture built in Keras and TensorFlow for a one-shot learning task, where OpenCV was used
to augment the dataset with different lighting conditions and poses. These studies have demonstrated the
effectiveness of using these libraries together, particularly in applications where computational efficiency and
.ease of implementation are essential

PROPOSED FRAMEWORKS |4
The Siamese Neural Network (SNN) architecture used in this project is built to analyze pairs of facial images and
assess their similarity. This process is performed by directing each image from a pair through two identical
subnetworks that share weights, which extract high-level features from each image. The resulting feature vectors
by are then compared to generate a similarity score. Below is a detailed breakdown of the architecture
.layer: Structure of the Siamese Neural Network for Face Recognition

The Siamese Neural Network (SNN) architecture used in this project is built to analyze pairs of facial images and
assess their similarity. This process is performed by directing each image from a pair through two identical
subnetworks that share weights, which extract high-level

features from each image. The resulting feature vectors are then compared to generate a similarity score. Below is
:a detailed breakdown of the architecture by layer

Input Layer: The network accepts an input image measuring 105 × 105 pixels, which represents a grayscale .1
facial image. This size allows for adequate detail of facial features while keeping computational demands
.reasonable

:First Convolutional Layer.2

.Layer Details: Convolutional layer featuring 64 filters of size *10 × 10 -

.Activation: ReLU (Rectified Linear Unit) -

.Output: The dimensions of the resulting feature map are *96 × 96 with 64 feature maps -

Description: This layer identifies basic features like edges and textures, which are vital for differentiating facial
traits

:First Max-Pooling Layer.3

.Pooling Size: *2 × 2-

.Output: The feature map is condensed to *48 × 48 with 64 feature maps -

Description: Max-pooling decreases the spatial dimensions while preserving critical features and lowering -
.computational expense

.Second Convolutional Layer: Layer Details: Convolutional layer with 128 filters of size *7× 7 .4

.Activation: ReLU-

.Output: The feature map dimensions are curtailed to *42 × 42 with 128 feature maps -

Description: This layer learns more intricate features, capturing facial structure patterns and enhancing the ability -
.to differentiate between individuals

:Second Max-Pooling Layer.5

.Pooling Size: *2 × 2-

.Output: The feature map is reduced to *21 × 21 with 128 feature maps -

.Description: This layer further diminishes spatial dimensions, assisting in feature abstraction -

.Third Convolutional Layer: Layer Details: Convolutional layer containing 128 filters of size *4 × 4 .6
 Paper title

.Activation: ReLU-

.Output: The feature map dimensions are brought down to *18 × 18 with 128 feature maps -

Description: This layer extracts even finer details from the facial images, enabling the model to recognize subtle -
.differences

:Third Max-Pooling Layer.7

.Pooling Size: *2 × 2-

.Output: The feature map is downsized to *9 × 9 with 128 feature maps -

Description: This max-pooling layer focuses the model on the most significant features by further reducing the -
.size

.Fourth Convolutional Layer: Layer Details: Convolutional layer with 256 filters of size *4 × 4 .8

.Activation: ReLU-

.Output: The dimensions of the feature map are *6 × 6 with 256 feature maps -

Description: This layer captures detailed features crucial for accurately differentiating facial attributes at a higher -
.abstraction level

.Fully Connected Layer: Layer Details: Fully connected layer consisting of 4096 units .9

.Activation: Sigmoid-

Description: This layer compresses the data into a 4096- dimensional feature vector that embodies a high-level -
representation of the facial image. This vector acts as the final output from each twin network prior to
.comparison

:Distance Calculation Layer.10

.Distance Metric: L1 (Manhattan) distance for the feature vectors of the image pairs -

Description: The feature vectors derived from the two sub networks are evaluated using the L1 distance, which -
computes the absolute differences between corresponding feature values. This distance indicates how similar the
.two input images are

.Final Fully Connected Layer with Output:-Activation: Sigmoid.11

.Output: A single value denoting the similarity score between the two images, which ranges from 0

Description: This output layer employs a sigmoid activation function to transform the measured distance into a -
.probability, reflecting whether the faces in the input pair belong to the same person or different individuals
 METHODOLOGY |5
This face recognition system is structured around several core stages, using CNNs for enhanced accuracy and
.adaptability across each step

Data Collection and Preprocessing.5.1

The system begins with collecting a large, labeled dataset covering a variety of facial expressions, lighting
conditions, and poses. Preprocessing includes resizing and normalizing images, with augmentation techniques (like
.rotations and brightness adjustments) added to increase dataset diversity and improve the model's generalization

Face Detection.5.2
Using deep learning, specifically CNNs, the model locates and isolates faces within images. CNNs improve upon
traditional methods by automatically learning hierarchical features, enabling accurate detection even under
.complex, real-world conditions

Facial Alignment.5.3
Facial alignment ensures each face is properly oriented before feature extraction, improving recognition accuracy.
CNNs handle challenges in alignment, like variations in pose, expression, and occlusions, by learning features that
.help maintain consistency

Feature Extraction.5.4
The model leverages CNNs to extract high-level facial features from the image, creating a compact, discriminative
feature vector for each face. This deep learning-based approach surpasses traditional methods by learning nuanced
.facial patterns directly from the raw data

Face Recognition.5.5
For recognition, the system compares the extracted feature vectors to those in its database using similarity metrics
(e.g., Euclidean distance). When the similarity score exceeds a set threshold, the face is matched to a known
.identity
 Paper title

Evaluation.5.6
The model’s accuracy is evaluated against labeled data and optimized through fine-tuning on a validation set. This
.step ensures robustness, enabling the system to adapt effectively to diverse real-world scenarios

RESULT AND ANALYSIS| 6

Face Detection and Alignment: - The CNN-based detection successfully recognized faces in various lighting .1
.situations and minor obstructions. Facial alignment adjusted poses, enhancing the accuracy of feature extraction

Feature Extraction:- The use of high-dimensional feature vectors effectively captured subtle facial .2
.characteristics, maintaining consistent performance across different lighting scenarios and angles
 Recognition Accuracy: - By employing similarity metrics, the system achieved high accuracy and low error .3
.rates, even in the presence of moderate obstructions

Calculation of metrics of the model used

Quantitative Metrics: - Exceptional precision, recall, and accuracy demonstrated effective identification, with .4
.few false positives and strong generalization capabilities

Fig: Graphical representation of metrics

Limitations: - The system's performance suffered under extreme obstructions or pronounced poses. The biases .5
.in the dataset indicated the necessity for more diverse training data

Comparison to Traditional Methods: - The CNN model surpassed traditional techniques, particularly in .6
.automated feature extraction and adaptability
 Paper title

Fig: Comparison between CNN, SNN, and DNN models

Summary: The system exhibited impressive accuracy and adaptability, making it appropriate for real-world
.applications, although there are minor aspects that could be enhanced
FUTURE WORK| 7
 Paper title

Future efforts should aim to expand the training dataset to ensure better demographic diversity, reducing biases
and improving the model's generalization across different user groups. Additionally, incorporating advanced
methods such as attention mechanisms or pose estimation modules can improve the system’s ability to handle
.occlusions and extreme poses effectively

Optimizing the model for real-time processing is another key area for enhancement, enabling its application in
scenarios requiring instant recognition. Furthermore, exploring privacy-preserving approaches like federated
learning can ensure ethical deployment by minimizing data exposure and maintaining user privacy. These
advancements will make the system more adaptable, secure, and ethically sound, further advancing the capabilities
.of AI-driven facial recognition

CONCLUSION| 8
This research highlights the potential of deep learning, particularly convolutional neural networks (CNNs), in
developing a highly accurate and robust face recognition system. The model excelled in diverse scenarios,
demonstrating resilience to variations in lighting, pose, and partial occlusions, outperforming traditional face
recognition techniques. By automating the feature extraction process and leveraging the power of deep learning,
.the system offers a reliable solution for applications in security, identity verification, and surveillance

However, certain challenges, such as handling extreme poses and substantial occlusions, were identified, along
with demographic biases due to limited representation in the training data. Addressing these issues is crucial for
.enhancing the system's reliability, inclusivity, and fairness, ensuring its broader applicability and social acceptance

REFERENCES| 9

C. Cruz, L. E. Sucar, and E. F. Morales, in 2008 8th IEEE International Conference on Automatic :2008 ]1[
Face &Gesture Recognition, IEEE, 2008, pp. 1–6
Maas, Andrew and Kemp, Charles. One-shot learning with bayesian networks. Cognitive Science :2009] 2[
.Society, 2009
.Mnih, Volodymyr. Cudamat: a cuda-based matrix class for python. 2009 :2009 ]3[
Simonyan, Karen and Zisserman, Andrew. Very deep convolutional networks for large-scale image :2014 ]4[
.recognition. arXiv preprint arXiv:1409.1556, 2014
X. Hao, G. Zhang, and S. Ma, vol. 10, no. 03, pp. 417–439, 2016 :2016 ]5[
.S. Singh and S. Prasad, Procedia Computer Science, vol. 143, pp. 536–543, 2018 :2018 ]6[
S. Saha, Dec. 15, 2018. A comprehensive guide to convolutional neural networks – the ELI5 way :2018 ]7[
.(accessed Aug. 11, 2023)
.Y. Kortli, M. Jridi, A. Al Falou, and M. Atri, Sensors, vol. 20, no. 2, p. 342, 2020 :2020 ]8[
 .Adjabi, A. Ouahabi, A. Benzaoui, and A. Taleb-Ahmed, Electronics, vol. 9, no. 8, p. 1188, 2020 :2020 ]9[
N. S. Singh, S. Hariharan, and M. Gupta, in Advances in Data Sciences, Security and :2020 ]10[
.Applications: Proceedings of ICDSSA 2019, Springer, 2020, pp. 375–382
P. Kaur, K. Krishan, S. K. Sharma, and T. Kanchan, Medicine, Science and the Law, vol. 60, no. :2020 ]11[
.2, pp. 131–139, 2020
K. H. Teoh, R. C. Ismail, S. Z. M. Naziri, R. Hussin, M. N. M. Isa, and M. Basir, in Journal of :2021 ]12[
.Physics: Conference Series, IOP Publishing, 2021, p. 012006
.H. Wang and L. Guo, IEEE, 2021, pp. 540–546 :2021 ]13[
E. M. Onyema, P. K. Shukla, S. Dalal, M. N. Mathur, M. Zakariah, and B. Tiwari, Journal of :2021 ]14[
.Healthcare Engineering, vol. 2021, 2021
X. Qi, C. Wu, Y. Shi, H. Qi, K. Duan, and X. Wang, Computational Intelligence and :2023 ]15[
.Neuroscience, vol. 2023, 2023
Paper title
Paper title