FACIAL RECOGNITION

TO TEXT TRANSLATION
TEAM MEMBERS:
• PRIYANKA.S – 2021103559
• MONIKA.K – 2021103313
• AISHWARYA.S - 2021103314
OBJECTIVE
The objective of the project is to develop a facial recognition system integrated
with emotion recognition and text generation capabilities. The system will
utilize computer vision techniques, including face detection and recognition
using Convolutional Neural Networks (CNNs), to identify faces in images or
video streams. Additionally, the system will incorporate emotion recognition
algorithms to analyze facial expressions and classify emotions. Finally, the
system will generate textual descriptions based on the recognized faces and
detected emotions using Natural Language Processing (NLP) techniques. The
project aims to provide a comprehensive solution for automatically translating
facial expressions into textual descriptions, which can be useful in various
applications such as human-computer interaction, surveillance, and emotion
analysis.
LITERATURE SURVEY

Publication
Title Authors Year Methodologies Setbacks Advantages
DeepFace: Closing the Deep neural networks, Computational
Yaniv Taigman, Ming Achieves human-level
Gap to Human-Level Convolutional Neural Networks complexity, need
Yang, Marc'Aurelio 2014 performance in face
Performance in Face (CNNs), training on large-scale for large-scale
Ranzato, Lior Wolf verification tasks
Verification datasets datasets

FaceNet: A Unified Computational

Florian Schroff, Dmitry Deep neural networks, Triplet loss Learns discriminative
Embedding for Face complexity, need
Kalenichenko, James 2015 function, training on large-scale embeddings for face
Recognition and for large-scale
Philbin datasets recognition and clustering
Clustering datasets

Deep Learning Face Yi Sun, Yuheng Chen, Deep neural networks, Prediction Computational Learns highly discriminative
Representation from Xiaogang Wang, 2014 of 10,000 classes, large-scale complexity, data face representations from
Predicting 10,000 Classes Xiaoou Tang datasets annotation effort large-scale datasets

Computational
Achieves state-of-the-art
SphereFace: Deep Weiyang Liu, Yandong Deep neural networks, complexity,
performance in face
Hypersphere Embedding Wen, Zhiding Yu, Ming 2017 Hypersphere manifold learning, hypersphere
recognition with a margin-
for Face Recognition Li, Bhiksha Raj large-margin cosine loss embedding
based loss
interpretation
 Publication
Title Authors Year Methodologies Setbacks Advantages
Jiankang Deng, Jia Guo, Deep neural networks, Angular margin- Computational Improves face recognition accuracy by
ArcFace: Additive Angular Margin Loss for
Niannan Xue, Stefanos 2019 based loss function, large-scale complexity, need for learning discriminative features with angular
Deep Face Recognition
Zafeiriou datasets large-scale datasets margin loss

Facial expression recognition with Andr Teixeira Lopes,

Convolutional Neural Networks (CNNs), Limited data availability, Addresses challenges of facial expression
Convolutional Neural Networks: Coping with Edilson de Aguiar, 2017
Data augmentation, training strategies overfitting recognition with limited dat
few data and the training sample order Alberto F. De Souza

Jan Van den Stock, Experimental study, Analysis of body Potential biases in
Body expressions influence recognition of Explores influence of body expressions on
Ruthger Righart, Béatrice 2007 expressions impact on facial and vocal experimental design,
emotions in the face and voice. facial and vocal emotion recognitio
de Gelder emotion recognition limited generalizability

Eigenface Algorithm-Based Facial Expression Eigenface algorithm, Facial expression Limited generalizability,
Zixiang Fei, Erfu Yang, Investigates facial expression recognition in
Recognition in Conversations - An 2018 recognition in conversational settings, sensitivity to lighting
David Li conversational settings using Eigenface
Experimental Study Experimental study conditions

Manfred Nusseck, Experimental study, Analysis of facial Potential biases in Provides insights into the role of different
The contribution of different facial regions to
Douglas W. Cunningham, 2008 regions' contribution to conversational dataset, limited facial regions in conversational expression
the recognition of conversational expressions
Christian Wallraven expression recognition generalizability recognition

Provides a toolbox for researchers to

The computer expression recognition toolbox Gwen Littlewort, Jacob Development of a computer expression Limited documentation,
2011 develop and evaluate facial expression
(CERT) Whitehill, Tingfan Wu recognition toolbox (CERT) potential bugs in software
recognition algorithms
 PUBLICATION
TITLE AUTHORS YEAR METHODOLOGIES SETBACKS ADVANTAGES

Provides a validated
Limited size of the
The MPI Facial Expression Database — A Database creation, validation of database for research on
K Kaulard, Douglas W. database, potential
Validated Database of Emotional and 2012 emotional and conversational facial emotional and
Cunningham, HH Bülthoff biases in data
Conversational Facial Expressions expressions conversational facial
collection
expressions

Peter Weyers, Andreas Electromyographic (EMG) recordings, Limited sample size, Investigates physiological
Electromyographic responses to static and
Mühlberger, Carolin 2006 analysis of static and dynamic avatar potential artifacts in responses to avatar
dynamic avatar emotional facial expressions
Hefele emotional facial expressions EMG recordings emotional

Limited
Statistical analysis of recognition Identifies differing impacts
A differing bidirectional impact on the Jingjing Wang, Weijian Li, generalizability,
2016 accuracy, comparison of different on facial expression
recognition accuracy of facial expression Xinyu Li potential biases in
recognition methods recognition accuracy
dataset

Development of pose-invariant
Pose-Invariant Facial Expression Recognition Shiro Kumano, Kazuhiro
2007 recognition algorithms, template
Using Variable-Intensity Templates Otsuka, Junji Yamato
matching
DETAILED
DESIGN
 PERFORMANCE
METRICS

1. Accuracy: Accuracy measures the proportion of correctly identified faces among all the faces processed by the system. It is calculated
as the ratio of the number of correctly identified faces to the total number of faces.

2. Precision: Precision measures the proportion of correctly identified faces among all the faces identified by the system. It is calculated
as the ratio of the number of true positive identifications to the total number of identifications made by the system.

3. Recall (Sensitivity): Recall measures the proportion of correctly identified faces among all the actual faces in the dataset. It is
calculated as the ratio of the number of true positive identifications to the total number of actual faces.

4. F1 Score: The F1 score is the harmonic mean of precision and recall. It provides a balanced measure of the system's performance,
taking into account both false positives and false negatives.

5. False Acceptance Rate (FAR): FAR measures the proportion of incorrectly identified faces among all the faces not belonging to the
target group. It represents the system's tendency to incorrectly accept non-matching faces.

6. False Rejection Rate (FRR): FRR measures the proportion of correctly identified faces among all the faces belonging to the target group
that are incorrectly rejected by the system. It represents the system's tendency to incorrectly reject matching faces.

7. Receiver Operating Characteristic (ROC) Curve: The ROC curve is a graphical plot that illustrates the performance of a facial
recognition system across different threshold values. It shows the trade-off between the true positive rate and the false positive rate.

8. Area Under the Curve (AUC): AUC represents the overall performance of the facial recognition system by measuring the area under
the ROC curve. A higher AUC value indicates better discrimination between positive and negative cases.
 Test Case ID Test Case Description Expected Result

TC_01 Capture image with a single face Successful recognition of the captured face

TC_02 Capture image with multiple faces Successful recognition of all detected faces

TC_03 Capture image with no faces No faces detected, system should return
appropriate error message

TEST CASES TC_04 Capture image in varying lighting

conditions
System should maintain consistent
recognition accuracy

TC_05 Test system performance with large System should handle large datasets
dataset of known faces efficiently without significant performance
degradation

TC_06 Test system performance with System should maintain acceptable

limited computational resources performance with limited computational
resources

TC_07 Translate facial recognition results Successful translation of recognized faces

into text description into textual descriptions
CONCLUSION
In conclusion, the integration of facial recognition with
text translation offers promising prospects for enhancing
human-computer interaction and communication
systems. By translating facial expressions into textual
descriptions, these systems can create more inclusive
interfaces and facilitate better understanding of human
emotions. However, challenges remain in accurately
interpreting facial cues, bridging the semantic gap
between expressions and text, and addressing ethical
considerations. Future research should focus on
improving accuracy, robustness, and ethical practices to
realize the full potential of facial recognition to text
translation systems.