Estimating Student Learning Affect Using Facial

Emotions *

Benisemeni Esther Zakka Hima Vadapalli

School of Computer Science and Applied Mathematics School of Computer Science and Applied Mathematics
2020 2nd International Multidisciplinary Information Technology and Engineering Conference (IMITEC) | 978-1-7281-9520-9/20/$31.00 ©2020 IEEE | DOI: 10.1109/IMITEC50163.2020.9334075

University of the Witwatersrand University of the Witwatersrand

Republic of South Africa Republic of South Africa
[email protected] [email protected]

Abstract— The current COVID-19 pandemic has seen a lot behavior aspects may include body language, eye gaze, facial
of higher institutions of learning embracing the e-learning expressions, and emotions exhibited and through vocal
systems. Although these e-learning systems promise to deliver feedback. These behavioral aspects are then decoded by
solutions to teaching and learning in this pandemic era, a key teachers based on their experience. Because affect can be a
challenge is motivating the learner to engage with the e-learning result of different psychological reasons and different affects
system continuously. Most e-learners quickly get bored and lose can be expressed by different behaviors, it is crucial to know
motivation in the course of learning. While there exist many which input modalities are crucial in decoding one’s learning
strategies such as chatrooms and sporadic question and answer affect and to correctly estimate the respective input modality
sessions to keep learners involved in e-learning platforms, they
under study.
have always achieved minimal connectedness among e-learners.
Facial emotions have been identified as an effective tool for With the huge surge in computational intelligence
interpreting learning experience in learners. This study, techniques, and their application in different fields such as
therefore, examines the use of facial emotions expressed by natural language processing, image understanding and
learners to interpret their learning affect in an e-learning labelling, health care, etc., education becomes the next area
session. This work also explores a standardized mapping where computational intelligence can have a huge impact. The
mechanism between facial emotions exhibited and their advent of e-learning technologies has brought a considerable
respective learning affects. The study identifies the physical
shift to teaching and learning at present. This type of education
changes in the face of a learner and uses it to estimate their facial
promotes learning versatility which includes learner-centered
emotions and then based on the mapping mechanism, maps
emotional states to a student's learning affect. Experiments learning and reduced geographic barriers. But this new
include the use of a convolutional neural network for the approach to learning also presents many challenges in terms
classification of seven facial emotions. The research study tests of cost and effective communication between the teacher and
different network architectures to find optimal architecture, student. Considering how teachers provide the learner with
using the FER2013 dataset. Results from the mapping are meaningful learning experiences in traditional learning due to
statistically analyzed and compared with responses provided by active classroom communication, modern ways of teaching
participants who participated in the live testing of the system. using technology tend to be a challenge. Unlike the
Results show that facial emotions, which are a form of non- conventional classroom setup, most of these e-learning
verbal communication, can be used to estimate the learning systems do not have features that can detect learners learning
affect of a student and provides a new avenue to enhance the affect: where learning affects are defined as learning skills
current e-learning platforms. mostly related to emotion. To close this void, multiple
researchers have proposed the use of natural language
Keywords—E-learning, Facial Emotions, Learning Affect, processing, hand gesture recognition, eye gaze estimation,
Convolutional Neural Networks facial emotion recognition, and body language detection to
estimate learners learning affect and provide a measure that
I. INTRODUCTION will provide more effective learning experience [2]–[4]. Of
With the ongoing trend of e-learning and online learning, these, facial expressions were classified as the most beneficial
several institutions have embraced e-learning platforms for and frequently used form of modality to estimate a students’
teaching and learning due to several occurrences one of which learning affect. Therefore, it is important to leverage this input
is the current COVID-19 pandemic that has ravaged the world. modality (facial emotion) to enhance teaching and learning in
However, keeping learners motivated and engaged on an e- e-learning platforms.
learning platform has been identified as a difficult task to
achieve. This is due to the lack of direct real-time II. RELATED WORK
communication between the instructors and the learners [1]. It Researchers have tried to evaluate students’ learning affect
is, therefore, crucial to understand how students learn and both in the traditional classroom settings and e-learning
comprehend material during an online session. By doing so, settings, to estimate and quantify a students’ comprehension
one can evaluate their learning skills based on the learner’s level and learning affect. Methods for the determination of
actions of awareness, interest, attention, and concern (learning learning affect include the use of questionnaires, biophysical
affect). In traditional classroom teaching, teachers evaluate sensors, eye-gaze, body gesture, facial emotions and others.
their students’ learning affect, the level of understanding and
comprehension, by mainly observing students’ behavior. The

XXX-X-XXXX-XXXX-X/XX/$XX.00 ©20XX IEEE

Authorized licensed use limited to: Univ of Calif Santa Barbara. Downloaded on June 17,2021 at 14:21:12 UTC from IEEE Xplore. Restrictions apply.
There is, however, limited work done on mapping facial non-verbal forms of communication. Ayvaz et al. [9] proposed
emotions to learning affects. a framework for an online video conference learning
environment to detect learners' motivation level. Their
A. Techniques in estimating learning affects program sensed the emotions of a learner and provided
Questionnaires: Questionnaires are administered to feedback to the teacher. The feedback helped to create an
external observers or to the learners and are known as self- immersive learning environment. Hammoumi et al. [10] have
reporting. Questionnaires are easy and mostly administered also introduced a framework for tracking the learner's
for estimating learner’s engagement or learning affect. emotions in an e-learning environment. The framework was
Zembylas et al. [3], explored the emotions learners exhibit incorporated into an instructional game software. Young
during online distance learning, and they achieved this by learners took part in the system evaluation. Their emotions
administering questionnaires in the form of daily diaries over were captured and analyzed in real-time using a web-camera.
six months. They further classified the learners' emotions Emotions were identified during the learning process, except
using open-coding techniques on the data obtained from the when the learners did not look into the camera. Their results
questionnaire. The results showed that while using e-learning demonstrated that learners displayed sad emotions when faced
platforms the learners experience both positive and negative with a challenge. Pan et al. [11] developed a system to record
emotions. As learning progressed, these emotions were found the attendance of learners using facial recognition. They
to change. From this analysis, it is evident that emotions and further developed a model to detect students’ engagement in a
learning correlate. Similarly, Sathik and Jonathan [5] used classroom using facial emotion expression. A stimulus-
questionnaires to assess the most used nonverbal response mechanism was used to classify the facial expression
communication strategies in the classroom and the association and attention of the learner. They mapped six emotions
between facial expression and learner level of comprehension. (concerned, curiosity, thinking, comprehension, disregard,
Results from their study show that facial emotions are the and disgust) onto learning affects. An antenna type learning
most frequently used non-verbal cues in learning. Further affects transfer model was designed to evaluate the teaching
analysis shows that a positive affect is expressed when the techniques. This technique made it possible to analyze
course content is understood, or the learner is satisfied with teaching methods in the classroom quantitatively. Their
the course delivery. On the other hand, when learners do not research gave a theoretical framework for analyzing learning
understand the content of the course, a negative affect is affect from the facial emotion expression of a learner, but it
expressed. was not automated.
A more complex and analytical approach to assess changes This study proposes the use of facial emotions to
in the affective state of the learner is to examine the emotional determine the learning affect experienced by the learners. This
changes exhibited by learner using sensors such as cameras, is achieved by using CNN for predicting facial emotions. The
microphones, pressure gages, special monitoring devices, and FER2013 dataset is used for the training of the network. The
other tools [6]. CNN architecture is used because of its ability to capture,
Biophysical signals: Shen et al. [7] used biophysical extract relevant and useful attributes from an image, as it
signals to determine how emotions evolve during teaching and achieved state-of-the-art performance on image classification
learning and the possibility of using the result to improve problems [10], [12]. The FER2013 emotion data set is adopted
learning. Their work applied Russell’s circumplex model of in this work as it contains un-posed images from Google
affect and machine learning techniques such as Support search API, and these images cut across different ethnicities,
Vector Machines (SVM) and K- Nearest Neighbors (KNN) to gender, and age groups. The dataset is a collection of images
analyze the emotions of learners who were engaged in a the system may see when used in the e-learning platform.
learning process. Their results showed that SVMs perform B. Mapping Emotions to learning affect
better with an accuracy of 86.3%. They concluded that the use There is minimal literature relating to the mapping of
of emotion data in e-learning would improve the performance facial expressions to learning affect. A study by Kapoor et al.
of learners within the system. Their research proved that [13] mapped facial action units (AU’s) to learners’ learning
emotional data is vital for an effective e-learning system. experience. The facial action units define the deformation of
However, the system was tested in a laboratory setting; the face based on movements of the facial muscles. Their
devices used, restricted the movement of learners and are findings show that smile (AU 6 + AU 12), tightening eyelids
uncomfortable to use. Implementation of such a system in an (AU 7), expanding eyes (AU 5) and rising eyebrows (AU 1 +
e-learning setting may prove difficult. AU 2) are clear indications indicating that the learner is
Body Gesture: Klein & Celik [2] used body movements to interested and enthusiastic about the learning phase or activity
detect student involvement inside a classroom environment. while lowering eyebrows (AU 1 + AU 4), wrinkling nose (AU
To determine whether the learners were "engaged" or "not 9), lower lip corners (AU 15) suggest lack of motivation.
engaged", they trained a CNN on image data acquired from Similarly, work by Sathik and Jonathan [3] suggested a
learning sessions. They got an accuracy of 89.7%. Similarly, mapping based on classroom observation and instructors'
Bidwell & Fuchs [8] used an automated gaze system to answers to questionnaires. Their work mapped facial
estimate the learner involvement. They used videos captured movements to the comprehension level of the learners. They
from a learning phase in the classroom to determine whether suggested that eyebrows and wide-open eyes were indications
a learner is engaged or not. that implied positive learning while narrowing eyes, wrinkles
Facial Emotions: Sathik and Jonathan [3] identified four on the forehead, narrowed eyes and curling lips showed a
non-verbal communication channels as the most popularly negative learning experience, a smile on the face or a neutral
used in learning, they further analyzed these mediums to facial expression showed an undecided learning state. In a
determine the most used for learning. The results of their related study, Pan et al. [11] proposed mapping of seven
survey show that facial expressions are the most widely used emotional states (concern, curiosity, thinking, comprehension,

Authorized licensed use limited to: Univ of Calif Santa Barbara. Downloaded on June 17,2021 at 14:21:12 UTC from IEEE Xplore. Restrictions apply.
disregard and disgust) to three learning effects (positive,
negative, and neutral) based on data collected from learning
videos in classrooms. Their findings did not include a clear
mapping of facial emotions to learning affects, however, a
description of the AU’s and facial movements from a
psychological perspective served as pointers that facial
emotion can be mapped to positive or negative learning
affects. Table I gives a summary of the mappings listed above.

III. DESIGN
The overall design of the proposed system includes three
phases: emotion recognition and evaluation, estimation of the Fig. 1. Overview of the proposed system
learning affect and feedback generation. The current work
discussed the first two phases of the proposed system. Fig. 1 A. Phase 1: Emotion Recognition and Evaluation
provides an overview of the proposed system, where phase 1) Data (Samples from existing database)
one includes: data acquirement, pre-processing, and emotion The samples for the initial training of the system are taken
recognition, while phase two maps and estimates the from the FER2013 dataset [14]. This dataset was introduced
recognized emotion to learning affects. The sections below at the International Conference on Machine Learning in 2013.
describe a detailed overview of each phase. Images in the dataset were collected by the Google image
search API automatically, which makes it a large and dynamic
dataset with population and environmental variations. The
dataset comprises of 35887 samples, which are 48 x 48
TABLE I.: SUMMARY OF FACIAL EXPRESSIONS TO LEARNING AFFECT BY
KAPOOR ET AL.; PAN ET AL.; SATHIK & JONATHAN [13], [3], [11] grayscale images each containing the following labels: anger,
disgust, fear, happiness, sadness, surprise and neutral. All
Kapoor et Sathik & Pan et al samples were split into training, validation and testing sets.
Possible Jonathan [3] [11]
Description
Emotion
al. [13] The distribution of samples is as shown in Table II. Fig. 2
provides an overview of the different emotions as depicted in
Lowering Sadness Negative Negative Nil the dataset. The samples include different face orientations
eyebrows and partial occlusions as expected in a live setup.
Nose Anger, Negative Nil Nil
wrinkling disgust TABLE II.DATA SPLIT FROM FER2013 DATASET FOR THE CURRENT STUDY

Depressing Sadness Negative Nil Nil Data Number of samples

lower lip Training 29065
Validation 3230
Smile/ Happy Positive Undecided Positive
Testing 3586
Pleasant
expression
Eye Anger Positive Negative Nil
lightening
Upper lid Fear, Positive Positive Nil
raiser surprise
Fear, Positive Positive Nil
Raising surprise
eyebrow

Wrinkle Anger Nil Negative Positive

forehead/
Frowning
No Neutral Nil Undecided Positive
expression
Not Disgust Nil Nil Negative
focused
Display Surprise Nil Nil Positive
surprise Fig. 2. Sample Images From FER2013 Dataset [13]

Fig. 3. Image Captured During Live Test Using A Computer Web-camera

Authorized licensed use limited to: Univ of Calif Santa Barbara. Downloaded on June 17,2021 at 14:21:12 UTC from IEEE Xplore. Restrictions apply.
 1) Data (Live testing) TABLE III. PROPOSED MAPPING
During a scheduled live session, proposed e-learning Emotion Kapoor et Sathik & Pan et al. Initial
model automatically activates video capture whenever a al. Jonathan [11] Mapping
learner plays a video from a given e-learning platform. The [13] [3]
OpenCV function, cap. read () is utilized to take video Anger Negative Negative Positive Negative
coverage of the learner using a computer webcam. A snapshot Disgust Negative Not stated Negative Negative
of video footage during live testing is shown in Fig. 3. Fear Positive Positive Not stated Positive
Happy Positive Undecided Positive Positive
The system splits the video data acquired from the web Sad Negative Negative Not stated Negative
camera into frames and converts them to grayscale using Surprise Positive Positive Positive Positive
OpenCV libraries. The system then detects the facial region Neutral Not stated Undecided Positive Neutral
using HAAR cascade classifier [15]. HAAR cascade is used
for its ability to detect faces almost in real-time on different A. Adaptive Mapping
scales. Detected faces were resized to 48X48 and normalized. It can be observed from Table III that the 'fear' emotion is
This action enabled the resized image to fit into the CNN mapped to a positive learning affect. This poses the following
model and ensured that pixel intensity values are normalized. question: Is fear an indicator of positive learning? Therefore,
These images are then fed into the trained convolutional to investigate further, this work also suggests an adaptive
neural network, which is used to predict the emotions mapping method in which the initial mapping in Table III
exhibited by a learner. would be compared to the responses provided by the learners
during a live testing session: that is the learning experience
2) Emotion Classification they had while using the proposed system. These responses
A convolutional neural network is a subset of deep neural will help to adapt the mapping and come up with a reliable
network proposed by Yann Lecun in the year 1988 [16]. It has mapping of facial emotions to learning affect.
been used in a wide range of problems such as image
classification, visual recognition and high dimensional V. LIVE TESTING
shallow feature encoding [17]. The network has achieved Nineteen participants volunteered to test the proposed
state-of-the-art accuracy in many image classifications system. The live testing was conducted in a laboratory setting
challenges, one of which is mentioned in the ICML 2013 where a video was played for each participant from a
workshop with an accuracy of 71% on the FER 2013 dataset computer system which had a web-camera. The webcam was
[14]. A trained CNN model is used in this work to predict and automatically triggered to capture the learner's facial images
classify the facial emotion of the learner. as the video starts. These images are pre-processed (resized
The CNN architecture comprises of three main layers: and reshaped) and fed into the trained CNN model. Classified
convolutional layer, pooling layer and fully connected layer. emotions are mapped to the respective learning affects based
The convolutional layer in this work consists of a 3x3 kernel; on the initial mapping as given in Table III. At the end of the
the kernel moves in strides of 2 performing matrix learning session, the program produces the facial emotions
multiplication between the kernel and the image to extract and maps the top three recognized facial emotions to
high-level features such as edges and gradient orientation. corresponding learning affects.
Adam optimizer, with an initial learning rate of 0.001, is used
in the network. Keras learning rate reducer function is used to VI. EXPERIMENTAL RESULT
reduce the learning rate by a factor of 0.9, whenever there is A. Emotion Recognition
no improvement in the validation loss after three epochs.
The trained CNN model was tested on a test set acquired
Batch normalization and drop out layers are inserted in the
from the FER2013 dataset. The model achieved a test
network to prevent overfitting and improve performance.
accuracy of 64.43%. This accuracy is close to some of the
The second layer in the CNN architecture (max-pooling state-of-the-art accuracies achieved by other researchers using
layers) are inserted after every two consecutive convolutional CNN models on FER2013 dataset [12], [18].
layers; this helped to reduce the spatial size of the
representation, control overfitting and suppress noise in the B. Learning Affect Estimation
images. The SoftMax activation function is used on the fully The impact of learning is estimated based on the facial
connected layer to classify the output, and Relu activation emotions recognized by the emotion recognition model. Each
function is used on the convolutional layers. facial emotion recognized by the system is mapped to a
corresponding learning affect as suggested in Table III.
IV. PHASE 2 EMOTION: LEARNING AFFECT MAPPING
Nineteen (19) participants volunteered for the live testing
The next phase of the work involves mapping an identified of the system. The system was tested in a laboratory setting,
emotion to a relevant learning affect. This study integrates the each participant watched a twenty (20) minute videos on the
mappings by Kapoor et al., Pan et al., and Sathik & Jonathan difference between deep learning and machine learning. At
([3], [11], [13]). The integration was done to collate the the end of the learning session, the participants voluntarily
mapping from the literature and come up with a direct responded to survey questions on their learning experience
mapping between facial emotions and learning affect. Based (learning affect). Fig. 4 gives a summary of emotions detected
on the mappings found in the literature, the current work came from each participant during the live test.
up with an initial mapping as shown in Table III. In the initial
mapping ‘anger’, ‘disgust’ and ‘sad’ were mapped as negative It was observed that surprise, fear and disgust emotions
learning affects, ‘fear’, ‘happy’ and ‘surprise’ were mapped as were never detected during the live testing sessions. This
positive learning affect, while ‘neutral’ is mapped as a neutral makes it difficult to establish the accuracy of the initial
learning affect. mapping between these emotions and their respective learning
affect. This also meant that one of the questionable initial

Authorized licensed use limited to: Univ of Calif Santa Barbara. Downloaded on June 17,2021 at 14:21:12 UTC from IEEE Xplore. Restrictions apply.
mapping i.e. fear to positive learning affect could not be Fig. 5. Correlation between the first, second and third estimated learning
verified. This warrants the need for a more intensive testing of affect with participants responses.
the proposed model to effectively adapt the initial mapping.
VII. CONCLUSION AND FUTURE WORK
This study proposes an e-learning model (system) that can
estimate learners learning affect to generate feedback in real-
time. This was achieved using CNN on FER2013 dataset to
generate a facial emotion recognition model which was used
to predict the facial emotion of the learners in real-time. The
predicted facial emotions were mapped to learning affects
based on the mapping proposed in this study.
Nineteen (19) participants participated in the live testing
of the proposed system. The system recognized the facial
emotion expression of the participants and mapped each
Fig. 4. Emotion detected per participant during live testing recognized facial emotion to its corresponding learning affect
according to the initial mapping. An analysis of the top 3 most
The 'neutral' facial emotion was predicted as the top-most estimated learning affects was compared with the responses
emotion regardless of the learners learning experience, i.e. acquired from survey responses from the learners. The result
both learners with positive and negative learning affect from the analysis indicate that there is a correlation of facial
expressed a neutral emotion during most of the learning emotions to learning affect, also our results shows that using
session (Fig. 4). Also, a close look at the images from the top two predictions facial emotion for learning affect detection
video frames indicated that most learners maintained a neutral is more ideal than using one or three predictions.
expression during most of the learning session, with only a
Further testing of the proposed mapping on more samples
few variations where the learners lose interest, are distracted,
could lead to a better-weighted mapping mechanism for
unable to understand or found a portion of the course content
learning affect predictions. Future studies in the e-learning
very engaging.
system should provide feedback to the learners based on the
The responses from the survey questions answered by the learning affect estimated and also include the use of multi-
learners at the end of the learning session were compared with model pattern analysis such as body expression, eye gaze, and
the estimated learning affect from the proposed system. head movements to achieve a more accurate result in the
Results from these analyses indicate that the first estimated identification of effects on learning.
learning affect did not correspond with the responses provided
by the learners, the second estimated learning affect REFERENCE
corresponded to the responses provided by the learners by [1] Brittany Gilbert, “Online Learning Revealing the
79% while the third estimate learning affect corresponded Benefits and Challenges,” 2015.
26% to the learners' response. Fig. 5 shows the top three [2] R. Klein and T. Celik, Tthe intelligent teaching
learning affects estimated and their correlation with the system: detecting student engagement during lectures
responses from the participants. using convolutional neural networks,” 2017 IEEE
It is worth noting that, compared to the first and third International Conference on Image Processing
predictions, the second prediction corresponded more with the (ICIP), 2017.
learner’s responses. However, because of the high occurrence [3] M. Sathik and S. G. Jonathan, “Effect of facial
of the neutral emotion, the first prediction cannot be expressions on student’s comprehension recognition
overlooked, for instance, sad and angry facial emotion were in virtual educational environments,” SpringerPlus,
detected by the system as the second most expressed emotion vol. 2, no. 1, pp. 1–9, 2013, doi: 10.1186/2193-1801-
from learners who indicated they had a negative learning 2-455.
experience. Learners who had a positive learning experience [4] G. Sofia and M. M. Sathik, “Pattern Recognition
expressed either a neutral emotion all through the learning System to Classify Student’s Emotion using
experience or happy the second most experienced emotion. Forehead Wrinkles,” 2011.
Therefore, a top-two emotion prediction for the mapping [5] M. Zembylas, M. Theodorou, and A. Pavlakis, “The
mechanism will be more accurate, and this recommendation role of emotions in the experience of online learning:
corresponds to the findings by Mollahosseini et al. and Challenges and opportunities,” Educational Media
Saravanan et al. [12], [18]. Also, worth noting is that none of
International, vol. 45, no. 2, pp. 107–117, 2008, doi:
the participants indicated that they had a neutral learning
experience. 10.1080/09523980802107237.
[6] R. W. Picard, “Affective Computing,” Cambridge,
150 massachustes institure of technology., 1997,
[Online].Available:
100 http://www.media.mit.edu/˜picard/.
50 [7] L. Shen, M. Wang, and R. Shen, “Affective e-
0 Learning: Using ‘Emotional’ Data to Improve
First Second Third Learning in Pervasive Learning Environment,” 2009.
[8] J. Bidwell and H. Fuchs, “Classroom analytics:
Correct Wrong Nill (Predictions) Measuring student engagement with automated gaze
tracking,” Behav Res Methods, vol. 49, 2011.

Authorized licensed use limited to: Univ of Calif Santa Barbara. Downloaded on June 17,2021 at 14:21:12 UTC from IEEE Xplore. Restrictions apply.
[9] U. Ayvaz, H. Gürüler, and M. O. Devrim, “Use of
Facial Emotion Recognition in E-Learning Systems,”
Information Technologies and Learning Tools, vol.
60, no. 4, p. 95, Sep. 2017, doi:
10.33407/itlt.v60i4.1743.
[10] O. el Hammoumi, F. Benmarrakchi, N. Ouherrou, J.
el Kafi, and A. el Hore, “Emotion Recognition in E-
learning Systems,” 2018 6th International
Conference on Multimedia Computing and Systems
(ICMCS), pp. 1–6, 2018.
[11] M. Pan, J. Wang, and Z. Luo, “Modelling Study on
Learning Affects for Classroom Teaching / Learning
Auto-Evaluation 2. Foundation of Research:
Automatic Analysis of Learning Behavior in
Classroom Teaching,” Science Journal of Education,
vol. 6, no. 3, pp. 81–86, 2018, doi:
10.11648/j.sjedu.20180603.12.
[12] A. Mollahosseini, D. Chan, and M. H. Mahoor,
“Going Deeper in Facial Expression Recognition
using Deep Neural Networks,” Nov. 2015, doi:
10.1109/WACV.2016.7477450.
[13] A. Kapoor, S. Mota, and R. W. Picard, “Towards a
Learning Companion that Recognizes Affect,” 2001.
[14] I. J. Goodfellow et al., “Challenges in representation
learning: A report on three machine learning
contests,” Neural Networks, vol. 64, pp. 59–63, Apr.
2015, doi: 10.1016/j.neunet.2014.09.005.
[15] P. Viola and M. Jones, “MERL-A Mitsubishi
Electric Research Laboratory Rapid Object Detection
Using a Boosted Cascade of Simple Features Rapid
Object Detection using a Boosted Cascade of Simple
Features,” 2004. [Online]. Available:
http://www.merl.com.
[16] Y. Lecun, L. Eon Bottou, Y. Bengio, and P. H.
Abstract|, “Gradient-Based Learning Applied to
Document Recognition,” in Proceedings of the IEEE,
1998, pp. 2278–2324.
[17] P. Agrawal, R. Girshick, and J. Malik, “Analyzing
the Performance of Multilayer Neural Networks for
Object Recognition,” The Visual Computer, pp. 1–8,
2019.
[18] A. Saravanan, G. Perichetla, and Dr K. S. Gayathri,
“Facial Emotion Recognition using Convolutional
Neural Networks,” Oct. 2019, [Online].
Available:http://arxiv.org/abs/1910.05602.

Authorized licensed use limited to: Univ of Calif Santa Barbara. Downloaded on June 17,2021 at 14:21:12 UTC from IEEE Xplore. Restrictions apply.