Face Recognition Algorithms As Models of Human Face Processing
Face Recognition Algorithms As Models of Human Face Processing
Heather A. Wild
The University of Texas at Dallas
Richardson TX 75083-0688
2
suitability of the algorithms as models of human face University (MSU) [12], the Media Lab at the Massa-
processing. chutsetts Institute of Technology (MIT95, MIT96)[4],
In the present study, we develop a method for com- University of Maryland (UMD97) [12], University of
paring human and model performance at the level of Southern California (USC)[2] and Rutgers University
individual faces. This method is based on a multi- (RUT)[11].
dimensional representation that combines the human- Six additional PCA-based algorithms, implemented
and model-generated measures of the \similarity" be- by researchers involved in designing the FERET evalu-
tween all pairs of faces in a data set. Multidimen- ations, were included : (a) to provide a performance
sional analysis is a method for creating a spatial map baseline control model using a standard PCA algo-
of \item locations" that tries to preserve the similar- rithm, and (b) to gain a better understanding of the
ities/distances between items. Similarities/distances impact of varying the \retrieval" stage of the model via
are assessed along a set of variables. Here, the items we variations of distance metric implemented in the near-
analyzed consisted of 13 of the 18 FERET algorithms2 est neighbor classier. These NIST control algorithms
and 22 human observers. The variables consisted of the are as follows : L1 distance (L1), L2 distance (L2),
human- or algorithm-generated typicality measures for Mahalanobis distance (MD), city block distance metric
a set of faces. This analysis yielded a map of the simi- (ML1), Euclidean distance (ML2), angular distance or
larity of performance of the models and human subjects cosine (AN). These algorithms were all based on PCA
in the same multidimensional space. representations of faces, and varied only in the distance
metrics used to recognize the faces. The overall accu-
2. Methods racy of these algorithms is detailed elsewhere[5] and
indicates that variations in the distance metric impact
This paper is organized as follows. First, we describe the recognition performance of the PCA substantially.
the stimuli for both the algorithm and human data
collection. Next, we present a brief description and The performance of the algorithms was assessed us-
classication of the subset of FERET-evaluated mod- ing the FERET September 1996 evaluation method [7].
els and describe the algorithm measures of face typi- This method supplies a similarity measure between all
cality. We then describe the psychological methods for possible pairs of images for each algorithm. These sim-
gathering data analogous to the algorithm measures. ilarity measures can be represented in the form of a
Finally, we present a set of combined analyses aimed matrix S . For example, S contained the similarity
i;j
at determing the accord between human subjects and between the i and j face. For each algorithm, we
th th
3
Table 1. Computational Algorithms
ALGORITHM REPRESENTATION DISTANCE METRIC
Excalibur Co. (EX) Unknown Unknown
MIT95 PCA-based L2
MIT96 PCA-dierence space MAP Bayesian Statistic
Michigan St. U. (MSU) Fischer discriminant L2
Rutgers U. Greyscale projection Weighted L1
U. of So. CA (USC) Dynamic Link Elastic matching
Architecture
U. of MD (UMD97) Fischer discriminant P
L2
NIST (L1) PCA P (jxx ,, yy )j
k
i =1 i i
, P Px y z
NIST (L2) PCA k
i =1 i i
2
, qP P
i
k
P jx , y jz
k k
x2 y2
i=1 i i=1 i
4
0.4
Face Typicality : Human Subjects and Models 3. Interpretation and Discussion
Human Subjects. As noted, an initial explantaion
RUT
0.3
MSU USC L1
5
common bimodal clustering is complicated by the fact References
that we did not have full access to implementation de-
tails (e.g., some were proprietary, and not all details [1] J. D. Cohen, B. McWhinney, M. Flatt, and J. Provost.
of the algorithms are reported). In any case, some Psyscope : A new graphic interactive environment
conclusions can be drawn. First, it is clear that the for designing psychology experiments. Behavior Re-
algorithms do not separate exclusively based on their search Methods, Instruments and Computers, 25:257{
underlying representation. From Figure 1, clearly algo- 271, 1993.
rithms with rather dierent representations can cluster [2] K. O. et. al. The Bochum/USC face recognition sys-
tem. In H. Wechsler, P. J. Phillips, V. Bruce, F. Fo-
together (and vice versa). Although some clustering gelman Soulie, and T. S. Huang, editors, Face Recog-
might relate to representation (e.g., the UMD97 and nition: From Theory to Applications. Springer-Verlag,
MSU algorithms, both based on linear Fischer discrim- Berlin, 1998.
inant analysis, are very close), representation is appar- [3] L. Light, F. Kayra-Stuart, and S. Hollander. Recogni-
ently not the only factor. tion memory for typical and unusual faces. Journal of
Second, the distance metrics, which comprised the Experimental Psychology: Human Learning and Mem-
ory, 5:212{228, 1979.
most controlled design feature in this study, had a po- [4] B. Moghaddam and A. Pentland. Beyond linear
tent eect on the response patterns.6 This is clear in eigenspaces: Bayesian matching for face recognition.
the distribution of the NIST control implementations, In H. Wechsler, P. J. Phillips, V. Bruce, F. Fogelman
which are scattered across the space. Given that only Soulie, and T. S. Huang, editors, Face Recognition:
the distance metric varies in these models, the way in From Theory to Applications. Springer-Verlag, Berlin,
which \similarity" is computed in the algorithms can 1998.
give rise to very dierent patterns of typicality at the [5] H. Moon and P. J. Phillips. Analysis of PCA-based
level of individual faces (see also [5]). face recognition algorithms. In K. W. Bowyer and P. J.
Phillips, editors, Empirical Evaluation Techniques in
Finally, it is worth noting that the extensive FERET Computer Vision. IEEE Computer Society Press, Los
tests suggest that performance accuracy probably did Alamitos, CA, 1998.
not contribute strongly to the algorithm clustering. [6] A. J. O'Toole, K. A. Deenbacher, D. Valentin, and
The three most accurate algorithms, MIT96, UMD97, H. Abdi. Structural aspects of face recognition and
and USC, are divided between the clusters. the other-race eect. Memory & Cognition, 22:208{
224, 1994.
Combined, it is clear that the bimodal algorithm [7] P. J. Phillips, H. Moon, S. Rizvi, and P. Rauss. The
structure is determined by complex trade-os between FERET evaluation. In H. Wechsler, P. J. Phillips,
the representations and distance metrics. Furthermore, V. Bruce, F. Fogelman Soulie, and T. S. Huang, edi-
given that the algorithm response patterns align with tors, Face Recognition: From Theory to Applications.
the human response patterns, and given that an ab- Springer-Verlag, Berlin, 1998.
stract interpretation of these human response patterns [8] T. Valentine and V. Bruce. The eects of distinctive-
was possible, it is perhaps reasonable to characterize ness in recognising and classifying faces. Perception,
the algorithms in these terms. The \abstract" dimen- 15:525{536, 1986.
[9] T. Vetter and T. Poggio. Image synthesis from a sin-
sions used by the subjects relate to real physical prop- gle example image. In B. Buxton and R. Cippola,
erties of the faces. Athletic/masculine faces are shaped editors, Computer Vision { ECCV'96: Lecture Notes
dierently than non-athletic/non-masculine faces. Hu- in Computer Science, pages 652{659. Springer Verlag,
man subjects are very comfortable using these terms Cambridge, UK, 1996.
to describe faces, and neural network models have also [10] J. R. Vokey and J. D. Read. Familiarity, memorability,
been shown to be sensitive to this shape distinction. and the eect of typicality on the recognition of faces.
The age of a face is also specied from physical infor- Memory & Cognition, 20:291{302, 1992.
mation in the face, e.g., from shapes and facial textures. [11] J. Wilder. Face recognition using transform coding
Therefore, it is not surprising that such dimensions re- of gray scale projections and the neural tree network.
In R. J. Mammone, editor, Artical Neural Networks
late to the algorithms' predictions of face similarity. with Applications in Speech and Vision, pages 520{
The possibility of classifying faces in this abstract man- 536. Chapman Hall, 1994.
ner provides a link between the information that hu- [12] W. Zhao, A. Krishnaswamy, R. Chellappa, D. Swets,
mans and algorithms use to recognize faces. and J. Weng. Discriminant analysis of principal com-
ponents for face recognition. In H. Wechsler, P. J.
Phillips, V. Bruce, F. Fogelman Soulie, and T. S.
6 See [5] for a quantitative performance assessment of the dis- Huang, editors, Face Recognition: From Theory to Ap-
tance metrics, which is consistent with the qualitative ndings plications. Springer-Verlag, Berlin, 1998.
reported here.