2010 International Conference on Advances in Recent Technologies in Communication and Computing

Face Detection and Localization in Skin Toned Color

Images Using Wavelet and Edge Detection
Techniques
H C Vijay Lakshmi S. PatilKulkarni
Department of Computer Science & Engineering Department of Electronics & Communication
[email protected] [email protected]
S J College Of Engineering Mysore, India S J College Of Engineering Mysore, India

Abstract─ Detection and localization of face in colour images with process. While segmenting faces of people wearing skin-tone
skin tone regions involves many challenges. This paper proposes dresses (“Fig. 1c” and “Fig. 1d”) using skin pixel
an algorithm that uses a combination of wavelets, edge detection segmentation pre-processing technique, the entire image of the
techniques to improve the efficiency during segmentation and person with skin-tone dress is detected as the face region and
feature extraction stage. Experimental results using the proposed
hence requires a further face localization step. Besides,
algorithm presented here show improved false acceptance and
false rejection rates. overlapping face regions also add additional constraints while
segmenting the faces. Due to variation in illumination, skin
Keywords─Skintone Wavelets, Edge-detection, Feature regions may not be properly identified as skin during skin
extraction, Detection. segmentation. Locating faces in these circumstances is more
complex as opposed to localizing faces with uniform, non
I. INTRODUCTION skin-tone background. The “Open CV” face detection [13]
Face detection and localization is the task of software based on Viola-Jones [12] correctly identifies the
checking whether the given input image contains any human face region mostly with images containing single face, but
face, and if so, returning the location of the human face in the fails to the dataset containing multiple faces with skin-tone
image. A large number of factors govern the problem of face background and dresses (refer Fig. 2).
detection [10].The long list of these factors include the pose, In spite of using combination of different colour
orientation, facial expressions, facial sizes found in the image, spaces during segmentation, it is tedious to demarcate region
luminance conditions, occlusion, structural components, boundaries between skin and pseudo skin regions and also
gender, ethnicity of the subject, the scene and complexity of eliminate these regions from searching process. The use of
image’s background. Faces appear totally different under colour space alone sometimes fails to segment the boundary
different lighting conditions. A thorough survey of face regions of the image. In order to overcome this problem
detection research work is available in [10]. In terms of combination of colour spaces for efficient skin segmentation
applications, face detection and good localization is an followed by Canny and Prewitt edge detection to demarcate
important pre-processing step in online face recognition. the region boundary is used for input image segmentation [4].
This paper is organized as follows. Section II briefly
highlights the various challenges encountered while detecting
and localizing the faces in skin tone regions. The need for
robust segmentation and feature extraction is highlighted in
section III. The proposed algorithm is explained in section IV
and it is followed by results in section V and conclusion in
section VI.

Figure 1a Input image with skin tone background

II. CHALLENGES
For the problem of face detection involving colour
images having complex scenes [1], the use of skin pixel
properties for segmentation reduces the search space to a
greater extent. Most of the detection algorithms have
considered images having non-skin tone background, people
wearing non-skin tone dresses etc. If images contain skin tone
background, then the entire region is identified as skin region
(“Fig. 1a” and “Fig. 1b”). In order to locate faces present in
the segmented regions, it calls for additional face localization Figure 1b Segmented image using HSI and YCbCr combination

978-0-7695-4201-0/10 $26.00 © 2010 IEEE 231

DOI 10.1109/ARTCom.2010.34
 by edge extraction for approximation image results in
prominent facial edges, which is evident from the images of
the facial edges extracted for raw face image “Fig. 3a”, “Fig.
3b” and edges extracted for the approximation images after
first and second level wavelet decomposition “Fig. 3c” , “Fig.
3d” respectively. The edges extracted for the second level
approximation eliminates all unwanted noisy edges present in
Figure 1c Input image with skin tone dress the image and also reduces the size of the edge feature set to
1/16th of the original size due to two level wavelet
decomposition of the input image. The Canny method applies
a high threshold for low edge sensitivity and a low threshold
for high edge sensitivity. Edge starts with the low sensitivity
result and then grows it to include connected edge pixels from
the high sensitivity result. This helps fill in gaps in the
detected edges. Prewitt, Sobel are used for detecting vertical
and horizontal edges in images. They return edges at those
points where the gradient of image is the maximum and also
Figure 1d Segmented image using HSI and YCbCr colour space
have the advantage of yielding zeros for uniform regions.
In this paper initial boundary segmentation is carried out
using the approach used in [4]. Selected segmented regions are
wavelet decomposed up to second level followed by extraction
of prominent facial edges as features using a combination of
Canny and Sobel edge extraction algorithm.

IV. PROPOSED ALGORITHM

As all the skin segmented regions are not face regions, each
Figure 2 Face detection using Open CV Software segmented region has to be checked whether the segmented
region contains face or not the following algorithm is proposed.
The proposed algorithm precisely locates where exactly the
III. SEGMENTATION AND FEATURE EXTRACTION actual face lies in each skin segmented region as the entire
segmented region does not represent the face. This algorithm is
Segmentation and feature extraction are the two important
implemented in two phases namely Feature extraction phase
pre-processing steps that play a vital role in face detection and and testing phase.
localisation. Segmentation is a process that partitions an image
into distinct regions. In the problem of face detection, skin
A. Feature Extraction Phase
segmentation helps in identifying the probable regions
containing the faces as all skin segmented regions are not face For feature extraction, images with only dominant facial
regions and segmentation aids in reducing the search space. features eyes, nose and mouth with variations in pose and
expressions which fit in to a window of size 88 X 80 are
Segmentation of an image based on human skin
considered. The proposed algorithm uses prominent facial
chromaticity using different colour spaces results in
edges as features, which is extracted using second level
identifying even pseudo skin like regions as skin regions.
wavelet decomposed approximation face image followed by
Hence there is a need for further eliminating these pseudo skin
edge extraction eliminating possible noisy edges.
regions. For a detailed survey of colour spaces refer [9].
Researchers are working on adaptive skin colour segmentation Step-1: Wavelet decompose each histogram equalized face
used for detection [2]. In [5, 8, 11] researchers have proposed image containing prominent facial features with slight
YCbCr and Robert cross edge algorithm for segmentation. In variation in pose and expression up to second level, retain only
[4] a combination of colour spaces (HSI and YCbCr) to approximation image and discard the horizontal, vertical and
identify the skin regions combined with Canny and Prewitt diagonal details .
edge detection algorithm for better boundary separation is
Step-2: Apply Canny and Sobel edge detection algorithm
used.
separately on the second level approximation images of each
For efficient classification of the segmented region as face training image considered for feature extraction.
or non-face, a robust feature set is very much essential. This
Step-3: Combine the edges of the approximation image
calls for an efficient feature extraction method. Edges play an
obtained in Step-2, into a single edge image using pixel by
important role in extracting prominent facial features. For
pixel image multiplication. On this edge image perform
detailed analysis of various edge detection algorithms refer [6,
morphological operations such as erosion and dilation to
7]. Two level wavelet decomposition of face images followed
extract the prominent facial edge features.

232
Step-4: The extracted edge feature is stacked into an edge data row sum (and column sum) of each projected and
matrix as column vectors. reconstructed window approximation edges with the
corresponding row sum (and column sum) original window
Step5: Apply PCA on edge data matrix obtained in step-4 for
wavelet approximation edges using ‘City block’ similarity
reducing the dimension and thus obtain Wavelet Eigen-edges.
measures.
To fix up the similarity distance thresholds, run the data set
used for feature extraction as test set and fix up the minimum
and maximum thresholds T1 and T2 for both row and column
sum. Distance lies between T1 and T2 for the window
Figure 3a Input Face image Figure 3b Edge extracted raw input image contents containing face images. Best match is found when the
threshold is closer to the average of T1 and T2.

V. RESULTS
Figure 3c Edge extraction of Figure 3d Edge extraction of
I level approximation image II level approximation image In the experiments 35 face images of size 88 X 80 were
used for extracting second level approximation feature set
edges. When experiment was tried with face image size less
B. Testing Phase
than 80 X 80 pixels for extracting feature edges, the false
Step-1: Segment the input colour image using the algorithm detection rate was higher. Group images containing faces with
proposed by the authors [4]. Select the segmented regions variation in pose, having moustache, structural components
satisfying constraints like aspect ratio for the face, area greater and slight variation in expressions were used for testing. In [3]
than 800 pixels, having holes due to the presence of eyes and approximation coefficients were used without extracting
mouth as the probable face regions and omit other segmented edges. The number of false acceptances was higher in skin
regions. tone regions and there were false rejections in skin tone
images with complex background. The number of false
Step-2: If the size of the selected segmented region is larger
acceptances and false rejections were reduced considerably
than the size of the face with prominent facial features, use a
when the edge strength of the second level approximation was
sliding window technique in order to locate the face in the
used as the feature set.
segmented region. (A window of size 88 X 80 is considered in
the experiment). Place the fixed size window on the selected In the proposed method, there are few false rejections and
segmented regions. If the length of the segmented region is the number of false acceptance is minimized. False
larger than the window size, slide the window horizontally. acceptances are noticed only when approximation images
Slide the window vertically if the width of the segmented produce similar edges at identical positions. The proposed
region is larger. Slide both horizontally and vertically if both algorithm was tried on a number of group images with
the length and width are larger. variations in illumination, pose and structural components.
Cumulative false rejection and acceptance cases are tabulated
Step-3: Wavelet decompose each histogram equalized
in Table 1 for various sliding window contents.
window content up to second level as the window slides
across the selected segmented region. Retain only False rejections were noticed for face images with wide
approximation image and discard the details. variations in facial expressions as evidenced by “Fig. 4f” of
“Fig. 4c”, while the face in the image “Fig. 4h” of “Fig. 4d”
Step-4: Apply Canny and Sobel edge detection algorithm
was properly localized as there was less variation in
separately on the second level approximation images of
expression. Use of only wavelet approximation coefficients
every two level wavelet decomposed window approximation
without edge extraction [3] sometimes reported false
image.
acceptance even for window contents of “Figure 4g”.
Step-5: Combine the edges of the approximation window
image into a single image using pixel by pixel
multiplication. On this perform morphological operations
such as erosion and dilation to extract the prominent edge
features of the wavelet decomposed window content.
Step-6: The edge features obtained in step-5 is projected
onto the Eigen-edges obtained in the Feature extraction
Figure 4a Input Image Figure 4b After Segmentation
phase.
Step-7: Compute the row sum and column sum of each
projected and reconstructed window approximation edges.
Classification is performed by calculating distance between

233
 International Conference on Signal Acquisition and Processing
February 2010, Bangalore February 9th -10th 2010, pp 162-166.
[5] Mohamed A. Berbar Hamdy M. Kelash, and Amany A. Kandeel “Faces
and Facial Features Detection in Color Images “ Proceedings of the
Geometric Modeling and Imaging― New Trends (GMAI'06) 0-7695-
2604-7/06
Figure 4c Figure 4d Selected Segmented Region for face localization [6] R. C. Gonzalez, R. E. Woods “Digital Image processing” Second edition
Prentice Hall India.
[7] Raman Maini & Dr. Himanshu Aggarwal “Study and Comparison of
Various Image Edge Detection Techniques “International Journal of
Image Processing (IJIP), Jan-Feb 2009, Volume (3) Issue (1)pp.1-11.
[8] Son Lam Phung, Abdesselam Bouzerdoum, and Douglas Chai “Skin
Figure 4e and Figure 4f Sliding window contents of Figure 4c Segmentation Using Colour and Edge Information” Perth, Western
Australia, Australia Proc. Int. Symposium on Signal Processing and its
Applications, 1-4 July 2003, Paris, France. 0-7803-7946-2/03
[9] V. Veznevets, V Sazonov, nad A Andreeva, “A survey on pixel-based
skin color detection techniques”, in Proceedings Graphicon-2003 pp.
85-92.
[10] Y. Ming-Hsuan, D J Kreigman, and N Ahuja, “ Detecting Faces in
Figure 4g and Figure 4h Sliding window contents of Figure 4d images: a survey”, IEEE Transactions on Pattern Analysis and Machine
intelligence, Vol 24, 2002, pp. 34-58.
VI. CONCLUSION [11] S. Phung,. A. Bouzerdoum and D. Chai, Skin segmentation using color
pixel classification: Analysis and comparison. IEEE Trans. on Pattern
In this paper an efficient algorithm that uses a Analysis and Machine Intelligence, 27(1), 2005,pp. 148–154.
combination of wavelet and edge detecting techniques to [12] P. Viola and M. Jones. "Robust real time object detection", International
detect and localize faces in colour images with skin-tone Journal of Computer Vision, 2001.
regions was proposed and experiment was conducted on [13] S. Krishna , “OpenCV face detection system” based on Viola-Jones
Method,
colour images. Results showed a considerable improvement in http://www.mathworks.com/matlabcentral/fileexchange/19912-open-cv-
false acceptance and false rejection rates compared to viola-jones-face-detection-in-matlab
algorithms that do not use the edge detection operations that
have been proposed.
TABLE 1 RESULTS

Method Distance Segmented Window False False

Measure Regions Tested Positive Rejection
Proposed
Method Cityblock 50 1700 30 3

Approximation
Image without Bhattacharya 40 1500 128 20
Edge
Extraction

Cityblock 40 1500 44 38

REFERENCES

[1] D. Petrescu and M. Gelgon “Face detection from complex Scenes in

colour images” Eurasip/Proceedings/Eusipco2000..
http://www.eurasip.org/Proceedings/Eusipco/Eusipco2000/sessions/Wed
Am/PO4/cr1871.pdf
[2] F Fritsch, S Lang, M Kleinehagenbrock, G A Fink and Sagerer,
“Improving Adaptive Skin Colour Segmentation by incorporating
Results from face detection”, Proceedings of IEEE International
workshop on Robot and Human Interactive Communication (ROMAN),
Berlin , Germany, September 2002, pp.337-343.
[3] H.C. Vijaylakshmi, S. PatilKulakarni “Face Localization and Detection
Algorithm for Colour Images using Wavelet Approximations”, 2nd
International Conference on Advances in Computer Vision and
Information Technology, December 2009, Aurangabad. India, pp 305-
311.
[4] H.C. Vijaylakshmi, S. PatilKulakarni “Segmentation Algorithm for
Multiple Face Detection for Color Images with Skin Tone Regions” 2nd

234