Computing and Informatics, Vol.

36, 2017, 1001–1020, V 2017-Aug-25

NOVEL APPROACH FOR DETECTION

AND REMOVAL OF MOVING CAST SHADOWS
BASED ON RGB, HSV AND YUV COLOR SPACES

Brahim Farou
Computer Science Department Badji Mokhtar-Annaba University
P.O. Box 12, 23000 Annaba, Algeria
&
LabSTIC, Guelma University
P.O. Box 401, 24000 Guelma, Algeria
e-mail: [email protected]

Houssam Rouabhia, Hamid Seridi

LabSTIC, Guelma University
P.O. Box 401, 24000 Guelma, Algeria
e-mail: [email protected], [email protected]

Herman Akdag
LIASD, Paris 8 University
93526 Saint-Denis, France
e-mail: [email protected]

Abstract. Cast shadow affects computer vision tasks such as image segmentation,
object detection and tracking since objects and shadows share the same visual mo-
tion characteristics. This unavoidable problem decreases video surveillance system
performance. The basic idea of this paper is to exploit the evidence that shadows
darken the surface which they are cast upon. For this reason, we propose a sim-
ple and accurate method for detection of moving cast shadows based on chromatic
properties in RGB, HSV and YUV color spaces. The method requires no a priori
assumptions regarding the scene or lighting source. Starting from a normalization
1002 B. Farou, H. Rouabhia, H. Seridi, H. Akdag

step, we apply canny filter to detect the boundary between self-shadow and cast
shadow. This treatment is devoted only for the first sequence. Then, we separate
between background and moving objects using an improved version of Gaussian
mixture model. In order to remove these unwanted shadows completely, we use
three change estimators calculated according to the intensity ratio in HSV color
space, chromaticity properties in RGB color space, and brightness ratio in YUV
color space. Only pixels that satisfy threshold of the three estimators are labeled as
shadow and will be removed. Experiments carried out on various video databases
prove that the proposed system is robust and efficient and can precisely remove
shadows for a wide class of environment and without any assumptions. Experimen-
tal results also show that our approach outperforms existing methods and can run
in real-time systems.

Keywords: Computer vision, shadow detection, chromaticity, GMM

1 INTRODUCTION

Computer vision systems dedicated to video processing require some paramount

procedures sash as detection and tracking of moving objects. When the objects
of interest have a well-defined shape, the advanced classifiers can be used to seg-
ment objects directly from the image. These techniques work well for objects with
well-defined contours, but are difficult to carry out for objects with flexible contour.
Gaussian mixtures model (GMM) are among the most commonly used approaches
for detecting moving objects in a video sequence. However, GMM generally have one
major drawback, shadows tend to be classified as part of the foreground leading to
confusion between the object and its shadow. Indeed, shadows share the same move-
ment patterns and have intensity change similar to moving objects, which influences
video surveillance systems performance.
In recent years, many works have been published to solve the problem of de-
tecting and removing shadows, and the contributions reported in the literature can
be organized into three types: those whose works focus on algorithms [1], others
according to the relationship between object/environment and implementation do-
main [2], and the latest based on choice of relevant characteristics [3]. Prati et al. [1]
have regrouped related works into two main classes: statistical algorithms with ver-
sions both parametric [4] and non-parametric [5], and deterministic algorithms with
model-based approach [6] and non-model-based approach [7]. Another point of view
is proposed by Al-Najdawi et al. [2] to categorize the contributions according to
a new taxonomy based on dependence between methods and objects [8, 9, 10, 11, 12],
methods and environment [13, 14, 15] or on both [16, 7]. Sanin et al. [3] observed
that the choice of features has a great influence on the results compared to the choice
of algorithms, and proposed a new taxonomy composed of two essential categories:
spectral and spatial characteristics. Moreover, spectral characteristics are divided
Novel Approach for Detection and Removal of Moving Cast Shadows . . . 1003

into intensity [14, 17], chromaticity [4, 7, 18, 19] and physical properties [20]; spatial
characteristics are split into geometry [21, 22, 23] and textures [24, 25].
The importance given to this field has prompted researchers to offer a large
amount of approaches to solve shadow problems in videos. However, the proposed
approaches give results only in very specific and well-defined environments [1]. In
addition, the conditions imposed by the authors for the proper functioning of these
systems restrict their use in wide public environments. Comparative studies in the
literature have shown that the results quality obtained by the spatial characteristics
is higher than spectral characteristics. However, the spatial characteristics consume
much computation time and need more memory space, which limits their use in
real-time and on machines with low power. Conversely, the spectral characteristics
offer high execution speed, but have the inconvenience to be sensitive to change in
light intensity and they give bad results when objects have an intensity or color
like shadow. To avoid the problems outlined above, we propose a novel method for
detecting and removing shadows in video surveillance taken from a fixed camera.
The approach is based on chromatic properties in RGB, HSV and YUV color spaces.
We also implemented Large Region texture-based method (LR) presented by Sanin
et al. [3] and in which they showed that LR method gives better results compared
with proposed methods in literature. The rest of this paper is organized as follows.
Section 2 presents an overview of the shadow model. Section 3 describes in detail
the proposed approach. Comparative experimental results are analyzed in Section
4. Finally, conclusion and perspectives are given in Section 5.

2 SHADOW

When light strikes an opaque object it is scattered, absorbed or reflected, but at

the back of the object light does not pass, it is a shadow area. So we can say that
shadow is a dark area created by the interposition of an opaque object between
a light source and the surface on which the light is reflected. This shadow takes
a shape of a silhouette without thickness which depends on the source intensity and
its location relative to the object. There are four types of shadows (Figure 1, [26]):

• The attached shadow is a region of the object that receives no light. It is located
behind the object, in the area where the light from the source does not arrive.
• The umbra is a region of space where the light rays from the source do not pass
because they are stopped by the object.
• The cast shadow is a region of a screen placed behind an object relative to the
light source and not receiving radius. The size and shape of the cast shadow
depend on the shape, size and position of the object relative to the source, but
also depend on the location and angle of the screen.
• The penumbra is a border area that appears between the illuminated part and
the shade part.
1004 B. Farou, H. Rouabhia, H. Seridi, H. Akdag

Cast Shadow

Attached Shadow
Light Source

Umbra

Figure 1. Shadow type representation

3 PROPOSITION

The ability to extract moving objects from a video sequence is a crucial issue in many
video surveillance systems. The primary role of image processing operations in such
system is not the correct detection of the object details, but the robust detection of
shapes in motion. Unfortunately, these shapes are generally deformed by their own
shadow. In addition, in dynamic scenes, all pixels of moving objects or shadows
are detected at the same time, shadows and objects share the same visual motion
characteristics. Taking into account of all the considerations mentioned before, we
propose a simple and reliable method able to detect and remove shadows generated
by one or more light sources. Figure 2 shows the overall architecture of our system.

Video
Fragmentation Normalization Edge detection

Preprocessing Dilatation

Background Subtraction

Video Shadows detection and removal

Figure 2. Architecture of the proposed system

Novel Approach for Detection and Removal of Moving Cast Shadows . . . 1005

3.1 Preprocessing

Preprocessing involves four steps: First thing to do is segment the video stream into
frames, where each frame represents an image of the video. Then all frames are
normalized to avoid disparities in size among the images taken by different cameras.
In the next step, canny filter is used to detect edges that will serve to segment the
image into a set of connected components and to separate between the attached
shadow and cast shadow. Finally, the size of edges is increased using morphological
to remove any discontinuities.

3.2 Background Subtraction

Tracking moving objects is hard task to realize in automated video surveillance

systems. In literature, there are two distinct contributions: Influence of discrimi-
native power of the features on system performance and the separation algorithm
between foreground and background with better variations management. Generally,
a good separation algorithm simplifies the further treatments, reduces run time and
consumes less memory space. Among the research conducted in this field, works
done in [27] showed that GMM offers a good compromise between extraction qual-
ity and runtime compared to other background subtraction methods. Despite their
widespread use in various applications, they still suffer from some problems such
as local variations, the instantaneous variations in brightness as well as the back-
ground complexity [27]. For background subtraction, we use an improved GMM
approach proposed in [28]. The following algorithm illustrates the process used for
background subtraction.
Algorithm:

Initialization:
Split the first image into several equal size areas
Assign a thread for each area
Convert all the image pixels from RGB to HSV
Calculate and store the color histogram of each zone
Initialize the GMM parameters

Iteration
FOR each new frame
Convert all image pixels from RGB to HSV
FOR each area
Calculate the color histogram
Measure the similarity degree between the
calculated histogram and the stored one
If the difference is greater than a threshold T
Memorize the new histogram
Updating the GMM parameters
1006 B. Farou, H. Rouabhia, H. Seridi, H. Akdag

END IF
END FOR
END FOR

3.3 Shadows Detection and Removal

In the literature several color spaces have been used to model separately the bright-
ness and chromaticity. Usually, the systems using chromaticity as a criterion for
detecting and removing shadows choose an adequate color space that allows a nat-
ural separation between intensity and chromaticity. In the current process, we use
RGB, HSV and YUV color spaces reported as the most robust among others to de-
tect the shadow [2, 3]. Each color space is presented to better understand its effect
on the shadow.
• RGB color space is the vector space generated by the three primary components
Red, Green and Blue. It is the basic color space. It easily allows switching from
one space to another, but it does not explain the influence of brightness and
saturation on color because their change affects all basic components.
• HSV color space characterizes the colors with more intuitive way, in accordance
with natural color perception. The hue (H) is the name used to describe the
color conveniently associated with a wavelength. Saturation (S) is the color
purity level, which should be between the maximum purity (bright color) and
achromatic (gray level). The value (V) is the color light-intensity measure, which
should be between the absolute black and white.
• YUV color space represents colors by using a luminance component Y, and two
chrominance components U and V. The luminance component Y is a weighted
average of relative human sensitivity to primary colors. The chrominance com-
ponents (U and V) are the contrast blue/yellow and the contrast red/cyan.
YUV color space provides a natural separation between the chromaticity and
brightness.
We conducted several studies and experiments to understand the shadow impact
on colors. The obtained results led to the following remarks:
• Shadows density is the most relevant and the most difficult characteristic for
modeling the shade.
• The shadow depends on the amount of light reflected by the surface on which
the shadow is projected.
• In HSV color space, shadow does not change the hue of the color, but tends to
decrease the value (V) and the saturation (S) components; for example, if a red
object is covered by shadow it becomes dark-red which is darker than red but
remains red.
• The shadow effect on the values of the three components R, G and B in RGB
space disturbs indirectly the values of H and S components in HSV space sup-
Novel Approach for Detection and Removal of Moving Cast Shadows . . . 1007

posed invariant to light intensity change. The changeover from RGB to HSV
color space makes modeling the shadow effect on colors more difficult.
• Using one color space is not enough to express all changes made by adding
shadow on color.
• The value of Y component in YUV color space is the best light-intensity direct
measurement to model the shadow effect.
• The shadows effect on objects causes the same change degree to the three com-
ponents of RGB color space. This condition is necessary but not sufficient to
deduce that there is a shadow.

Proposed solutions:
To detect and remove moving shadows, using a background model as reference is
required. For each new frame we extract moving objects, and then we calculate
the change degree of pixels values for all components in RGB, HSV and YUV color
spaces. This process is guaranteed by calculating the ratio change between pixels
values in the background model (BG) and the pixels values of current frame (F) using
the following empirical equations. The ratio change of R, G and B components in
RGB color space are described by Equations (1), (2) and (3):
RBG − RF
RCR = , (1)
RBG
GBG − GF
RCG = , (2)
GBG
BBG − BF
RCB = (3)
BBG
where (RBG , GBG , BBG ) and (RF , GF , BF ) are the pixels values for red, green and
blue components in RGB color space. The ratio change of hue and saturation in
HVS color space are described by Equations (4) and (5):

RCH = HF − HBG , (4)

RCS = SF − SBG (5)

where (HBG , SBG ) and (HF , SF ) are the pixels values for hue and saturation compo-
nents in HSV color space. The ratio change of luminance component in YUV color
space is given by Equation (6):
YBG − YF
RCY = (6)
YBG
where (YBG , YF ) are the pixels values for luminance component in YUV color space.
Based on the remarks mentioned above, a pixel is considered part of the shadow if
it satisfies simultaneously the following three rules:
1008 B. Farou, H. Rouabhia, H. Seridi, H. Akdag

|RCR − RCG | < 3 and |RCB − RCG | < 3 and |RCR − RCB | < 3, (7)

RCH < P 1 and RCS < P 2, (8)

0 < RCY < RCH − RCS (9)
where P 1, P 2 represent thresholds optimized empirically.

4 EXPERIMENTS

4.1 Settings

The algorithms presented in this paper are implemented in Java on a computer

with an Intel Core i5 2.67 GHz and a 4 GB memory capacity. This section seeks to
highlight the results obtained from tests performed on videos. The used videos are
different with respect to the contexts, light intensity, fluctuations in objects, number
and kind of objects, movements caused by nature elements (clouds, dust and noise),
and camera movements. To rate system performance we used four databases in
which three are public.
The first one (DBA) has six videos taken in four environments representing:
a campus, a highway (I, I2 and II), an intelligent room and a laboratory [1]. The
second (DBB) is constituted of nine videos representing: a bootstrap, a campus,
a curtain, an escalator, a fountain, a hall, a lobby, a shopping mall and a water
surface [29]. The third (DBC) has two videos representing: a highway and a hall-
way [30]. Our own database (DBD) is constituted of four environments representing
a campus, a hallway, a highway and a public park where each environment is taken
with three videos. Each sequence gives a new challenge to our method for detecting
shadows and for testing robustness. Table 1 shows some details about the used
videos in the test.

4.2 Performance Evaluation

In-depth comparative works done by Sanin et al. [3] showed that the method based
on large region texture (LR) gives better results compared to other methods. In all
cases, the large region texture-based method performs considerably better than all the
others, obtaining high values for both the detection and discrimination rates in all
sequences. For this reason and to better situate our approach over what exists in the
state of the art, we implemented the LR method which will be used as a reference
method.

4.2.1 Qualitative Results

In this section we present some qualitative results performed on video databases

mentioned above. For a good visual comparison, the blue color areas show the
Novel Approach for Detection and Removal of Moving Cast Shadows . . . 1009
Data Environment Number Resolution Length Frame rate
Base of Frame (mn) (frame/s)
Campus 1178 352 × 288 01:57 10
Highway I 439 320 × 240 00:29 10
Highway I2 439 320 × 240 00:29 14
DBA
Highway II 499 320 × 240 00:33 14
Intelligent room 299 320 × 240 00:30 10
Laboratory 886 320 × 240 01:28 10
Bootstrap 3054 160 × 120 / /
Campus 2438 160 × 120 / /
Curtain 23963 160 × 120 / /
Escalator 4814 160 × 130 / /
DBB Fountain 1522 160 × 128 / /
Hall 4583 176 × 144 / /
Lobby 2545 160 × 128 / /
Shopping Mall 2285 320 × 256 / /
Water surface 1632 160 × 128 / /
Hallway 1799 320 × 240 03:00 10
DBC
Highway III 2055 320 × 240 03:42 10
526 00:17
Campus 896 320 × 240 01:30 30
896 02:38
616 00:20
Hallway 893 640 × 480 00:29 29
382 00:12
DBD
1210 00:40
Highway 420 640 × 480 00:14 29
896 00:29
559 00:18
Public Park 585 640 × 480 00:19 29
555 00:18

Table 1. Description of used databases

detected shadow on the target image and the shadow mask in the neighbor frame.
The sequences of images below show the shadow detection in indoor/outdoor videos
taken under different constraints, noise and light intensity variations.
It is clear to see (Figure 3) that our approach can detect shadows for all moving
objects without any prior assumptions about the nature of the environment or on the
moving objects and produces better results compared to LR method. In frame 275,
one can notice that textures have failed to distinguish between the object and its
shadow. The frame 135 shows that our system can differentiate between cast shadow
and attached shadow, which is not the case in systems based on LR.
In Figure 4, the first video was taken into Guelma University campus in a sunny
day with frequent cloud passage. This phenomenon creates an instant brightness
1010 B. Farou, H. Rouabhia, H. Seridi, H. Akdag

LR Our Approach
Frame 275
Frame 603

Public Park
Frame 853
Frame 135
Frame 165

Campus I
Frame 267

Figure 3. Shadow detection of multiple moving objects in an outdoor environment with

multiple variations

change affecting all background colors. Once again, results show the effectiveness
of this approach and its adaptability to multiple color changes in the background.
The second video was taken into a highway with the same conditions as the first
video. The frames 339 and 347 show that LR method cannot distinct between cast
shadow and attached shadow, and also has completely distorted the moving object
when removing shadows, while our system has perfectly separated the shadow from
object in both frames.
Even in partially dark environments with light reflection effects on the wall and
floor, our method is effective as shown in three frames (Figure 5). Frame 23 shows
that our method differentiates between attached shadow and cast shadow. Therefore
it possesses a superior performance than LR method.
These frames (Figure 6) highlight the ability of our system to detect shadows in
public video databases. The frame 248 shows that the proposed method is able to
Novel Approach for Detection and Removal of Moving Cast Shadows . . . 1011

8 LR8 Our88Approach
8 8
8 8 8 8
Frame8279
8

Highway8I8
8 8 8 8
Frame8320
8

8 8 8 8
Frame8103
8

8 8 8 8

Intelligentroom8
Frame8248
8

8 8 8 8
Frame8253
8

Figure 4. Shadow detection of a single moving object with brightness variations in envi-
ronment

LR Our Approach
Frame 23
Frame 65

Hallway
Frame 136

Figure 5. Shadow detection in a dark indoor environment with reflection effects

1012 B. Farou, H. Rouabhia, H. Seridi, H. Akdag

Frame 249
Frame 339 LR Our Approach

Highway
Frame 347
Frame 87
Frame 335 Frame 128

Campus II

Figure 6. Shadow detection in DBA database

detect completely shadow in the case where the floor color is substantially similar to
shadow color. This scenario is one of the most difficult cases to treat in chromaticity-
based method.
In summary, the qualitative results presented through these experiments show
that our approach is effective without any assumptions on the nature of the envi-
ronment, image quality, and light variations or color. Further, it outperforms LR
method almost in all considered videos.

4.2.2 Quantitative Results

In order to evaluate quantitatively shadow detection, we used two metrics [1]. The
shadow detection rate η, which indicates how well the algorithm detects shadows.
The shadow discrimination rate ξ, which describes how the system can differentiate
between shadows and foreground pixels. They are evaluated by Equations (10)
Novel Approach for Detection and Removal of Moving Cast Shadows . . . 1013

and (11), respectively. For further evaluation, we also calculated the processing
time per sequence for each method.
T PS
η= , (10)
T P S + F NS
T PF
ξ= (11)
T PF + F NF
where TP is the true positive pixels representing the number of pixels correctly
detected and the FN is the false negative pixels representing the number of pixels
incorrectly detected. Subscripts S and F denote shadow and foreground, respec-
tively. T PF is the correct number of points on foreground objects minus the number
of points on foreground objects marked as shadow. TPS and FNS are calculated
according to ground-truth shadows pixels; T PF and F NF are calculated using
ground-truth objects pixels.
To test and evaluate the performances of the proposed approach in our video
data base (DBD), a ground truth data set is also necessary. Objects ground truth
and cast shadows ground truth are obtained by manually labeling objects and cast
shadows after extracting backgrounds with GMM. During producing ground truth
data set for the captured videos, we noticed that labeling shadows is a hard task to
accomplish accurately especially in the scene where floor or objects color is similar
to shadow. For giving more credibility to tests, all frames are taken randomly to
produce ground truth data set. Public video ground truth for DBA, DBB and DBC
are available respectively in [1, 29, 30].
A selection of well-known methods is compared to our method in terms of quan-
titative measures. Seventeen selected methods were evaluated quantitatively based
on η and ξ.
Table 2 compares the result obtained by our method with results achieved by
the other methods. The experimental results show that our method provides both
a good detection and discrimination rate relative to other methods. However, we
cannot draw deep conclusions because improving shadow detection performance is
proportional to improvement in background subtraction performance.
For rational comparison, we also implemented the LR method using the same
background subtraction method as proposed in our approach. The comparison re-
sults are shown in Table 3. Quantitative results clearly show that the proposed
system allows both good detection with an average of 91.37 % and better discrim-
ination with an average of 95.98 %, which means that our method obtained a gain
of 3.44 % in detection and 7.98 % in discrimination relative to the LR method [3].
We note that there is a drop in performance especially in Highway I and Highway
III videos due to a poor contours detection. The qualitative results also showed this
decreased performance because our system has badly detected edges between the
object and its shadow. In return, the LR method gives good shadow detection with
a stable performance in all cases. However, discrimination is a little weak due to
non-availability of discriminative texture in the treated cases.
1014 B. Farou, H. Rouabhia, H. Seridi, H. Akdag

Highway I Intel-Room Laboratory Campus

η% ξ% η% ξ% η% ξ% η% ξ%
Mikic[31] 59.59 84.70 76.27 90.74 64.85 95.39 72.43 74.08
Haritaoglu[32] 81.59 63.76 72.82 88.90 84.03 92.35 82.87 86.65
Cucchiara[33] 69.72 76.93 78.61 90.29 76.26 89.87 82.87 86.65
Stauder[34] 75.49 62.38 62.00 93.89 60.34 81.57 69.10 62.96
Salvador[35] 71.82 79.29 73.45 86.52 88.24 93.57 72.4 72.4
Martel.Brisson[36] 75.43 74.67 73.60 79.10 76.62 75.14 66.2 72.3
Al-Najdawi[37] N/A N/A 87.24 95.85 90.22 92.83 90.67 93.34
Horprasert[38] N/A N/A 72.82 88.90 84.03 92.35 80.58 69.37
Joshi[39] 88.21 97.00 91.02 97.66 N/A N/A N/A N/A
Jung[40] N/A N/A 97.67 86.21 85.84 95.1 87.69 92.18
Zhang[41] 67.17 90.19 88.63 88.97 86.28 92.64 87.95 97.74
Siala[42] 83.30 68.92 N/A N/A N/A N/A N/A N/A
Song[43] 76.86 80.52 N/A N/A N/A N/A N/A N/A
Martel.Brisson[44] 72.10 79.70 N/A N/A N/A N/A N/A N/A
Celik[45] 79.74 90.07 86.24 98.96 67.18 96.52 N/A N/A
Choi[46] 84.98 88.97 95.01 91.39 90.63 94.00 N/A N/A
Proposed method 79.29 98,78 93,15 96,39 95,39 98,92 95,55 96,68

Table 2. Comparison of the proposed method with quantitative evaluation

Shadow Detection Shadow Discrimination

Data Base Videos
Rate (η %) Rate (ξ %)
LR Our Method LR Our Method
Campus 89.63 95.55 92.93 96.68
HighwayI 87.93 79.29 93.25 98.78
DBA
Intelligentroom 90.19 93.15 88.14 96.39
Laboratory 87.13 95.39 92.67 98.92
Bootstrap 77.86 84.06 91.89 96.73
Campus 88.63 88.93 91.95 96.99
DBB
Fountain 88.91 90.73 89.94 97.46
Hall 86.27 89.66 82.91 95.07
Hallway 91.75 91.39 90.92 93.01
DBC
HighwayIII 66.60 79.08 89.94 94.68
Campus 88.2 96.77 77.88 97.98
DBD
Hallway 86.8 91.47 95.80 96.57
Highway 86.52 94.28 68.95 89.52
PublicPark 89.17 97.44 84.96 95.05
Average 87.93 91.37 88.00 95.98

Table 3. Comparison of the proposed method with LR method in both public and personal
video data base
Novel Approach for Detection and Removal of Moving Cast Shadows . . . 1015

Table 4 shows that our method consumes less time compared to the LR method
although both algorithms have the same operations complexity.
This is due to the extra steps required by the LR method to produce the candidate
shadow regions and to calculate the gradients for each pixel. For further comparison,
we computed the processing time.

LR (ms/frame) Our Method (ms/frame)

Campus 10.76 4.72
HighwayI 27.71 7.73
Intelligentroom 6.25 4.14
Laboratory 12.73 4.95
Bootstrap 7.35 2.56
Campus 8.69 2.76
Fountain 8.27 2.23
Hall 4.8 1.98
Hallway 13.59 4.13
HighwayIII 4.75 2.82
Campus 23.79 7.2
Hallway 22.07 5.67
Highway 22.66 7.01
PublicPark 22.43 6.44
Average 13.99 4.6

Table 4. Processing time calculated in milliseconds per frame for all used video databases

5 CONCLUSIONS

The paper proposes an effective method based on RGB, HSV and YUV color mod-
els to detect and remove shadows of moving objects in both indoor and outdoor
environment and without any prior assumptions on illumination conditions.
In our method, we first applied canny algorithm to detect the boundaries for
objects and shadows. This information is used later as a criterion to separate be-
tween attached and cast shadows. After extracting moving objects with an improved
Gaussian mixture model, we used three criteria calculated in RGB, HSV and YUV
color space to decide if a pixel belongs to shadow or to an object and to remove cast
shadows.
Through the experiments and analysis, our results show an enhancement in
detecting shadows compared to the existing methods. They also show the algorithm
ability to deal with textured surfaces and complex environments which fall beyond
the scope of numerous shadow detection methods.
The low complexity of the proposed algorithms can also save significant process-
ing time for probable use in real time applications. Unfortunately, our method may
fail to detect shadows since the latter is miss classed and considered as attached
shadows if the boundaries cannot be well defined. However, the performance can be
1016 B. Farou, H. Rouabhia, H. Seridi, H. Akdag

improved with a sophisticate edge detection method. Our conclusions are supported
by both quantitative and qualitative experiments on shadow detection carried out
on various video databases.

REFERENCES

[1] Prati, A.—Mikic, I.—Trivedi, M.—Cucchiara, R.: Detecting Moving Shad-

ows: Algorithms and Evaluation. IEEE Transactions on Pattern Analysis and Ma-
chine Intelligence, 2003, Vol. 25, No. 7, pp. 918–923.
[2] Al-Najdawi, N.—Bez, H. E.—Singhai, J.—Edirisinghe, E. A.: A Survey of
Cast Shadow Detection Algorithms. Pattern Recognition Letters, Vol. 33, 2012, No. 6,
pp. 752–764.
[3] Sanin, A.—Sanderson, C.—Lovell, B. C.: Shadow Detection: A Survey and
Comparative Evaluation of Recent Methods. Pattern Recognition, Vol. 45, 2012,
No. 4, pp. 1684–1695.
[4] Wang, N.—Lang, C.—Xu, D.: Image-Based Shadow Removal Via Illumination
Chromaticity Estimation. Proceedings of the Third International Conference on Mul-
timedia Information Networking and Security (MINES), 2011, pp. 33–36.
[5] Irie, K.—McKinnon, A.—Unsworth, K.—Woodhead, I.: An Investigation
into Noise-Bound Shadow Detection and Removal. Proceedings of the 23rd Inter-
national Conference Image and Vision Computing (IVCNZ), New Zealand, 2008,
pp. 1–6.
[6] McFeely, R.—Hughes, C.—Jones, E.—Glavin, M.: Removal of Non-Uniform
Complex and Compound Shadows from Textured Surfaces Using Adaptive Direc-
tional Smoothing and the Thin Plate Model. IET Image Processing, Vol. 5, 2011,
No. 3, pp. 233–248.
[7] Ahmad, K.—Mohd Noor, M.—Hussain, Z.—Idin, M.—Abdullah, N.: Im-
provement Moving Vehicle Detection Using RGB Removal Shadow Segmentation.
Proceedings of the IEEE International Conference on Control System, Computing
and Engineering (ICCSCE), 2011, pp. 22–26.
[8] Zhu, Z.—Lu, X.: An Accurate Shadow Removal Method for Vehicle Tracking. Pro-
ceedings of the International Conference on Artificial Intelligence and Computational
Intelligence (AICI), 2010, Vol. 2, pp. 59–62.
[9] Chen, C.-C.—Aggarwal, J.: Human Shadow Removal with Unknown Light
Source. Proceedings of the 20th International Conference on Pattern Recognition
(ICPR), 2010, pp. 2407–2410.
[10] Hafiz, F.—Shafie, A.—Khalifa, O.—Ali, M. H.: Foreground Segmentation-
Based Human Detection with Shadow Removal. Proceedings of the International
Conference on Computer and Communication Engineering (ICCCE), 2010, pp. 1–6.
[11] As’ari, M. A.—Sheikh, U.—Abu-Bakar, S. A. R.: Object’s Shadow Removal
with Removal Validation. Proceedings of the IEEE International Symposium on Sig-
nal Processing and Information Technology, 2007, pp. 841–845.
Novel Approach for Detection and Removal of Moving Cast Shadows . . . 1017

[12] Sun, B.—Li, S.: Moving Cast Shadow Detection of Vehicle Using Combined Color
Models. Proceedings of the Chinese Conference on Pattern Recognition (CCPR),
2010, pp. 1–5.
[13] Sofka, M.: Commentary Paper on “Shadow Removal in Indoor Scenes”. Proceedings
of the Fifth IEEE International Conference on Advanced Video and Signal Based
Surveillance (AVSS), 2008, pp. 299–300.
[14] Gallego, J.—Pardàs, M.: Enhanced Bayesian Foreground Segmentation Using
Brightness and Color Distortion Region-Based Model for Shadow Removal. Proceed-
ings of the 17th IEEE International Conference on Image Processing (ICIP), 2010,
pp. 3449–3452.
[15] Amin, R.—Gould, R.—Hou, W.—Arnone, R.—Lee, Z.: Optical Algorithm for
Cloud Shadow Detection over Water. IEEE Transactions on Geoscience and Remote
Sensing, 2013, Vol. 51, No. 2, pp. 732–741.
[16] Bian, J.—Yang, R.—Yang, Y.: A Novel Vehicle’s Shadow Detection and Removal
Algorithm. Proceedings of the 2nd International Conference on Consumer Electronics,
Communications and Networks (CECNet), 2012, pp. 822–826.
[17] Yu, H.-Y.—Sun, J.-G.—Liu, L.-N.—Wang, Y.-H.—Wang, Y.-D.: MSER
Based Shadow Detection in High Resolution Remote Sensing Image. Proceedings of
the International Conference on Machine Learning and Cybernetics (ICMLC), 2010,
Vol. 2, pp. 780–783.
[18] Song, X.—Ding, Y.—Gen, J.—Chen, Y.: Shadow Removal of Vehicles in a Video
System Based on RGB Chroma Model. Proceedings of the International Conference
on Computer Science and Software Engineering, 2008, Vol. 1, pp. 977–980.
[19] Yang, Q.—Tan, K.-H.—Ahuja, N.: Shadow Removal Using Bilateral Filtering.
IEEE Transactions on Image Processing, 2012, Vol. 21, No. 10, pp. 4361–4368.
[20] Makarau, A.—Richter, R.—Muller, R.—Reinartz, P.: Adaptive Shadow
Detection Using a Black Body Radiator Model. IEEE Transactions on Geoscience
and Remote Sensing, 2011, Vol. 49, No. 6, pp. 2049–2059.
[21] Panagopoulos, A.—Wang, C.—Samaras, D.—Paragios, N.: Simultaneous
Cast Shadows, Illumination and Geometry Inference Using Hypergraphs. IEEE
Transactions on Pattern Analysis and Machine Intelligence, 2013, Vol. 35, No. 2,
pp. 437–449.
[22] Xu, M.—Lu, L.—Jia, T.—Ren, J.—Smith, J.: Cast Shadow Removal in Motion
Detection by Exploiting Multiview Geometry. Proceedings of the IEEE International
Conference on Systems, Man, and Cybernetics (SMC), 2012, pp. 762–766.
[23] Fang, L. Z.—Qiong, W. Y.—Sheng, Y. Z.: A Method to Segment Moving Vehicle
Cast Shadow Based on Wavelet Transform. Pattern Recognition Letters, Vol. 29,
2008, No. 16, pp. 2182–2188.
[24] Leone, A.—Distante, C.: Shadow Detection for Moving Objects Based on Texture
Analysis. Journal of Pattern Recognition Society, Vol. 40, 2007, No. 4, pp. 1222–1233.
[25] Leone, A.—Distante, C.—Buccolieri, F.: A Shadow Elimination Approach
in Video-Surveillance Context. Pattern Recognition Letters, Vol. 27, 2006, No. 5,
pp. 345–355.
1018 B. Farou, H. Rouabhia, H. Seridi, H. Akdag

[26] Mamassian, P.—Knill, D. C.—Kersten, D.: The Perception of Cast Shadows.

Trends in Cognitive Sciences, Vol. 2, 1998, No. 8, pp. 288–295.
[27] Hedayati, M.—Zaki, W.—Hussain, A.: Real-Time Background Subtraction for
Video Surveillance: From Research to Reality. Proceedings of the 6th International
Colloquium on Signal Processing and Its Applications (CSPA), 2010, pp. 1–6.
[28] Farou, B.—Seridi, H.—Akdag, H.: A New Approach for the Extraction of Mov-
ing Objects. In: Amine, A., Otmane, A. M., Bellatreche, L. (Eds.): Modeling Ap-
proaches and Algorithms for Advanced Computer Applications. Springer Interna-
tional Publishing, Studies in Computational Intelligence, Vol. 488, 2013, pp. 27–36.
[29] Martel-Brisson, N.—Zaccarin, A.: Learning and Removing Cast Shadows
Through a Multidistribution Approach. IEEE Transactions on Pattern Analysis and
Machine Intelligence, Vol. 29, 2007, No. 7, pp. 1133–1146.
[30] Li, L.—Huang, W.—Gu, I.-H.—Tian, Q.: Statistical Modeling of Complex Back-
grounds for Foreground Object Detection. IEEE Transactions on Image Process, 2004,
Vol. 13, No. 11, pp. 1459–1472.
[31] Mikic, I.—Cosman, P.—Kogut, G.—Trivedi, M.: Moving Shadow and Ob-
ject Detection in Traffic Scenes. Proceedings of the 15th International Conference on
Pattern Recognition, 2000, Vol. 1, pp. 321–324.
[32] Haritaoglu, I.—Harwood, D.—Davis, L.: W4: Real-Time Surveillance of Peo-
ple and Their Activities. IEEE Transactions on Pattern Analysis and Machine Intel-
ligence, 2000, Vol. 22, No. 8, pp. 809–830.
[33] Cucchiara, R.—Grana, C.—Neri, G.—Piccardi, M.—Prati, A.: The
Sakbot System for Moving Object Detection and Tracking. In: Remagnino, P.,
Jones, G. A., Paragios, N., Regazzoni, C. S. (Eds.): Video-Based Surveillance Sys-
tems. Springer US, 2002, pp. 145–157.
[34] Stauder, J.—Mech, R.—Ostermann, J.: Detection of Moving Cast Shadows for
Object Segmentation. IEEE Transactions on Multimedia, 1999, Vol. 1, pp. 65–76.
[35] Salvador, E.—Cavallaro, A.—Ebrahimi, T.: Cast Shadow Segmentation Us-
ing Invariant Color Features. Computer Vision and Image Understanding, Vol. 95,
2004, No. 2, pp. 238–259.
[36] Martel-Brisson, N.—Zaccarin, A.: Moving Cast Shadow Detection from
a Gaussian Mixture Shadow Model. Proceedings of the IEEE Computer Society
Conference on Computer Vision and Pattern Recognition (CVPR), 2005, Vol. 2,
pp. 643–648.
[37] Al-Najdawi, N.—Bez, H.—Edirisinghe, E.: A Novel Approach for Cast Shadow
Modelling and Detection. Proceedings of the IET International Conference on Visual
Information Engineering (VIE), 2006, pp. 553–558.
[38] Horprasert, T.—Harwood, D.—Davis, L. S.: A Statistical Approach for Real-
Time Robust Background Subtraction and Shadow Detection. Proceedings of the
IEEE International Conference on Computer Vision FRAME-RATE Workshop, 1999,
pp. 1–19.
[39] Joshi, A.—Papanikolopoulos, N.: Learning to Detect Moving Shadows in Dy-
namic Environments. IEEE Transactions on Pattern Analysis and Machine, 2008,
Vol. 30, No. 11, pp. 2055–2063.
Novel Approach for Detection and Removal of Moving Cast Shadows . . . 1019

[40] Jung, C.: Efficient Background Subtraction and Shadow Removal for Monochro-
matic Video Sequences. IEEE Transactions on Multimedia, 2009, Vol. 11, No. 3,
pp. 571–577.
[41] Zhang, W.—Fang, X. Z.—Yang, X.—Wu, Q. M. J.: Moving Cast Shadows De-
tection Using Ratio Edge. IEEE Transactions on Multimedia, 2007, Vol. 9, No. 6,
pp. 1202–1214.
[42] Siala, K.—Chakchouk, M.—Chaieb, F.—Besbes, O.: Moving Shadow Detec-
tion with Support Vector Domain Description in the Color Ratios Space. Proceedings
of the 17th International Conference on Pattern Recognition (ICPR), 2004, Vol. 4,
pp. 384–387.
[43] Song, K.-T.—Tai, J.-C.: Image-Based Traffic Monitoring with Shadow Suppres-
sion. Proceedings of the IEEE, 2007, Vol. 95, No. 2, pp. 413–426.
[44] Martel-Brisson, N.—Zaccarin, A.: Kernel-Based Learning of Cast Shadows
from a Physical Model of Light Sources and Surfaces for Low-Level Segmentation.
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
(CVPR), 2008, pp. 1–8.
[45] Celik, H.—Ortigosa, A.—Hanjalic, A.—Hendriks, E.: Autonomous and
Adaptive Learning of Shadows for Surveillance. Proceedings of the Ninth Interna-
tional Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS),
2008, pp. 59–62.
[46] Choi, J.—Yoo, Y. J.—Choi, J. Y.: Adaptive Shadow Estimator for Removing
Shadow of Moving Object. Computer Vision and Image Understanding, Vol. 114,
2010, No. 9, pp. 1017–1029.

Brahim Farou received his Master’s degree in computer scien-

ce from Guelma University. He received D.Sc. in computer
science with distinction from the University of Badji Mokhtar,
Algeria. He is Associate Professor in Computer Science Depart-
ment and member of the LabSTIC laboratory at the University
of Guelma. His research interests include, video mining, human
behavior, the extraction of moving objects, shadow detection
and removal, image segmentation.

Houssam Rouabhia is a Ph.D. student in the Computer Scien-

ce Department and member of a LabSTIC laboratory at the
University of Guelma. His research interests include pattern
recognition, image segmentation and shadow detection.
1020 B. Farou, H. Rouabhia, H. Seridi, H. Akdag

Hamid Seridi received his Master’s degree from the Polytech-

nic Institute of New-York, USA in 1984 in electrical engineer-
ing. He received his Ph.D. in computer science with distinction
from the University of Reims, France. He was responsible of the
National Graduate School of Science and Information Technolo-
gies and Communication. From August 2005 through December
2010, he was Vice Dean of the Post Graduation, Scientific Re-
search and External Relations at the University of Guelma. Cur-
rently he is Professor and Director of Laboratory of Science and
Information Technologies and Communication (LabSTIC) at the
University of Guelma. He is an expert member at the national committee for evaluation
and accreditation national projects research. His research interests include approximate
knowledge management, pattern recognition and artificial intelligence, data mining, video
mining, machine learning and cryptography.

Herman Akdag received his Ph.D. and H.Dr. degrees from

Paris VI University in 1980 and 1992, respectively. He was As-
sistant Professor since 1980 and obtained a Full Professor po-
sition at Reims University (France) in 1995. Currently he is
Senior Researcher at LIASD laboratory at University of Paris
8, France. His research interests include fuzzy set theory and
machine learning approaches to decision-making, image classifi-
cation and image retrieval. He also works on approximate rea-
soning, fuzzy abduction, data mining, and user modelling.