2009 - Fire Detectionin Video Sequences Using A Generic Color Model
2009 - Fire Detectionin Video Sequences Using A Generic Color Model
a r t i c l e in fo abstract
Article history: In this paper, a rule-based generic color model for flame pixel classification is proposed. The proposed
Received 8 December 2006 algorithm uses YCbCr color space to separate the luminance from the chrominance more effectively than
Received in revised form color spaces such as RGB or rgb. The performance of the proposed algorithm is tested on two sets of
7 May 2008
images, one of which contains fire, the other containing fire-like regions. The proposed method achieves
Accepted 8 May 2008
Available online 7 July 2008
up to 99% fire detection rate. The results are compared with two other methods in the literature and the
proposed method is shown to have both a higher detection rate and a lower false alarm rate.
Keywords: Furthermore the proposed color model can be used for real-time fire detection in color video sequences,
Fire detection and we also present results for segmentation of fire in video using only the color model proposed in this
Generic color model
paper.
Image processing
& 2008 Elsevier Ltd. All rights reserved.
1. Introduction grayscale and color video sequences. Krull et al. [5] used low-cost
CCD cameras to detect fires in the cargo bay of long range
Fire detection systems are one of the most important passenger aircraft. The method uses statistical features, based on
components in surveillance systems used to monitor buildings grayscale video frames, including mean pixel intensity, standard
and environment as part of an early warning mechanism that deviation, and second-order moments, along with non-image
reports preferably the start of fire. Currently, almost all fire features such as humidity and temperature to detect fire in the
detection systems use built-in sensors that primarily depend on cargo compartment. The system is commercially used in parallel
the reliability and the positional distribution of the sensors. The to standard smoke detectors to reduce the false alarms caused by
sensors should be distributed densely for a high precision fire the smoke detectors. The system also provides visual inspection
detector system. In a sensor-based fire detection system, coverage capability which helps the aircraft crew to confirm the presence or
of large areas in outdoor applications is impractical due to the absence of fire. However, the statistical image features are not
requirement of regular distribution of sensors in close proximity. considered to be used as part of a standalone fire detection
Due to the rapid developments in digital camera technology system. Most of the works on flame pixel classification in color
and video processing techniques, there is a big trend to replace video sequences are rule based. Chen et al. [1] used raw R, G, and B
conventional fire detection techniques with computer vision- information and developed a set of rules to classify the flame
based systems. In general computer vision-based fire detection pixels. Instead of using the rule-based color model as in Chen
systems employ three major stages [1–4]. First stage is the flame et al., Töreyin et al. [2] used a mixture of Gaussians in RGB space
pixel classification; the second stage is the moving object which is obtained from a training set of flame pixels. In a recent
segmentation, and the last part is the analysis of candidate paper, the authors employed Chen’s flame pixel classification
regions. This analysis is usually based on two figures of merit; method along with a motion information and Markov field
shape of the region and the temporal changes of the region. modeling of the flame flicker process [3]. Marbach et al. [6] used
The fire detection performance depends critically on the YUV color model for the representation of video data, where time
performance of the flame pixel classifier which generates seed derivative of luminance component Y was used to declare the
areas on which the rest of the system operates. The flame pixel candidate fire pixels and the Chrominance components U and V
classifier is thus required to have a very high detection rate and were used to classify the candidate pixels to be in the fire sector or
preferably a low false alarm rate. There exist few algorithms not. In addition to luminance and chrominance they have
which directly deal with the flame pixel classification in the incorporated motion into their work. They report that their
literature. The flame pixel classification can be considered both in algorithm detects less than one false alarm per week; however,
they do not mention the number of tests conducted. Homg et al.
[7] used HSI color model to roughly segment the fire-like regions
Corresponding author. Tel.: +90 392 6301436; fax: +90 392 3650240. for brighter and darker environments. Initial segmentation is
- elik).
E-mail address: [email protected] (T. C followed by removing lower intensity and lower saturation pixels
0379-7112/$ - see front matter & 2008 Elsevier Ltd. All rights reserved.
doi:10.1016/j.firesaf.2008.05.005
ARTICLE IN PRESS
in order to get rid of the spurious fire-like regions such as smoke. is no attempt to reduce the false positives and false negatives by
They also introduced a metric based on binary contour difference changing their threshold values. Celik et al. [4] used normalized
images to measure the burning degree of fire flames into classes RGB (rgb) values for a generic color model for the flame. The
such as ‘‘no fire’’, ‘‘small’’, ‘‘medium’’ and ‘‘big’’ fires. They report normalized RGB is proposed in order to alleviate the effects of
96.94% detection rate, together with results including false changing illumination. The generic model is obtained using
positives and false negatives for their algorithms. However, there statistical analysis carried out in r–g, r–b, and g–b planes. Due to
Fig. 1. Original RGB color images in column (a), and R, G, and B channels in columns (b)–(d), respectively.
ARTICLE IN PRESS
the distribution nature of the sample fire pixels in each 2. Classification of flame pixels
plane, three lines are used to specify a triangular region
representing the region of interest for the fire pixels. Therefore, Each digital color image is composed of three color planes: red,
triangular regions in respective r–g, r–b, and g–b planes are green, and blue (R, G, and B). Each color plane represents a color-
used to classify a pixel. A pixel is declared to be a fire pixel receptor in human eye working on different wavelength. The
if it falls into three of the triangular regions in r–g, r–b, and g–b combination of RGB color planes gives ability to devices to
planes. Even though normalized RGB color space overcomes to represent a color in digital environment. Each color plane is
some extent the effects of variations in illumination, further quantized into discrete levels. Generally 256 (8 bits per color
improvement can be achieved if one uses YCbCr color space which plane) quantization levels are used for each plane, for instance
makes it possible to separate luminance/illumination from white is represented by (R, G, B) ¼ (255, 255, 255) and black is
chrominance. represented by (R, G, B) ¼ (0, 0, 0). A color image consists of pixels,
In this paper we propose to use the YCbCr color space to where each pixel is represented by spatial location in rectangular
construct a generic chrominance model for flame pixel classifica- grid (x, y), and a color vector (R(x, y), G(x, y), B(x, y)) corresponding
tion. In addition to translating the rules developed in the RGB and to spatial location (x, y). Each pixel in a color image containing a
normalized rgb to YCbCr color space, new rules are developed in
YCbCr color space which further alleviate the harmful effects of
Table 1
changing illumination and improves detection performance. The
Mean values of R, G, and B planes of fire regions for images given in Fig. 2
flame pixel classification rates of the proposed system with new
rules and new generic chrominance model is compared with the Row index in Fig. 2 Mean of R Mean of G Mean of B
previously introduced flame pixel classification models. The
1 218 137 97
proposed model gives 99.0% correct flame pixel classification rate
2 152 84 75
with a 31.5% false alarm rate. This is a significant improvement 3 211 158 105
over other methods used in the literature.
Fig. 2. Original RGB images are given in column (a) and corresponding fire regions, manually labeled with green color, are given in column (b).
ARTICLE IN PRESS
fire blob (region containing fire), the value of Red channel is [1,4]. For instance in Fig. 1 column (a) shows samples of digital
greater than the Green channel, and the value of Green channel is color images, and columns (b)–(d) show R, G, and B color planes
greater than the value of Blue channel for the spatial location. (channels), respectively. It can be noticed from Fig. 1 that for the
Furthermore, the flame color has high saturation in Red channel fire regions, R channel has higher intensity values than the G
Fig. 3. RGB color images in column (a) and its Y, Cb, and Cr channels in columns (b)–(d), respectively.
ARTICLE IN PRESS
channel, and G channel has higher intensity values than the B where Y is luminance, Cb and Cr are ChrominanceBlue and
channel. ChrominanceRed components, respectively. The range of Y is
In order to explain this idea better, we picked sample images [16 235], Cb and Cr are equal to [16 240].
from Fig. 1(a), and segmented its fire pixels as shown in Fig. 2(b) For a given image, one can define the mean values of the three
with green color. Then we calculate mean values of R, G, and B components in YCbCr color space as
planes in the segmented fire regions of the original images. The
results are given in Table 1 for the images given in Fig. 2. It is clear 1X K
Y mean ¼ Yðxi ; yi Þ
that, on the average, the fire pixels show the characteristics that K i¼1
their R intensity value is greater than G and G intensity value is 1X K
greater than the B. Cbmean ¼ Cbðxi ; yi Þ
K i¼1
Even though RGB color space can be used for pixel classifica-
tion, it has disadvantages of illumination dependence. It means 1X K
Cr mean ¼ Crðxi ; yi Þ (2)
that if the illumination of image changes, the fire pixel classifica- K i¼1
tion rules can not perform well. Furthermore, it is not possible to
where (xi, yi) is the spatial location of the pixel, Ymean, Cbmean, and
separate a pixel’s value into intensity and chrominance. The
Crmean are the mean values of luminance, ChrominanceBlue, and
chrominance can be used in modeling color of fire rather than
ChrominanceRed channels of pixels, and K is the total number of
modeling its intensity. This gives more robust representation for
pixels in image.
fire pixels. So it is needed to transform RGB color space to one of
The rules defined for RGB color space, i.e. RXGXB, and
the color spaces where the separation between intensity and
RXRmean [4,1], can be translated into YCbCr space as
chrominance is more discriminate. Because of the linear conver-
sion between RGB and YCbCr color spaces, we use YCbCr color Yðx; yÞ4Cbðx; yÞ (3)
space to model fire pixels. The conversion from RGB to YCbCr color
space is formulated as follows [8]: Crðx; yÞ4Cbðx; yÞ (4)
2 3 2 32 3 2 3 where Y(x, y), Cb(x, y), and Cr(x, y) are luminance, Chrominance-
Y 0:2568 0:5041 0:0979 R 16
6 7 6 0:1482 Blue and ChrominanceRed values at the spatial location (x, y).
4 Cb 5 4
¼ 0:2910 7 6 7
0:4392 54 G 5 þ 4 128 7
6
5 (1) Eqs. (3) and (4) imply, respectively, that flame luminance should
Cr 0:4392 0:3678 0:0714 B 128 be greater than ChrominanceBlue and ChrominanceRed should
be greater than the ChrominanceBlue. Eqs. (3) and (4) can be
interpreted to be a consequence of the fact that the flame has
Table 2
Mean values of Y, Cb, and Cr planes of fire regions of images given in Fig. 2
saturation in red color channel (R). In Fig. 3, we show the RGB
images and its corresponding Y, Cb, and Cr channel responses for
Row index in Fig. 2 Mean of Y Mean of Cb Mean of Cr the images shown in Fig. 1. The validity of Eqs. (3) and (4) can
easily been observed for fire regions.
1 151 98 166
Similar to Table 1, we picked sample images from Fig. 1(a), and
2 125 114 158
3 160 97 155 segmented its fire pixels as shown in Fig. 2(b). Then we calculate
mean values of Y, Cb, and Cr planes in the segmented fire regions
Fig. 4. RGB input image and its Y, Cb, and Cr channels: (a) original RGB image, (b) Y channel, (c) Cb channel, and (d) Cr channel.
ARTICLE IN PRESS
of the original images. The results are given in Table 2 for the consisting of 1000 images. Fig. 5 shows a few samples from this
images given in Fig. 2. It is clear that, on the average, the fire pixels set. Note that in Fig. 5, there are a variety of images including ones
shows the characteristics that their Y color value is greater than Cb with changing illumination and lighting. Furthermore, the images
color value and Cr color value is greater than the Cb color value. are selected so that fire-like colored objects are also included in
Besides these two rules (Eqs. (3) and (4)), since the flame the set. For instance the Sun in the image that produces fire-like
region is generally the brightest region in the observed scene, the color. There are some images in the set which do not contain any
mean values of the three channels, in the overall image Ymean, fire. The image set consists of random images collected from the
Cbmean, and Crmean contain valuable information. For the flame internet. Images are from both indoor and outdoor environments.
region the value of the Y component is bigger than the mean Y The ROC curve for the image set is given in Fig. 6 where hand
component of the overall image while the value of Cb component segmented fire images are used in order to create the ROC curve.
is in general smaller than the mean Cb value of the overall image. The rules (2) through (6) are applied to hand segmented fire
Furthermore, the Cr component of the flame region is bigger than
the mean Cr component. These observations which are verified
over countless experiments with images containing fire regions
are formulated as the following rule:
(
1; if Yðx; yÞ4Y mean ; Cbðx; yÞoCbmean ; Crðx; yÞ4Cr mean
Fðx; yÞ ¼
0; otherwise
(5)
where F(x, y) indicates that any pixel which satisfies condition
given in Eq. (5) is labeled as fire pixel.
Fig. 4 shows the three channels for a representative image
containing fire in more detail. The rule in (5) can be easily verified.
It can easily be observed from the representative fire image
(Fig. 4(c) and (d)) that there is a significant difference between the
Cb and Cr components of the flame pixels. The Cb component is
predominantly ‘‘black’’ while the Cr component is predominantly
‘‘white’’. This fact is formulated as a rule as follows:
(
1; if jCbðx; yÞ Crðx; yÞjXt
F t ðx; yÞ ¼ (6)
0; otherwise
where t is a constant.
The value of t is determined using a receiver operating
characteristics (ROC) [9] analysis of Eq. (6) on an image set Fig. 6. Receiver operating characteristics for t.
images with different values of t changing from 1 to 100. For each that high false positive rate. Using this tradeoff, in our experi-
value of t, we calculate corresponding true and false positive rates ments the value of t is picked such that the detection rate is over
on the image set and tabulate it. The true positive is defined as the 90% and false alarm rate is less than 40% (point d) which
decision when an image contains a fire, and false positive is corresponds to t ¼ 40.
defined as the decision when an image contains no fire but In addition to the above rules a statistical analysis of
classified as having fire. The ROC curve consists of 100 data points chrominance information in flame pixels over a larger set of
corresponding to different t values and some of them are labeled images is performed. For this purpose a set of 1000 images,
in Fig. 6 with blue letters, i.e. a–e. For each point in the ROC curve containing fire at different resolutions are collected from the
there are three values; true positive rate, false positive rate, and t. Internet. Samples from this set are shown in Fig. 7. The
For instance, for the point labeled with a, the true positive rate is collected set of images has a wide range of illumination and
60%, false positive rate is 6% and corresponding t is 96. Using the camera effects. The fire regions in the 1000 images are manually
ROC curve, different values of t can be selected with respect to segmented and the histogram of a total of 16,309,070 pixels is
required true positive and false positive rates. created in the Cb–Cr chrominance plane. Fig. 8 shows the
Since fire detection systems should not miss any fire alarm, the distribution of flame pixels in Cb–Cr plane. The area containing
value of t should be selected so that systems true positive rate is flame pixels in Cb–Cr plane can be modeled using intersections of
high enough. It is clear from Fig. 6 that, high positive rate means three polynomials denoted by fu(Cr), fl(Cr), and fd(Cr). The
Fig. 7. Samples from set of images used in extracting chrominance model for fire-pixels.
Fig. 8. 3-D distribution of hand labeled flame pixels in Cb–Cr color plane and three polynomials, fu(Cr), fl(Cr), and fd(Cr), bounding the flame region.
ARTICLE IN PRESS
Fig. 9. Fire detection in a still image: (a) original image, (b) fire segmentation using only (3), (c) fire segmentation using only (4), (d) fire segmentation using only (5), (e) fire
segmentation using only (6), (f) fire segmentation using only (8), (g) fire segmentation using combination (3)–(6) and (8), and (h) segmented color image which consists of
fire.
Fig. 10. Demonstration of fire segmentation in four different video sequences (a–d) with frame numbers of 1, 11, 21, 31, 41, 51, 61, and 71 are selected for visualization.
ARTICLE IN PRESS
In Table 3, we have tabulated flame detection results with false We have demonstrated fire segmentation for outdoor uncon-
alarm rates. It is clear that the proposed model outperforms the trolled environment using four different video sequences
model which uses rgb values proposed by Celik et al. both in from Wildland Fire Operations Research Group in Canada [11].
detection rates and false alarm rate. The proposed model has The sequences consist of views of different forest fire scenes
better performance than the model proposed by Chen et al. which recorded from a helicopter. The segmentation results using the
operates in RGB color space. proposed generic color model are shown in Fig. 10. Each video
The performance improvement is expected since YCbCr color sequence consists of frames recorded consecutively. The model
space has the ability of discriminating luminance from chromi- defined in this paper is applied to each frame of the video
nance information. Since the chrominance dominantly represents sequences. Each frame and its corresponding binary map showing
information without effect of luminance, the chrominance-based fire pixels is shown in Fig. 10 where it is clear that the proposed
rules and the color model defined in the chrominance plane are color model robustly detects fire regions in the given video
more descriptive for flame behavior. sequences.
Fig. 11. Demonstration of fire segmentation in an indoor video sequence collected from Celik et al. [4].
ARTICLE IN PRESS