Automatic Forest Fire Detection Based

on a Machine Learning and Image

Analysis Pipeline

João Alves1 , Christophe Soares1,2 , José M. Torres1,2(B) , Pedro Sobral1,2 ,

and Rui S. Moreira1,2,3
1
ISUS Unit, University Fernando Pessoa, Porto, Portugal
{joao.alves,csoares,jtorres,pmsobral,rmoreira}@ufp.edu.pt
2
LIACC, University of Porto, Porto, Portugal
3
INESC-TEC, FEUP - University of Porto, Porto, Portugal
http://isus.ufp.pt

Abstract. Forest fires can have devastating consequences if not detected

and fought before they spread. This paper presents an automatic fire
detection system designed to identify forest fires, preferably, in their early
stages. The system pipeline processes images of the forest environment
and is able to detect the presence of smoke or flames. Additionally, the
system is able to produce an estimation of the area under ignition so that
its size can be evaluated. In the process of classification of a fire image,
one Deep Convolutional Neural Network was used to extract, from the
images, the descriptors which are then applied to a Logistic Regression
classifier. At a later stage of the pipeline, image analysis and processing
techniques at color level were applied to assess the area under ignition.
In order to better understand the influence of specific image features in
the classification task, the organized dataset, composed by 882 images,
was associated with relevant image metadata (eg presence of flames,
smoke, fog, clouds, human elements). In the tests, the system obtained a
classification accuracy of 94.1% in 695 images of daytime scenarios and
94.8% in 187 images of nighttime scenarios. It presents good accuracy
in estimating the flame area when compared with other approaches in
the literature, substantially reducing the number of false positives and
nearly keeping the same false negatives stats.

Keywords: Forest fire detection · Computer vision · Deep learning ·

Feature extraction · Classification · Machine learning

1 Introduction
In southern Europe, and more specifically in the Mediterranean basin, the forest
fires are a top concern as the summer approaches. In fact, the risk period of
forest fires has been increasing in Portugal [1], regardless of the year season,
as a result of climate change fueled by global warming [2]. The consequences
c Springer Nature Switzerland AG 2019
Á. Rocha et al. (Eds.): WorldCIST'19 2019, AISC 931, pp. 240–251, 2019.
https://doi.org/10.1007/978-3-030-16184-2_24
 Automatic Forest Fire Detection Based on a Machine Learning 241

of forest fires can be devastating if not detected, combated and extinguished

in the initial phase [3]. According to [2], the prospect of an overall increase in
temperature by 2 ◦ C accompanied by the decrease in humidity levels will cause
higher levels of dryness in the environment. As a result, the risk of climate-
related forest fires is expected to increase in the future, particularly around the
Mediterranean. The use of an automatic forest fire detection system, targeted
for its initial phase, would be a desirable solution to mitigate this problem. Such
system could complement and enhance the current work of human observers on
fire detection. The use of automatic surveillance technologies is a very promising
option, backed-up by a significant number of studies presented in [4].
Among the available fire detection techniques, we emphasize the use of video
cameras as input source for automatic detection systems. The optical systems
may mimic the human eye looking for apparent signs of fire. However, the forest
does not present uniform landscape scenarios, which makes fire detection a very
challenging process. Dynamic phenomena such as reflections, cloud shadows and
fog, or even the presence of human apparatus or activities translate into more
demanding problems and increase the possibility of false alarms [5]. With these
problems in mind, the present work was carried out in order to implement a low
cost solution able to tackle these inherent challenges of forest fire detection with
the use of video cameras feeding machine learning algorithms.
The automatic fire detection process described in this article aims identify-
ing a fire situation. This ability requires two initial steps, first the extraction
of image descriptors which then feeds a posterior classification. Together, these
extraction and classification phases form the pipeline of a Deep Learning (DL)
model. The extraction of descriptors returns unique properties that characterize
an image by means of a numerical vector. This vector is the output of a Deep
Convolutional Neural Network (DCNN) module. The obtained descriptors are
then used to train a Machine Learning (ML) classifier. The aim is to carefully
plan the training process of the classification model, addressing particular situ-
ations in a variety of scenarios, so that the model is as best fitted as possible to
detect fires in real situations. On the final stage, after obtaining a fire classifica-
tion, the pipeline focuses on spotting the image areas with flames, through the
application of Computational Vision (CV) techniques.

2 State of the Art

In this section a more focused study is conducted for projects that propose the
use of mobile or static video cameras in fire detection. The systems presented
are divided in two classes: one based on CV techniques and other using DL
techniques. CV systems mostly seek to detect only flames or only smoke, while
in DL systems there is no such restriction.

2.1 Deep Learning Systems in Fire Detection

In [6] the best choice of optimizers, reduction functions and learning rates for
the DCNN Inception-V3 is analyzed. By default, Inception-V3 is implemented
242 J. Alves et al.

according to certain parameters, however, the existing alternatives have been

analyzed. They conclude that the best combination of parameters encompasses
the Adam optimizer, the function reduce max and a learning rate of 0.1. As
demonstrated in [7], it is proposed to use two DCNNs whose goal is to improve
detection accuracy by fine-tuning the fully connected layer of both. The two
DCNNs involved are VGG16 and Resnet50. The results demonstrate that by
adding Fully Connected layers at the end of networks, an accuracy improvement
of only 1% occurs. In [8], the best DCNN for the problem of detection of forest
fires is analyzed. As such, five DCNNs are studied: AlexNet, GoogLeNet, mod-
ified GoogLeNet, VGG13 and modified VGG13. After training, its concluded
that GoogLeNet (version prior to Inception-V3), has the best result, with 99%
accuracy, following the modified VGG13. On the other hand, the original VGG13
has the worst accuracy and the longest training time due to overfitting and the
large number of parameters.

Table 1. Comparison between DCNN based fire detection systems

System DCNN Flame Smoke Night Precision Dataset

[6] Inception-V3   ✗ 99.7% ++
[7] Resnet50 modified    92.15% ++
[7] VGG16 modified    91.18% ++
[8] GoogLeNet   n/a 99% ++
[8] AlexNet   n/a 94.8% ++
[8] VGG13 modified   n/a 96.2% ++

Most of the systems presented in Table 1 detect both fire flames and the
presence of smoke, however, some do not address nocturnal scenarios. All the
systems exhibit high precision (above 90%) which proves the ability of the DCNN
in the detection of forest fires. It is noted that all systems only mention the
adoption of a large dataset. They do not perform any analysis of the image set,
in order to understand what types of situations are used and which ones should
be added in order to make the dataset more comprehensive. Given that the
systems presented used different datasets, it is not fair to compare their relative
precision. Still, the best results in terms of accuracy were obtained in [6], which
is based on the DCNN Inception-V3. However, the dataset used does not address
several forest scenarios, small fires and night scenarios.

2.2 Computer Vision Systems in Fire Detection

Flame Detection: A method for detecting flames and smoke is proposed in [9].
It is suitable for daytime scenarios and is intended to be applied to drones. The
method consists in the use of a new color index called, FFDI, whose algorithm
only involves the manipulation of the RGB color space through the application
 Automatic Forest Fire Detection Based on a Machine Learning 243

of a set of color indexes. A method for flame detection based on image process-
ing is shown in [10]. Here, flame detection is achieved by manipulating RGB
and YCrCb color spaces. The algorithm consists of seven rules to be applied
sequentially. Another method is proposed in [11] which seeks to improve the
accuracy of flames detection through fixed surveillance cameras. The system
uses two parallel image streams, one for the detection of the flame and the other
for confirmation of the results. The analysis of the HSI, YCbCr and RGB color
spaces of dynamic textures and spatial-temporal frequency are used in the first
stream, while in the second stream a Background Subtraction algorithm is used
to detect and trace moving objects.

Smoke Detection: A method for smoke detection based on color and motion
analysis is presented in [12]. This system provides rapid detection in daytime
scenarios and is designed to allow multi-camera signal processing without large
computational complexities. The detection process involves signal preprocess-
ing, feature extraction and classification. In [13] a method is proposed based on
the segmentation of zones with movement, extraction of characteristics of the
smoke and, finally, a classifier. For the segmentation of motion zones, the Visual
Background extractor method is used. Next, the detection is based on certain
characteristics, such as the high frequency energy based on the wave transfor-
mation, the consistency and the direction of the smoke movement. Finally, in
[14], an approach is proposed to detect smoke through static surveillance cam-
eras. The Support Vector Machine classification template is used based on the
previously obtained data. They perform a preprocessing to reduce the noise and
to divide the image into small blocks. The idea is to determine the suspected
regions of smoke based on the detection of color and slow movements and, finally,
to analyze the regions through a co-occurrence matrix. The co-occurrence matrix
is aimed at detecting three characteristics that are representative of the presence
of smoke, such as contrast, energy and correlation.

Table 2. Comparison between CV based fire detection systems

System Camera Flame Smoke Night Cost Precision Dataset

[9] Mobile  ✗ ✗ + 96.82% +++
[10] Mobile  ✗ ✗ + 99% +
[11] Static  ✗ ✗ +++ 91.1% ++
[12] Static ✗  ✗ ++ 90% +++
[13] Static ✗  ✗ +++ 92.5% +
[14] Static ✗  ✗ ++ 87% +++

Table 2 summarises several analysed systems where the detection accuracy is

very good, some of them above 95%, as is the case of [9,10]. Again, given that
244 J. Alves et al.

each system uses a different dataset, it is not fair to directly compare their pre-
cision, but other features such as their cost of implementation or the quality of
applied techniques can be evaluated. Daytime flame detection systems [9–11], in
general, have a lower implementation cost than those that perform smoke detec-
tion [12–14]. This is because the process of identifying the smoke characteristics
is usually more complex than simply applying color rules, as is the case in some
systems [9,10] in the detection of flames.
Regarding the quality of the used dataset, that is, if it is vast and varied
in terms of situations, the [9,12,14] systems are the ones that best meet these
requirements.

3 Implementation of the Fire Detection Pipeline

The proposed fire detection system aims to fulfill, in sequence, two specific phases
over the collected images. First, to correctly recognize and classify a fire situation.
Second, to detect the flame areas present in the image classified with fire. The
whole process of detection is carried out autonomously from the moment it
receives an image until its final decision.

3.1 Dataset Description and Organization

Before deploying the fire detection system, a varied set of images was collected
and selected to cover a range of real world scenarios. The image selection process
was carried out carefully to train and test the classifier, and maximize the correct
classification of future image samples, containing images from a multitude of
landscape regions and seasons of the year. The set of relevant images was selected
from the Internet, and contained a total of 882 images organized into four classes,
as described in Table 3.

Table 3. Dataset classes statistics

Classes Number of images

Day + Fire 247
Day + No fire 448
Night + Fire 101
Night + No fire 86
882

In the construction of the dataset, specific image content characteristics were

taken into consideration in order to obtain a dataset that is most representative
of reality. This aimed also to increase the training set variability and, hopefully,
reduce the number of future false positives classified by the system. The selected
images contained several types of forest zones, daytime and nighttime scenarios
 Automatic Forest Fire Detection Based on a Machine Learning 245

and fires with several dimensions, among other features. In order to keep track of
all these variants, a metadata set of information was created, containing all the
characteristics associated to each image. Table 4 depicts the variables and asso-
ciated values considered in the collected metadata information. The advantage
of using this metadata is the possibility of performing various tests depending on
the desired characteristics. With this, one could verify, to what extent, a certain
characteristic negatively influences the performance of the classifier.

Table 4. Metadata associate with images

Variables Values
Mode Day; Night
Flames 0; 1
Smoke 0; 1
Clouds 0; 1
Fog 0; 1
Human elements 0; 1
Terrestrial surface type 0 (other); 1 (forest)
Vegetation color 0 (other); 1 (green)
Fire stage None; Initial; Advanced

The images have mainly, but not exclusively, an aerial perspective and were
taken away from the area of interest. Due to the adoption of this research crite-
rion, the images available for the collection were more restricted. In addition, it
was a concern to collect as many images as possible with initial fire, both for day
and night scenarios, since the main purpose is that the system is able to detect
small spots of fire. None of the images used were submitted to any preprocessing,
and therefore they possess their original varied resolutions.

3.2 System Pipeline

The automatic fire detection system proposed in this paper, uses a pipe and filter
architecture with three sequential modules, as illustrated in Fig. 1. Initially, the
images are acquired and then analyzed in sequence by the features extraction
module, followed by the classification module. At this point, a correct decision is
expected on the existence or not of a fire. In the last module, all images previously
classified as fire are subjected to the flame detection module. However, flames
are not always visible and detectable, which in such cases, leads to the conclusion
that the fire was detected due to the presence of smoke (Fig. 2).

3.3 Features Extraction

The extraction of descriptors from an image can be performed through a DCNN.
The chosen DCNN for this module was Inception-V3 which output vector serves
246 J. Alves et al.

Begin

Image
Yes Estimation of
Feature Flames
Images
Extraction detected
area

No

Image just
End Alert
with smoke

Fig. 1. Diagram of the fire detection system pipeline

Feature Extraction

Inception-V3

Fig. 2. Flowchart of the features extraction module

as the input basis for the next classification module. All DCNNs have a descriptor
extraction layer and a classification layer at the end of their network. However, it
is possible to disassociate the classification layer in order to test various learning
algorithms, to find the best solution. Briefly, this module extracts from each
image a numerical vector of features. Two different images will originate two
different vectors. To acquire a numerical representation of the images, they are
fed to the DCNN Inception-V3, which in this case will discard the classification
layer and consider only the output of the penultimate layer of the network.
This layer returns a vector with 2048 descriptive values of each image (cf. base
extracted features) (Fig. 3).

Fire !
Logistic
Regression
No Fire !

Fig. 3. Flowchart of the classification module

3.4 Classification
The classification module follows the descriptor extraction module, receiving as
input the 2048 descriptors of each image. The goal is now to obtain a binary
response, whether or not there is a fire in the image, provided the descriptors
supplied as input to the classifier. The classifier model was tuned by a training
stage. Generically speaking, the training followed a supervised learning model by
using a set of input images, for which, the output associated class (of each image)
was known (cf. training set). After testing several classifiers, it was concluded
 Automatic Forest Fire Detection Based on a Machine Learning 247

that the Logistic Regression (LR) model offered the better performance for our
purpose. However, since the set of images has both daytime and nighttime situ-
ations, and taking into account the significant differences between them, it was
decided to use separated classifiers for day and night images.

Flames are detected

If Fire PLT FFDI

Fig. 4. Flowchart of the flame area estimation module

3.5 Flame Area Estimation

The flame area estimation module builds on the previously developed CAFE
project [15]. In order to achieve a more efficient detection and, at the same
time, reduce the number of false positive situations, instead of using only the
FFDI color index, the CAFE approach introduces a preprocessing module, in the
Lab color space, which provides better results. This module, generically, aims to
estimate the area of visible flames in an image. Since the classification module
does not specify whether a fire was detected with smoke, with flames, or both,
in this module, such omission can be solved through the following rule: if flames
are detected in the image, then the image is composed of smoke and flames; if
no flames are detected then it may be presumed that the module is facing an
image where the fire signal is just smoke (Fig. 4).

4 Evaluation
With respect to the classification task of the system, two different evaluations
are carried out. On the one hand we intend to analyze the performance of the
classification model when trained with the whole set of images. On the other
hand, a second evaluation is carried out to study the influence of each character-
istic on the classification task, with the help of metadata. Finally, the system’s
ability to correctly detect flaming areas is analyzed.

4.1 Performance of Trained Models with the Entire Set of Images

The two classification models (day and night classifiers), were trained and eval-
uated with the whole set of images, by following the Cross Validation evaluation
methodology.
The evaluation of the day and night models culminated in an accuracy of
94.1% and 94.8% respectively. Given the number of images used in the training
stage, the results obtained are very promising and clearly demonstrate the model
capability to distinguish fire situations (Fig. 5).
248 J. Alves et al.

(a) Daily Model (b) Nightly Model

Fig. 5. Confusion matrices

4.2 Influence of Image Characteristics in the Classification Process

In order to analyze the influence of the characteristics present in the images, in

the classification process, a test was performed for each of the metadata repre-
sented in Table 4. The process consisted in, for each characteristic, to separate
on a 70/30 proportion, the images characterized by having or not that specific
characteristic.
For example, 70% of the fog images are separated for training while the
remaining 30% are reserved for testing purposes. Thus, the training set is com-
posed by the union of those 70% with the remaining dataset. In the end, the
30% of the fog images previously reserved, are used to test the performance of
the model with the evaluation of a specific feature in mind (Table 5).

Table 5. Influence of image characteristics in the classification process

Tested characteristic Negative influence?

Fog YES
Clouds NO
Humanized elements (day) NO
Open field landscapes NO
Varied tonality vegetation NO
Initial fire YES
Advanced fire NO
Humanized elements (night) YES

Through the tests performed with the metadata variables, it was verified
that, for daily scenarios, FP situations are especially due to the presence of fog.
FN situations, however, occur mainly in cases of very small initial fires. During
the night, it has been proven that artificial lighting contributes to the increase
in the number of FP cases.
 Automatic Forest Fire Detection Based on a Machine Learning 249

4.3 Estimation of the Area of the Flames

Considering that the CAFE approach was used to detect the flaming zones, the
tests performed attempted to compare the CAFE solution, that is, the PLT
component followed by the application of the FFDI block, versus the use of
FFDI only (FFDI [9] approach). The images used come from 4 daytime scenarios
with different characteristics, as shown in the Table 6, which allows to evaluate
the CAFE approach in different situations. The detailed methodology used is
described in a previous work [15].

Table 6. Characteristics of the considered daytime scenarios

A Little vegetation, rocky area

B Burned vegetation
C Dry brownish surface
D Housing area

Table 7. CAFE versus FDDI approach in daily scenarios (TP and FP results)

Flame area in source IMG TP FFDI TP CAFE FP FFDI FP CAFE

A 28 28 27 3 0
B 7 7 5 241 0
C 25 25 23 106 0
D 6 6 5 114 4

In Table 7 it is verified that in the scenarios presented we obtain similar

results as to the identified TPs, however, there is a noticeable difference in the
number of detected FPs. In this way we have been able to improve our flame
detection mechanism and the respective regions affected by the fire.

5 Conclusion

This work presents a system of automatic detection of forest fires, oriented to

operate continuously, day and night, and in different forest scenarios. Its devel-
opment took into account the possibility of being applied in mobile cameras, as
in drones, and static, as in watchtowers.
The architecture of the detection system starts by extracting descriptors from
an image. The choice of DCNN Inception-V3 was based on being the better one,
in terms of precision, when compared with others. In the classification process,
the supervised learning model LR demonstrated, when trained with the whole
set of images, a high detection accuracy in both daytime (94.1%) and nighttime
(94.8%) scenarios. However, there are cases of false negatives that should be
avoided because they can not occur when the system is actually used.
250 J. Alves et al.

The results obtained after tests with the metadata, clarify that the presence
of fog during the day, and of artificial illumination lights at night, are the char-
acteristics with greater responsibility for the occurrence of false positives by the
classification model. They further clarify that detecting small fire spots on an
image is a challenging task and requires a more voluminous training set.
The estimation of the area of the flames is the last step to be applied in
this process of detection of forest fires. The proposal used, the CAFE, is an
improvement of an index used in the detection of flames, the great difference
being translated into a smaller number of false positives detected.

Acknowledgements. This work was partially funded by: FCT-Fundação para a

Ciência e Tecnologia in the scope of the strategic project LIACC-Artificial Intelligence
and Computer Science Laboratory (PEst-UID/CEC/00027/2013); and by Fundação
Ensino e Cultura Fernando Pessoa.

References
1. Gabinete do Secretário de Estado das Florestas e do Desenvolvimento Rural,
Despacho n.o 9081-E/2017 de 13 de Outubro, Portugal (2017)
2. de Rigo, D., Libertá, G., Houston Durrant, T., Artés Vivancos, T., San-Miguel-
Ayanz, J.: Forest fire danger extremes in Europe under climate change: variability
and uncertainty, Luxembourg (2017)
3. EUFOFINET: Project Detection Synthesis of Good Practices, Zvolen
4. Li, M., Xu, W., Xu, K.E., Fan, J., Hou, D.: Review of fire detection technologies
based on video image. J. Theor. Appl. Inf. Technol. 49(2), 700–707 (2013)
5. Alkhatib, A.A.A.: A review on forest fire detection techniques. Int. J. Distrib. Sens.
Netw. 10(3), 597368 (2014)
6. Huttner, V., Steffens, C.R., da Costa Botelho, S.S.: First response fire combat: deep
leaning based visible fire detection. In: 2017 Latin American Robotics Symposium
(LARS) and 2017 Brazilian Symposium on Robotics (SBR), pp. 1–6 (2017)
7. Sharma, J., Granmo, O., Goodwin, M., Fidje, J.T.: Deep convolutional neural net-
works for fire detection in images. In: Engineering Applications of Neural Networks,
vol. 744, pp. 183–193. Springer International Publishing (2017)
8. Lee, W., Kim, S., Lee, Y.-T., Lee, H.-W., Choi, M.: Deep neural networks for wild
fire detection with unmanned aerial vehicle. In: 2017 IEEE International Confer-
ence on Consumer Electronics (ICCE), vol. 10132 LNCS, pp. 252–253 (2017)
9. Cruz, H., Eckert, M., Meneses, J., Martı́nez, J.-F.: Efficient forest fire detection
index for application in unmanned aerial systems (UASs). Sensors 16(6), 893
(2016)
10. Vipin, V.: Image processing based forest fire detection. Int. J. Emerg. Technol.
Adv. Eng. 2(2), 87–95 (2012)
11. Gomes, P., Santana, P., Barata, J.: A vision-based approach to fire detection. Int.
J. Adv. Rob. Syst. 11, 1–12 (2014)
12. Sedlak, V., Stopjakova, V., Brenkus, J.: A real-time method for smoke detection in
monitored forest areas. In: International Conference on Applied Electronics (AE),
pp. 1–6 (2017)
 Automatic Forest Fire Detection Based on a Machine Learning 251

13. Cai, M., Lu, X., Wu, X., Feng, Y.: Intelligent video analysis-based forest fires
smoke detection algorithms. In: 2016 12th International Conference on Natural
Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD), pp. 1504–
1508 (2016)
14. Luu-Duc, H., Vo, D.T., Do-Hong, T.: Wildfire smoke detection based on co-
occurrence matrix and dynamic feature. In: International Conference on Advanced
Technologies for Communications (ATC), pp. 277–281 (2016)
15. Alves, J., Soares, C., Torres, J., Sobral, P., Moreira, R.S.: Color algorithm for flame
exposure (CAFE). In: 2018 13th Iberian Conference on Information Systems and
Technologies (CISTI), pp. 1–7 (2018)