Article

A Forest Fire Identification System Based on Weighted

Fusion Algorithm
Jingjing Qian and Haifeng Lin *

College of Information Science and Technology, Nanjing Forestry University, Nanjing 210037, China
* Correspondence: [email protected]; Tel.: +86-25-85427827

Abstract: The occurrence of forest fires causes serious damage to ecological diversity and the safety
of people’s property and life. However, due to the complex forest environment, the changeable
shape of forest fires, and the uncertainty of flame color and texture, forest fire detection becomes
very difficult. Traditional image processing methods rely heavily on artificial features and are not
generally applicable to different forest fire scenes. In order to solve the problem of inaccurate forest fire
recognition caused by the manual extraction of features, some scholars use deep learning technology
to adaptively learn and extract forest fire features, but they often use a single target detection model,
and their lack of learning and perception makes it difficult for them to accurately identify forest fires
in a complex forest fire environment. Therefore, in order to overcome the shortcomings of the manual
extraction of features and achieve a higher accuracy of forest fire recognition, this paper proposes
an algorithm based on weighted fusion to identify forest fire sources in different scenarios, fuses
two independent weakly supervised models Yolov5 and EfficientDet, completes the training and
prediction of data sets in parallel, and uses the weighted boxes fusion algorithm (WBF) to process
the prediction results to obtain the fusion frame. Finally, the model is evaluated by Microsoft COCO
standard. Experimental results show that compared with Yolov5 and EfficientDet, the proposed
Y4SED improves the detection performance by 2.5% to 4.5%. The fused algorithm proposed in this
paper has better feature extraction ability, can extract more forest fire feature information, and better
Citation: Qian, J.; Lin, H. A Forest balances the recognition accuracy and complexity of the model, which provides a reference for forest
Fire Identification System Based on fire target detection in the real environment.
Weighted Fusion Algorithm. Forests
2022, 13, 1301. https://doi.org/ Keywords: forest fire identification; feature extraction; integrated learning; deep learning;
10.3390/f13081301 artificial intelligence
Academic Editors: Costantino Sirca
and Olga Viedma

Received: 2 July 2022

1. Introduction
Accepted: 12 August 2022
Published: 16 August 2022
In the world, forest fires occur frequently every year, which not only cause serious
economic losses and destroy the ecological environment, but also pose a certain threat to
Publisher’s Note: MDPI stays neutral human life and safety. Forest fires usually spread quickly and are difficult to control in a
with regard to jurisdictional claims in short time. Therefore, the real-time warning of forest fire sources can help people to put
published maps and institutional affil-
out the fire in the early stage of the fire, greatly reducing the cost and loss of firefighting.
iations.
However, the traditional forest fire source identification method has obvious shortcomings.
The detection system based on smoke sensors [1–3] has good performance in indoor spaces
and is only suitable for installation in places where there is burning and a lot of smoke,
Copyright: © 2022 by the authors.
but it is difficult to install outdoors. Infrared or ultraviolet detectors [4] are susceptible to
Licensee MDPI, Basel, Switzerland.
environmental interference, and given their short detection distance, they are not suitable
This article is an open access article for detecting large areas. Satellite remote sensing [5,6] is good at detecting large-area forest
distributed under the terms and fires, but cannot detect early regional fire detection.
conditions of the Creative Commons With the development of computer technology, more and more scholars use image
Attribution (CC BY) license (https:// processing technology to monitor forest fire sources. Chen et al. [7] combined RGB and his
creativecommons.org/licenses/by/ color criteria to segment the fire candidate area and identified whether there was a fire by
4.0/). studying the area change and centroid stability of the fire candidate area. Horng et al. [8]

Forests 2022, 13, 1301. https://doi.org/10.3390/f13081301 https://www.mdpi.com/journal/forests

Forests 2022, 13, 1301 2 of 11

used the interframe difference method and color masking technology to remove the false
fire area and obtain the suspected area of the fire in the color space model. Based on this,
they constructed a simple method to estimate the burning degree of the fire flame so as to
receive an appropriate early warning. Celik et al. [9] studied diverse video sequences and
images and proposed a fuzzy color model using statistical analysis. Combined with motion
analysis, this model can realize good discrimination of fire-like objects. In short, most
traditional fire detection methods based on image processing focus on creating artificial
features, such as flame color and texture, to detect fires [10,11].
The arrival of the era of artificial intelligence has made everything intelligent and
in-depth. Models based on convolutional neural networks have more advantages in feature
learning than traditional manual recognition, and the features they extract contain deeper
semantic information. Studies on forest fire identification based on deep learning have been
in progress at home and abroad. Zheng et al. [12] studied the feasibility of using Faster
R-CNN [13], YOLOv3 [14], SSD [15], and EfficientDet deep convolution neural network to
detect forest fire smoke. They found that YOLOv3 has a detection speed of up to 27 FPS
and better smoke detection accuracy. For fire detection tasks, most researchers habitually
use a single target detector to complete, such as use the improvement of a single model,
and few use the idea of integrated learning to solve the problem of missed detection in
actual fire detection. However, forest fire detection is a complex and spaced task, and it
is impractical to use a separate individual learner to detect fire in different scenes. Each
individual learner has its own expertise and can extract different features from images.
Yolov5 has the best detection performance in the Yolo series [16], with small depth and high
image reasoning speed. EfficientDet is a target detection model proposed by the Google
brain team in 2020. Under extensive resource constraints, it is always more efficient than
existing technologies [17]. In the eight models of the Efficientdet series D0~D7, with the
gradual improvement of the accuracy of the network, the computational time complexity
and spatial complexity will increase accordingly. Therefore, this paper proposes a new
forest fire detection method based on the weighted fusion algorithm, which integrates
Yolov5s and EfficientDet-D2, two single-stage models with higher real-time performance,
which can significantly improve the robustness of the model and improve the detection
performance, so as to effectively solve the problem of missing forest fire detection.

2. Materials and Methods

2.1. Experimental Environment
For details about how to configure the experimental environment, see Table 1. Before
the training data, the experimental data set was divided into training set and test set
according to 9:1. Only the training set participates in the actual model training process, and
the test set is only used to evaluate the accuracy of the model.

Table 1. Experimental environment configuration.

Experimental Environment Configuration Parameters

Programming language Python3.8
Deep learning framework PyTorch1.7.1
GPU NVIDIA GeForce RTX 3060
GPU accelerating package CUDA: 11.0
Operating system Windows10
CPU processor AMD Ryzen 7 5800H

2.2. Data Set

The data set established in this paper includes various forest fire images and images
containing fire disruptors (sun). In order to ensure that our model can handle complex
forest fire sources (ground fire, trunk fire, and canopy fire), in addition to obtaining data
sets from the open-source fire data set website BoWFire [18] and others, we also used web
crawler technology to obtain forest fire images from the Internet. The obtained data set
 2.2. Data Set
The data set established in this paper includes various forest fire images and images
containing fire disruptors (sun). In order to ensure that our model can handle complex
Forests 2022, 13, 1301
forest fire sources (ground fire, trunk fire, and canopy fire), in addition to obtaining 3dataof 11

sets from the open-source fire data set website BoWFire [18] and others, we also used web
crawler technology to obtain forest fire images from the Internet. The obtained data set
wasthen
was thenmanually
manuallyfiltered,
filtered,and
andaaforest
forestfire
firedata
dataset
setcontaining
containing2976
2976images
imageswas
wascreated.
created.
Representativesamples
Representative samplesfrom
fromthe
theforest
forest fire
fire source
source image
image data
data set
set are
are randomly
randomly shown
shown
belowin
below inFigure
Figure 1.
1.

(a) (b)

(c) (d)
Figure
Figure1. 1. Representative
Representativeimages
imagesfrom
fromthe
theforest
forestfire
firedata
dataset
setincluding
including(a)
(a)ground
groundfires
fires(b)
(b)ground
ground
fires (c) trunk fires, and (d) sun distractor.
fires (c) trunk fires, and (d) sun distractor.

2.3.Integrated
2.3. IntegratedLearning
Learning
Indeep
In deeplearning,
learning,our
ourgoal
goalis is
toto train
train a model
a model withwith good
good performance
performance andand strong
strong ro-
robustness,
bustness, butbut
thethe actual
actual situation
situation is isnot
notso;
so;usually,
usually,different
differentindividual
individuallearners
learnersshow
show
theirown
their own“preferences”
“preferences” for
for feature
feature learning.
learning. Ensemble
EnsembleLearning
Learning[19]
[19]isisthe
theintegration
integrationofof
multiple weak supervision models with “preferences” to obtain a more
multiple weak supervision models with “preferences” to obtain a more efficient strong efficient strong
supervision model. Tables
supervision Tables2–42–4show
show thethe
principles. In addition,
principles. In addition, 𝑚 indicates
mi indicates the iththe
model.
ith
model.
Table 2. Positive effects of integration.
Table 2. Positive
Model effects of integration.
Test Case 1 Test Case 2 Test Case 3
Model
m1 Test Case
3 1 Test Case
3 2 Test Case
5 3
m12
m 3 5 3
m 5 3 3
m23   
Integration
m3 3 3 3
Integration   
Table 3. Integration does not work.

Model Test Case 1 Test Case 2 Test Case 3

m1 3 3 5
m2 3 3 5
m3 3 3 5
Integration 3 3 5
Forests 2022, 13, 1301 4 of 11

Table 4. Integration has a “negative effect”.

Model Test Case 1 Test Case 2 Test Case 3

m1 3 5 5
m2 5 3 5
m3 5 5 3
Integration 5 5 5

Therefore, ensemble learning generally generates several individual learners first and
then adopts some strategy to combine them effectively [20]. In order to maximize the
integration effect, the integrated individual learner should be homogenous. The greater the
difference and accuracy of the individual learner are, the better the integration effect will
be. In other words, the integration in Table 1 plays a “positive role”.
In view of the reality of forest fire detection, algorithm accuracy and real-time require-
ments are high, and forest fire target size is different. Therefore, we chose the single-stage
model with higher real-time performance, and its typical representatives are Yolo, SSD,
and EfficientDet.
1. According to the network depth and network width, Yolov5 can be divided into
Yolov5s, Yolov5m, Yolov5l, and Yolov5x. The Yolov5s network was used in this paper.
The depth of Yolov5s network is the smallest in the Yolov5 series [21]. The image
inference speed of the Yolov5s model reaches 455FPS, which is widely used by a large
number of scholars with this advantage.
2. SSD is another single-order target recognition model after Yolo. It uses the method of
direct regression bbox and classification probability in Yolo. At the same time, it also
refers to Fast R-CNN and uses anchor extensively to improve the recognition accuracy.
It has the advantages of high precision and high real-time, but its recognition effect
on small targets is general.
3. Limited by hardware computing resources, Efficientdet-D2 was used for the experi-
ment in this paper. For forest fire detection, the advantage of the EfficientDet model is
that it has different trunk network, feature fusion network, and output network [22],
which can select different detection frameworks according to the cost performance of
software and hardware and the actual requirements for accuracy and efficiency in the
real environment, so as to design a more efficient forest fire detector.

2.4. Fusion Model Y4SED

The traditional method of filtering prediction boxes is Non-Maximum Suppression
(NMS). The filtering process of boxes depends on the selection of this single threshold
IOU [23]. However, different thresholds may affect the final results of the model. If multiple
objects exist side by side, one of them will be deleted. The NMS discards the redundancy
box and therefore cannot effectively generate average local predictions from different
models. It can be seen from Figure 2 that unlike NMS, the WBF [24] algorithm uses the
confidence (score) of all prediction boxes to construct the fused box.
Taking two prediction boxes as an example, the calculation method of the weighted
box generated by the fusion of two prediction boxes is described. Assuming that the two
prediction boxes are bboxA : [ Ax1, Ay1, Ax2, Ay2, As]and bboxB : [ Bx1, By1, Bx2, By2, Bs],
( Ax1, Ay1) represent the coordinates of the upper left corner of the bboxA frame; ( Ax2, Ay2)
represent the coordinates of the lower right corner of the bboxA frame, and As represents
the confidence degree of the bboxA frame. The same is true for bboxB. bboxC is obtained
by combining bboxA and bboxB, as shown in Figure 3:
Forests 2022, 13, x FOR PEER REVIEW
Forests 2022, 13, 1301 5 of 11

Figure 2. Schematic diagram of WBF and NMS processing multiple prediction results. The red box
represents the real red box, and the blue box represents the prediction made by multiple models.

Taking two prediction boxes as an example, the calculation method of the weighted
box generated by the fusion of two prediction boxes is described. Assuming that the two
prediction boxes are 𝑏𝑏𝑜𝑥𝐴: 𝐴𝑥1, 𝐴𝑦1, 𝐴𝑥2, 𝐴𝑦2, 𝐴𝑠 and 𝑏𝑏𝑜𝑥𝐵: 𝐵𝑥1, 𝐵𝑦1, 𝐵𝑥2, 𝐵𝑦2, 𝐵𝑠 ,
𝐴𝑥1, 𝐴𝑦1 represent the coordinates of the upper left corner of the 𝑏𝑏𝑜𝑥𝐴 frame;
𝐴𝑥2, 𝐴𝑦2 represent the coordinates of the lower right corner of the 𝑏𝑏𝑜𝑥𝐴 frame, and
𝐴𝑠 represents the confidence degree of the 𝑏𝑏𝑜𝑥𝐴 frame. The same is true for 𝑏𝑏𝑜𝑥𝐵.
Schematic diagram of WBF
ofand NMS processing multiple prediction
𝑏𝑏𝑜𝑥𝐶 is obtained by
Figure
Figure 2.2.
combining 𝑏𝑏𝑜𝑥𝐴
Schematic diagramand 𝑏𝑏𝑜𝑥𝐵,
WBF and NMS
as processing
shown 3:results.
multiple
in Figure The red results.
prediction box The
represents
represents thethe
realreal
red box,
red and
box,the blue
and boxblue
the represents the prediction
box represents themade by multiple
prediction mademodels.
by multiple m

Taking two prediction boxes as an example, the calculation method of the w

box generated by the fusion of two prediction boxes is described. Assuming that
prediction boxes are 𝑏𝑏𝑜𝑥𝐴: 𝐴𝑥1, 𝐴𝑦1, 𝐴𝑥2, 𝐴𝑦2, 𝐴𝑠 and 𝑏𝑏𝑜𝑥𝐵: 𝐵𝑥1, 𝐵𝑦1, 𝐵𝑥2, 𝐵
𝐴𝑥1, 𝐴𝑦1 represent the coordinates of the upper left corner of the 𝑏𝑏𝑜𝑥𝐴
𝐴𝑥2, 𝐴𝑦2 represent the coordinates of the lower right corner of the 𝑏𝑏𝑜𝑥𝐴 fram
𝐴𝑠 represents the confidence degree of the 𝑏𝑏𝑜𝑥𝐴 frame. The same is true for
𝑏𝑏𝑜𝑥𝐶 is obtained by combining 𝑏𝑏𝑜𝑥𝐴 and 𝑏𝑏𝑜𝑥𝐵, as shown in Figure 3:

Figure 3. Process of two prediction boxes fusing into one box through the fusion box formula.
Figure 3. Process of two prediction boxes fusing into one box through the fusion box formula.
The value calculated by Formulas (1) and (2) constitutes the upper-left coordinate of
the bboxC
The value calculated byfusion box; the (1)
Formulas value calculated
and by Formulas
(2) constitutes (3) upper-left
the and (4) constitutes the lower-
coordinate of
right coordinate of the bboxC fusion box, and the confidence of the bboxC box is calculated
the 𝑏𝑏𝑜𝑥𝐶 fusion box; the value
by Formula (5). calculated by Formulas (3) and (4) constitutes the lower-
right coordinate of the 𝑏𝑏𝑜𝑥𝐶 fusion box, Cx1 and=theAx1confidence
× As + Bx1 ×of Bs the 𝑏𝑏𝑜𝑥𝐶 box is calcu-
(1)
lated by Formula (5). As + Bs
Ay1 × As + By1 × Bs
𝐴𝑥1Cy1𝐴𝑠 = + 𝐵𝑥1 𝐵𝑠
As + Bs
(2)
𝐶𝑥1 = one box through the fusion (1)
Figure 3. Process of two prediction 𝐴𝑠
Cx2 =
+
Ax2 𝐵𝑠
boxes fusing
× As + Bx2into
× Bs box form
(3)
As + Bs
The value𝐶𝑦1
𝐴𝑦1 by𝐴𝑠
calculated
+ 𝐵𝑦1 (1)𝐵𝑠
Formulas and
= Ay2 × As + By2 × (2)
Bs constitutes the upper-left (2) coord
Cy2𝐴𝑠= + 𝐵𝑠 (4)
the 𝑏𝑏𝑜𝑥𝐶 fusion box; the value calculated As + Bs by Formulas (3) and (4) constitutes th
right coordinate of the 𝐴𝑥2𝑏𝑏𝑜𝑥𝐶 𝐴𝑠Cs + =𝐵𝑥2
fusionAsbox,𝐵𝑠and the confidence of the 𝑏𝑏𝑜𝑥𝐶
+ Bs box i
lated by Formula 𝐶𝑥2(5).= 2
(5)
(3)
𝐴𝑠 + 𝐵𝑠
This paper proposes a new Yolov5-S and Efficientdet-D2 integration model, Y5SED, to
𝐴𝑦2 + 𝐵𝑦2by𝐴𝑥1
𝐴𝑠 caused 𝐵𝑠 𝐴𝑠 + 𝐵𝑥1 𝐵𝑠
solve the problem of missing
𝐶𝑦2 =
detection 𝐶𝑥1 = a single weakly supervised learning model (4)
and ensure that our model has better 𝐴𝑠 +robustness
𝐵𝑠 𝐴𝑠 + 𝐵𝑠
to different scenarios. Figure 4 shows
that this paper fused two separate weakly supervised
𝐴𝑦1 models,
𝐴𝑠 + 𝐵𝑦1 Yolov5-S
𝐵𝑠 and Efficientdet-
𝐴𝑠 +𝐶𝑦1
D2, to train forest fire data sets, saved 𝐵𝑠predicted
the = results in Jason files, and then used
𝐶𝑠 =
the weighted boxes fusion algorithm 2 (WBF) to fuse the𝐴𝑠 + 𝐵𝑠 boxes of the two mod-
prediction
(5)
𝐴𝑥2 𝐴𝑠 + 𝐵𝑥2 𝐵𝑠
𝐶𝑥2 =
This paper proposes a new Yolov5-S and Efficientdet-D2 integration model, Y5SED,
𝐴𝑠 + 𝐵𝑠
to solve the problem of missing detection caused by a single weakly supervised learning
𝐴𝑦2 𝐴𝑠 + 𝐵𝑦2 𝐵𝑠
𝐶𝑦2 =
 model and ensure that our model has better robustness to different scenarios. Figure 4
shows that this paper fused two separate weakly supervised models, Yolov5-S and Effi-
Forests 2022, 13,cientdet-D2,
1301 to train forest fire data sets, saved the predicted results in Jason files, 6and of 11

then used the weighted boxes fusion algorithm (WBF) to fuse the prediction boxes of the
two models. Experiments show that this method can greatly improve the detection accu-
els. Experiments show that this method can greatly improve the detection accuracy of
racy of the model.
the model.

Figure 4. Integration
Figuremodel architecture
4. Integration diagram. diagram.
model architecture

2.5. Evaluation Indicators

2.5. Evaluation Indicators
In order to verify the model proposed in this paper, Microsoft COCO [25], the most
In order toauthoritative
verify thestandard
model recognized
proposedininthethis
fieldpaper,
of targetMicrosoft
recognition,COCO [25], tothe
was adopted most
evaluate
authoritative standard
the model recognized in the
(as shown in Table 5).field of target recognition, was adopted to eval-
uate the modelTable
(as shown in Table 5).
5. Accuracy and recall under Microsoft COCO standard.

Average
Table 5. Accuracy and recall under Microsoft COCO accuracy (AP)
standard.
AP0.5 The average accuracy when IOU = 0.5
Average accuracy at multiple scales
Average accuracy (AP)
APS AP0.5 of small target (size < 322 )
AP0.5 APM The average
AP0.5 ofaccuracy
medium target when (322 IOU
< size = 0.5
< 962)

APLscales AP0.5 of big target (size > 96 ) 2

Average accuracy at multiple
Average recall rate (AR)
APS AR0.5 AP0.5Theof average
small target (size
recall rate when<IOU322=) 0.5
Average
APM recall rate at multiple scales AP0.5 of medium target (322 < size <2 962)
ARS AR0.5 of small target (size < 32 )
APL ARM APAR0.5 of big target (size 2> 962)
0.5 of medium target (32 < size < 96 )
2
2)
Average recall rate (AR)L
AR AR 0.5 of big target (size > 96
AR0.5 The average recall rate when IOU = 0.5
Average recall The
rateAverage Precision
at multiple (AP) value in Table 6 is the area enclosed by the P–R curve. P is
scales
accuracy rate; R is recall rate. The IOU threshold is 0.5. Generally, the larger the2 value is,
ARS AR0.5 of small target (size < 32 )
the better the model learning effect is and vice versa.
ARM AR0.5 of medium target (322 < size < 962)
Table 6. Details of model training.
ARL AR0.5 of big target (size > 962)
Model Training Test Optimizer Vector Batch Size Number of Iterations
Yolov5-S The2678
Average Precision
298 (AP)SGD
value
[26] in
Table
1 ×6 10
is−the
2 area enclosed
12 by the P–R
300curve. P
−4
is accuracy rate; R is recall rate. The IOU
EfficientDet-D2 2678 298 AdamW [27]threshold is 0.5. Generally, the larger300the value
1 × 10 12
is, the better the model learning effect is and vice versa.
Accuracy (P) measures the percentage of positive predictions among all positive
Table 6. Details of model In
samples. training.
the forest fire identification task, the recall rate represents the ratio of the
number of correctly predicted forest fire images ( TP) to the total number of predicted forest
l Training Test Optimizer
fire images ( TP +Vector
FP), as shown Batch Size (6).
in Formula Number of Iterations
-S 2678 298 SGD [26] 1 × 10−2 12 TP 300
et-D2 2678 298 AdamW [27] 1 × 10−4 12P = TP + FP 300 (6)

The recall rate (R) measures the percentage of all forecast samples that are forecast
Accuracy to(P)bemeasures thea percentage
positive. On of positive
forest fire identification predictions
mission, among
the accuracy allispositive
rate sam-
the ratio of the
ples. In the forest fire identification task, the recall rate represents the ratio of the number
of correctly predicted forest fire images 𝑇𝑃 to the total number of predicted forest fire
images 𝑇𝑃 + 𝐹𝑃 , as shown in Formula (6).
𝑇𝑃
𝑃= (6)
𝑇𝑃 + 𝐹𝑃
Forests 2022, 13, x FOR PEER REVIEW 7 of 11

Forests 2022, 13, 1301 7 of 11

The recall rate (R) measures the percentage of all forecast samples that are forecast to
be positive. On a forest fire identification mission, the accuracy rate is the ratio of the
number of forest fire images correctly predicted by the model 𝑇𝑃 to all predicted im-
number
ages 𝑇𝑃of+forest
𝐹𝑁 . This
fire images
index iscorrectly
shown in predicted
Formulaby(7).the model ( TP) to all predicted images
( TP + FN ). This index is shown in Formula (7).
𝑇𝑃
𝑅= (7)
𝑇𝑃TP + 𝐹𝑁
R= (7)
TP + FN
3. Experimental Results and Analysis
3. Experimental Results and Analysis
3.1.
3.1. Parameter
ParameterSetting
Setting
For
For details of
details of the
the Yolov5
Yolov5 model
model and and EfficientDet
EfficientDet model
model training
trainingin inthe
theexperiment,
experiment,
please refer to Table
please refer to Table 6. 6.
In
Inthe
the WBF
WBF algorithm,
algorithm, the IOU threshold
the IOU threshold isisset
setto
to0.5,
0.5,and
andthe
theresults
resultsofofthe
thetwo
twomodels
mod-
els
areare given
given thethe same
same weight,
weight, thatthat is, the
is, the weights
weights of both
of both models
models areare
set set
to 1.to 1.

3.2.
3.2. Experimental
ExperimentalAnalysis
Analysis
Through experimental observations,
Through experimental observations,we wefound
foundthatthat Yolov5
Yolov5 is better
is better at large-target
at large-target fires
fires as shown
as shown in Figure
in Figure 5a,b,sometimes
5a,b, but but sometimes
missesmisses
objectsobjects
(Figure(Figure 6). Meanwhile,
6). Meanwhile, even
even though
though EfficientDet
EfficientDet is not sensitive
is not sensitive to large-target
to large-target fires
fires (Figure (Figure
5c,d), 5c,d),careful
it is more it is more
than careful
YoloV5
than YoloV5
and can andmore
identify can identify more fires.
fires. Therefore, we Therefore,
believe thatwe thebelieve that the
combination of combination
these two weakof
monitoring models with different specialties can effectively solve the
these two weak monitoring models with different specialties can effectively solve the problem of missing
forest fireofdetection.
problem missing forest fire detection.

(a) (b)

(c) (d)
Figure
Figure5.
5.(a)
(a)True
True positive predictions generated
positive predictions generatedbybyYolov5;
Yolov5;(b)(b)True
Truepositive
positive predictions
predictions generated
generated by
by Yolov5; (c) EfficientDet false positive prediction; (d) EfficientDet-generated false positive predic-
Yolov5; (c) EfficientDet false positive prediction; (d) EfficientDet-generated false positive prediction.
tion.
Therefore, this paper adopts the Yolov5-S and EfficientDet-D2 model to complete
forest fire identification. The yolov5-S model is extremely fast, so the time complexity of the
algorithm will be reduced, but the experiment shows that the learning is not as efficient as
EfficientDet-D2. Yolov5 is better at detecting large targets, and EfficientDet is more careful
than Yolov5. Therefore, the combination of the two can train a strong supervision model
with better robustness, which can effectively solve the problem of missing detection in
forest fire identification.
Forests 2022,
Forests 13, x1301
2022, 13, FOR PEER REVIEW 8 8ofof11
11

(a) (b)

(c) (d)
Figure
Figure 6.
6. EfficientDet
EfficientDet isisaamore
more“careful”
“careful” target
target detector
detector than
than Yolov5.
Yolov5. (a) Yolov5
(a) Yolov5 missed
missed a flame
a flame target;
target; (b) Yolov5 missed two flame targets; (c) EfficientDet detection of all flame areas; (d)
(b) Yolov5 missed two flame targets; (c) EfficientDet detection of all flame areas; (d) EfficientDet Effi-
cientDet detected five flame targets.
detected five flame targets.

Therefore, this
3.3. Experimental paper adopts the Yolov5-S and EfficientDet-D2 model to complete
Analysis
forestAccording
fire identification.
to Table 7, The yolov5-Swith
compared model
the is extremelynon-maximum
traditional fast, so the time complexity
suppression of
algo-
the algorithm will be reduced, but the experiment shows that
rithm (NMS) and the screening prediction frame method for forest fire identification, the the learning is not as effi-
cient as EfficientDet-D2.
integrated model (Y4SED)Yolov5 is better
established at detecting
by using the weighted largeboxestargets,
fusion andalgorithm
EfficientDet is
(WBF)
more
in thiscareful
paper than Yolov5.
obtains a higherTherefore,
averagethe combination
accuracy in forest of the
firetwo can train aand
identification strong super-
effectively
vision
solves model with better
the problem of missed robustness,
detectionwhich canfire.
of forest effectively
In addition, solve thethe problem ofresults
experimental missingin
detection in forest fire identification.
Table 8 show that compared with the original individual model (Yolov5 and EfficientDet),
the integrated model proposed in this paper (Y4SED) has significantly improved all indexes.
3.3.
WhenExperimental Analysis is 0.5, the AP value of forest fire recognition by the integrated
the IOU threshold
model can reach
According to 87%.
TableYolov5-S
7, compared and with
Efficientdet-D2
the traditional were improved by suppression
non-maximum 4.5% and 2.5%, al-
respectively, compared with a single weakly supervised learner.
gorithm (NMS) and the screening prediction frame method for forest fire identification, At the same time, the
integration
the integrated model
model in (Y4SED)
values ofestablished
AP0.5 , APSby, AP M , AP
using theL , weighted
AR0.5 , ARboxes S , ARM , and algorithm
fusion ARL have
significant
(WBF) in thisadvantages.
paper obtains Therefore,
a higherconsidering the result
average accuracy of thefire
in forest experiment,
identificationin terms of
and ef-
detectionsolves
fectively accuracythe and
problem algorithm operation
of missed spaceofcomplexity,
detection forest fire. this article proposes
In addition, the
the experi-
Y4SED results
mental model, in which
Table has a stronger
8 show anti-interference
that compared with the ability
originaland is better able
individual to distinguish
model (Yolov5
between complex background and different kinds of forest
and EfficientDet), the integrated model proposed in this paper (Y4SED) has significantly fires (small fire, medium fire,
large fire, strip-type fire, surface trunk, crown fire, and fire at
improved all indexes. When the IOU threshold is 0.5, the AP value of forest fire recogni- night). In addition, there will
be no
tion byfalse detection in
the integrated modelthe face of interfering
can reach objectsand
87%. Yolov5-S similar to fire (suchwere
Efficientdet-D2 as the sun) (as
improved
shown in Figure 7), so it can more effectively complete the forest
by 4.5% and 2.5%, respectively, compared with a single weakly supervised learner. At the fire identification work. In
addition,
time,the model can notmodel only detect whether
of AP there is a forest fire in real time through
same the integration in values . , AP , AP , AP , AR . , AR , AR ,
video (as shown in Figure 8), but also
and AR have significant advantages. Therefore, consideringaccurately locate the specific
the resultlocation of the
of the fire,
experi-
which plays a vital role in judging whether there is a forest fire
ment, in terms of detection accuracy and algorithm operation space complexity, this arti- and forest protection.
cle proposes the Y4SED model, which has a stronger anti-interference ability and is better
Table 7. Comparison of average accuracy of forest fire identification using WBF algorithm and NMS
able to distinguish between complex background and different kinds of forest fires (small
algorithm models.
fire, medium fire, large fire, strip-type fire, surface trunk, crown fire, and fire at night). In
addition,
Algorithmthere willinbe
Used theno false detection
Integration Model in the face of interfering APobjects similar to fire
0.5
(such as the sun) (as shown in Figure 7), so it can more effectively complete the forest fire
NMS 79
identification work. WBF In addition, the model can not only detect whether 87
there is a forest
fire in real time through video (as shown in Figure 8), but also accurately locate the specific
 Algorithm Used Algorithm Used inModel
in the Integration the Integration Model AP0.5 AP0.5
NMS NMS 79 79
WBF WBF 87 87
Forests 2022, 13, 1301 9 of 11

Table
Table 8. Experimental 8. Experimental
results—Models results—Models
were were
evaluated using evaluated
Microsoft usingstandards.
COCO Microsoft COCO standards.

Model
AP0.5 APS AP0.5Table Experimental
AP8.MAP S results—Models
AP AP
L M AR0.5wereAP
evaluated
L ARSusing
ARMicrosoft
0.5 ARMCOCO
ARS standards.
ARLARM A
S Yolov5-S
82.5
Model 36.0 82.5
AP 0.5
36.0
48.7AP S
66.0
AP M
48.7 AP
69.2
L
66.0 48 69.2ARS 59 48
AR0.5 ARM 76 AR 59L 7
-D2 EfficientDet-D2
84.5
Yolov5-S 36.3 84.5 50.236.0
82.5 36.3 64.150.2 66.0
48.7 68.6 64.169.2 49.8 68.6 48 64.2 49.859 73 64.2
76
Y4SED
EfficientDet-D2
87 36 87 50.136.3
84.5 36 50.2
68.6 50.1 64.1
71.5 68.668.6
52.4 71.5 49.8 63.4 52.4
64.2 63.4
77.3 73 7
Y4SED 87 36 50.1 68.6 71.5 52.4 63.4 77.3
𝐴𝑃 . , 𝐴𝑃 , 𝐴𝑃 , 𝐴𝑃𝐴𝑃, . 𝐴𝑅
, 𝐴𝑃 𝐴𝑃 , ,𝐴𝑅
. , , 𝐴𝑅 𝐴𝑃 ,, and
𝐴𝑅 .𝐴𝑅 , 𝐴𝑅are , 𝐴𝑅 , and 𝐴𝑅 are
all percentages. Theallbest
percentages.
data for each The indi-
best data for ea
AP0.5 , APS , APM , APL , AR0.5 , ARS , ARM , and ARL are all percentages. The best data for each indicator are shown
cator are shown in cator
in bold.are shown in bold.
bold.

(a) (a) (b) (b)

Figure
Figure 7. Recognition 7.7.Recognition
effect
Figure Recognition
of Y4SEDeffect effect
model of pictures
on
of Y4SED Y4SED
model onmodel on containing
containing
pictures pictures containing
interferences interferences
(sun).(sun).
interferences (a) The (sun).
(a) The
model has no falsemodel has
detection no
and false detection
strong and strong
anti-interference anti-interference
ability; (b) The modelability;
has (b)
no The
false model
detec- has no fals
model has no false detection and strong anti-interference ability; (b) The model has no false detection
tion and strong
tion and strong anti-interference
and strong anti-interference
ability.
anti-interference ability. ability.

(a) (a)
Figure 8. Real-time detection (b)
of forest fire video. (b)
Figure 8. Real-timeFigure
4. 8. Real-time
Conclusions
detection detection
of forest fire video.of forest fire video.
The identification of forest fires is of great significance for forest protection, but the
4. Conclusions 4. Conclusions
actual environment of the forest is complex and changeable; the background is complex;
the shape and color of the flame change all the time; there is no fixed form, and the
The identification The identification
of forest
identification firesfire
of forest ofgreat
forest
is often
of fires is ofofgreat
significance
has the problem significance
for forest
missed for forest
protection,
detection. In view but
of protection,
thethe
above
actual environment actual environment
of the
problems, forest
this paper of thea forest
isproposes
complex is recognition
complex the
and changeable;
forest fire and changeable;
background
system based on theis the background
complex;
weighted fusion is co
the shape and colorthe shape
algorithm,
of and color
the which
flame solves
changeof
thethe flame
problem
all of change
the time; missing all
is the
detection
there time; there
of forest
no fixed firesis
form, noto
due
and fixed form, and th
theiden-
the complex
tification of forestbackground
tification
fire often and
of has the
forest diversity
thefire of flame
often has
problem types under
the problem
of missed actual conditions.
of missed
detection. In view A new
detection. parallel
In view of the
of the above
execution method using the Yolov5 and EfficientDet models is proposed, and this paper
does not use the traditional non-maximum suppression algorithm, but uses the weighted
boxes fusion algorithm (WBF) to fuse the prediction frames of the two models to obtain
Forests 2022, 13, 1301 10 of 11

the fused prediction frame. Since Yolov5 is suitable for forest fire identification of large
targets, EfficientDet is more “careful” than Yolov5. The two can complement each other
and effectively improve the performance of the model. Experimental results show that
the average accuracy of the proposed model for forest fire identification can reach 87%.
The integration of learning strategies significantly improved the forest fire recognition
task of widespread missing detection problem. Compared with the single target detector
and the model using the traditional non-maximum suppression algorithm, our model
achieves a better compromise between the average accuracy and the spatial complexity
of the algorithm. These major improvements make the model perform well in the actual
application of forest fire identification and play an important role in the timely detection of
forest fires and forest protection in reality.

Author Contributions: J.Q. devised the programs and drafted the initial manuscript. H.L. designed
the project and revised the manuscript. All authors have read and agreed to the published version of
the manuscript.
Funding: This work was supported by Key Research and Development plan of Jiangsu Province
(Grant No.BE2021716) and Jiangsu Modern Agricultural Machinery Equipment and Technology
Demonstration and Promotion Project (NJ2021-19).
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Conflicts of Interest: The authors declare no conflict of interest.

References
1. Chowdhury, N.; Mushfiq, D.R.; Chowdhury, A.E. Computer Vision and Smoke Sensor Based Fire Detection System. In Proceedings
of the 2019 1st International Conference on Advances in Science, Engineering and Robotics Technology (ICASERT), Dhaka,
Bangladesh, 3–5 May 2019; pp. 1–5.
2. Varela, N.; Ospino, A.; Zelaya, N.A.L. Wireless sensor network for forest fire detection. Procedia Comput. Sci. 2020, 175, 435–440.
[CrossRef]
3. Lin, H.; Liu, X.; Wang, X.; Liu, Y. A fuzzy inference and big data analysis algorithm for the prediction of forest fire based on
rechargeable wireless sensor networks. Sustain. Comput. Inform. Syst. 2018, 18, 101–111. [CrossRef]
4. Sun, F.; Yang, Y.; Lin, C.; Liu, Z.; Chi, L. Forest Fire Compound Feature Monitoring Technology Based on Infrared and Visible
Binocular Vision. J. Phys. Conf. Ser. 2021, 1792, 012022. [CrossRef]
5. Barmpoutis, P.; Papaioannou, P.; Dimitropoulos, K.; Grammalidis, N. A Review on Early Forest Fire Detection Systems Using
Optical Remote Sensing. Sensors 2020, 20, 6442. [CrossRef]
6. Zhan, J.; Hu, Y.; Cai, W.; Zhou, G.; Li, L. PDAM–STPNNet: A Small Target Detection Approach for Wildland Fire Smoke through
Remote Sensing Images. Symmetry 2021, 13, 2260. [CrossRef]
7. Chen, T.H.; Wu, P.H.; Chiou, Y.C. An early fire-detection method based on image processing. In Proceedings of the 2004
International Conference on Image Processing, Singapore, 24–27 October 2004; Volume 3, pp. 1707–1710.
8. Horng, W.B.; Peng, J.W.; Chen, C.Y. A new image-based real-time flame detection method using color analysis. In Proceedings of
the 2005 IEEE Networking, Sensing and Control, Tucson, AZ, USA, 19–22 March 2005; pp. 100–105.
9. Çelik, T.; Özkaramanlı, H.; Demirel, H. Fire and smoke detection without sensors: Image processing based approach. In
Proceedings of the 2007 15th European Signal Processing Conference, Poznan, Poland, 3–7 September 2007; pp. 1794–1798.
10. Khan, M.N.A.; Tanveer, T.; Khurshid, K.; Zaki, H.; Zaidi, S.S.I. Fire Detection System using Raspberry Pi. In Proceedings of the
2019 International Conference on Information Science and Communication Technology (ICISCT), Karachi, Pakistan, 9 March
2019; pp. 1–6.
11. Priya, R.S.; Vani, K. Deep Learning Based Forest Fire Classification and Detection in Satellite Images. In Proceedings of the 2019
11th International Conference on Advanced Computing (ICoAC), Chennai, India, 18–20 December 2019; pp. 61–65.
12. Zheng, X.; Chen, F.; Lou, L.; Cheng, P.; Huang, Y. Real-Time Detection of Full-Scale Forest Fire Smoke Based on Deep Convolution
Neural Network. Remote Sens. 2022, 14, 536. [CrossRef]
13. Ren, S.; He, K.; Girshick, R.; Sun, J. Faster r-cnn: Towards real-time object detection with region proposal networks. Adv. Neural
Inf. Processing Syst. 2015, 28, 91–99. [CrossRef] [PubMed]
14. Redmon, J.; Divvala, S.; Girshick, R.; Farhadi, A. You only look once: Unified, real-time object detection. In Proceedings of the
IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 26–30 June 2016; IEEE: Piscataway, NJ, USA,
2016; pp. 779–788.
Forests 2022, 13, 1301 11 of 11

15. Liu, W.; Anguelov, D.; Erhan, D.; Szegedy, C.; Reed, S.; Fu, C.Y.; Berg, A.C. Ssd: Single shot multibox detector. In Proceedings of
the European Conference on Computer Vision, Amsterdam, The Netherlands, 8–16 October 2016; Springer: Cham, Switzerland,
2016; pp. 21–37.
16. Ultralytics. Yolov5. Available online: https://github.com/ultralytics/yolov5 (accessed on 1 May 2022).
17. Tan, M.; Pang, R.; Le, Q.V. Efficientdet: Scalable and efficient object detection. In Proceedings of the IEEE/CVF Conference on
Computer Vision and Pattern Recognition, Seattle, WA, USA, 13–19 June 2020; IEEE: Piscataway, NJ, USA, 2020; pp. 10781–10790.
18. BoWFire Dataset. Available online: https://bitbucket.org/gbdi/bowfifire-dataset/downloads/ (accessed on 1 May 2022).
19. Xie, Y.; Peng, M. Forest fire forecasting using ensemble learning approaches. Neural Comput. Appl. 2019, 31, 4541–4550. [CrossRef]
20. Dong, X.; Yu, Z.; Cao, W.; Shi, Y.; Ma, Q. A survey on ensemble learning. Front. Comput. Sci. 2020, 14, 241–258. [CrossRef]
21. Yang, G.; Feng, W.; Jin, J.; Lei, Q.; Li, X.; Gui, G.; Wang, W. Face mask recognition system with YOLOV5 based on image
recognition. In Proceedings of the 2020 IEEE 6th International Conference on Computer and Communications (ICCC), Chengdu,
China, 11–14 December 2020; pp. 1398–1404.
22. Song, S.; Jing, J.; Huang, Y.; Shi, M. EfficientDet for fabric defect detection based on edge computing. J. Eng. Fibers Fabr. 2021, 16,
15589250211008346. [CrossRef]
23. Zhou, D.; Fang, J.; Song, X.; Guan, C.; Yin, J.; Dai, Y.; Yang, R. Iou loss for 2d/3d object detection. In Proceedings of the 2019
International Conference on 3D Vision (3DV), Quebec City, QC, Canada, 16–19 September 2019; pp. 85–94.
24. Solovyev, R.; Wang, W.; Gabruseva, T. Weighted boxes fusion: Ensembling boxes from different object detection models. Image
Vis. Comput. 2021, 107, 104117. [CrossRef]
25. Rezatofighi, H.; Tsoi, N.; Gwak, J.; Sadeghian, A.; Reid, I.; Savarese, S. Generalized intersection over union: A metric and a loss
for bounding box regression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long
Beach, CA, USA, 15–20 June 2019; IEEE: Piscataway, NJ, USA, 2019; pp. 658–666.
26. Lydia, A.; Francis, S. Adagrad—An optimizer for stochastic gradient descent. Int. J. Inf. Comput. Sci. 2019, 5, 566–568.
27. Yao, Z.; Gholami, A.; Shen, S.; Mustafa, M.; Keutzer, K.; Mahoney, M. Adahessian: An adaptive second order optimizationer for
machine learning. Proc. AAAI Conf. Artif. Intell. 2021, 35, 10665–10673.