Eight Files of Neural3
Eight Files of Neural3
Fuel
journal homepage: www.elsevier.com/locate/fuel
h i g h l i g h t s
a r t i c l e i n f o a b s t r a c t
Article history: There is increasing need for tighter controls of coal-fired plants due to more stringent regulations and
Received 5 October 2014 addition of more renewable sources in the electricity grid. Achieving this will require better process
Received in revised form 11 January 2015 knowledge which can be facilitated through the use of plant models. Drum-boilers, a key component
Accepted 27 January 2015
of coal-fired subcritical power plants, have complicated characteristics and require highly complex rou-
Available online 8 February 2015
tines for the dynamic characteristics to be accurately modelled. Development of such routines is labori-
ous and due to computational requirements they are often unfit for control purposes. On the other hand,
Keywords:
simpler lumped and semi empirical models may not represent the process well. As a result, data-driven
NARX neural networks
Subcritical coal-fired power plant
approach based on neural networks is chosen in this study. Models derived with this approach incorpo-
Drum-boiler rate all the complex underlying physics and performs very well so long as it is used within the range of
gPROMS modelling and simulation conditions on which it was developed. The model can be used for studying plant dynamics and design of
controllers. Dynamic model of the drum-boiler was developed in this study using NARX neural networks.
The model predictions showed good agreement with actual outputs of the drum-boiler (drum pressure
and water level).
Ó 2015 Elsevier Ltd. All rights reserved.
1. Introduction the option of experimenting with the actual plants for safety and
economic reasons.
1.1. Background
1.2. Motivation
Drum-boiler (Fig. 1) is a critical component of thermal power
plants such as coal-fired subcritical power plants and many indus- Drum-boilers in coal-fired subcritical power plants have com-
trial processes. In the power industry in many countries, it has plicated geometry with complex phase equilibrium and steam
become needful for thermal power plants to be more tightly con- bubbles distributed below water level in the drum. Ideally, ade-
trolled to follow changes in electricity demand. This is due to more quate representation of the dynamic nature of such system will
stiff regulations and addition of renewable energy systems into the involve laborious and computationally-intensive distributed
electricity grid. Achieving this will require better process knowl- parameter modelling. Models of such complexity are unfit for con-
edge and more robust control systems. This can be facilitated trol purposes. Simpler lumped and semi-empirical models have
through modelling and simulation. This approach is preferred to been shown to considerably capture the complex dynamics of
drum-boilers [1–4]. However, for control purposes these non-
⇑ Corresponding author. Tel.: +44 1482 466688. linear models still have to be reduced in model order and then lin-
E-mail address: [email protected] (M. Wang). earized [5]. The performance of linear models usually deteriorates
http://dx.doi.org/10.1016/j.fuel.2015.01.091
0016-2361/Ó 2015 Elsevier Ltd. All rights reserved.
140 E. Oko et al. / Fuel 151 (2015) 139–145
away from operating point and as a result the model cannot be Most of the studies so far on application of neural networks in
trusted if big changes in operating conditions are expected. boiler modelling either as stand-alone or as a component of a ther-
For an already existing plant where operating data can easily be mal power plant are based on feedforward neural networks. In
obtained, it is considered that a data-driven approach commonly contrast, NARX neural network (recurrent neural networks) was
referred to as system identification is more convenient. Data- used in this study. Recurrent neural networks such as NARX neural
driven models incorporates all the complex underlying physics network have been shown to outperform feedforward neural net-
and performs very well so long as it is used within the range of works in predicting time-series data [20] and thus are more suit-
conditions on which it was developed. More importantly, the able for dynamic modelling [21].
approach avoids exact determination of model parameters which NARX neural networks have been used for dynamic modelling
often vary unpredictably. The methodology is already widely in of reactor–exchangers [22], crude preheater [23], hydraulic
use: conventional system identification is commonly used for con- suspension dampers [24], unsteady separation control [25], gas
troller design in the industry [6] and commercially available turbines [26,27], magnetic levitation [28] among others. There is
ESMER multiphase flow meter is based on advanced system iden- yet to be a case of data-driven drum-boiler models based on NARX
tification technique (neural networks) [7]. neural networks to the best of our knowledge.
Neural network models have been found to be less difficult to
develop compared to models based on conventional system identi- 2. Neural networks
fication. This is because more careful and rigorous design of the
test experiment for data acquisition is required in conventional Neural Network (NN) is a computational paradigm inspired
system identification. Also, in some cases, neural network models from the structure of biological neural networks and their way of
have shown better prediction accuracy compared to models based encoding and solving problems. They are able to identify underly-
on conventional system identification [8]. Neural network-based ing highly complex relationships based on input–output data only.
models are adaptive and have rapid response with good accuracy NN comprises of interconnections of the basic building blocks
if developed properly and can be used for real-time simulation called neurones (Fig. 2) organised in layers: the input, hidden and
among others [9]. output layers. The inputs to a neurone, (u1 ðtÞ, u2 ðtÞ; u3 ðtÞ . . . uk ðtÞ),
are either the network inputs or outputs of neurones in the previous
1.3. Aims and novelty layer and an externally applied bias (h). The bias can either increase
or lower the sum of the inputs (u) depending on its value. Also,
The aim of the study is to model subcritical boiler drum level the input channels are associated with synaptic weights
and pressure dynamics using NARX (Nonlinear AutoRegressive (w1; w2 ; w3 . . . wk ) which can have both positive (excitatory) and
with eXogenous inputs) neural networks. Neural networks have negative (inhibitory) values. The bias and weights are both adjust-
been used for predicting boiler performance in the past. Yusoff able parameters and development (training) of NN is about deter-
[10] used neural network for emission monitoring from biomass- mining optimal values for the parameters for specific cases. The
fired boilers. Romeo and Gareta [11] and Teruel et al. [12] used activation (or transfer) function is typically sigmoid function in
neural networks for predicting fouling and slagging in boiler fur- the hidden layer and either linear or sigmoid functions in the output
nace. Li and Fang [13] identified superheater model of an ultra- layer. More details on NN can be found in Haykin [29] among several
supercritical boiler using neural networks, and Rusinowski and other books.
Stanek [14] used neural network to develop correlations for pre- Depending on signal flow configuration, NN can be classified
dicting flue gas temperature. Whole boiler/thermal power plant into feedforward and recurrent NN. In feedforward NN, the outputs
models built with neural networks have also been reported are calculated directly from the inputs through feedforward con-
[8,9,15–19]. nections [21]. Feedforward NN is mostly static networks. Recurrent
NN on the other hand are dynamic and have at least one feedback
loop. The network outputs are therefore not the result of the exter-
nal inputs only.
NARX NN belongs to the recurrent NN class. They have a feed-
back connection enclosing several layers of the network (Fig. 3).
The architecture includes tapped delay lines (TDL) which plays
the role of holding past values of the input. This feature makes them
more suitable for multi-step-ahead predictions (time-series predic-
tion) than feedforward networks [21]. It is therefore more appropri-
ate to use them for dynamic modelling. The inputs are normally a
sequence of input vectors that occur in a certain time order.
A NARX model is generally defined by the equation:
yðtÞ ¼ f yðt 1Þ;yðt 2Þ; .. .; yðt ny Þ; uðt 1Þ; uðt 2Þ; .. .; uðt nu Þ
ð1Þ
In the equation, yðtÞ is the current value of predicted output signal
expressed as a function of the previous values of the output signal
(yðt 1Þ; yðt 2Þ; . . . ; yðt ny Þ) and previous values of an indepen-
dent (exogenous) input signal (uðt 1Þ; uðt 2Þ; . . . ; uðt nu Þ). The
terms ny and nu are respectively the orders of the output and exoge-
nous input respectively. The previous values are recorded using TDL
and the nonlinear polynomial function (f ) approximated using a
Fig. 1. Drum-boiler (⁄) illustrations and text are taken from the Spirax Sarco
feedforward NN. Consequently, typical architecture for a first order
website ‘Steam Engineering Tutorials’ at <http://www.spiraxsarco.com/resources/
steam-engineering-tutorials.asp>. Illustrations and text are copyright, remains the NARX NN (where ny and nu in Eq. (1) are both equal to 1) has the form
intellectual property of Spirax Sarco, and have been used with their full permission. shown in Fig. 3.
E. Oko et al. / Fuel 151 (2015) 139–145 141
that the main inputs to the drum boiler include the heat input, feed-
120
water flowrate and steam flowrate and the outputs are drum level
and drum pressure. The heat input is obtained from steady state 100
calculations when the values of the drum pressure, volume of water
in the loop and steam flowrate are specified (the values of the drum 80
pressure, volume of water in the loop and steam flowrate used for
the steady state calculations were obtained from Åström and Bell 60
[3]). For complete coal-fired subcritical power plant, heat input will 40
be replaced with coal flowrate and steam flowrate could be substi- 0 1000 2000 3000
tuted with governor valve opening. The input–output set up identi- Time (s)
fied from the first principles model will be used for the NN model
development. Fig. 4. Perturbations in feedwater flowrate.
142 E. Oko et al. / Fuel 151 (2015) 139–145
Train
Validation
100 Test
60
-2
10
40
3500 4500 5500 6500
Time (s) 10
-4
-6
10
2.10E+08
2.00E+08 0 50 100 150 200 250 300
3
1.40E+08 Correlations
Zero Correlation
1.30E+08 2.5 Confidence Limit
1.20E+08 2
7100 8100 9100 10100
Correlation
0.5
0.045
0
Drum water level (m)
0.04
-
20 -15 -10 -5 0 5 10 15 20
0.035 Lag
0.03
Fig. 10. Drum pressure prediction error autocorrelation plot.
0.025
0.02
0.015
0 2000 4000 6000 8000 10000
x 10-11
Time (s) 7
Correlations
Zero Correlation
6 Confidence Limit
Fig. 7. Drum level.
5
4
1.3E+07
3
1.2E+07
Drum Pressure (Pa)
2
1.2E+07
1
1.1E+07
1.1E+07 0
1.0E+07
-20 -15 -10 -5 0 5 10 15 20
9.5E+06
Lag
9.0E+06
8.5E+06 Fig. 11. Drum level prediction error autocorrelation plot.
0 2000 4000 6000 8000 10000
Time (s)
by performing backward calculations through the network
Fig. 8. Drum pressure.
starting from the output layer. In MATLAB Neural Network
Toolbox, the various optimization algorithms are implemented
different optimization algorithm such as gradient descent, as training functions, namely trainlm function (Levenberg–
Levenberg–Marquardt, Bayesian regularization, scaled conjugate Marquardt), trainbr function (Bayesian regularization), trainscg
gradient among others. These algorithms are usually executed function (scaled conjugate gradient) etc.
E. Oko et al. / Fuel 151 (2015) 139–145 143
0.03
0.025
0.02
0.015 -4
4 x 10
Targets - Outputs
2
Error
0
-2
1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 11000
Time
1.05
0.95
0.9
0.85 4
4 x 10
Targets - Outputs
2
Error
0
-2
-4
1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 11000
Time
For dynamic NN with a feedback loop such as NARX NN, train- pre-processing function becomes part of the network object, so
ing is complicated because some of the inputs (feedback) are also that whenever the network is used, the data coming into the net-
functions of the weights (Fig. 3). To avoid this complication, NARX work is pre-processed in a similar way. The NN output is similarly
NN is trained in open loop (without the feedback loop). This is post-processed to transform the output to the same form as the
based on series–parallel architecture where the actual output, actual output. In this study, the mapminmax and removeconstant-
rather than the estimated output fed back to the network, is used rows processing functions in MATLAB have been used. The map-
as the input. On this basis, the NARX NN is then purely feedforward minmax function transforms the data so that their values fall into
network and can be trained as such. Details of this procedure can the interval [1, 1]. On the other hand, removeconstantrows func-
be found in Beale et al. [21]. After training, the closeloop function tions removes the rows of the data vector that are constant (if
in MATLAB can be used to convert the NN from the series–parallel any) since they will not provide useful information to the NN. Also,
configuration (open loop) to close loop configuration for multi-step pre-processing for dynamic networks include shifting the data to
ahead predictions. initialize the TDL. In MATLAB, this is accomplished using preparets
Prior to training, the available data (input and target vectors) is function. The function uses the network object to initialize the TDL
pre-processed to transform the data to more suitable form for NN by shifting the data accordingly to create the correct inputs and
training. This makes the learning process faster and efficient with- targets to use in training or simulating the network.
out the possibility of saturation of the sigmoid transfer function Commonly, overfitting occurs during NN training. This is a situ-
often used in the hidden layers [21]. Some training algorithm also ation where the NN memorises the training examples including
requires particular pre-processing for optimal performance, e.g. noises such that it is not able to generalize to new conditions. This
data transformation to a form where their values fall into the inter- can be avoided using either early stopping or regularization
val [1, 1] for trainbr algorithm. When the network is created, the techniques. Early stopping technique was used in this study after
144 E. Oko et al. / Fuel 151 (2015) 139–145
0.042 1.07000E+07
0.041
0.038 1.01000E+07
0.037
9.90000E+06
0.036
0.035 9.70000E+06
0 100 200 0 50 100 150 200
Time (s) Time (s)
Actual NARX Actual NARX
Fig. 14. Drum pressure and level response to +30 kg/s step change in feedwater flowrate.
0.043
1.08000E+07
0.042
1.06000E+07
1.04000E+07
0.04
1.02000E+07
0.039
1.00000E+07
0.038
9.80000E+06
0.037
9.60000E+06
0.036
9.40000E+06
0.035
0 100 200 300 9.20000E+06
0 100 200 300
Time (s) Time (s)
Actual NARX Actual NARX
Fig. 15. Drum pressure and level response to +10 kg/s step change in steam flowrate.
1.16000E+07 0.0365
0.036
1.14000E+07
Drum Pressure (Pa)
0.0355
Drum Level (m)
1.12000E+07
0.035
1.10000E+07 0.0345
0.034
1.08000E+07
0.0335
1.06000E+07
0.033
1.04000E+07 0.0325
0 100 200 300 0 100 200 300
Time (s) Time (s)
Actual NARX Actual NARX
Fig. 16. Drum pressure and level response to +10 MWth step change in heat input.
exploring the two techniques; regularization technique gave poorer (i.e. feedwater flowrate, steam flowrate and heat input) and two
result. In early stopping method the available data is divided into outputs (i.e. drum level and drum pressure) was developed in
three subsets, namely training, validation and testing sets. The error MATLAB using the simulation data from gPROMS ModelBuilderÒ
normally decreases during the initial phase of the training. Overfit- (Figs. 4–8). There are 100 neurones in the hidden layer each utiliz-
ting begins to set in when the validation error begins to increase. ing sigmoid activation function while each of the two outer layer
The optimal network weights and biases are obtained at the mini- neurones utilize linear activation function. The Levenberg–Marqu-
mum validation set error before overfitting begins to set in. ardt algorithm (trainlm training function in MATLAB) was used to
obtain the optimal values of the adjustable parameters, weights
5. Results and discussion and biases. The MSE performance function (Eq. (2)) was used to
assess the network performance. In Eq. (2), zi = the targets,
5.1. Training results yi = network outputs and N = data size.
1X N
Based on the discussions above, a two-layer first order MSE ¼ ðzi yi Þ2 ! min ð2Þ
N i¼1
NARX NN dynamic model of the drum-boiler with three inputs
E. Oko et al. / Fuel 151 (2015) 139–145 145