Artificial Neural

Networks for Engineers

and Scientists
Solving Ordinary Differential
Equations
 Artificial Neural
Networks for Engineers
and Scientists
Solving Ordinary Differential
Equations

Snehashish Chakraverty and Susmita Mall

CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742

© 2017 by Taylor & Francis Group, LLC

CRC Press is an imprint of Taylor & Francis Group, an Informa business

No claim to original U.S. Government works

Printed on acid-free paper

International Standard Book Number-13: 978-1-4987-8138-1 (Hardback)

This book contains information obtained from authentic and highly regarded sources. Reasonable
efforts have been made to publish reliable data and information, but the author and publisher cannot
assume responsibility for the validity of all materials or the consequences of their use. The authors
and publishers have attempted to trace the copyright holders of all material reproduced in this
publication and apologize to copyright holders if permission to publish in this form has not been
obtained. If any copyright material has not been acknowledged please write and let us know so we
may rectify in any future reprint.

Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced,
transmitted, or utilized in any form by any electronic, mechanical, or other means, now known
or hereafter invented, including photocopying, microfilming, and recording, or in any information
storage or retrieval system, without written permission from the publishers.

For permission to photocopy or use material electronically from this work, please access www.
copyright.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC),
222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that
provides licenses and registration for a variety of users. For organizations that have been granted a
photocopy license by the CCC, a separate system of payment has been arranged.

Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and
are used only for identification and explanation without intent to infringe.

Library of Congress Cataloging-in-Publication Data

Names: Chakraverty, Snehashish. | Mall, Susmita.

Title: Artificial neural networks for engineers and scientists : solving
ordinary differential equations / Snehashish Chakraverty & Susmita Mall.
Description: Boca Raton : CRC Press, 2017. | Includes bibliographical references.
Identifiers: LCCN 2017004411| ISBN 9781498781381 (hardback) | ISBN
9781498781404 (ebook)
Subjects: LCSH: Differential equations--Data processing. | Engineering
mathematics--Data processing. | Artificial intelligence.
Classification: LCC QA372 .C42527 2017 | DDC 515/.3520285632--dc23
LC record available at https://lccn.loc.gov/2017004411

Visit the Taylor & Francis Web site at

http://www.taylorandfrancis.com

and the CRC Press Web site at

http://www.crcpress.com
Contents

Preface.......................................................................................................................ix
Acknowledgments............................................................................................... xiii
Authors....................................................................................................................xv
Reviewers............................................................................................................. xvii

1. Preliminaries of Artificial Neural Network................................................1

1.1 Introduction............................................................................................1
1.2 Architecture of ANN.............................................................................2
1.2.1 Feed-Forward Neural Network..............................................3
1.2.2 Feedback Neural Network......................................................3
1.3 Paradigms of Learning..........................................................................4
1.3.1 Supervised Learning or Associative Learning......................4
1.3.2 Unsupervised or Self-Organization Learning......................4
1.4 Learning Rules or Learning Processes................................................5
1.4.1 Error Back-Propagation Learning Algorithm or Delta
Learning Rule............................................................................5
1.5 Activation Functions.............................................................................8
1.5.1 Sigmoid Function......................................................................8
1.5.1.1 Unipolar Sigmoid Function.....................................8
1.5.1.2 Bipolar Sigmoid Function........................................9
1.5.2 Tangent Hyperbolic Function.................................................9
References..........................................................................................................9

2. Preliminaries of Ordinary Differential Equations.................................. 11

2.1 Definitions.............................................................................................12
2.1.1 Order and Degree of DEs.......................................................12
2.1.2 Ordinary Differential Equation.............................................12
2.1.3 Partial Differential Equation.................................................12
2.1.4 Linear and Nonlinear Differential Equations.....................13
2.1.5 Initial Value Problem..............................................................13
2.1.6 Boundary Value Problem.......................................................14
References........................................................................................................15

3. Multilayer Artificial Neural Network........................................................17

3.1 Structure of Multilayer ANN Model.................................................18
3.2 Formulations and Learning Algorithm of Multilayer
ANN Model..........................................................................................18
3.2.1 General Formulation of ODEs Based on ANN Model......18

v
vi Contents

3.2.2Formulation of nth-Order IVPs.............................................20

3.2.2.1 Formulation of First-Order IVPs...........................21
3.2.2.2 Formulation of Second-Order IVPs......................21
3.2.3 Formulation of BVPs..............................................................22
3.2.3.1 Formulation of Second-Order BVPs.....................22
3.2.3.2 Formulation of Fourth-Order BVPs......................23
3.2.4 Formulation of a System of First-Order ODEs...................24
3.2.5 Computation of Gradient of ODEs for Multilayer
ANN Model.............................................................................25
3.3 First-Order Linear ODEs.....................................................................27
3.4 Higher-Order ODEs.............................................................................32
3.5 System of ODEs....................................................................................34
References........................................................................................................36

4. Regression-Based ANN................................................................................37
4.1 Algorithm of RBNN Model...............................................................37
4.2 Structure of RBNN Model..................................................................39
4.3 Formulation and Learning Algorithm of RBNN Model................39
4.4 Computation of Gradient for RBNN Model....................................40
4.5 First-Order Linear ODEs.....................................................................40
4.6 Higher-Order Linear ODEs................................................................50
References........................................................................................................56

5. Single-Layer Functional Link Artificial Neural Network......................57

5.1 Single-Layer FLANN Models............................................................58
5.1.1 ChNN Model...........................................................................58
5.1.1.1 Structure of the ChNN Model...............................58
5.1.1.2 Formulation of the ChNN Model.........................59
5.1.1.3 Gradient Computation of the ChNN Model.......60
5.1.2 LeNN Model............................................................................62
5.1.2.1 Structure of the LeNN Model................................62
5.1.2.2 Formulation of the LeNN Model..........................63
5.1.2.3 Gradient Computation of the LeNN Model........63
5.1.3 HeNN Model...........................................................................64
5.1.3.1 Architecture of the HeNN Model.........................64
5.1.3.2 Formulation of the HeNN Model.........................65
5.1.4 Simple Orthogonal Polynomial–Based Neural
Network (SOPNN) Model.....................................................66
5.1.4.1 Structure of the SOPNN Model.............................66
5.1.4.2 Formulation of the SOPNN Model.......................67
5.1.4.3 Gradient Computation of the SOPNN Model......68
5.2 First-Order Linear ODEs.....................................................................68
5.3 Higher-Order ODEs.............................................................................69
5.4 System of ODEs....................................................................................71
References........................................................................................................74
Contents vii

6. Single-Layer Functional Link Artificial Neural Network

with Regression-Based Weights..................................................................77
6.1 ChNN Model with Regression-Based Weights................................78
6.1.1 Structure of the ChNN Model...............................................78
6.1.2 Formulation and Gradient Computation
of the ChNN Model................................................................79
6.2 First-Order Linear ODEs.....................................................................79
6.3 Higher-Order ODEs.............................................................................83
References........................................................................................................85

7. Lane–Emden Equations................................................................................87
7.1 Multilayer ANN-Based Solution of Lane–Emden Equations........89
7.2 FLANN-Based Solution of Lane–Emden Equations.......................93
7.2.1 Homogeneous Lane–Emden Equations..............................94
7.2.2 Nonhomogeneous Lane–Emden Equation.......................101
References......................................................................................................102

8. Emden–Fowler Equations..........................................................................105
8.1 Multilayer ANN-Based Solution of Emden–Fowler
Equations.............................................................................................106
8.2 FLANN-Based Solution of Emden–Fowler Equations................. 110
References...................................................................................................... 113

9. Duffing Oscillator Equations.................................................................... 117

9.1 Governing Equation.......................................................................... 117
9.2 Unforced Duffing Oscillator Equations.......................................... 118
9.3 Forced Duffing Oscillator Equations...............................................123
References......................................................................................................131

10. Van der Pol–Duffing Oscillator Equation...............................................133

10.1 Model Equation..................................................................................134
10.2 Unforced Van der Pol–Duffing Oscillator Equation.....................135
10.3 Forced Van der Pol–Duffing Oscillator Equation..........................135
References......................................................................................................144

Index......................................................................................................................147
Preface

Differential equations play a vital role in various fields of science and engi-
neering. Many real-world problems in engineering, mathematics, physics,
chemistry, economics, psychology, defense, etc., may be modeled by ordi-
nary or partial differential equations. In most of the cases, an analytical/
exact solution of differential equations may not be obtained easily. Therefore,
various types of numerical techniques have been developed by researchers
to solve such equations, such as Euler, Runge–Kutta, predictor-corrector,
finite difference, finite element, and finite volume techniques. Although
these methods provide good approximations to the solution, they require
the discretization of the domain into a number of finite points/elements, in
general. These methods provide solution values at predefined points, and
computational complexity increases with the number of sampling points.
In recent decades, among the various machine intelligence procedures,
artificial neural network (ANN) methods have been established as power-
ful techniques to solve a variety of real-world problems because of ANN’s
excellent learning capacity. ANN is a computational model or an information
processing paradigm inspired by the biological nervous system. Recently,
a lot of attention has been devoted to the study of ANN for solving dif-
ferential equations. The approximate solution of differential equations by
ANN is found to be advantageous but it depends upon the ANN model that
one considers. Here, our target is to handle ordinary differential equations
(ODEs) using ANN. The traditional numerical methods are usually iterative
in nature, where we fix the step size before the start of the computation. After
the solution is obtained, if we want to know the solution between steps, the
procedure needs to be repeated from the initial stage. ANN may be one of the
ways to overcome this repetition of iterations.
In solving differential equations by ANN, no desired values are known and
the output of the model can be generated by training only. As per the existing
training algorithm, the architecture of a neural model is problem dependent
and the number of nodes, etc., is taken by trial-and-error method where the
training depends upon the weights of the connecting nodes. In general, these
weights are taken as random numbers that dictate the training. This book
includes recently developed new ANN models to handle various types of
ODES. A brief outline of each chapter is given next.
The preliminaries of ANN are presented in Chapter 1. Definitions of ANN
architecture, paradigm of learning, activation functions, leaning rules and
learning processes, etc., are reviewed here. Chapter 2 describes preliminaries
of ODEs. We recall the definitions that are relevant to the present book such

ix
x Preface

as linear ODEs, nonlinear ODEs, initial and boundary value problems, etc.
Chapter 3 deals with traditional multilayer ANN models to solve first- and
higher-order ODEs. In the training algorithm, the number of nodes in the
hidden layer is chosen by a trial-and-error method. The initial weights are
random numbers as per the desired number of nodes. A simple feed-forward
neural network and an unsupervised error back-propagation algorithm have
been used. The chapter also addresses the general formulation of ODEs using
multilayer ANN, formulation of nth-order initial value as well as boundary
value problems, system of ODEs, and computation of gradient.
In Chapter 4, the recently developed regression-based neural network
(RBNN) model is used to handle ODEs. In the RBNN model, the number of
nodes in the hidden layer may be fixed according to the degree of polyno-
mial in the regression. The coefficients involved are taken as initial weights
to start with the neural training. A variety of first- and higher-order ODEs are
solved as example problems. A single-layer functional link artificial neural
network (FLANN) is presented in Chapter 5. In FLANN, the hidden layer
is replaced by a functional expansion block for enhancement of the input
patterns using orthogonal polynomials such as Chebyshev, Legendre, and
Hermite. In this chapter, we discuss various types of single-layer FLANNs
such as Chebyshev neural network (ChNN), Legendre neural network
(LeNN), simple orthogonal polynomial–based neural network (SOPNN),
and Hermite neural network (HeNN) used to solve linear and nonlinear
ODEs. Chapter 6 introduces a single-layer FLANN model with regression-
based weights to solve initial value problems. A second-order singular
nonlinear initial value problem, namely, a Lane–Emden equation, has been
solved using multi- and single-layer ANNs in Chapter 7. Next, Chapter 8
addresses the Emden–Fowler equations and their solution using multilayer
ANN and single-layer FLANN methods.
Duffing oscillator equations have been considered in Chapter 9. Single-
layer FLANN models, namely, SOPNN and HeNN models, have been used
in this chapter to handle Duffing oscillator equations. Finally, Chapter 10
presents an ANN solution to the nonlinear van der Pol–Duffing oscilla-
tor equation where single-layer SOPNN and HeNN methods have been
employed.
This book aims to provide a systematic understanding of ANN along with
the new developments in ANN training. A variety of differential equations
are solved here using the methods described in each chapter to show their
reliability, powerfulness, and easy computer implementation. The book pro-
vides comprehensive results and up-to-date and self-contained review of
the topic along with an application-oriented treatment of the use of newly
developed ANN methods to solve various types of differential equations.
It may be worth mentioning that the methods presented may very well be
adapted for use in various other science and engineering disciplines where
Preface xi

problems are modeled as differential equations and when exact and/or other
traditional numerical methods may not be suitable. As such, this book will
certainly prove to be essential for students, scholars, practitioners, research-
ers, and academicians in the assorted fields of engineering and sciences inter-
ested in modeling physical problems with ease.

Snehashish Chakraverty
Susmita Mall
National Institute of Technology, Rourkela
Acknowledgments

We express our sincere appreciation and gratitude to those outstanding peo-

ple who have supported and happened to be the continuous source of inspi-
ration throughout this book writing project.
The first author expresses his sincere gratitude to his parents, who always
inspired him during the writing of this book. He also thanks his wife, Shewli
Chakraborty, and his daughters, Shreyati and Susprihaa, for their continuous
motivation. His PhD students and the NIT Rourkela facilities were also an
important source of support in completing this book.
The second author expresses sincere gratitude and special appreciation to
her father Sunil Kanti Mall, mother Snehalata Mall, father-in-law Gobinda
Chandra Sahoo, mother-in-law Rama Mani Sahoo, and family members for
their help, motivation, and invaluable advice. Writing this book would not
have been possible without the love, unconditional support, and encourage-
ment of her husband Srinath Sahoo and daughter Saswati.
The second author of this book conveys her heartfelt thanks to the family
members of the first author, especially to his wife and daughters for their
love, support, and inspiration.
Also, the authors gratefully acknowledge all the contributors and the
authors of the books and journals/conferences papers listed in the book.
Finally, they greatly appreciate the efforts of the entire editorial team of the
publisher for their continuous support and help.

Snehashish Chakraverty
Susmita Mall

xiii
Authors

Dr. Snehashish Chakraverty is a professor of mathematics at National

Institute of Technology, Rourkela, Odisha, India. He earned his PhD from
the Indian Institute of Technology—Roorkee in 1992, and did postdoctoral
research at ISVR, the University of Southampton, UK, and at Concordia
University, Canada. He was a visiting professor at Concordia University and
at McGill University, in Canada, and also at the University of Johannesburg,
South Africa. Dr. Chakraverty has authored ten books, including two that
will be published in 2017, and has published more than 250 research papers
in journals and for conferences. He has served as president of the Section of
Mathematical Sciences (including Statistics) of the Indian Science Congress
(2015–2016) and as the vice president of the Orissa Mathematical Society
(2011–2013). Dr. Chakraverty has received several prestigious awards,
including the INSA International Bilateral Exchange Program, Platinum
Jubilee ISCA Lecture, CSIR Young Scientist, BOYSCAST, UCOST Young
Scientist, Golden Jubilee CBRI Director’s Award, and the Roorkee University
Gold Medals. He has undertaken around 16 major research projects as prin-
ciple investigator, funded by different agencies. His present research areas
include soft computing, numerical analysis, differential equations, vibration
and inverse vibration problems, and mathematical computation.

Dr. Susmita Mall received her PhD from the National Institute of Tech­
nology, Rourkela, Odisha, India, in 2016 and MSc in mathematics from
Ravenshaw University, Cuttack, Odisha. She was awarded the Women
Scientist Scheme-A (WOS-A) fellowship under the Department of Science
and Technology (DST), Government of India, to pursue her PhD studies. She
has published ten research papers in international refereed journals and five
in conferences. Her current research interest includes mathematical model-
ing, artificial neural network, differential equations, and numerical analysis.

xv
Reviewers

Rama Bhata Tshilidzi Marwala

Mechanical Engineering Department of Electrical and
Concordia University Electronic Engineering
Montreal, Quebec, Canada University of Johannesburg
Johannesburg, South Africa
Fernando Buarque
Polytechnic School of Pernambuco Sushmita Mitra
University of Pernambuco Machine Intelligence Unit
Pernambuco, Brazil Indian Statistical Institute
Kolkata, West Bengal, India
T.R. Gulati
A. Sahu
Department of Mathematics
Mathematics & Computer Science
Indian Institute of Technology
Coppin State University
Roorkee
Baltimore, Maryland
Roorkee, Uttarakhand, India
Tanmoy Som
Aliakbar Montazer Haghighi Department of Mathematical
Department of Mathematics Science
Prairie View A&M University Indian Institute of Technology
Prairie View, Texas (BHU)
Varanasi, Uttar Pradesh, India
Nikos D. Lagaros
Institute of Structural Analysis & A.M. Wazwaz
Seismic Research Department of Mathematics
National Technical University Saint Xavier University
Athens, Greece Chicago, Illinois

xvii
1
Preliminaries of Artificial
Neural Network

This chapter addresses basics of artificial neural network (ANN) architec-

ture, paradigms of learning, activation functions, and leaning rules.

1.1 Introduction
Artificial neural network (ANN) is one of the popular areas of artificial
intelligence (AI) research and also an abstract computational model based
on the organizational structure of the human brain [1]. The simplest defini-
tion of ANN is provided by the inventor of one of the first neurocomputers,
Dr. Robert Hecht-Nielsen. He defines a neural network as

a computing system made up of a number of simple, highly intercon-

nected processing elements, which process information by their dynamic
state response to external inputs.

ANNs are processing devices (algorithms) that are loosely modeled after
the neuronal structure of the mammalian cerebral cortex but on much
smaller scales. Computer scientists have always been inspired by the
human brain. In 1943, Warren S. McCulloch, a neuroscientist, and Walter
Pitts, a logician, developed the first conceptual model of an ANN [1]. In
their paper, they describe the concept of a neuron, a single cell living in a
network of cells that receives inputs, processes those inputs, and generates
an output.
ANN is a data modeling tool that depends upon various parameters and
learning methods [2–8]. Neural networks are typically organized in lay-
ers. Layers are made up of a number of interconnected “neurons/nodes,”
which contain “activation functions.” ANN processes information through
neurons/nodes in a parallel manner to solve specific problems. ANN
acquires knowledge through learning, and this knowledge is stored within
interneuron connections’ strength, which is expressed by numerical values
called “weights.” These weights are used to compute output signal values

1
2 Artificial Neural Networks for Engineers and Scientists

vi
wij
xj

Input layer Output layer

Hidden layers

FIGURE 1.1
Structure of artificial neural network.

for a new testing input signal value. Patterns are presented to the network
via the “input layer,” which communicates to one or more “hidden layers,”
where the actual processing is done via a system of weighted “connections.”
The hidden layers then link to an “output layer,” where the answer is output,
as shown in Figure 1.1.
Here, xj are input nodes, wij are weights from the input layer to the hidden
layer, and vi and y denote the weights from the hidden layer to the output
layer and the output node, respectively.
The ANN method has been established as a powerful technique to solve
a variety of real-world problems because of its excellent learning capac-
ity [9–12]. This method has been successfully applied in various fields
[8,13–25] such as function approximation, clustering, prediction, identifi-
cation, pattern recognition, solving ordinary and partial differential equa-
tions, etc.

1.2 Architecture of ANN
It is a technique that seeks to build an intelligent program using models that
simulate the working of neurons in the human brain. The key element of the
network is the structure of the information processing system. ANN pro-
cesses information in a similar way as the human brain does. The network
is composed of a large number of highly interconnected processing elements
(neurons) working in parallel to solve a specific problem.
Neural computing is a mathematical model inspired by the biological
model. This computing system is made up of a large number of artificial neu-
rons and a still larger number of interconnections among them. According to
Preliminaries of Artificial Neural Network 3

the structure of these interconnections, different classes of neural network

architecture can be identified, as discussed next.

1.2.1 Feed-Forward Neural Network

In a feed-forward neural network, neurons are organized in the form of lay-
ers. Neurons in a layer receive input from the previous layer and feed their
output to the next layer. Network connections to the same or previous layers
are not allowed. Here, the data goes from the input node to the output node
in a strictly feed-forward way. There is no feedback (back loops); that is, the
output of any layer does not affect the same layer. Figure 1.2 shows the block
diagram of the feed-forward ANN [8].
Here, X = (x1, x2, … , xn) denotes the input vector, f is the activation function,
and O = (o1, o2, … , om) is the output vector. Symbol W = wij denotes the weight
matrix or connection matrix, and [WX] is the net input value, which is a sca-
lar product of input vectors and weight vectors wij

é w11 w12 ¼ w1n ù

êw w22 ¼ w2 n úú
W=ê
21

ê   ¼  ú
ê ú
ë w m1 wm 2 ¼ wmn û

1.2.2 Feedback Neural Network

In a feedback neural network, the output of one layer routes back to the pre-
vious layer. This network can have signals traveling in both directions by the
introduction of loops in the network. This network is very powerful and, at
times, gets extremely complicated. All possible connections between neurons
are allowed. Feedback neural networks are used in optimization problems,
where the network looks for the best arrangement of interconnected factors.
They are dynamic and their state changes continuously until they reach an
equilibrium point (Figure 1.3).

X f = [WX] O

FIGURE 1.2
Block diagram of feed-forward ANN.
4 Artificial Neural Networks for Engineers and Scientists

Input layer Hidden layer Output layer

FIGURE 1.3
Diagram of feedback neural network.

1.3 Paradigms of Learning
The ability to learn and generalize from a set of training data is one of the
most powerful features of ANN. The learning situations in neural networks
may be classified into two types, namely, supervised and unsupervised.

1.3.1 Supervised Learning or Associative Learning

In supervised training, both inputs and outputs are provided. The net-
work then processes the inputs and compares its resulting outputs against
the desired outputs. A comparison is made between the network’s com-
puted output and the corrected expected output to determine the error. The
error can then be used to change network parameters, which results in an
improvement in performance. In other words, inputs are assumed to be at
the beginning and outputs at the end of the causal chain. Models can include
mediating variables between inputs and outputs.

1.3.2 Unsupervised or Self-Organization Learning

The other type of training is called unsupervised training. In unsupervised
training, the network is provided with inputs but not with desired out-
puts. The system itself must then decide what features it will use to group
the input data. This is often referred to as self-organization or adaptation.
Unsupervised learning seems much harder than supervised learning, and
this type of training generally fits into the decision problem framework
because the goal is not to produce a classification but to make decisions that
maximize rewards. An unsupervised learning task is to try to find the hidden
Preliminaries of Artificial Neural Network 5

structure in unlabeled data. Since the examples given to the learner are unla-
beled, there is no error or reward signal to evaluate a potential solution.

1.4 Learning Rules or Learning Processes

Learning is the most important characteristic of the ANN model. Every
neural network possesses knowledge that is contained in the values of the
connection weights. Most ANNs contain some form of “learning rule”
that modifies the weights of the connections according to the input patterns
that it is presented with. Although there are various kinds of learning rules
used by neural networks, the delta learning rule is often utilized by the most
common class of ANNs called back-propagation neural networks (BPNNs).
Back propagation is an abbreviation for the backward propagation of error.
There are various types of learning rules for ANN [8,13] such as the
following:

1. Error back-propagation learning algorithm or delta learning rule

2. Hebbian learning rule
3. Perceptron learning rule
4. Widrow–Hoff learning rule
5. Winner-Take-All learning rule, etc.

The details of the above-listed types of learning may be found in any stan-
dard ANN books. As such, here, we will discuss only a bit about the first one
because the same has been used in most of the investigations in this book.

1.4.1 Error Back-Propagation Learning Algorithm

or Delta Learning Rule
Error back-propagation learning algorithm has been introduced by
Rumelhart et al. [3]. It is also known as the delta learning rule [5,8,14] and
is one of the most commonly used learning rules. This learning algorithm
is valid for continuous activation function and is used in the supervised/
unsupervised training method.
A simple perceptron can handle linearly separable or linearly independent
problems. Taking the partial derivative of error of the network with respect
to each of its weights, we can know the flow of error direction in the network.
If we take the negative derivative and then proceed to add it to the weights,
the error will decrease until it approaches a local minimum. We have to add a
negative value to the weight or the reverse if the derivative is negative. Then,
these partial derivatives are applied to each of the weights, starting from the
6 Artificial Neural Networks for Engineers and Scientists

y1
w11 v11

w21 v12
x y2 O

w31 v13
Input layer y3 Output layer

Hidden layer

FIGURE 1.4
Architecture of multilayer feed forward neural network.

output layer weights to the hidden layer weights, and then from the hidden
layer weights to the input layer weights.
In general, the training of the network involves feeding samples as input
vectors, calculating the error of the output layer, and then adjusting the
weights of the network to minimize the error. The average of all the squared
errors E for the outputs is computed to make the derivative simpler. After
the error is computed, the weights can be updated one by one. The descent
depends on the gradient ∇E for the training of the network.
Let us consider a multilayer neural architecture containing one input node
x, three nodes in the hidden layer yj, j = 1 , 2 , 3, and one output node O. Now
by applying feed-forward recall with error back-propagation learning to the
model presented in Figure 1.4, we have the following algorithm [8]:

Step 1: Initialize the weights W from the input layer to the hidden layer
and weights V from the hidden layer to the output layer. Choose the
learning parameter η (lies between 0 and 1) and error Emax.
Next, initially, error is taken as E = 0.
Step 2: Training steps start here.
Outputs of the hidden layer and the output layer are computed as
follows:
y j ¬ f ( wjx ) , j = 1, 2, 3

( )
ok ¬ f v k y , k = 1

where
wj is the jth row of W for j = 1, 2, 3
vk is the kth row of V for k = 1
f is the activation function
Preliminaries of Artificial Neural Network 7

Step 3: Error value is computed as

1
( dk - ok ) + E
2
E=
2

Here,
dk is the desired output
ok is the output of ANN
Step 4: The error signal terms of the output layer and the hidden layer
are computed as

dok = éë( dk - ok ) f ¢ ( vk y ) ùû ( error signal of the output layer )

d yj = éë( 1 - y j ) f ( w j x ) ùû dok v kj ( error signal of the hidden layer )

where ok = f(vky),   j = 1 , 2 , 3 ,   and k = 1.

Step 5: Compute components of error gradient vectors as
∂E/∂wji = δyjxi for j = 1, 2, 3 and i = 1. (For the particular ANN model,
see Figure 1.4.)
∂E/∂vkj = δokyj for j = 1, 2, 3 and k = 1. (For the particular ANN model,
see Figure 1.4.)
Step 6: Weights are modified using the gradient descent method from
the input layer to the hidden layer and from the hidden layer to the
output layer as

æ ¶E ö
w nji+1 = w nji + Dw nji = w nji + çç -h n ÷÷
è ¶w ji ø

æ ¶E ö
vkjn+1 = vkjn + Dvkjn = vkjn + çç -h n ÷÷
è ¶vkj ø

where
η is the learning parameter
n is the iteration step
E is the error function
Step 7: If E = Emax, terminate the training session; otherwise, go to step 2
with E ← 0 and initiate a new training.

The generalized delta learning rule propagates the error back by one layer,
allowing the same process to be repeated for every layer.
8 Artificial Neural Networks for Engineers and Scientists

1.5 Activation Functions
An activation or transfer function is a function that acts upon the net (input)
to get the output of the network. It translates input signals to output signals.
It acts as a squashing function such that the output of the neural network
lies between certain values (usually between 0 and 1, or −1 and 1).
Five types of activation functions are commonly used:

1. Unit step (threshold) function

2. Piecewise linear function
3. Gaussian function
4. Sigmoid function
a. Unipolar sigmoid function
b. Bipolar sigmoid function
5. Tangent hyperbolic function

Throughout this book, we have used sigmoid and tangent hyperbolic func-
tions only, which are nonlinear, monotonic, and continuously differentiable.

1.5.1 Sigmoid Function
The sigmoid function is defined as a strictly increasing and continu-
ously differentiable function.
It exhibits a graceful balance between linear and nonlinear behavior.

1.5.1.1 Unipolar Sigmoid Function

The unipolar sigmoid function is shown in Figure 1.5 and defined by the formula
1
f (x) =
1 + e - lx
where λ > 0 is the slope of the function.
The output of the uniploar sigmoid function lies in [0, 1].

0.5

–0.5 0 0.5

FIGURE 1.5
Plot of unipolar sigmoid function.
Preliminaries of Artificial Neural Network 9

–1 0 1

–1

FIGURE 1.6
Plot of bipolar sigmoid function.

1.5.1.2 Bipolar Sigmoid Function

The bipolar sigmoid function is formulated as

2 1 - e - lx
f (x) = - 1 =
1 + e - lx 1 + e - lx

The output of the bipolar sigmoid function lies between [−1, 1]. Figure 1.6
shows the plot of the function.

1.5.2 Tangent Hyperbolic Function

The tangent hyperbolic function is defined as

e x - e-x e2x - 1
f (x) = =
e x + e-x e2x + 1

The output of the tangent hyperbolic function lies in [−1, 1] and plot of tan-
gent hyperbolic function is same as bipolar sigmoid function.
Further details of ANN architecture, paradigms of learning, learning algo-
rithms, activation functions, etc., may be found in standard ANN books.

References
1. W.S. McCulloch and W. Pitts. A logical calculus of the ideas immanent in ner-
vous activity. Bulletin of Mathematical Biophysics, 5(4): 115–133, December 1943.
2. M. Minsky and S. Papert. Perceptrons. MIT Press, Cambridge, MA, 1969.
3. D.E. Rumelhart, G.E. Hinton, and J.L. McClelland. Parallel Distributed Processing.
MIT Press, Cambridge, MA, 1986.
4. R.P. Lippmann. An introduction to computing with neural nets. IEEE ASSP
Magazine, 4: 4–22, 1987.
5. J.A. Freeman and D.M. Skapura. Neural Networks: Algorithms, Applications, and
Programming Techniques. Addison-Wesley Publishing Company, Boston, MA,
1991.
10 Artificial Neural Networks for Engineers and Scientists

6. J.A. Anderson. An Introduction to Neural Networks. MIT Press, Cambridge,

London, England, U.K., 1995.
7. G.E. Hinton, S. Osindero, and Y.-W. Teh. A fast learning algorithm for deep
belief nets. Neural Computation, 18: 1527–1554, 2006.
8. J.M. Zurada. Introduction to Artificial Neural Systems. West Publishing Co., St.
Paul, MN, 1992.
9. K. Hornik, M. Stinchcombe, and H. White. Multilayer feed forward networks
are universal approximators. Neural Networks, 2(5): 359–366, January 1989.
10. T. Khanna. Foundations of Neural Networks. Addison-Wesley Press, Boston, USA,
1990.
11. R.J. Schalkoff. Artificial Neural Networks. McGraw-Hill, New York, 1997.
12. P.D. Picton. Neural Networks, 2nd edn. Palgrave Macmillan, New York, 2000.
13. R. Rojas. Neural Networks: A Systematic Introduction. Springer, Berlin, Germany,
1996.
14. S.S. Haykin. Neural Networks: A Comprehensive Foundation. Prentice Hall Inc.,
Upper Saddle River, NJ, 1999.
15. B. Yegnanarayana. Artificial Neural Networks. Eastern Economy Edition, Prentice-
Hall of India Pvt. Ltd., India, 1999.
16. M. Hajek. Neural Network. University of KwaZulu-Natal Press, Durban, South
Africa, 2005.
17. D. Graupe. Principles of Artificial Neural Networks. World Scientific Publishing
Co. Ltd., Toh Tuck Link, Singapore, 2007.
18. N.D. Lagaros and M. Papadrakakis. Learning improvement of neural networks
used in structural optimization. Advances in Engineering Software, 35(1): 9–25,
January 2004.
19. S. Chakraverty. Identification of structural parameters of two-storey shear
building by the iterative training of neural networks. Architectural Science Review
Australia, 50(4): 380–384, July 2007.
20. S. Mall and S. Chakraverty. Comparison of artificial neural network architecture
in solving ordinary differential equations. Advances in Artificial Neural Systems,
2013: 1–24, October 2013.
21. S. Mall and S. Chakraverty. Regression-based neural network training for the
solution of ordinary differential equations. International Journal of Mathematical
Modelling and Numerical Optimization, 4(2): 136–149, 2013.
22. S. Chakraverty and S. Mall. Regression based weight generation algorithm in
neural network for solution of initial and boundary value problems. Neural
Computing and Applications, 25(3): 585–594, September 2014.
23. S. Mall and S. Chakraverty. Regression based neural network model for the
solution of initial value problem. National Conference on Computational and
Applied Mathematics in Science and Engineering (CAMSE-2012), VNIT, Nagpur,
India, December 2012.
24. S. Mall and S. Chakraverty. Chebyshev neural network based model for solving
Lane–Emden type equations. Applied Mathematics and Computation, 247: 100–114,
November 2014.
25. S. Mall and S. Chakraverty. Numerical solution of nonlinear singular ini-
tial value problems of Emden–Fowler type using Chebyshev neural network
method. Neurocomputing, 149: 975–982, February 2015.
Preliminaries of Artificial Neural Network
W.S. McCulloch and W. Pitts . A logical calculus of the ideas immanent in nervous activity.
Bulletin of Mathematical Biophysics, 5(4): 115–133, December 1943.
M. Minsky and S. Papert . Perceptrons. MIT Press, Cambridge, MA, 1969.
D.E. Rumelhart , G.E. Hinton , and J.L. McClelland . Parallel Distributed Processing. MIT Press,
Cambridge, MA, 1986.
R.P. Lippmann . An introduction to computing with neural nets. IEEE ASSP Magazine, 4: 4–22,
1987.
J.A. Freeman and D.M. Skapura . Neural Networks: Algorithms, Applications, and Programming
Techniques. Addison-Wesley Publishing Company, Boston, MA, 1991.
J.A. Anderson . An Introduction to Neural Networks. MIT Press, Cambridge, London, England,
U.K., 1995.
G.E. Hinton , S. Osindero , and Y.-W. Teh . A fast learning algorithm for deep belief nets.
Neural Computation, 18: 1527–1554, 2006.
J.M. Zurada . Introduction to Artificial Neural Systems. West Publishing Co., St. Paul, MN,
1992.
K. Hornik , M. Stinchcombe , and H. White . Multilayer feed forward networks are universal
approximators. Neural Networks, 2(5): 359–366, January 1989.
T. Khanna . Foundations of Neural Networks. Addison-Wesley Press, Boston, USA, 1990.
R.J. Schalkoff . Artificial Neural Networks. McGraw-Hill, New York, 1997.
P.D. Picton . Neural Networks, 2nd edn. Palgrave Macmillan, New York, 2000.
R. Rojas . Neural Networks: A Systematic Introduction. Springer, Berlin, Germany, 1996.
S.S. Haykin . Neural Networks: A Comprehensive Foundation. Prentice Hall Inc., Upper Saddle
River, NJ, 1999.
B. Yegnanarayana . Artificial Neural Networks. Eastern Economy Edition, Prentice-Hall of India
Pvt. Ltd., India, 1999.
M. Hajek . Neural Network. University of KwaZulu-Natal Press, Durban, South Africa, 2005.
D. Graupe . Principles of Artificial Neural Networks. World Scientific Publishing Co. Ltd., Toh
Tuck Link, Singapore, 2007.
N.D. Lagaros and M. Papadrakakis . Learning improvement of neural networks used in
structural optimization. Advances in Engineering Software, 35(1): 9–25, January 2004.
S. Chakraverty . Identification of structural parameters of two-storey shear building by the
iterative training of neural networks. Architectural Science Review Australia, 50(4): 380–384,
July 2007.
S. Mall and S. Chakraverty . Comparison of artificial neural network architecture in solving
ordinary differential equations. Advances in Artificial Neural Systems, 2013: 1–24, October
2013.
S. Mall and S. Chakraverty . Regression-based neural network training for the solution of
ordinary differential equations. International Journal of Mathematical Modelling and Numerical
Optimization, 4(2): 136–149, 2013.
S. Chakraverty and S. Mall . Regression based weight generation algorithm in neural network
for solution of initial and boundary value problems. Neural Computing and Applications, 25(3):
585–594, September 2014.
S. Mall and S. Chakraverty . Regression based neural network model for the solution of initial
value problem. National Conference on Computational and Applied Mathematics in Science
and Engineering (CAMSE-2012), VNIT, Nagpur, India, December 2012.
S. Mall and S. Chakraverty . Chebyshev neural network based model for solving Lane–Emden
type equations. Applied Mathematics and Computation, 247: 100–114, November 2014.
S. Mall and S. Chakraverty . Numerical solution of nonlinear singular initial value problems of
Emden–Fowler type using Chebyshev neural network method. Neurocomputing, 149: 975–982,
February 2015.
Preliminaries of Ordinary Differential Equations
W.E. Boyce and R.C. Diprima . Elementary Differential Equations and Boundary Value
Problems. John Wiley & Sons, Inc., New York, 2001.
Y. Pinchover and J. Rubinsteinan . Introduction to Partial Differential Equations. Cambridge
University Press, Cambridge, England, 2005.
D.W. Jordan and P. Smith . Nonlinear Ordinary Differential Equations: Problems and Solutions.
Oxford University Press, Oxford, U.K., 2007.
R.P. Agrawal and D. O’Regan . An Introduction to Ordinary Differential Equations. Springer,
New York, 2008.
D.G. Zill and M.R. Cullen . Differential Equations with Boundary Value Problems. Brooks/Cole,
Cengage Learning, Belmont, CA, 2008.
H.J. Ricardo . A Modern Introduction to Differential Equations, 2nd edn. Elsevier Academic
Press, Ca, 2009.
M.V. Soare , P.P. Teodorescu , and I. Toma . Ordinary Differential Equations with Applications
to Mechanics. Springer, New York, 2007.
A.D. Poyanin and V.F. Zaitsev . Handbook of Exact Solutions for Ordinary Differential
Equations. Chapman & Hall/CRC, Boca Raton, FL, 2000.
J. Douglas and B.F. Jones . Predictor-Corrector methods for nonlinear parabolic differential
equations. Journal for Industrial and Applied Mathematics, 11(1): 195–204, March 1963.
G.F. Simmons . Differential Equations with Applications and Historical Notes. Tata McGraw-Hill,
Inc., New Delhi, India, 1972.
J. Sinharoy and S. Padhy . A Course on Ordinary and Partial Differential Equations with
Applications. Kalyani Publishers, New Delhi, India, 1986.
A. Wambecq . Rational Runge–Kutta methods for solving systems of ordinary differential
equations. Computing, 20(4): 333–342, December 1978.
R. Norberg . Differential equations for moments of present values in life insurance. Insurance:
Mathematics and Economics, 17(2): 171–180, October 1995.
C.J. Budd and A. Iserles . Geometric integration: Numerical solution of differential equations on
manifolds. Philosophical Transactions: Royal Society, 357: 945–956, November 1999.

Multilayer Artificial Neural Network

J.M. Zurada . Introduction to Artificial Neural Systems. West Publishing Co., St. Paul, MN,
1992.
S.S. Haykin . Neural Networks: A Comprehensive Foundation. Prentice Hall Inc., Upper Saddle
River, NJ, 1999.
H. Lee and I.S. Kang . Neural algorithm for solving differential equations. Journal of
Computational Physics, 91(1): 110–131, November 1990.
A.J. Meade and A.A. Fernandez . The numerical solution of linear ordinary differential equations
by feed forward neural networks. Mathematical and Computer Modelling, 19(2): 1–25, June
1994.
A.J. Meade and A.A. Fernandez . Solution of nonlinear ordinary differential equations by feed
forward neural networks. Mathematical and Computer Modelling, 20(9): 19–44, November
1994.
E. Lagaris , A. Likas , and D.I. Fotiadis . Artificial neural networks for solving ordinary and partial
differential equations. IEEE Transactions on Neural Networks, 9(5): 987–1000, September
1998.
M. Kumar and N. Yadav . Multilayer perceptrons and radial basis function neural network
methods for the solution of differential equations a survey. Computers and Mathematics with
Applications, 62(10): 3796–3811, November 2011.
K.I. Ibraheem and B.M. Khalaf . Shooting neural networks algorithm for solving boundary value
problems in ODEs. Applications and Applied Mathematics, 6(11): 1927–1941, June 2011.
B.-A. Liu and B. Jammes . Solving ordinary differential equations by neural networks.
Proceeding of 13th European Simulation Multi-Conference Modelling and Simulation: A Tool for
the Next Millennium, Warsaw, Poland, June 1999.
S. Mall and S. Chakraverty . Comparison of artificial neural network architecture in solving
ordinary differential equations. Advances in Artificial Neural Systems, 2013: 1–24, October
2013.
S. Chakraverty and S. Mall . Regression based weight generation algorithm in neural network
for solution of initial and boundary value problems. Neural Computing and Applications, 25(3):
585–594, September 2014.
A. Malek and R.S. Beidokhti . Numerical solution for high order deferential equations, using a
hybrid neural network—Optimization method. Applied Mathematics and Computation, 183(1):
260–271, December 2006.
H.S. Yazdi , M. Pakdaman , and H. Modaghegh . Unsupervised kernel least mean square
algorithm for solving ordinary differential equations. Nerocomputing, 74(12–13): 2062–2071,
June 2011.
G. Tsoulos and I.E. Lagaris . Solving differential equations with genetic programming. Genetic
Programming and Evolvable Machines, 7(1): 33–54, March 2006.
D.R. Parisi , M.C. Mariani , and M.A. Laborde . Solving differential equations with unsupervised
neural networks. Chemical Engineering and Processing 42: 715–721, September 2003.

Regression-Based ANN
S. Mall and S. Chakraverty . Comparison of artificial neural network architecture in solving
ordinary differential equations. Advances in Artificial Neural Systems, 2013: 1–24, October
2013.
S. Mall and S. Chakraverty . Regression-based neural network training for the solution of
ordinary differential equations. International Journal of Mathematical Modelling and Numerical
Optimization, 4(2): 136–149, 2013.
S. Chakraverty and S. Mall . Regression based weight generation algorithm in neural network
for solution of initial and boundary value problems. Neural Computing and Applications, 25(3):
585–594, September 2014.
S. Mall and S. Chakraverty . Regression based neural network model for the solution of initial
value problem. National Conference on Computational and Applied Mathematics in Science
and Engineering (CAMSE-2012), VNIT, Nagpur, India, December 2012.
S. Chakraverty , V.P. Singh , and R.K. Sharma . Regression based weight generation algorithm
in neural network for estimation of frequencies of vibrating plates. Journal of Computer Methods
in Applied Mechanics and Engineering, 195: 4194–4202, July 2006.
S. Chakraverty , V.P. Singh , R.K. Sharma , and G.K. Sharma . Modelling vibration frequencies
of annular plates by regression based neural Network. Applied Soft Computing, 9(1): 439–447,
January 2009.
E. Lagaris , A. Likas , and D.I. Fotiadis . Artificial neural networks for solving ordinary and partial
differential equations. IEEE Transactions on Neural Networks, 9(5): 987–1000, September
1998.
D.R. Parisi , M.C. Mariani , and M.A. Laborde . Solving differential equations with unsupervised
neural networks. Chemical Engineering and Processing, 42: 715–721, 2003.
S.T. Karris . Numerical Analysis: Using MATLAB and Spreadsheets. Orchard Publication,
Fremont, CA, 2004.
R.B. Bhat and S. Chakraverty . Numerical Analysis in Engineering. Alpha Science Int., Ltd.,
Oxford, U.K., 2004.

Single-Layer Functional Link Artificial Neural Network

S. Mall and S. Chakraverty . Chebyshev neural network based model for solving Lane–Emden
type equations. Applied Mathematics and Computation, 247: 100–114, November 2014.
S. Mall and S. Chakraverty . Numerical solution of nonlinear singular initial value problems of
Emden–Fowler type using Chebyshev neural network method. Neurocomputing, 149: 975–982,
February 2015.
S. Mall and S. Chakraverty . Application of Legendre neural network for solving ordinary
differential equations, Applied Soft Computing, 43: 347–356, 2016.
S. Mall and S. Chakraverty . Multi layer versus functional link single layer neural network for
solving nonlinear singular initial value problems. Third International Symposium on Women
computing and Informatics (WCI-2015), SCMS College, Kochi, Kerala, India, August 10–13,
Published in Association for Computing Machinery (ACM) Proceedings, pp. 678–683, 2015.
S. Mall and S. Chakraverty . Hermite functional link neural network for solving the Van der Pol-
Duffing oscillator equation. Neural Computation, 28(8): 1574–1598, July 2016.
Y.H. Pao and S.M. Philips . The functional link net and learning optimal control.
Neurocomputing, 9(2): 149–164, October 1995.
D.R. Parisi , M.C. Mariani , and M.A. Laborde . Solving differential equations with unsupervised
neural networks. Chemical Engineering and Processing, 42: 715–721, September 2003.
S. Purwar , I.N. Kar , and A.N. Jha . Online system identification of complex systems using
Chebyshev neural network. Applied Soft Computing, 7(1): 364–372, January 2007.
J.C. Patra , R.N. Pal , B.N. Chatterji , and G. Panda . Identification of nonlinear dynamic
systems using functional link artificial neural networks. IEEE Transactions on Systems, Man,
and Cybernetics, Part B—Cybernetics, 29(2): 254–262, April 1999.
J.C. Patra and A.C. Kot . Nonlinear dynamic system identification using Chebyshev functional
link artificial neural network. IEEE Transactions on Systems, Man and Cybernetics, Part
B—Cybernetics, 32(4): 505–511, August 2002.
J.C. Patra , M. Juhola , and P.K. Meher . Intelligent sensors using computationally efficient
Chebyshev neural networks. IET Science, Measurement and Technology, 2(2): 68–75, March
2008.
W.-D. Weng , C.-S. Yang , and R.-C. Lin . A channel equalizer using reduced decision feedback
Chebyshev functional link artificial neural networks. Information Sciences, 177(13): 2642–2654,
July 2007.
T.T. Lee and J.T. Jeng . The Chebyshev-polynomials-based unified model neural networks for
function approximation. IEEE Transactions on Systems, Man and Cybernetics, Part
B—Cybernetics, 28(6): 925–935, December 1998.
S.S. Yang and C.S. Tseng . An orthogonal neural network for function approximation. IEEE
Transactions on Systems, Man, and Cybernetics, Part B—Cybernetics, 26(5): 779–784,
October 1996.
J.C. Patra , W.C. Chin , P.K. Meher , and G. Chakraborty . Legendre-FLANN-based nonlinear
channel equalization in wireless. IEEE International Conference on Systems, Man and
Cybernetics, Singapore, pp. 1826–1831, October 2008.
J.C. Patra , P.K. Meher , and G. Chakraborty . Nonlinear channel equalization for wireless
communication systems using Legendre neural networks. Signal Processing, 89(11):
2251–2262, November 2009.
J.C. Patra and C. Bornand . Nonlinear dynamic system identification using Legendre neural
network. IEEE International Joint Conference on Neural Networks, Barcelona, Spain, pp. 1–7,
July 2010.
S.K. Nanda and D.P. Tripathy . Application of functional link artificial neural network for
prediction of machinery noise in opencast mines. Advances in Fuzzy Systems, 2011: 1–11,
April 2011.
R.B. Bhat and S. Chakraverty . Numerical Analysis in Engineering. Alpha Science Int. Ltd.,
Oxford, U.K., 2004.
C.F. Gerald and P.O. Wheatley . Applied Numerical Analysis. Pearson Education Inc., Boston,
MA, 2004.
E. Suli and D.F. Mayers . An Introduction to Numerical Analysis. Cambridge University Press,
Cambridge, U.K., 2003.
O.P. Singh , R.K. Pandey , and V.K. Singh . Analytical algorithm of Lane-Emden type equation
arising in astrophysics using modified homotopy analysis method. Computer Physics
Communications, 180(7): 1116–1124, July 2009.
Single-Layer Functional Link Artificial Neural Network with Regression-
Based Weights
A. Namatame and N. Ueda . Pattern classification with Chebyshev neural network. International
Journal Neural Network, 3: 23–31, 1992.
J.C. Patra . Chebyshev neural network-based model for dual-junction solar cells. IEEE
Transactions on Energy Conversion, 26: 132–140, 2011.
S. Purwar , I.N. Kar , and A.N. Jha . Online system identification of complex systems using
Chebyshev neural network. Applied Soft Computing, 7: 364–372, 2007.
J.C. Patra and A.C. Kot . Nonlinear dynamic system identification using Chebyshev functional
link artificial neural network. IEEE Transactions on Systems, Man, and Cybernetics, Part
B—Cybernetics, 32(4): 505–511, 2002.
T.T. Lee and J.T. Jeng . The Chebyshev-polynomials-based unified model neural networks for
function approximation. IEEE Transactions on Systems, Man, and Cybernetics, Part
B—Cybernetics, 28(6): 925–935, 1998.
J.C. Patra , M. Juhola , and P.K. Meher . Intelligent sensors using computationally efficient
Chebyshev neural networks. IET Science, Measurement & Technology, 2(2): 68–75, 2008.
S. Mall and S. Chakraverty . Chebyshev neural network based model for solving Lane–Emden
type equations. Applied Mathematics and Computation, 247: 100–114, November 2014.
S. Mall and S. Chakraverty . Numerical solution of nonlinear singular initial value problems of
Emden–Fowler type using Chebyshev neural network method. Neurocomputing, 149: 975–982,
February 2015.
S. Mall and S. Chakraverty . Application of Legendre neural network for solving ordinary
differential equations. Applied Soft Computing, 43: 347–356, 2016.
S. Mall and S. Chakraverty . Multi layer versus functional link single layer neural network for
solving nonlinear singular initial value problems. Third International Symposium on Women
Computing and Informatics (WCI-2015), SCMS College, Kochi, Kerala, India, August 10–13,
Published in Association for Computing Machinery (ACM) Proceedings, pp. 678–683, 2015.
S. Mall and S. Chakraverty . Hermite functional link neural network for solving the Van der Pol-
Duffing oscillator equation. Neural Computation, 28(8): 1574–1598, July 2016.
S. Chakraverty , V.P. Singh , and R.K. Sharma . Regression based weight generation algorithm
in neural network for estimation of frequencies of vibrating plates. Journal of Computer Methods
in Applied Mechanics and Engineering, 195: 4194–4202, July 2006.
S. Chakraverty , V.P. Singh , R.K. Sharma , and G.K. Sharma . Modelling vibration frequencies
of annular plates by regression based neural Network. Applied Soft Computing, 9(1): 439–447,
January 2009.
S. Mall and S. Chakraverty . Comparison of artificial neural network architecture in solving
ordinary differential equations. Advances in Artificial Neural Systems, 2013: 1–24, October
2013.
S. Mall and S. Chakraverty . Regression-based neural network training for the solution of
ordinary differential equations. International Journal of Mathematical Modelling and Numerical
Optimization, 4(2): 136–149, 2013.
S. Chakraverty and S. Mall . Regression based weight generation algorithm in neural network
for solution of initial and boundary value problems. Neural Computing and Applications, 25(3):
585–594, September 2014.
S. Mall and S. Chakraverty . Regression based neural network model for the solution of initial
value problem. National Conference on Computational and Applied Mathematics in Science
and Engineering (CAMSE-2012), VNIT, Nagpur, India, December 2012.
S.T. Karris . Numerical Analysis: Using MATLAB and Spreadsheets. Orchard Publication,
Fremont, CA, 2004.
R.B. Bhat and S. Chakraverty . Numerical Analysis in Engineering. Alpha Science Int. Ltd.,
Oxford, U.K., 2004.
Lane–Emden Equations
J.H. Lane . On the theoretical temperature of the sun under the hypothesis of a gaseous mass
maintaining its volume by its internal heat and depending on the laws of gases known to
terrestrial experiment. The American Journal of Science and Arts, 50(2): 57–74, 1870.
R. Emden . Gaskugeln, Anwendungen der mechanischenWarmen-theorie auf Kosmologie and
meteorologische Problem. Teubner, Leipzig, Germany, 1907.
H.T. Davis . Introduction to Nonlinear Differential and Integral Equations. Dover, New York,
1962.
O.W. Richardson . The Emission of Electricity from Hot Bodies. Longman, Green & Co.,
London, U.K., 1921.
S. Chandrasekhar . Introduction to Study of Stellar Structure. Dover, New York, 1967.
B.K. Datta . Analytic solution to the Lane-Emden equation. Nuovo Cimento, 111B: 1385–1388,
1996.
L. Dresner . Similarity Solutions of Nonlinear Partial Differential Equations. Pitman Advanced
Publishing Program, London, U.K., 1983.
M. Wazwaz . A new algorithm for solving differential equation Lane–Emden type. Applied
Mathematics and Computation, 118(3): 287–310, March 2001.
A.M. Wazwaz . Adomian decomposition method for a reliable treatment of the Emden–Fowler
equation. Applied Mathematics and Computation, 161(2): 543–560, February 2005.
A.M. Wazwaz . The modified decomposition method for analytical treatment of differential
equations. Applied Mathematics and Computation, 173(1): 165–176, February 2006.
N.T. Shawagfeh . Non perturbative approximate solution for Lane–Emden equation. Journal of
Mathematical Physics, 34(9): 4364–4369, September 1993.
M.S.H. Chowdhury and I. Hashim . Solutions of a class of singular second order initial value
problems by homotopy-perturbation Method. Physics Letters A, 365(5–6): 439–447, June 2007.
M.S.H. Chowdhury and I. Hashim . Solutions of Emden-Fowler Equations by homotopy-
perturbation Method. Nonlinear Analysis: Real Word Applications, 10(1): 104–115, February
2009.
S.J. Liao . A new analytic algorithm of Lane–Emden type equations. Applied Mathematics and
Computation, 142(1): 1–16, September 2003.
M. Dehghan and F. Shakeri . Approximate solution of a differential equation arising in
astrophysics using the variational iteration method. New Astronomy, 13(1): 53–59, January
2008.
K.S. Govinder and P.G.L. Leach . Integrability analysis of the Emden-Fowler equation. Journal
of Nonlinear Mathematical Physics, 14(3): 435–453, 2007.
O.P. Singh , R.K. Pandey , and V.K. Singh . Analytical algorithm of Lane-Emden type equation
arising in astrophysics using modified homotopy analysis method. Computer Physics
Communications, 180: 1116–1124, July 2009.
B. Muatjetjeja and C.M. Khalique . Exact solutions of the generalized Lane-Emden equations of
the first and second kind. Pramana, 77: 545–554, 2011.
S. Mall and S. Chakraverty . Chebyshev neural network based model for solving Lane–Emden
type equations. Applied Mathematics and Computation, 247: 100–114, November 2014.

Emden–Fowler Equations
J.H. Lane . On the theoretical temperature of the sun under the hypothesis of a gaseous mass
maintaining its volume by its internal heat and depending on the laws of gases known to
terrestrial experiment. The American Journal of Science and Arts, 2nd series, 4: 57–74, 1870.
R. Emden . Gaskugeln Anwendungen der mechanischen Warmen-theorie auf Kosmologie and
meteorologische Probleme. Teubner, Leipzig, Germany, 1907.
R.H. Fowler . The form near infinity of real, continuous solutions of a certain differential
equation of the second order. Quarterly Journal of Mathematics (Oxford), 45: 341–371, 1914.
R.H. Fowler . Further studies of Emden’s and similar differential equations. Quarterly Journal of
Mathematics (Oxford), 2: 259–288, 1931.
H.T. Davis . Introduction to Nonlinear Differential and Integral Equations. Dover Publications
Inc., New York, 1962.
S. Chandrasekhar . Introduction to Study of Stellar Structure. Dover Publications Inc., New
York, 1967.
B.K. Datta . Analytic solution to the Lane-Emden equation. Nuovo Cimento, 111B: 1385–1388,
1996.
A.M. Wazwaz . A new algorithm for solving differential equation Lane–Emden type. Applied
Mathematics and Computation, 118: 287–310, 2001.
A.M. Wazwaz . Adomian decomposition method for a reliable treatment of the Emden–Fowler
equation. Applied Mathematics and Computation, 161: 543–560, 2005.
A.M. Wazwaz . The modified decomposition method for analytical treatment of differential
equations. Applied Mathematics and Computation, 173: 165–176, 2006.
M.S.H. Chowdhury and I. Hashim . Solutions of a class of singular second order initial value
problems by homotopy-perturbation method. Physics Letters A, 365: 439–447, 2007.
M.S.H. Chowdhury and I. Hashim . Solutions of Emden-Fowler equations by homotopy-
perturbation method. Nonlinear Analysis: Real Word Applications, 10: 104–115, 2009.
J.I. Ramos . Linearization techniques for singular initial-value problems of ordinary differential
equations. Applied Mathematics and Computation, 161: 525–542, 2005.
S.J. Liao . A new analytic algorithm of Lane–Emden type equations. Applied Mathematics and
Computation, 142: 1–16, 2003.
M. Dehghan and F. Shakeri . Approximate solution of a differential equation arising in
astrophysics using the variational iteration method. New Astronomy, 13: 53–59, 2008.
K.S. Govinder and P.G.L. Leach . Integrability analysis of the Emden-Fowler equation. Journal
of Nonlinear Mathematical Physics, 14: 435–453, 2007.
O.P. Singh , R.K. Pandey , and V.K. Singh . Analytical algorithm of Lane-Emden type equation
arising in astrophysics using modified homotopy analysis method. Computer Physics
Communications, 180: 1116–1124, 2009.
B. Muatjetjeja and C.M. Khalique . Exact solutions of the generalized Lane-Emden equations of
the first and second kind. Pramana, 77: 545–554, 2011.
C.M. Mellin , F.M. Mahomed , and P.G.L. Leach . Solution of generalized Emden-Fowler
equations with two symmetries. International Journal of NonLinear Mechanics, 29: 529–538,
1994.
S.K. Vanani and A. Aminataei . On the numerical solution of differential equations of Lane-
Emden type. Computers and Mathematics with applications, 59: 2815–2820, 2010.
H. Demir and I.C. Sungu . Numerical solution of a class of nonlinear Emden-Fowler equations
by using differential transformation method. Journal of Arts and Science, 12: 75–81, 2009.
T. Kusano and J. Manojlovic . Asymptotic behavior of positive solutions of sub linear differential
equations of Emden–Fowler type. Computers and Mathematics with Applications, 62: 551–565,
2011.
A.H. Bhrawy and A.S. Alofi . A Jacobi-Gauss collocation method for solving nonlinear Lane-
Emden type equations. Communications in Nonlinear Science and Numerical Simulation, 17:
62–70, 2012.
A.S. Bataineh , M.S.M. Noorani , and I. Hashim . Homotopy analysis method for singular initial
value problems of Emden-Fowler type. Communications in Nonlinear Science and Numerical
Simulation, 14: 1121–1131, 2009.
B. Muatjetjeja and C.M. Khalique . Conservation laws for a generalized coupled bi dimensional
Lane–Emden system. Communications in Nonlinear Science and Numerical Simulation, 18:
851–857, 2013.
S. Mall and S. Chakraverty . Numerical solution of nonlinear singular initial value problems of
Emden–Fowler type using Chebyshev neural network method. Neurocomputing, 149: 975–982,
2015.
S. Mall and S. Chakraverty . Multi layer versus functional link single layer neural network for
solving nonlinear singular initial value problems. Third International Symposium on Women
Computing and Informatics (WCI-2015), SCMS College, Kochi, Kerala, India, August 10–13,
Published in Association for Computing Machinery (ACM) Proceedings, pp. 678–683, 2015.
Duffing Oscillator Equations
M.T. Ahmadian , M. Mojahedi , and H. Moeenfard . Free vibration analysis of a nonlinear beam
using homotopy and modified Lindstedt–Poincare methods. Journal of Solid Mechanics, 1:
29–36, 2009.
J. Guckenheimer and P. Holmes . Nonlinear Oscillations, Dynamical Systems and Bifurcations
of Vector Fields. Springer-Verlag, New York, 1983.
N. Srinil and H. Zanganeh . Modelling of coupled cross-flow/in-line vortex-induced vibrations
using double Duffing and van der Pol oscillators. Ocean Engineering, 53: 83–97, 2012.
S. Nourazar and A. Mirzabeigy . Approximate solution for nonlinear Duffing oscillator with
damping effect using the modified differential transform method. Scientia Iranica B, 20:
364–368, 2013.
N.A. Khan , M. Jamil , A.S. Anwar , and N.A. Khan . Solutions of the force-free Duffing-van der
pol oscillator equation. International Journal of Differential Equations, 2011: 1–9, 2011.
D. Younesian , H. Askari , Z. Saadatnia , and M.K. Yazdi . Free vibration analysis of strongly
nonlinear generalized Duffing oscillators using He’s variational approach and homotopy
perturbation method. Nonlinear Science Letters A, 2: 11–16, 2011.
Y.M. Chen and J.K. Liu . Uniformly valid solution of limit cycle of the Duffing–van der Pol
equation. Mechanics Research Communications, 36: 845–850, 2009.
A. Kimiaeifar , A.R. Saidi , G.H. Bagheri , M. Rahimpour , and D.G. Domairr . Analytical solution
for Van der Pol–Duffing oscillators. Chaos, Solitons and Fractals, 42: 2660–2666, 2009.
M. Akbarzade and D.D. Ganji . Coupled method of homotopy perturbation method and
variational approach for solution to Nonlinear Cubic-Quintic Duffing oscillator. Advances in
Theoretical and Applied Mechanics, 3: 329–337, 2010.
S. Mukherjee , B. Roy , and S. Dutta . Solution of the Duffing–van der Pol oscillator equation by
a differential transform method. Physica Scripta, 83: 1–12, 2010.
A.N. Njah and U.E. Vincent . Chaos synchronization between single and double wells Duffing
Van der Pol oscillators using active control. Chaos, Solitons and Fractals, 37: 1356–1361,
2008.
D.D. Ganji , M. Gorji , S. Soleimani , and M. Esmaeilpour . Solution of nonlinear cubic-quintic
Duffing oscillators using He’s Energy Balance Method. Journal of Zhejiang University—Science
A, 10(9): 1263–1268, 2009.
M.R. Akbari , D.D. Ganji , A. Majidian , and A.R. Ahmadi . Solving nonlinear differential
equations of Van der pol. Rayleigh and Duffing by AGM. Frontiers of Mechanical Engineering,
9(2): 177–190, 2014.
N.Q. Hu and X.S. Wen . The application of Duffing oscillator in characteristic signal detection of
early fault. Journal of Sound and Vibration, 268: 917–931, 2003.

Van der Pol–Duffing Oscillator Equation

J. Guckenheimer and P. Holmes . Nonlinear Oscillations, Dynamical Systems and Bifurcations
of Vector Fields. Springer-Verlag, New York, 1983.
M. Tsatssos . Theoretical and numerical study of the Van der Pol equation, Dissertation,
Thessaloniki, Greece, 2006.
S.S. Motsa and P. Sibanda . A note on the solutions of the Van der Pol and Duffing equations
using a linearization method. Mathematical Problems in Engineering, 2012: 1–10, July 2012.
M.R. Akbari , D.D. Ganji , A. Majidian , and A.R. Ahmadi . Solving nonlinear differential
equations of Vander Pol, Rayleigh and Duffing by AGM. Frontiers of Mechanical Engineering,
9(2): 177–190, June 2014.
S. Nourazar and A. Mirzabeigy . Approximate solution for nonlinear Duffing oscillator with
damping effect using the modified differential transform method. Scientia Iranica, 20(2):
364–368, April 2013.
S. Mukherjee , B. Roy , and S. Dutta . Solutions of the Duffing–van der Pol oscillator equation
by a differential transform method. Physica Scripta, 83: 1–12, December 2010.
A. Kimiaeifar , A.R. Saidi , G.H. Bagheri , M. Rahimpour , and D.G. Domairr . Analytical solution
for Van der Pol–Duffing oscillators. Chaos, Solutions and Fractals, 42(5): 2660–2666,
December 2009.
A.N. Njah and U.E. Vincent . Chaos synchronization between single and double wells Duffing
Van der Pol oscillators using active control. Chaos, Solitons and Fractals, 37(5): 1356–1361,
September 2008.
H. Molaei and S. Kheybari . A numerical solution of classical Van der Pol-Duffing oscillator by
He’s parameter-expansion method. International Journal of Contemporary Mathematical
Sciences, 8(15): 709–714, 2013.
N.H. Sweilam and R.F. Al-Bar . Implementation of the parameter-expansion method for the
coupled van der Pol oscillators. International Journal of Nonlinear Sciences and Numerical
Simulation, 10(2): 259–264, 2009.
C. Zhang and Y. Zeng . A simple numerical method For Van der Pol-Duffing Oscillator
Equation. International Conference on Mechatronics, Control and Electronic Engineering,
Shenyang, China, Atlantis Press, pp. 476–480, September 2014.
K. Hu and K.W. Chung . On the stability analysis of a pair of Van der Pol oscillators with
delayed self-connection position and velocity couplings. AIP Advances, 3: 112–118, 2013.
M.I. Qaisi . Analytical solution of the forced Duffing oscillator. Journal of Sound and Vibration,
194(4): 513–520, July 1996.
N.A. Khan , M. Jamil , S.A. Ali , and N.A. Khan . Solutions of the force-free Duffing-van der pol
oscillator equation. International Journal of Differential Equations, 2011: 1–9, August 2011.
V. Marinca and N. Herisanu . Periodic solutions of Duffing equation with strong non-linearity.
Chaos, Solitons and Fractals, 37(1): 144–149, July 2008.
N.Q. Hu and X.S. Wen . The application of Duffing oscillator in characteristic signal detection of
early fault. Journal of Sound and Vibration, 268(5): 917–931, December 2003.
Z. Zhihong and Y. Shaopu . Application of van der Pol–Duffing oscillator in weak signal
detection. Computers and Electrical Engineering, 41: 1–8, January 2015.
G. Wang , W. Zheng , and S. He . Estimation of amplitude and phase of a weak signal by using
the property of sensitive dependence on initial conditions of a nonlinear oscillator. Signal
Processing, 82(1): 103–115, January 2002.
E. Tamaseviciute , A. Tamasevicius , G. Mykolaitis , and E. Lindberg . Analogue electrical
circuit for simulation of the Duffing–Holmes Equation. Nonlinear Analysis: Modelling and
Control, 13(2): 241–252, June 2008.
C. Li and L. Qu . Applications of chaotic oscillator in machinery fault diagnosis. Mechanical
Systems and Signal Processing, 21(1): 257–269, January 2007.
S. Mall and S. Chakraverty . Hermite functional link neural network for solving the Van der Pol-
Duffing oscillator equation. Neural Computation, 28(8): 1574–1598, July 2016.