17

NUMERICAL
ANALYSIS - II
MUHAMMAD USMAN HAMID

MUHAMMAD USMAN HAMID

 1

PREFACE
In the beginning of university life I used only those books which were
recommended by my teachers, soon I realized, that books were not
enough to fulfill my requirements then I used every possible way to get
any information about my topics.
Now I want to say thanks to the writers of the books which I used.
I also say thank to my parents for their valuable prayers,
To my teachers for their guidance,
To my friends for their appreciation,
And I dedicated this book to MSc Mathematics Batch 2015-2017 the
students of INSTITUTES OF LEADERSHIP AND MANAGEMENT
(ILM) SARGODHA.

MUHAMMAD USMAN HAMID

UNIVERSITY OF SARGODHA
PUNJAB, PAKISTAN
(0323-6032785)

Visit us @ Youtube “learning with Usman Hamid”

 2

Objectives of the course:

This course is designed to teach the students about numerical methods and their theoretical
bases. The students are expected to know computer programming to be able to write program for
each numerical method. Knowledge of calculus and linear algebra would help in learning these
methods

Course Contents:

Forward, backward and centered difference formulae, Lagrange interpolation, Newton‟s divided
difference formula, Interpolation with a cubic spline, Hermite interpolation, least squares
approximation. Richardson‟s extrapolation, Newton-Cotes formulae, Numerical integration:
Rectangular rule, trapezoidal rule, Simpson‟s 1/3 and 3/8 rules, Boole‟s and Weddle‟s rules,
Gaussian quadrature.

Difference and Differential Equation: Formulation of difference equations, solution of

linear(homogeneous and inhomogeneous) difference equations with constant coefficients.

The Euler and modified Euler method, Runge-Kutta methods and predictor-corrector type
methods for solving initial value problems along with convergence and instability criteria. Finite
difference, collocation and variational method for boundary value problems.

Visit us @ Youtube “learning with Usman Hamid”

 3

INTRODUCTION

NUMERICAL ANALYSIS:

Numerical Analysis is the branch of mathematics that provides tools and methods for solving
mathematical problems in numerical form. In numerical analysis we are mainly interested in
implementation and analysis of numerical algorithms for finding an approximate solution to a
mathematical problem.

It is the study of algorithms that use numerical approximations (as opposed to general symbolic
manipulation) for the problem of mathematical analysis (as distinguished from discrete
mathematics)

IMPORTANCE: Numerical analysis naturally finds applications in all fields of engineering and
the physical sciences, but in the 21st century, the life science and even the arts have adopted
elements of scientific computations. Ordinary differential equations appear in the movements of
heavenly bodies (planets, stars, galaxies); optimization occurs in portfolio management;
numerical linear algebra is important for data analysis; stochastic differential equations and
Markov chains are essential in simulating living cells for medicine and biology.
Before the advent of modern computers numerical methods often depended
on hand interpolation in large printed tables. Since the mid-20th century, computers calculate the
required functions instead. These same interpolation formulas nevertheless continues to be used
a part of the software algorithms for solving differential equations.

The overall goal of the field of numerical analysis is the design and analysis of techniques to
give approximate but accurate solutions to hard problems, the variety of which is suggested by
the following;

 Advanced numerical methods are essential in making numerical weather prediction

feasible.
 Computing the trajectory of a space craft requires the accurate numerical solutions of a
system of ordinary differential equations.
 Car companies can improve the crash safety of their vehicle by using computer
simulations of car crashes. Such simulations essentially consist of solving partial
differential equations numerically.
 Hedge funds (Private investment funds) use tools from all fields of numerical analysis to
calculate the value of stocks and derivatives more precisely than other market
participants.
 Airlines use sophisticated optimization algorithms to decide ticket prices, airplane and
crew assignments and fuel needs. Historically, such algorithms were developed within
the overlapping fields of operations research.
 Insurance companies use numerical programs for actuarial analysis.

Visit us @ Youtube “learning with Usman Hamid”

 4

HISTORY:

The field of numerical analysis predates the invention of modern computers by many centuries.
Linear interpolation was already in use more than 2000 year ago. Many great mathematicians of
the past were preoccupied by numerical analysis, as is obvious from the names of important
algorithms like Newton‟s method, Lagrange‟s interpolation polynomial, Gaussian elimination or
Euler‟s method.To facilitate computations by hand, large books were produced with formulas
and tables of data such as interpolation points and function coefficients. Using these tables, often
calculated out to 16 decimal places or more for some functions, one could look up values to plug
into the formulas given and achieve very good numerical estimates of some functions. The
canonical work in the field is the NIST publications edited by Abramovitz and Stegun, a 1000
plus page book of a very large number of commonly used formulas and functions and their
values at many points. The function values are no longer very useful when a computer is
available, but the large listing of formulas can still be very handy.
The mechanical calculator was also developed as a tool for hand computation. These calculators
evolved into electronic computers in the 1940‟s and it was then found that these computers were
also useful for administrative purposes. But the invention of the computer also influenced the
field of numerical analysis, since now longer and more complicated calculations could be done.
Direct methods compute the solution to a problem in a finite number of
steps. These methods would give the precise answer if they were performed in infinite precision
arithmetic. Examples include Gaussian elimination, the QR factorization method for solving the
system of linear equations and the simplex method of linear programming. In practice, finite
precision is used and the result is an approximation of the true solution (assuming stability)

In contrast to direct methods, iterative methods are not expected to terminate in a number of
steps. Starting from an initial guess, iterative methods from successive approximations that
converge to the exact solution only in the limit. A convergence test is specified in order to decide
when a sufficiently accurate solution has (hopefully) been found. Even using infinite precision
arithmetic these methods would not reach the solution within a finite number of steps (in
general). Examples include Newton‟s method, the Bisection method and Jacobi iteration. In
computational matrix algebra, iterative methods are generally needed for large problems.
Iterative methods are more common than direct methods in numerical analysis. Some methods
are direct in principle but are usually use as though they were not, e.g. GMRES and the
conjugate gradient method. For these methods the number of steps needed to obtain the exact
solution is so large that an approximation is accepted in the same manner as for an iterative
method.

Visit us @ Youtube “learning with Usman Hamid”

 5

NUMERICAL METHOD (NUMERICAL ITERATION METHOD):

A complete set of rules for solving a problem or problems of a particular type involving only the
operation of arithmetic.

OR: A mathematical procedure that generates a sequence of improving approximate solution for
a class of problems i.e. the process of finding successive approximations.

NUMERICAL ALGORITHM (ALGORITHM OF ITERATION METHOD):

A complete set of procedures which gives an approximate solution to a mathematical problem. A

specific way of implementation of an iteration method, including to termination criteria is called
algorithm of an iteration method. In the problem of finding the solution of an equation, an
iteration method uses as initial guess to generate successive approximation to the solution.
CRITERIA FOR A GOOD METHOD

1) Number of computations i.e. Addition, Subtraction, Multiplication and Division.

2) Applicable to a class of problems. 4) Error management.
3) Speed of convergence. 5) Stability.

STABLE ALGORITHM:

Algorithm for which the cumulative effect of errors is limited, so that a useful result is generated
is called stable algorithm. Otherwise Unstable.

NUMERICAL STABILITY:

Numerical stability is a procedure that is about how a numerical scheme propagates error.

Why we use numerical iterative methods for solving equations?

As analytic solutions are often either too tiresome or simply do not exist, we need to find an
approximate method of solution. This is where numerical analysis comes into picture.

CONVERGENCE CRITERIA FOR A NUMERICAL COMPUTATION:

If the method leads to the value close to the exact solution, then we say that the method is
convergent otherwise the method is divergent. i.e.

LOCAL CONVERGENCE:

An iterative method is called locally convergent to a root, if the method converges to root for
initial guesses sufficiently close to root.

Visit us @ Youtube “learning with Usman Hamid”

 6

RATE OF CONVERGENCE OF AN ITERATIVE METHOD:

Suppose that the sequence (xn) converges to “r” then the sequence (xn) is said to converge to “r”
with order of convergence “a” if there exist a positive constant “p” such that
Thus if , the convergence is
linear. If , the convergence is quadratic and so on. Where the number “a” is called
convergence factor.

REMARK

 Rate of convergence for fixed point iteration method is linear with

 Rate of convergence for Newton Raphson method is quadratic with
 Rate of convergence for Secant method is super linear with

ORDER OF CONVERGENCE OF THE SEQUENCE

Let (x0, x1,x2,.......)be a sequence that converges to a number “a” and set Ɛn=a - xn

If there exist a number “k” and a positive constant “c” such that
Then “k” is called order of convergence of the sequence and “c” the asymptotic error constant.

CONSISTENT METHOD:

A multi – step method is consistent if it has order at least one.

Let , and the function ƒ:[a,b]×Rd×R+ Rd may be thought of as the

approximate increment per unit step, Or the approximate difference quotient and it defines the
method and consider T(x,y:h) is truncation error then the method “ƒ” is called consistent if
T(x,y:h) 0 as h 0 uniformly for (x,y) [a,b]×Rd

PRECISION:

Precision mean how close are the measurements obtained from successive iterations.

ACCURACY:

Accuracy means how close are our approximations from exact value.

DEGREE OF ACCURACY OF A QUADRATURE FORMULA:

It is the largest positive integer “n” such that the formula is exact for “xk ” for each
(k = 0,1,2,……n). i.e. Polynomial integrated exactly by method.

Visit us @ Youtube “learning with Usman Hamid”

 7

CONDITION OF A NUMERICAL PROBLEM:

A problem is well conditioned if small change in the input information causes small change in
the output. Otherwise it is ill conditioned.

ZERO STABILITY:

A numerical IVP solver is Zero Stable if small perturbation in the initial conditions do not cause
the numerical approximation to diverge away from the true solution provided the true solution of
the IVP is bounded.

A_STABILITY:

A numerical IVP solver is A- Stable if its region of absolute stability includes the entire complete
half plane with negative real part,

STEP SIZE, STEP COUNT, INTERVAL GAP

The common difference between the points i.e. h = = ti+1 - ti is called step-size.

Visit us @ Youtube “learning with Usman Hamid”

 8

SOLUTION OF NON-LINEAR EQUATIONS

There are two types of methods to find the roots of Algebraic and Transcendental equations.

(i) DIRECT METHODS (ii) INDIRECT (ITERATIVE) METHODS

DIRECT METHODS

1. Methods which calculate the required solution without any initial approximation in finite
number of steps.
2. Direct methods give the exact value of the roots in a finite number of steps.
3. These methods determine all the roots at the same time assuming no round off errors.
4. In the category of direct methods; Elimination Methods are advantageous because they
can be applied when the system is large.
5. Examples; Gauss‟s Elimination method, Gauss‟s Jordan method, Cholesky‟s method.

INDIRECT (ITERATIVE) METHODS

1. Methods based on the concept of successive approximations. The general procedure is to

start with one or more approximation to the root and obtain a sequence of iterates “x”
which in the limit converges to the actual or true solution to the root.
2. Indirect Methods determine one or two roots at a time.
3. Rounding error have less effect
4. These are self-correcting methods.
5. Easier to program and can be implemented on the computer.
6. Examples; Gauss‟s Jacobi method, Gauss‟s Seidal method, Relaxation method.

REMEMBER: Indirect Methods are further divided into two categories

I. BRACKETING METHODS II. OPEN METHODS

BRACKETING METHODS

These methods require the limits between which the root lies. e.g. Bisection method, False
position method.

OPEN METHODS

These methods require the initial estimation of the solution. e.g. Newton Raphson method.

ADVANTAGES AND DISADVANTAGES OF BRACKETING METHODS

Bracket methods always converge.

The main disadvantage is, if it is not possible to bracket the root, the method cannot applicable.

Visit us @ Youtube “learning with Usman Hamid”

 9

GEOMETRICAL ILLUSTRATION OF BRACKET FUNCTIONS

In these methods we choose two points “xn” and “xn-1” such that f(xn) and f(xn-1) are of opposite
signs. Intermediate value property suggests that the graph of “y=f(x)” crosses the x-axis between
these two points, therefore a root (say) “x=xr” lies between these two points.

REMARK: Always set your calculator at radian mod while solving Transcendental or
Trigonometric equations.

How to get first approximation? :

We can find the approximate value of the root of f(x)=0 by “Graphical method” or by
“Analytical method”.

Visit us @ Youtube “learning with Usman Hamid”

 10

DIFFERENCE OPERATORS

DIFFERENCE EQUATION
Equation involving differences is called Difference Equation.
OR an equation that consist of an independent variable „t‟, a dependent variable „y(t)‟ and one or
more several differences of the dependent variable as is called difference
equation.
The functional relationship of the difference equation is
Solution of differential equation will be sequence of values for which the equation is true for
some set of consecutive integer „t‟.
Importance: difference equation plays an important role in problem where there is a quantity „y‟
that depends on a continuous independent variable „t‟
In polynomial dynamics, modeling the rate of change of the population or modeling the growth
rate yields a differential equation for the population „y‟ as the function of time „t‟
i.e.
In differential equation models, usually the population is assumed to vary continuously I time.
Difference equation model arise when the population is modeled only at certain discrete time.
DIFFERENCE OF A POLYNOMIAL
The “nth” difference of a polynomial of degree „n‟ is constant, when the values of the
independent variable are given at equal intervals.

EXAMPLES:

i.
ii.
Order of differential equation is the difference between the largest and smallest argument „t‟
appearing in it.
For example; if then
Degree of differential equation is the highest power of „y‟
For example; if then
FINITE DIFFERENCES: Let we have a following linear D. Equation

Subject to the boundary conditions

Then the finite difference method consists of replacing every derivative in above Equation by
finite difference approximations such as the central divided difference approximations

and

Shooting Method is a finite difference method.

Visit us @ Youtube “learning with Usman Hamid”

 11

FINITE DIFFERENCES OF DIFFERENT ORDERS: Supposing the argument equally

spaced so that the difference of the values are denoted as

And are called First differences.

Second differences are as follows

In General: And are called nth differences

DIFFERENCE TABLE: The standard format for displaying finite differences is called
difference table.

DIFFERENCE FORMULAS: Difference formulas for elementary functions somewhat parallel

those of calculus. Example include the following

The differences of a constant function are zero. In symbol where „c‟ denotes a
constant. For a constant time another function we have

The difference of a sum of two functions is the sum of their differences

The „linearity property‟ generalizes the two previous results.

Where and are constants.

PROVE THAT

FOR A CONSTANT FUNCTION ALL DIFFERENCES ARE ZERO, PROVE!

Let then for all „k‟

Where is a constant function.

REMEMBER: The fundamental idea behind finite difference methods is the replace derivatives
in the differential equation by discrete approximations, and evaluate on a grid to develop a
system of equations.

COLLOCATION : Like the finite difference methods, the idea behind the collocation is to
reduce the boundary value problem to a set of solvable algebraic equations.

However, instead of discretizing the differential equation by replacing derivative with finite
differences, the solution is given a functional from whose parameters are fit by the method.

Visit us @ Youtube “learning with Usman Hamid”

 12

CRITERION OF APPROXIMATION: Some methods are as follows

i. collocation ii. Osculation iii. Least square

FORWARD DIFFERENCE OPERATOR „ (DELTA)

We define forward difference operator as

Where y=f(x)
For first order: Given function y=f(x) and a value of argument „x‟ as x=a, a+h…… a+nh etc.
Where „h‟ is the step size (increment) first order Forward Difference Operator is

For Second Order

Let =

For Third Order

In General:

Remark:

QUESTION:

Express and in terms of the value of function y.

SOLUTION
= - = ( - ) - ( - ) = -2 +

= - = - -( - )
= - -( - ) -( - )+( - ) = -4 + -4 +
QUESTION: Compute the missing values of and in the following table.
=6
=5
=1 =4 =13 =18 =24

Visit us @ Youtube “learning with Usman Hamid”

 13

SOLUTION
=1

=4

=13

=18

=24

=4 and =5 5 =4

(1) =1 =0

(3) And =18

(4)

(5)

Now since we know that

Since By table and

5=6 and

QUESTION :

Show that the value of „yn‟ can be expressed in terms of the leading value‟ „and the Binomial
leading differences .....

Visit us @ Youtube “learning with Usman Hamid”

 14

SOLUTION

Similarly

Similarly

Also from (2) and (3) we can write as

From (1) and (4) we can write as

Similarly, we can symbolically write

In general

Hence

BACKWARD DIFFERENCE OPERATOR (NEBLA)

We Define Backward Difference Operator as

= (OR)

(OR)

Visit us @ Youtube “learning with Usman Hamid”

 15

QUESTION: Show that any value of „y‟ can be expressed in terms of and its backward
differences.

SOLUTION: Since And (1)

Also (2)

Thus = (Rearranging Above)

(1)

Similarly We Can Show That

Symbolically above results can be written as

In General

i.e.

SHIFT OPERATOR “E”: Shift Operator defined as for y=f(x)

OR OR

” CENTRAL DIFFERENT OPERATOR (DELTA IN LOWER CASE)

Central Different Operator for y=f(x) defined as

(OR) ( ) ( ) (OR)

AVERAGE OPERATOR “

For y=f(x) Differential Operator defined as

[ ]

(OR) ( ) ( ) (OR)

Visit us @ Youtube “learning with Usman Hamid”

 16

DIFFERENTIAL OPERATOR “D”: For y=f(x) Differential Operator defined as

SOME USEFUL RELATIONS

From the Definition of “ and “E” we have

Now by definitions of and we have

The definition of Operators „ ‟and „E‟ gives

The definition of „ and „E‟ Yields

[ ]

Now Relation between „D‟ and „E‟ is as follows

Since

Using Taylor series expansion, we have

Taking „Log‟ on both sides we get

Hence, all the operators are expressed in terms of „E‟

⁄ ⁄ ⁄ ⁄ ⁄ ⁄

Visit us @ Youtube “learning with Usman Hamid”

 17

⁄ ⁄
( )

⁄ ⁄ ⁄ ⁄
( ) ( ) ( )

⁄ ⁄

Visit us @ Youtube “learning with Usman Hamid”

 18

⁄ ⁄
( )

⁄ ⁄ ⁄ ⁄
( )( )

PROVE THAT

Since therefore

⁄ ⁄ ⁄ ⁄
( )( )

( )

Visit us @ Youtube “learning with Usman Hamid”

 19

then

( )

√ √( )

√ √( ) √ ( )

. / . /. /
√

( )
√

Visit us @ Youtube “learning with Usman Hamid”

 20

. / . /
* + 0 1 [ ] [. / ]
√

( )

( )( )

( ) ( )

Visit us @ Youtube “learning with Usman Hamid”

 21

Visit us @ Youtube “learning with Usman Hamid”

 22

Visit us @ Youtube “learning with Usman Hamid”

 23

EXTRAPOLATION

The method of computing the values of „y‟ for a given value of „x‟ lying outside the table of
values of „x‟ is called Extrapolation.

REMEMBER THAT

RATE OF CONVERGENCE OF AN ITERATIVE METHOD: Suppose that the sequence

(xn) converges to “r” then the sequence (xn) is said to converge to “r” with order of convergence
“a” if there exist a positive constant “p” such that

AITKIN‟S EXTRAPOLATION (no need) : Since we have rate of convergence of an iterative

method

Now suppose that then it approximately yields;

also

Dividing both we get

NOTE: This method is used to accelerate convergence of sequence. i.e. sequence obtained from
iterative method.

Visit us @ Youtube “learning with Usman Hamid”

 24

INTERPOLATION

For a given table of values the process of estimating the values of

“y=f(x)” for any intermediate values of “x = g(x)” is called “interpolation”.
If g(x) is a Polynomial, Then the process is called “Polynomial” Interpolation.

ERROR OF APPROXIMATION
The deviation of g(x) from f(x) i.e. |f(x) – g (x)| is called Error of Approximation.
REMARK: A function is said to interpolate a set of data points if it passes through those points.

INVERSE INTERPOLATION
Suppose on [a, b] and has non- zero „p‟ in [a, b]
Let be „n+1‟ distinct numbers in [a, b] with for each
.
To approximate „p‟ construct the interpolating polynomial of degree „n‟ on the nodes
“y0, y1...........yn” for
Since “yk=f (xk)” and f (p) =0, it follows that (yk) = Xk and p = (0).

“Using iterated interpolation to approximate is called iterated Inverse interpolation”

OR for given the set of values of x and y , the process of finding the values of x for certain
values of y is called inverse interpolation.

LINEAR INTERPOLATION FORMULA

Where

QUADRATIC INTERPOLATION FORMULA

Where

ERRORS IN POLYNOMIAL INTERPOLATION

Given a function f(x) and a set of distinct points and xi ϵ [a, b]
Let be a polynomial of degree ≤ n that interpolates f(x) at
i.e.
Then Error define as –

Visit us @ Youtube “learning with Usman Hamid”

 25

REMARK
Sometime when a function is given as a data of some experiments in the form of tabular values
corresponding to the values of independent variable „X‟ then
1. Either we interpolate the data and obtain the function “f(x)” as a polynomial in „x‟ and
then differentiate according to the usual calculus formulas.
2. Or we use Numerical Differentiation which is easier to perform in case of Tabular form
of the data.

DISADVANTAGES OF POLYNOMIAL INTERPOLATION

 n-time differentiable  No convergence result
 big error in certain intervals  Heavy to compute for large “n”
(especially near the ends)

EXISTENCE AND UNIQUENESS THEOREM FOR POLYNOMIAL INTERPOLATION

Given with Xi‟s distinct there exists one and only one Polynomial of degree
such that
PROOF
Existence Ok from construction.
For Uniqueness:
Assume we have two polynomials P(x), q(x) of degree n both interpolate the data i.e.

Now let which will be a polynomial of degree n

Furthermore, we have
So g(x) has „n+1‟ Zeros. We must have g(x) 0. Therefor p(x) g(x).

REMEMBER:

Using Newton‟s Forward difference interpolation formula we find the n-degree polynomial
which approximate the function in such a way that and agrees at „n+1‟ equally
Spaced „X‟ Values. So that

Where Are the values of „ f ‟ in table.

Visit us @ Youtube “learning with Usman Hamid”

 26

INTERPOLATION BY ITERATION:

Given the (n+1) points where the values of x need

not necessarily be equally spaced, then to find the value of „ ‟ corresponding to any
given value of „x‟ we proceed iteratively as follows;

Obtain a first approximation to „ ‟ by considering the first two points only i.e.

| |

| |

| |

And so on ………………

Then obtain a first approximation to „ ‟ by considering the first three points only i.e.

| | Similarly And so on ………………

And at the nth stage of approximation we obtain;

| |

Then using Table of Aitken‟s Scheme

X Y= f(x)

Visit us @ Youtube “learning with Usman Hamid”

 27

QUESTION: Use Aitkin‟s iteration formula to find the value of as accurately as

possible from the data

x : 300 304 305 307

Y = f(x) : 2.4771 2.4829 2.4843 2.4871
Solution:

X Y= f(x)
300 2.4771

304 2.4829

305 2.4843

307 2.4871

Hence required solution is

QUESTION: Use Aitkin‟s iteration formula to find the value of as accurately as

possible from the data

x : 4 4.2 4.4 4.6

Y = f(x) : 0.60206 0.62325 0.64345 0.66276

And Table of Neville‟s Scheme

X Y= f(x)

Visit us @ Youtube “learning with Usman Hamid”

 28

Visit us @ Youtube “learning with Usman Hamid”

 29

” CENTRAL DIFFERENT OPERATOR (DELTA IN LOWER CASE)

Central Different Operator for y=f(x) defined as

(OR) ( ) ( ) (OR)

TABLE

X Y

CONSTRUCTION OF FORWARD DIFFERENCE TABLE (Also called Diagonal

difference table)

X Y

Visit us @ Youtube “learning with Usman Hamid”

 30

QUESTION: Construct forward difference Table for the following value of „X‟ and „Y‟

X 0.1 0.3 0.5 0.7 0.9 1.1 1.3

Y 0.003 0.067 0.148 0.248 0.370 0.518 0.697
SOLUTION

X y
0.1 0.003
0.064
0.3 0.067 0.017
0.081 0.002
0.5 0.148 0.019 0.001
0.100 0.003 0
0.7 0.248 0.022 0.001 0
0.122 0.004 0
0.9 0.370 0.026 0.001
0.148 0.005
1.1 0.518 0.031
0.179
1.3 0.697

Visit us @ Youtube “learning with Usman Hamid”

 31

BACKWARD DIFFERENCE OPERATOR (NEBLA)

We Define Backward Difference Operator as

= (OR)

(OR)

BACKWARD DIFFERENCE TABLE

NEWTON FORWARD DIFFERENCE INTERPOLATION FORMULA

Newton‟s Forward Difference Interpolation formula is

Where

DERIVATION:
Let

Visit us @ Youtube “learning with Usman Hamid”

 32

CONDITION FOR THIS METHOD

 Values of „x‟ must have equal distance i.e. equally spaced.

 Value on which we find the function check either it is near to start or end.
 If near to start, then use forward method.
 If near to end, then use backward method.

QUESTION: Evaluate given the following table of values

X : 10 20 30 40 50
f(x) : 46 66 81 93 101

SOLUTION: Here „15‟ nearest to starting point we use Newtown‟s Forward Difference
Interpolation.

X Y Y 2Y 3Y 4Y

10 46
20
20 66 -5
15 2
30 81 -3 -3
12 -1
40 93 -4
8
50 101

Visit us @ Youtube “learning with Usman Hamid”

 33

Visit us @ Youtube “learning with Usman Hamid”

 34

i.e. is the required cubic polynomial.

Visit us @ Youtube “learning with Usman Hamid”

 35

NEWTONS‟S BACKWARD DIFFERENCE INTERPOLATION FORMULA

Newton‟s Backward Difference Interpolation formula is

DERIVATION: Let Then

* +

This is required Newton‟s Gregory Backward Difference Interpolation formula.

QUESTION: For the following table of values estimate f(7.5)

X 1 2 3 4 5 6 7 8
f(x) 1 8 27 64 125 216 343 512
SOLUTION

Since „7.5‟ is nearest to End of table, So We use Newton‟s Backward Interpolation.

X Y Y 2Y 3Y 4Y

1 1
7
2 8 12
19 6
3 27 18 0
37 6
4 64 24 0
61 6
5 125 30 0
91 6
6 216 36 0
127 6
7 243 42
169
8 512

Visit us @ Youtube “learning with Usman Hamid”

 36

Since

Now

Visit us @ Youtube “learning with Usman Hamid”

 37

LAGRANGE‟S INTERPOLATION FORMULA

For points define the cardinal Function

(polynomial of n-degree)

( ) ,

The Lagrange form of interpolation Polynomial is =∑

DERIVATION OF FORMULA

Let y=f (x) be a function which takes the values so we will obtain an
n-degree polynomial = 0 + +…………..+ n

Now

Now we find the constants

Put x= in (i)

Now Put x=

Similarly

Visit us @ Youtube “learning with Usman Hamid”

 38

Putting all the values in (i) we get

+ +……………

Where

ALTERNATIVELY DEFINE

Then

Then

Visit us @ Youtube “learning with Usman Hamid”

 39

CONVERGENCE CRITERIA

Assume a triangular array of interpolation nodes exactly „ ‟ distinct nodes for

“

Further assume that all nodes are contained in finite interval then for each „n‟ we
define ( )

Then we say method “converges” if uniformly for

(OR)

Lagrange‟s interpolation converges uniformly on [a, b] for on arbitrary triangular ret if nodes of
„f‟ is analytic in the circular disk „ ‟ centered at and having radius „r‟ sufficiently large.
So that holds.

𝑎 𝑏

Visit us @ Youtube “learning with Usman Hamid”

 40

PROVE THAT ∫

PROOF: Using Lagrange‟s formula for ∑

Integrating over when

∫ ∫ ∫

∫ ∫ ∫

Now

∫ ∫ ∫

Let

∫ ∫ ∫
∫ ∫ ∫

∫ ∫

∫ ∫ ∫ ∫

[| | ] | |

( ) ( )

Since Hence the result .

PROS AND CONS OF LAGRANGE‟S POLYNOMIAL

 Elegant formula
 Slow to compute, each is different
 Not flexible; if one change a point xj , or add an additional point xn+1 one must
re-compute all

Visit us @ Youtube “learning with Usman Hamid”

 41

INVERSE LAGRANGIAN INTERPOLATION : Interchanging „x‟ and „y‟ in Lagrange‟s

interpolation formula we obtain the inverse given by ∑

QUESTION: Find langrage‟s Interpolation polynomial fitting the points

Hence find

X: =1 =3 =4 =6

Y: -3 0 30 132

ANSWER: Since + + +

+ + +

By putting values, we get

Put to get Y (5) =75

Visit us @ Youtube “learning with Usman Hamid”

 42

DIVIDED DIFFRENCE: Assume that for a given value of ( )

Then the first order divided Difference is defined as

[ ]
The 2nd Order Difference is [ ]

[ ] [ ]
Similarly [ ]

DIVIDED DIFFERENCE IS SYMMETRIC

[ ] [ ]

 Also Newton Divided Difference is Symmetric

NEWTON‟S DIVIDED DIFFRENCE INTERPOLATION FORMULA

If are arbitrarily Spaced (unequal spaced) Then the polynomial of degree „n‟ through
( ) ) where is given by the newton‟s Devided difference
Interpolation formula (Also known as Newton‟s General Interpolation formula) given by.

[ ] [ ]

[ ]

DERIVATION OF FORMULA

Let ……………..

Put

Put

( ) using above values

Visit us @ Youtube “learning with Usman Hamid”

 43

Similarly

TABLE

X Y st
Order 2nd Order 3nd Order

EXAMPLE:

X Y

10

Visit us @ Youtube “learning with Usman Hamid”

 44

A RELATIONSHIP BETWEEN nth DIVIDED DIFFERENCE AND THE nth

DARIVATIVE
Suppose “f” is n-time continuously differentiable and are (n + 1) distinct numbers in
[a, b] then there exist a number in (a, b) such that

THEOREM : nth differences of a polynomial of degree „n‟ are constant.

PROOF Let us consider a polynomial of degree „n‟ in the form

Then

We now examine the difference of polynomial

Binomial expansion yields

Therefore

Where are constants involving „h‟ but not „x‟

Thus the first difference of a polynomial of degree „n‟ is another polynomial of degree

Similarly

Therefore is a polynomial of degree in „x‟

Similarly, we can find the higher order differences and every time we observe that the degree of
polynomial is reduced by one.
After differencing n-time we get

This constant is independent of „x‟ since is constant ,

Hence The and higher order differences of a polynomial of degree „n‟ are zero.

Visit us @ Youtube “learning with Usman Hamid”

 45

NEWTION‟S DIVIDED DIFERENCE FORMULA WITH ERROR TERM

Multiplying (ii) by (iii) By

And adding all Equation‟s

Also last term will be

LIMITATIONS OF NEWTON‟S INTERPOLATION.

This formula used only when the values of independent variable „x‟ are equally spaced. Also the
differences of „y‟ must ultimately become small. Its accuracy same as Lagrange‟s Formula but
has the advantage of being computationally economical in the sense that it involves less numbers
of Arithmetic Operations.

Visit us @ Youtube “learning with Usman Hamid”

 46

ERROR TERM IN INTERPOLATION

As we know that

Approximated by polynomial of degree „n‟ the error term is

Let

And

Vanish for Choose arbitrarily ̅ from them.

Consider an interval which span the points ̅ . Total number of points

Then vanish time by Roll’s theorem

Vanish time, vanish n-time. Hence vanish 1-time choose

arbitrarily

Visit us @ Youtube “learning with Usman Hamid”

 47

THEOREM:

NEWTON‟S DIVIDED DIFFERENCE AND LAGRANGE‟S INTERPOLATION FORMULA

ARE IDENTICAL, PROVE!

PROOF: Consider y = f(x) is given at the sample points

Since by Newton‟s divided difference interpolation for is given as

( )

( ) ( )( )

( ) ( )( , - )

( ) ( )( , - )

( ) ( )( , - )

( ) ( )

( ) ( )

* ( ) + *( ) + * +

* + * + * +

* + * + * +

This is Lagrange‟s form of interpolation polynomial.

Hence both Divided Difference and Lagrange‟s are identical.

Visit us @ Youtube “learning with Usman Hamid”

 48

SPLINE

A function „S‟ is called a spline of degree „k‟ if it satisfied the following conditions.

(i) S is defined in the interval

(ii) is continuous on ;
(iii) S is polynomial of degree on each subinterval

CUBIC SPLINE INTERPOLATION

A function denoted by over the interval [ ]

Is called a cubic spline interpolant if following conditions hold.

 ( )
 ( )
 ( )
 ( )

 A spline of degree “3” is cubic spline.

NATURAL SPLINE

A cubic spline satisfying these two additional conditions

Visit us @ Youtube “learning with Usman Hamid”

 49

HERMIT INTERPOLATION

In Hermit interpolation we use the expansion involving not only the function values but also its
first derivative.

Hermit Interpolation formula is given as follows

∑ ( ) ∑ ̂ ( )

Algorithm:

 Make table

k
…. …… …… ……
….. …… …… ……

 Find jth Lagrange‟s differential polynomial of degree „n‟

 Find ( )[ ] and ̂ [ ]
 Use formula ∑ ( ) ∑ ̂ ( )

EXAMPLE

Estimate the value of using hermit interpolation formula from the following data

k
0 1.3 0.6200860 -0.5220232
1 1.6 0.4554022 -005698959
2 1.9 0.2810186 -0.5811571

Solution: we first compute Lagrange‟s Polynomials and their derivatives.

and

and

and

Now we will find the Polynomials and ̂ for n = 2 and j = 0,1,2

Visit us @ Youtube “learning with Usman Hamid”

 50

[ ][ ] * +

* +

Similarly

* + and * +

Also ̂ [ ] * +

Similarly ̂ [ ] * +

And ̂ [ ] * +

Finally using the formula

∑ ( ) ∑̂ ( )

∑ ( ) ∑̂ ( )

̂
̂ ̂

Visit us @ Youtube “learning with Usman Hamid”

 51

EXAMPLE

Estimate the value of using hermit interpolation formula from the following data

X Y= =
1.00 1.00000 0.5000
1.10 1.04881 0.47673

Solution:

At first we compute and

And and

Now putting the values in Hermit Formula

We find

* ( ) +( ) ( ) * ( ) +( )

( )

Visit us @ Youtube “learning with Usman Hamid”

 52

NUMARICAL DIFFERENTIATION

The problem of numerical differentiation is the determination of approximate values the

derivatives of a function at a given point.

FORWARD – BACKWARD DIFFERENCE FORMULAE

The formula given as follows is known as forward difference formula if and backward
difference formula if

Where is error bound.

METHOD:

 Write formula
 Use values in given formula.
 To find error bound write absolute value of less than to its answer. i.e.
| |
 Compare actual value of given function at given point if needed and approximated value.

EXAMPLE:

Use forward difference formula to approximate the derivative of at using

and and determine bound for the approximation error.

SOLUTION:

Since we have and

Then using given values;

Also for actual value ̅

Visit us @ Youtube “learning with Usman Hamid”

 53

DIFFERENTIATION USING DIFFERENCE OPERATORS

We assume that the function is given for the equally spaced „x‟ values
for to find the darivatives of such a tabular function, we proceed as follows;

USING FORWARD DIFFERENCE OPERATOR

Since

Where D is differential operator.

* +

Therefore * +

* +

* +

Similarly, for second derivative * +

* + After solving

* +

USING BACKWARD DIFFERENCE OPERATOR “ ”

Since

Since

therefore * +

Now * +

* +

* +

Similarly, for second derivative squaring (i) we get

* +

* +

Visit us @ Youtube “learning with Usman Hamid”

 54

TO COMPUTE DARIVATIVE OF A TABULAR FUNCTION AT POINT NOT FOUND

IN THE TABLE

Since

Differentiate with respect to „x‟ and using (i) & (ii)

* + ( )

* +( )

* ( ) ( ) +

Differentiate with respect to „x‟

* ( ) +

Equation & are Newton‟s backward interpolation formulae which can be used to
st nd
compute 1 and 2 derivatives of a tabular function near the end of table similarly

Expression of Newton‟s forward interpolation formulae can be derived to compute the 1st, 2nd
and higher order derivatives near the beginning of table of values.

Visit us @ Youtube “learning with Usman Hamid”

 55

DIFFERENTIATION USING CENTRAL DIFFERENCE OPERATOR

Since

Since therefore

Also as therefore ( )

( ) ( ) ( )

Since by Maclaurin series

( ) ………..

( ) ( ) ( )
[ . / ]

* +

Similarly, for second derivatives squaring (i) and simplifying

* +

* +

For calculating first and second derivative at an inter tabular form (point) we use (i) and (ii)
while 1st derivative can be computed by another convergent form for which can derived as
follows

Since * +

Multiplying R.H.S by which is unity and noting the binomial expansion

√

( )

We get * +* +

0 1

Visit us @ Youtube “learning with Usman Hamid”

 56

Therefore * +

Equation (ii) and (iii) are called STERLING FORMULAE for computing the derivative of a
tabular function. Equation (iii) can also be written as

* +

STERLING FORMULA

Sterling‟s formula is

( ) ( ) ( )
( ) * +

Where

DERIVATION:

Since by Gauss forward interpolation formula

…………(i)

Also by Gauss backward interpolation formula

…………(ii)

Taking mean of both values

( ) ( ) ( )
( ) * +

Visit us @ Youtube “learning with Usman Hamid”

 57

TWO AND THREE POINT FORMULAE

Since

Similarly

Adding (i) and (ii) we get

Subtracting (i) and (iii) we get two point formulae for the first derivative

Similarly, we know that

And

Similarly

By subtracting (iv) and (vi) we get three point formulae for computing the 2nd derivative.

Visit us @ Youtube “learning with Usman Hamid”

 58

Visit us @ Youtube “learning with Usman Hamid”

 59

Visit us @ Youtube “learning with Usman Hamid”

 60

Visit us @ Youtube “learning with Usman Hamid”

 61

NUMERICAL INTEGRATION

The process of producing a numerical value for the defining integral ∫ is called
Numerical Integration. Integration is the process of measuring the Area under a function plotted
on a graph. Numerical Integration is the study of how the numerical value of an integral can be
found.

Also called Numerical Quadrature if ∫ ∑ which refers to finding a

square whose area is the same as the area under the curve.

A GENERAL FORMULA FOR SOLVING NUMERICAL INTEGRATION

This formula is also called a general quadrature formula.

Suppose f(x) is given for equidistant value of „x‟ say a=x0, x0+h,x0+2h …. x0+nh = b

Let the range of integration (a,b) is divided into „n‟ equal parts each of width „h‟ so that
“b-a=nh”.

By using fundamental theorem of numerical analysis It has been proved the general quadrature
formula which is as follows

* ( ) ( ) (

) +

Bu putting n into different values various formulae is used to solve numerical integration.

That are Trapezoidal Rule, Simpson‟s 1/3, Simpson‟s 3/8, Boole‟s, Weddle‟s etc.

IMPORTANCE: Numerical integration is useful when

 Function cannot be integrated analytically.

 Function is defined by a table of values.
 Function can be integrated analytically but resulting expression is so complicated.

COMPOSITE (MODIFIED) NUMERICAL INTEGRATION

Trapezoidal and Simpson‟s rules are limited to operating on a single interval. Of course, since
definite integrals are additive over subinterval, we can evaluate an integral by dividing the
interval up into several subintervals, applying the rule separately on each one and then totaling
up. This strategy is called Composite Numerical Integration.

Visit us @ Youtube “learning with Usman Hamid”

 62

TRAPEZOIDAL RULE

Rule is based on approximating by a piecewise linear polynomial that interpolates

at the nodes

Trapezoidal Rule defined as follows

∫ And this is called Elementary Trapezoidal Rule.

Composite form of Trapezoidal Rule is ∫

DARIVATION (1st METHOD)

Consider a curve bounded by and we have to find ∫ i.e.

Area under the curve then for one Trapezium under the area i.e. n = 1
Y
Y

f(x2)

F(x0) F(x1)
f(x0)

h h

O a=x0 B=x1 X

O a= x0 x1 b= x2 X

For two trapeziums i. e. n = 2

For n = 3 ∫

In general for n – trapezium the points will be and function will be

Visit us @ Youtube “learning with Usman Hamid”

 63

Trapezium rule is valid for n (number of trapezium) is even or odd.

The accuracy will be increase if number of trapezium will be increased OR step size will be
decreased mean number of step size will be increased.

DARIVATION (2nd METHOD)

Define y = f(x) in an interval then

∫ ∫ ∫ ∫

∫ * + * + * +

Where is global error.

Therefore ∫ Where

REMEMBER: The maximum incurred in approximate value obtained by Trapezoidal Rule is

nearly equal to where

EXAMPLE: Evaluate ∫ using Trapezoidal Rule when

SOLUTION

X 0 1/4 1/2 3/4 1

F(x) 1 0.9412 0.8000 0.6400 0.5000
Since by Trapezoidal Rule ∫

Visit us @ Youtube “learning with Usman Hamid”

 64

SIMPSON‟S RULE

Rule is based on approximating f(x) by a Quadratic Polynomial that interpolate f(x) at

Simpson‟s Rule is defined as for simple case ∫

While in composite form it is defined as

Global error for Simpson‟s Rule is defined as

REMARK

In Simpson Rule number of trapezium must of Even and number of points must of Odd.

DERIVATION OF SIMPSON‟S RULE (1st method)

Consider a curve bounded by x = a and x = b and let „c‟ is the mid-point between and such
that we have to find ∫ i.e. Area under the curve.

f(c )
f(a) f(b)

A B C
0 a c b X

Consider

Now

Visit us @ Youtube “learning with Usman Hamid”

 65

Now ∫ ∫ where y is small change

Using Taylor Series Formula

∫ ∫ * +

Neglecting higher derivatives

∫ ∫ * +

∫ | | * +

Put this value in (i)

For n = 4

∫ ∫ ∫

In General

Visit us @ Youtube “learning with Usman Hamid”

 66

DERIVATION OF SIMPSON‟S RULE (2nd method)

∫ ∫ ∫ ∫

This is required formula for Simpson‟s (1/3) Rule

EXAMPLE

Compute √ ∫ using Simpson‟s (1/3) Rule when

SOLUTION

X 0 0.125 0.250 0.375 0.5 0.625 0.750 0.875 1

F(x) 0.798 0.792 0.773 0.744 0.704 0.656 0.602 0.544 0.484
Since by Simpson‟s Rule

√ ∫

EXERCISE:

i. Compute ∫ using Simpson‟s (1/3) Rule when

ii. Compute ∫ using Simpson‟s (1/3) Rule

iii. Compute ∫ using Simpson‟s (1/3) Rule and also obtain the error bound by
taking
iv. Compute ∫ using Simpson‟s (1/3) Rule by dividing the interval of
integration into eight equal parts.

Visit us @ Youtube “learning with Usman Hamid”

 67

SIMPSON‟S RULE

Rule is based on fitting four points by a cubic.

Simpson‟s Rule is defined as for simple case

While in composite form (“n” must be divisible by 3) it is defined as

DERIVATION

∫ ∫ ∫ ∫

This is required formula for Simpson‟s (3/8) Rule.

REMARK: Global error in Simpson‟s (1/3) and (3/8) rule are of the same order but if we
consider the magnitude of error then Simpson (1/3) rule is superior to Simpson‟s (3/8) rule.

IN NUMERICAL INTEGRATION, WHICH METHOD IS BETTER THAN OTHERS?

Simpson 1/3 Rule is sufficiently accurate method. It based on fitting 3 points with a quadratic. It
is best because it has low error than others. i.e. . The global error of Simpson 1/3
and 3/8 rule is same but if we consider the magnitude of error term we notice that Simpson 1/3
rule is superior than Simpson 3/8 rule.

Visit us @ Youtube “learning with Usman Hamid”

 68

TRAPEZOIDAL AND SIMPSON‟S RULE ARE CONVERGENT

If we assume Truncation error, then in the case of Trapezoidal Rule

Where is the exact integral and the approximation. If

then assuming “ ” bounded

(This the definition of convergence of Trapezoidal Rule)

For Simpson‟s Rule we have the similar result

If then assuming “ ” bounded

(This the definition of convergence of Simpson‟s Rule)

ERROR TERMS

Rectangular Rule

Trapezoidal Rule

Simpson‟s (1/3) Rule

Simpson‟s (3/8) Rule

Visit us @ Youtube “learning with Usman Hamid”

 69

Visit us @ Youtube “learning with Usman Hamid”

 70

Visit us @ Youtube “learning with Usman Hamid”

 71

Visit us @ Youtube “learning with Usman Hamid”

 72

Visit us @ Youtube “learning with Usman Hamid”

 73

Visit us @ Youtube “learning with Usman Hamid”

 74

Visit us @ Youtube “learning with Usman Hamid”

 75

Visit us @ Youtube “learning with Usman Hamid”

 76

Visit us @ Youtube “learning with Usman Hamid”

 77

WEDDLE‟S

In this method “n” should be the multiple of 6. Rather function will not applicable. This method
also called sixth order closed Newton‟s cotes (or) the first step of Romberg integration.

First and last terms have no coefficients and other move with 5, then 1, then 6.

Weddle‟s Rule is given by formula

∫ [ ]

EXAMPLE: for ∫ at n = 6

X 0.25 0.5 0.75 1 1.25 1.5 1.75

F(x) 0.9411 0.8 0.64 0.5 0.4 0.3 0.2
Now using formula∫

BOOLE‟S RULE

The method approximate ∫ for „5‟ equally spaced values. Rule is given by George
Bool. Rule is given by following formula

EXAMPLE: Evaluate ∫ at n = 4 and h = 0.1

SOLUTION

X 0.2 0.3 0.4 0.5 0.6

F(x) 0.96 0.92 0.86 0.80 0.74
Now using formula ∫

∫ After putting the values.

Visit us @ Youtube “learning with Usman Hamid”

 78

RECTANGULAR RULE

Rule is also known as Mid-Point Rule. And is defined as follows for „n + 1‟ points.

In general ∫ ∑

REMEMBER

 As we increased „n‟ or decreased „h‟ the accuracy improved and the approximate solution
becomes closer and closer to the exact value.
 If „n‟ is given, then use it. If „h‟ is given, then we can easily get „n‟.
 If „n‟ is not given and only „points‟ are discussed, then „1‟ less that points will be „n‟. For
example, if „3‟ points are given then „n‟ will be „2‟.
 If only table is given, then by counting the points we can tell about „n‟.one point will be
greater than „n‟ in table.

EXAMPLE

Evaluate ∫ for n = 4 using Rectangular Rule.

SOLUTION

Here a = 1, b = 3 then

X 1 3/2 2 5/2 3
F(x) 1 4/9 1/4 4/25 1/5

Visit us @ Youtube “learning with Usman Hamid”

 79

DOUBLE INTEGRATION

Y/X 2 2.1 2.2 2.3 2.4

1 0.5 0.4762 0.4545 0.4348 0.4167
1.1 0.4545 0.4329 0.4132 0.3953 0.3788
1.2 0.4167 0.3968 0.3788 0.3623 0.3472
1.3 0.3846 0.3663 0.3497 0.3344 0.3205
1.4 0.3571 0.3401 0.3247 0.3106 0.2976
Now using previous formulae we get the required results

FOR TRAPEZOIDAL RULE: FOR SIMPSON’S RULE:

Verify actual integration by yourself.

Visit us @ Youtube “learning with Usman Hamid”

 80

QUESTION: Evaluate ∫ ∫ by Trapezoidal rule for h = 0.25 = k

SOLUTION:

And

STEP – I:

Y/X 1 1.25 1.50 1.75 2

1 0.5 0.4444 0.4 0.3636 0.3333
1.25 0.4444 0.4 0.3636 0.3333 0.3077
1.50 0.4 0.3636 0.3333 0.3077 0.2857
1.75 0.3636 0.3333 0.3077 0.2857 0.2667
2 0.3333 0.3077 0.2857 0.2667 0.25

STEP –I I:

STEP –III:

∫ ∫

Visit us @ Youtube “learning with Usman Hamid”

 81

⁄ ⁄
QUESTION: Evaluate ∫ ∫ √

SOLUTION: Take n = 4 (by own choice) then

And

STEP – I: √

Y/X 0

0 0 0.6186 0.8409 0.9612 1

0.6186 0.8409 0.9612 1 0.9612

0.8409 0.9612 1 0.9612 0.8409

0.9612 1 0.9612 0.8409 0.6186

1 0.9612 0.8409 0.6186 0

STEP –I I:

∫ ( ) ( ) ( ) * ( ) ( ) ( )+

∫ ( ) ( ) ( ) * ( ) ( ) ( )+

∫ ( ) ( ) ( ) * ( ) ( )
( )+

∫ ( ) ( ) ( ) * ( ) ( ) ( )+

⁄ ⁄
STEP –III: ∫ ∫ √

Visit us @ Youtube “learning with Usman Hamid”

 82

QUESTION: Evaluate ∬ where D is the square with comes at (1,1) , (2,1) ,(2,2) , (1,2)

SOLUTION: Take n = 4 (by own choice) then

Also

STEP – I:

Y/X 1 1.25 1.50 1.75 2

1 0.5 0.3902 0.3077 0.2462 0.2
1.25 0.3902 0.3200 0.2623 0.2162 0.1798
1.50 0.3077 0.2623 0.2222 0.1882 0.1600
1.75 0.2462 0.2162 0.1882 0.1633 0.1416
2 0.2 0.1798 0.1600 0.1416 0.1250
STEP –I I:

STEP –III:

∫ ∫

Visit us @ Youtube “learning with Usman Hamid”

 83

QUESTION: Evaluate ∫ ∫ by using Simpson (1/3) rule

SOLUTION: Take n = 4 (by own choice) then

Also

STEP – I:

Y/X 0 0.25 0.50 0.75 1

1 1 1.6250 1.25 1.5625 2
1.25 1.5625 1.6250 1.8125 2.1250 2.5625
1.50 2.25 2.3125 2.5 2.8125 3.25
1.75 3.0625 3.1250 3.3125 3.6250 4.0625
2 4 4.0625 4.2500 4.5625 5

STEP –I I:

∫
∫
∫

STEP –III:

∫ ∫

Visit us @ Youtube “learning with Usman Hamid”

 84

GUASSIAN QUADRATURE FORMULAE

DERIVATION OF TWO-POINT GAUSS QUADRATURE RULE

Method 1:
The two-point Gauss quadrature rule is an extension of the trapezoidal rule approximation where
the arguments of the function are not predetermined as a and b, but as unknowns x1 and x2. So in
the two-point Gauss quadrature rule, the integral is approximated as
∫ ≈ c1 f (x1) + c2 f (x2)
There are four unknowns x1, x2, c1 and c2. These are found by assuming that the formula gives
exact results for integrating a general third order polynomial,
.
Hence ∫ ∫

∫ | |

∫ * +

The formula would then give

∫ ≈ c1 f (x1) + c2 f (x2)

Equating Equations (i) and (ii) gives

( ) ( ) ( )
[ ]

This will give us

∫

Since in Equation (iii), the constants a0, a1, a2, and a3 are arbitrary, the coefficients of a0, a1, a2,
and a3 are equal. This gives us four equations as follows

( )

( )

{( )

Visit us @ Youtube “learning with Usman Hamid”

 85

we can find that the above four simultaneous nonlinear equations have only one acceptable
solution

, , ( )( ) , ( )( )
√ √

Hence

∫ ≈ c1 f (x1) + c2 f (x2) *( )( ) + *( )( ) +
√ √

Method 2 : We can derive the same formula by assuming that the expression gives exact values
for the individual integrals of∫ ,∫ ,∫ , and ∫ The reason the formula can
also be derived using this method is that the linear combination of the above integrands is a
general third order polynomial given by .

These will give four equations as follows

∫
∫ ( )

∫ ( )

{∫ ( )

These four simultaneous nonlinear equations can be solved to give a single acceptable solution
, , ( )( ) , ( )( )
√ √

Hence ∫ ≈ c1 f (x1) + c2 f (x2) *( )( ) + *( )( )

√ √

Since two points are chosen, it is called the two-point Gauss quadrature rule. Higher point
versions can also be developed.
HIGHER POINT GAUSS QUADRATURE FORMULAS
∫ ≈ c1 f (x1) + c2 f (x2) + c3 f (x3) is called the three-point Gauss quadrature rule. The
coefficients c1, c2 and c3, and the function arguments x1, x2 and x3 are calculated by assuming the
formula gives exact expressions for integrating a fifth order polynomial
∫
General n -point rules would approximate the integral
∫ ≈ c1 f (x1) + c2 f (x2) + . . . . . .. + cn f (xn )

Visit us @ Youtube “learning with Usman Hamid”

 86

A number of particular types of Gaussian formulae are given as follows.

GUASSIAN LEGENDER FORMULA: This formula takes the form ∫ ∑

And Truncation error for formula is ∑

Where “ ” is the approximate integral obtained by n – point formula.

GUASS – LAGURRE FORMULA

This formula takes the form ∫ ∑

GUASS – HERMITE FORMULA

This formula takes the form ∫ ∑

GUASS – CHEBYSHEV FORMULA

This formula takes the form ∫ ∑

√

Where “ ” is zero n – Chebysheves polynomial

EXAMPLE: Find Gauss 2 and 3 point formula for ∫ and compare with exact value.

Solution: Firstly interval of integration are transformed to [0,1] to [-1,1]

For this , let

Since

( ) ( )
Now ∫ ∫ ∫ ………….(i)

Now Gauss 2 point formula is ∫ ( ) ( )

√ √

( ) ( ) ( )
∫ √ √

( )
∫

( )
∫ ∫ ∫

Visit us @ Youtube “learning with Usman Hamid”

 87

Also Gauss 3 point formula is ∫ .√ / . √ /

( ) .√ / ( ) . √ /
∫

( )
∫

( )
∫

( )
∫ ∫ ∫

Exact value: ∫ | | * +

EXAMPLE: Find Gauss quadrature formula for ∫

Solution: Firstly interval of integration are transformed to [0,1] to [-1,1]

For this , let

Since

Also

( ) ( )
Now ∫ ∫ ∫ ∫ ………….(i)
( )

Now Gauss 2 point formula is ∫ ( ) ( )

√ √

(√ ) ( √ )
( )
∫
√ √

( )
∫
( )
∫

Visit us @ Youtube “learning with Usman Hamid”

 88

NEWTON‟S COTES FORMULA

A quadrature formula of the form ∫ ∑ is called a Newton‟s Cotes Formula

if the nodes are equally spaced. Where ∫ ∫ ∏

General Newton‟s Cotes Formula has the form

∫ ∑ ∫ ∏ ∫ ∏

REMARK: Trapezoidal and Simpson‟s Rule Are Close Newton Cotes formulae while
Rectangular Rule is Open Newton Cotes formula.

LIMITATION OF NEWTON‟S COTES: Newton‟s Cotes formulae (Simpson‟s, Rectangular

Rule, and Trapezoidal Rule) are not suitable for Numerical integration over large intervals. Also
Newton‟s Cotes formulas which are based on polynomial interpolation would be inaccurate over
a large interval because of oscillatory nature of high degree polynomials. To solve this problem,
we use composite Numerical integration.

FORMULA DARIVATION: We shall approximate the given tabulated function by a

polynomial and then integrate this polynomial.

Suppose we are given the data at equispaced points with spacing

we can represent the polynomial by any standard interpolation polynomial.

Now by using Lagrange‟s formula ∑

With associated error term

And

Where

Integrating (i) from w. r. to „x‟

∫ ∫ ∑ ∫

∫ ∑ ∫ ∑ (∫ ) ∑ ∫

And are called Newton‟s Cotes

Visit us @ Youtube “learning with Usman Hamid”

 89

HOW TO FIND NEWTON‟S COTES?

Let equispaced nodes are defined as and

change the variable

Since , …………….. And

Using above values in (IV) we get

So

Now and

When

If x = b, 𝑥 = a
𝑥 𝑥
𝑝

𝑥 𝑥
𝑝
𝑏 𝑎
𝑛
n=p

Now putting in we get

Visit us @ Youtube “learning with Usman Hamid”

 90

Since ∫ therefore after putting and “dx”

As then and if

This is required formula for Newton Cotes.

ERROR TERM let

Using in we get

Integrating both sides ∫ ∫

E(x) is called integral error.

Visit us @ Youtube “learning with Usman Hamid”

 91

ALTERNATIVE METHOD FOR DARIVATION OF TRAPEZOIDAL RULE AND ITS

ERROR TERM

For trapezoidal rule put ∑

Integrating both sides

∫ ∫ ∫ ∫

∫ ∫ ∫ ∫

Now by changing variables

∫ ∫ ∫

∫ ∫ ∫ ∫

∫ | | | | | |

∫ As required.

SIMPSON‟S RULE AND ERROR TERM

Since ∫

And ∫

Visit us @ Youtube “learning with Usman Hamid”

 92

Putting n = 2, k = 0 in (i) we get

∫ ∫ ∫

| | ( )

Now Putting n = 2, k = 1 in (i) we get

∫ ∫ | |

Now Putting n = 2, k =2 in (i) we get

∫ | | ( )

ERROR TERM FOR SIMPSON‟S RULE:

Now Putting n = 2 in (ii) we get

∫ ∫

| | ( )

Error term is zero so we find Global error term

Now for n = 3

∫ ∑

If then (i) becomes

Visit us @ Youtube “learning with Usman Hamid”

 93

DIFFERENTIAL EQUATIONS

DIFFERENCE EQUATION
Equation involving differences is called Difference Equation.
OR an equation that consist of an independent variable „t‟, a dependent variable „y(t)‟ and one or
more several differences of the dependent variable as is called difference
equation.
The functional relationship of the difference equation is
Solution of differential equation will be sequence of values for which the equation is true for
some set of consecutive integer „t‟.
Importance: difference equation plays an important role in problem where there is a quantity „y‟
that depends on a continuous independent variable „t‟
In polynomial dynamics, modeling the rate of change of the population or modeling the growth
rate yields a differential equation for the population „y‟ as the function of time „t‟
i.e.
In differential equation models, usually the population is assumed to vary continuously I time.
Difference equation model arise when the population is modeled only at certain discrete time.
DIFFERENCE OF A POLYNOMIAL
The “nth” difference of a polynomial of degree „n‟ is constant, when the values of the
independent variable are given at equal intervals.

EXAMPLES:

i.
ii.

ORDER AND DEGREE OF DE:

The highest derivative involved in the equation determines the order of Differential Equation and
the power of highest derivative in Differential Eq. is called degree of D.E. for example

( ) has order “2” and degree “1”

Order of differential equation is the difference between the largest and smallest argument „t‟
appearing in it.
For example; if then

Degree of differential equation is the highest power of „y‟

For example; if then

Visit us @ Youtube “learning with Usman Hamid”

 94

DIFFERENTIAL EQUATION: It is the relation which involves the dependent variable,

independent variable and Differential co-efficient i.e.

ORDINARY DIFFERENTIAL EQUATION: If differential co-efficient of Differential

Equation are total, then Differential Equation is called Ordinary Differential equation.
e. g.

PARTIAL DIFFERENTIAL EQUATION: If differential co-efficient of Differential Equation

are partial, then Differential Equation is called Ordinary Differential equation.
e. g.

SOLUTION OF DIFFERENTIAL EQUATION: A solution to a difference equation is any

function which satisfies it. It is the relation which satisfies the Differential Equation as consider

Then Are all solution of above equation.

THE MOST GENERAL SOLUTION

It is the solution which contains as many arbitrary constants as the order of differential equation.
e.g. Is a 2nd order Differential Eq. with constant co-efficient and general solution is

PARTICULAR SOLUTION: Solution which can be obtained from General Solution by giving
different values to the arbitrary constants in
For example

SINGULAR SOLUTION: Solution which cannot be obtained from General Solution by giving
different values to the arbitrary constants.

HOMOGENOUS DIFFERENTIAL EQUATION: A differential equation for which

is a solution is called a Homogenous Differential Equation where „u‟ is unknown function. In
other words, a differential equation which always possesses the trivial solution is
called Homogenous Differential Equation.

NON-HOMOGENOUS DIFFERENTIAL EQUATION: A differential equation for which

(i.e. Non-Trivial solution) is a solution is called a Nonhomogeneous Differential
Equation where „u‟ is unknown function.

Visit us @ Youtube “learning with Usman Hamid”

 95

INITIAL AND BOUNDARY CONDITIONS: To evaluate arbitrary constant in the General

solution we need some conditions on the unknown function or solution corresponding to some
values of the independent variables. Such conditions are called Boundary or Initial conditions.

If all the conditions are given at the same value of the independent variable, then they are called
Initial conditions. If the conditions are given at the end points of the independent variable, then
they are called Boundary conditions.

INITIAL VALUE PROBLEM: An initial value problem for a first order Ordinary Differential
Equation is the equation together with an initial condition on a specific interval

Such that

The equation is Autonomous if is independent of „x‟

BOUNDARY VALUE PROBLEM: A problem in which we solve an Ordinary Differential

Equation of order two subject to condition on or at two different points is called a
two point boundary value problem or simply a Boundary value problem.
OR A differential equation along with one or more boundary conditions defines a boundary
value problem.

CONVEX SET: A set is said to be convex if whenever and belong to

„D‟ then also belong to „D‟ for every in [0, 1]

LIPSCHITZ CONDITION:A function is said to satisfy a Lipschitz condition in the

variable „y‟ on a set if a constant exists with

Whenever and are in and is called

Lipschitz constant for

WELL – POSED PROBLEM: The initial value problem

is said to be a well - posed problem if (i) A unique solution y(x) to the problem exist.

There exist constants and such that for any with whenever
is continuous with ⌈ ⌉ and when the initial value problem
has a unique solution that satisfies

The problem is called a Perturbed problem associated with

Visit us @ Youtube “learning with Usman Hamid”

 96

SOME STANDARD TECHNIQUES FOR SOLVING ELEMENTARY DIFFERENTIAL

EQUATIONS ANALYTICALLY

SECOND ORDER HOMOGENEOUS LINEAR DIFFERENCE EQUATION

The homogeneous difference equation of order 2 is

Suppose that be the non – trivial solution of (i) then it must satisfies the eq (i) i.e.

Since is supposed to be non – trivial solution therefore

This quadratic equation is called auxiliary equation or characteristic equation.

Let be the roots of equation (ii) then for these values there corresponds two solutions,
which are given by

Now three cases appear

CASE I : BOTH ROOTS ARE REAL AND UNEQUAL(DISTINCT):

i.e. then general solution of (i) is

or

EXAMPLE : Solve the following homogeneous differential equation

SOLUTION : Given

Let be the non – trivial solution of (i) then it must satisfies the eq (i) i.e.

Since is supposed to be non – trivial solution therefore

So

Visit us @ Youtube “learning with Usman Hamid”

 97

EXAMPLE : Solve the following homogeneous differential equation

SOLUTION : Given

Let be the non – trivial solution of (i) then it must satisfies the eq (i) i.e.

Since is supposed to be non – trivial solution therefore

Here then using quadratic formula we get √

√ √

then general solution of (i) is ( √ ) ( √ )

CASE II : BOTH ROOTS ARE REAL AND EQUAL:

i.e.

then general solution of (i) is

or in general.

EXAMPLE : Solve the following homogeneous differential equation

SOLUTION : Given

Let be the non – trivial solution of (i) then it must satisfies the eq (i) i.e.

Since is supposed to be non – trivial solution therefore

then general solution of (i) is ( )

Visit us @ Youtube “learning with Usman Hamid”

 98

CASE III : BOTH ROOTS ARE IMAGINARY:

i.e. , the roots are complex (say)

let

then general solution of (i) is

where √ and ( )

EXAMPLE : Solve the following homogeneous differential equation

SOLUTION : Given

Let be the non – trivial solution of (i) then it must satisfies the eq (i) i.e.

Since is supposed to be non – trivial solution therefore

where √ √ √ for both above

and ( ) also ( )

then general solution of (i) is

( ) (√ ) (

) (√ )

EXAMPLE :

Solve the following homogeneous differential equation

SOLUTION : Given

Let be the non – trivial solution of (i) then it must satisfies the eq (i) i.e.

Visit us @ Youtube “learning with Usman Hamid”

 99

Since is supposed to be non – trivial solution therefore

-1 1 1 -1 -1
-1 0 1
1 0 -1 0

then general solution of (i) is

EXAMPLE :

Solve the following homogeneous differential equation with

SOLUTION : Given

Let be the non – trivial solution of (i) then it must satisfies the eq (i) i.e.

Since is supposed to be non – trivial solution therefore

So ( )

Now using ( ) ………..(iii)

( ) ………..(iv)

then

then general solution of (i) is ( )

EXAMPLE : Solve the following homogeneous differential equation

SOLUTION : Given

Let be the non – trivial solution of (i) then it must satisfies the eq (i) i.e.

Visit us @ Youtube “learning with Usman Hamid”

 100

( ) ( )

Since is supposed to be non – trivial solution therefore ( )

where √ √ ( ) √ also √ √ ( ) √

and ( ) . /

also ( ) . /

then general solution of (i) is ( )( )

EXAMPLE :

Solve the following homogeneous differential equation

SOLUTION : Given

Let be the non – trivial solution of (i) then it must satisfies the eq (i) i.e.

Since is supposed to be non – trivial solution therefore

- 1 -6 14 -14 5
1 1 -5 9 -5
1 -5 9 -5 0
1 -4 5
1 1 -4 5 0
using quadratic formula.

where √ √ √ also √ √ √

Visit us @ Youtube “learning with Usman Hamid”

 101

and ( ) ( ) also ( ) ( )

EXAMPLE :

Solve the following homogeneous differential equation

with

SOLUTION : Given

Let be the non – trivial solution of (i) then it must satisfies the eq (i) i.e.

Since is supposed to be non – trivial solution therefore

-1 1 6 11 6
-1 -5 -6
1 5 6 0

then general solution of (i) is

Now using ………..(iii)

………..(iv)

………..(v)

Eq (iii) – Eq (iv) ………..(vi) and Eq (iv) – Eq (v)

………..(vii)

Eq (vi) – Eq (vii)

Put in (vi) Put in (iii)

then general solution of (i) is

EXERCISE:

i.
ii.

Visit us @ Youtube “learning with Usman Hamid”

 102

SECOND ORDER INHOMOGENEOUS LINEAR DIFFERENCE EQUATION

The non - homogeneous difference equation has the form as

If is the solution of nth order homogeneous difference equation and is the solution of non
– homogeneous linear difference equation then where is also called
complimentary function and is particular integral.

TYPE I : When R.H.S of given non – homogeneous difference equation is constants. i.e.
then to find put and then evaluate the value of A.

EXAMPLE : Solve the following homogeneous differential equation

SOLUTION : Given

Firstly we find characteristic solution.

Let be the non – trivial solution of (i) then it must satisfies the eq (i) i.e.

Since is supposed to be non – trivial solution therefore

So
To find particular integral. Let
Now general solution is

EXAMPLE :

Solve the following homogeneous differential equation

SOLUTION : Given

Firstly we find characteristic solution.

Let be the non – trivial solution of (i) then it must satisfies the eq (i) i.e.

Visit us @ Youtube “learning with Usman Hamid”

 103

Since is supposed to be non – trivial solution therefore

So

TO FIND PARTICULAR INTEGRAL.: Since in characteristic solution and

involves in homogeneous solution of given equation. So it does not particular
integral for equation. So, we take higher power for „k‟‟ then put in (ii)

(after solving)
Thus
Now general solution is

TYPE II : When R.H.S of given non – homogeneous difference equation has the form as
where are constants then to find put and then evaluate
the value of A.

EXAMPLE : Solve the following homogeneous differential equation

SOLUTION : Given

Firstly we find characteristic solution.

Let be the non – trivial solution of (i) then it must satisfies the eq (i) i.e.

Since is supposed to be non – trivial solution therefore

√ using quadratic formula. So ( √ ) ( √ )

To find particular integral. Let

Now general solution is ( √ ) ( √ )

Visit us @ Youtube “learning with Usman Hamid”

 104

EXAMPLE :

Solve the following homogeneous differential equation

SOLUTION : Given

Firstly we find characteristic solution.

Let be the non – trivial solution of (i) then it must satisfies the eq (i) i.e.

Since is supposed to be non – trivial solution therefore

where √ √ √

and ( ) ( )

( )√

To find particular integral. Let

Now general solution is ( )√

EXAMPLE :

Solve the following homogeneous differential equation

SOLUTION : Given

Firstly we find characteristic solution.

Let be the non – trivial solution of (i) then it must satisfies the eq (i) i.e.

Visit us @ Youtube “learning with Usman Hamid”

 105

Since is supposed to be non – trivial solution therefore

To find particular integral. put in (i)

(after solving)
Thus
Now general solution is

EXAMPLE :

Solve the following homogeneous differential equation

SOLUTION : Given

Firstly we find characteristic solution.

Let be the non – trivial solution of (i) then it must satisfies the eq (i) i.e.

Since is supposed to be non – trivial solution therefore

using Quadratic formula.

where √ √ √ and ( ) ( )

. ( ( )) ( ( ))/

To find particular integral. put in (i)

(after solving)
Thus
Now general solution is

. ( ( )) ( ( ))/

Visit us @ Youtube “learning with Usman Hamid”

 106

EXAMPLE :

Solve the following homogeneous differential equation

SOLUTION : Given

Firstly we find characteristic solution.

Let be the non – trivial solution of (i) then it must satisfies the eq (i) i.e.

Since is supposed to be non – trivial solution therefore

factorizing. So

To find particular integral. put in (i)

(after solving)
Thus
Now general solution is

TYPE III : When R.H.S of given non – homogeneous difference equation is a polynomial i.e
etc. then to find
put ……….. upto the highest power of „k‟ defined in the
difference equation and the evaluate the values of ……………..

EXAMPLE :

Solve the following homogeneous differential equation

SOLUTION : Given

Firstly we find characteristic solution.

Let be the non – trivial solution of (i) then it must satisfies the eq (i) i.e.

Since is supposed to be non – trivial solution therefore

Visit us @ Youtube “learning with Usman Hamid”

 107

So

To find particular integral. Let in (i)

Comparing the coefficient of same powers of „k‟ we get

put in (iv) put in (iii)

Now general solution is ( )

EXERCISE : Solve the following homogeneous differential equation

i.
ii.
iii.

TYPE IV : When R.H.S of given non – homogeneous difference equation has the form as
where is a polynomial in „k‟ then to find
put ……….. upto the highest power of „k‟
defined in the difference equation and the evaluate the values of ……………..

EXAMPLE :

Solve the following homogeneous differential equation

SOLUTION : Given

Firstly we find characteristic solution.

Let be the non – trivial solution of (i) then it must satisfies the eq (i) i.e.

Since is supposed to be non – trivial solution therefore

Visit us @ Youtube “learning with Usman Hamid”

 108

So

To find particular integral. Let in (i)

[
]

[
]

[
]

Comparing the coefficient of same powers of „k‟ we get

put in (iv) put in (iii)

Now general solution is ( )

EXERCISE : Solve the following homogeneous differential equation

i.
ii.
iii.
iv.

Visit us @ Youtube “learning with Usman Hamid”

 109

TYPE V : When R.H.S of given non – homogeneous difference equation has the form as
where are constants then to find
put or then evaluate the values of
……………..

EXAMPLE :

Solve the following homogeneous differential equation

SOLUTION : Given

Firstly we find characteristic solution.

Let be the non – trivial solution of (i) then it must satisfies the eq (i) i.e.

Since is supposed to be non – trivial solution therefore

So

To find particular integral. Let in (i)

Comparing like terms

……………….(iii)

……………….(iv)

Let and

Visit us @ Youtube “learning with Usman Hamid”

 110

Then (iii) and (iv) becomes

and

then and

Then

Now general solution is

Where and

EXERCISE : Solve the following homogeneous differential equation

i.
ii.
iii.

TYPE VI : When R.H.S of given non – homogeneous difference equation has the form as
where are constants then to find
put or then evaluate the
values of ……………..

EXAMPLE :

Solve the following homogeneous differential equation

SOLUTION : Given

Firstly we find characteristic solution.

Let be the non – trivial solution of (i) then it must satisfies the eq (i) i.e.

Since is supposed to be non – trivial solution therefore

√
using quadratic formula.
√ √
So ( ) ( )

Visit us @ Youtube “learning with Usman Hamid”

 111

To find particular integral. Let in (i)

[
]

Comparing like terms

……………….(iii)

……………….(iv)

Let and

Then (iii) and (iv) becomes and

then and

Then ( )
Now general solution is
√ √
( ) ( ) ( )

Where and

EXERCISE : Solve the following homogeneous differential equation

i.
ii.
iii.
iv.

Visit us @ Youtube “learning with Usman Hamid”

 112

Visit us @ Youtube “learning with Usman Hamid”

 113

Visit us @ Youtube “learning with Usman Hamid”

 114

Visit us @ Youtube “learning with Usman Hamid”

 115

Visit us @ Youtube “learning with Usman Hamid”

 116

Visit us @ Youtube “learning with Usman Hamid”

 117

Visit us @ Youtube “learning with Usman Hamid”

 118

Visit us @ Youtube “learning with Usman Hamid”

 119

Visit us @ Youtube “learning with Usman Hamid”

 120

Visit us @ Youtube “learning with Usman Hamid”

 121

Visit us @ Youtube “learning with Usman Hamid”

 122

Visit us @ Youtube “learning with Usman Hamid”

 123

Visit us @ Youtube “learning with Usman Hamid”

 124

Visit us @ Youtube “learning with Usman Hamid”

 125

Visit us @ Youtube “learning with Usman Hamid”

 126

Visit us @ Youtube “learning with Usman Hamid”

 127

Visit us @ Youtube “learning with Usman Hamid”

 128

STIFF DIGFFERENTIAL EQUATIONS: Those equations whose exact solutions has a term
of the form where c is a large positive constant.

REGION OF ABSOLUTE STABILITY: Region R of absolute stability for a one step method
is and for a multistep method it is

A-STABLE METHOD: A numerical method is said to be A-Stable method if its region R of

absolute stability contains the entire left half plane.

HOW TO REMOVE STIFFNESS OF IVP?: by increasing magnitude of derivative but not of

solution. IN this situation error can grow so large that it dominates the calculations IVP.

Visit us @ Youtube “learning with Usman Hamid”

 129

METHODS FOR NUMERICAL SOLUTIONS OF ORDINARY

DIFFERENTIAL EQUATIONS

SINGLE STEP METHODS: A series for „y‟ in terms of power of „x‟ form which the value of
„y‟ at a particular value of „x‟ can be obtained by direct substitution
e.g. Taylor‟s, Picard‟s, Euler‟s, Modified Euler‟s Method.
MULTI – STEP METHODS: In multi-step methods, the solution at any point „x‟ is obtained
using the solution at a number of previous points.
(Predictor- corrector method, Adam‟s Moulton Method, Adam‟s Bash forth Method)
REMARK: There are some ODE that cannot be solved using the standard methods. In such
situations we apply numerical methods. These methods yield the solutions in one of two forms.
(i) A series for „y‟ in terms of powers of „x‟ from which the value of „y‟ can be obtained
by direct substitution. e.g. Taylor‟s and Picard‟s method
(ii) A set of tabulated values of „x‟ and „y‟. e.g. and Euler‟s, Runge Kutta

ADVANTAGE/DISADVANTAGE OF MULTI - STEP METHODS

They are not self-starting. To overcome this problem, the single step method with some order of
accuracy is used to determine the starting values.
Using these methods one step method clears after the first few steps.

LIMITATION (DISADVANTAGE) OF SINGLE STEP METHODS.

For one step method it is typical, for several functions evaluation to be needed.
IMPLICIT METHODS: Method that does not directly give a formula to the new
approximation. A need to get it, need an implicit formula for new approximation in term of
known data. These methods also known as close methods. It is possible to get stable 3rd order
implicit method.
EXPLICIT METHODS; Methods that not directly give a formula to new approximation and
need an explicit formula for new approximation in terms of known data. These are also
called open methods.
Most Authorities proclaim that it is not necessary to go to a higher order method. Explain.
Because the increased accuracy is offset by additional computational effort.
If more accuracy is required, then either a smaller step size. OR an adaptive method should be
used.
CONSISTENT METHOD: A multi-step method is consistent if it has order at least one “1”

Visit us @ Youtube “learning with Usman Hamid”

 130

TAYLOR‟S SERIES EXPANSION

Given , smooth function. Expand it at point then

∑ This is called Taylor‟s series of „f‟ at „c‟

If

MECLAURIN SERIES FROM TAYLOR‟S: If we put in Taylor‟s series then

ADVANTAGE OF TAYLOR‟S SERIES

(1) One step, Explicit.

(2) Can be high order.
(3) Easy to show that global error is the same as local truncation error.
(4) Applicable to keep the error small.

DISADVANTAGE: Need to explicit form of the derivatives of function. That is why not
practical.

ERROR IN TAYLOR‟S SERIES

Assume are continuous functions. Call

∑ Then first term is Taylor series

Then the error is

Where „ is some point between „x‟ and „c‟ .

CONVERGENCE

A Taylor‟s series converges rapidly if „x‟ is nears „c‟ and slowly (or not at all) if „x‟ is for away
form „c‟.

Visit us @ Youtube “learning with Usman Hamid”

 131

EXAMPLE: Obtain numerically the solution of using Taylor

Series method to find „y‟ at 1.3

SOLUTION

[ ]

...... ................................and so on.

where

therefore , , , ,...........................

Now by using formula

we get as required.

QUESTION: Explain Taylor Series method for solving an initial value problem described by

SOLUTION: Here we assume that f(x,y) is sufficiently differentiable with respect to „x‟ and „y‟
If y(x) is exact solution of (i) we can expand by Taylor Series about the point and obtain

Since the solution is not known, the derivatives in the above expansion are known explicitly.
However „f‟ is assume to be sufficiently differentiable and therefore the derivatives can be
obtained directly from the given differentiable equation itself. Noting that „f‟ is an implicit
function of „y‟ . we have

( )

Now

( )

Using (a) and (b) in (ii) we get

[ ]

Continuing in this manner we can express any derivative of „y‟ in term of and its partial
derivatives.

Visit us @ Youtube “learning with Usman Hamid”

 132

EULER‟S METHOD

To find the solution of the given Differential Equation in the form of a recurrence relation
( ) Is called Euler Method

FORMULA DERIVATION

Consider the differential Equation of the first order

Let and be two points of approximation curve. Then

Given That ( ) ( ) ( )

Similarly

BASE OF EULER‟S METHOD

In this method we use the property that in a small interval, a curve is nearly a Straight Line. Thus
at We approximate the Curve by a tangent at that point.

OBJECT (PURPOSE) OF METHOD

The object of Euler‟s Method is to obtain approximations to the well posed initial value problem

Visit us @ Youtube “learning with Usman Hamid”

 133

GEOMETRICAL INTERPRETATION

Geometrically, this method has a very simple meaning. The desired function curve is
approximated by a polygon train. Where the direction of each part is determined by the value of
the function at its starting point

Also Shows that the next approximation is obtained at the

point where the tangent to the graph of interest with the vertical line

LIMITATION OF EULER METHOD

There is too much inertia in Euler Method. One should not follow the same initial slope over the
whole interval of length “h”.

EULER METHOD IN VECTOR NOTATION

Consider the system where and

if we are given the initial condition then Euler method approximate a solution (x,
y) by

ADVANTAGE/DISADVANTAGE OF EULER METHOD

The advantage of Euler‟s method is that it requires only one slope evaluation and is simple to
apply, especially for discretely sampled (experimental) data points. The disadvantage is that
errors accumulate during successive iterations and the results are not very accurate.

EXAMPLE: Obtain numerically the solution of using simple

Euler method to find „y‟ at 0.1

SOLUTION: where

Then

Now by using formula we get

as required.

Visit us @ Youtube “learning with Usman Hamid”

 134

MODIFIED EULER METHOD

Modified Euler‟s Method is given by the iteration formula

* ( ) ( )+

Method also known as Improved Euler method sometime known as Runge Kutta method of
order 2

CONVERGENCE FOR EULER METHOD: Assume that f has a Lipschitz constant L,

for the variable „y, and that the solution of the initial value problem
at is Approximated by using Euler Method

Let „M‟ be an upper bound for on then ( )

Visit us @ Youtube “learning with Usman Hamid”

 135

DARIVATION OF MODIFIED EULER METHOD

Consider the differential Equation of 1st order and

Then by Euler‟s Method

( )

* ( ) ( )+

* ( ) ( )+

EXAMPLE: Obtain numerically the solution of using

modified Euler method to find „y‟ at 0.2

SOLUTION: Take h = 0.1 (own choice) and

Now using Euler‟s method

Then by using Euler‟s modified method

* ( )+

Again using Euler‟s method

Then by using Euler‟s modified method

* ( )+

Visit us @ Youtube “learning with Usman Hamid”

 136

Visit us @ Youtube “learning with Usman Hamid”

 137

RUNGE KUTTA METHODS

Basic idea of Runge Kutta Methods can be explained by using Modified Euler‟s Method by
Equation

Here we find the slope not only at „ ‟ but also at several other interior points and take the
weighted average of these slopes and add to „ „ to get „ ‟.

ALSO RK-Approach is to aim for the desirable features for the Taylor Series method but with
the replacement of the requirement for the evaluation of the higher order derivatives with the
requirement to evaluate at some points with in the steps to „ ‟

IMPORTANCE: Quite Accurate, Stable and easy to program but requires four slopes evaluation
at four different points of (x,y): these slope evaluations are not possible for discretely sampled
data points, because we have is what is given to us and we do not get to choose at will where to
evaluate slopes. These methods do not demand prior computation of higher derivatives of
as in Taylor Series Method. Easy for automatic Error control. Global and local errors have
same order in it.

DIFFERENCE B/W TAYLOR SERIES AND RK-METHOD (ADVANTAGE OF RK

OVER TAYLOR SERIES)

Taylor Series needs to explicit form of derivative of but in RK-method this is not in
demand. RK-method very extensively used.

SECOND ORDER RUNGE KUTTA METHOD

WORKING RULE: For a given initial value problem of first order

Suppose be equally spaced „x‟ values with interval „h‟

i.e.

Also denote

Then for until termination do:

,, ,,

Then ( Is the formula for second order RK-method.

REMARK: Modified Euler Method is a special case of second order RK-Method.

Visit us @ Youtube “learning with Usman Hamid”

 138

IN ANOTHER WAY:

If

Then Equation for second order method is

This is called Heun‟s Method(third order RK method)

ANOTHER FORMULA FOR SECOND ORDER RK-METHOD

Where ( )

LOCAL TRUNCATION ERROR IN RK-METHOD.

LTE in RK-method is the error that arises in each step simply because of the truncated Taylor
series. This error is inevitable. Error of Runge Kutta method of order two involves an error of
O .

In General RK-method of order „m‟ takes the form

Where

………………….….. (
∑ )

MULTI STEP METHODS OVER RK-METHOD (PREFRENCE):

Determination of require only on evaluation of per step. Whereas RK-method for

require four or more function evaluations. For this reason, multi-step methods can be
twice as fast as RK-method of comparable Accuracy.

EXAMPLE:

use second order RK method to solve at x =0.4 and h=0.2

SOLUTION:

If „h‟ is not given then use by own choice for 4 – step take h=0.1 and for 1 – step take h=0.4

Given that h=0.2 ,

Now using formula of order two

Visit us @ Youtube “learning with Usman Hamid”

 139

Where ( )
( )

For n = 0; ( ) 𝑥 𝑥
𝑛

n = 2 steps

For n = 1; ( )

Visit us @ Youtube “learning with Usman Hamid”

 140

Visit us @ Youtube “learning with Usman Hamid”

 141

Visit us @ Youtube “learning with Usman Hamid”

 142

Visit us @ Youtube “learning with Usman Hamid”

 143

CLASSICAL RUNGE KUTTA METHOD (RK – METHOD OF ORDER FOUR)

ALORITHM: Given the initial value problem of first order

Suppose be equally spaced „x‟ values with interval „h‟

i.e.
Also denote

Then for until termination do:

,, ( )
( )

Then

Is the formula for Runge Kutta method of order four and its error is

ADVANTAGE OF METHOD

Accurate method.  It lakes in estimating the error.

Easy to compute for the use of  Easy to program and is efficient.
computer.
COMPUTATIONAL COMPARISON:

The main computational effort in applying the Runge Kutta method is the evaluation of . In
RK – 2 the cost is two function evaluation per step. In RK – 4 require four evaluations per step.

EXAMPLE:

use 4th order RK method to solve from t =0 to 0.4 taking h = 0.4

SOLUTION:

Now using formulas for the RK method of 4th order

Where , ( ) , ( )

Visit us @ Youtube “learning with Usman Hamid”

 144

STEP I : for n=0;

, ( )
( )

STEP II : for n=1;

, ( )
( )

STEP III : for n=2;

, ( )
( )

this is required answer

Visit us @ Youtube “learning with Usman Hamid”

 145

Visit us @ Youtube “learning with Usman Hamid”

 146

PREDICTOR - CORRECTOR METHODS

A predictor corrector method refers to the use of the predictor equation with one subsequent
application of the corrector equation and the values so obtained are the final solution at the grid
point.
PREDICTOR FORMULA: The explicit (open) formula used to predict approximation is
called a predictor formula.
CORRECTOR FORMULA: The implicit (closed) formula used to determine is called
Corrector Formula. This used to improve
IN GENERAL: Explicit and Implicit formula are used as pair of formulas. The explicit formula
is called „predictor‟ and implicit formula is called „corrector‟
Implicit methods are often used as „corrector‟ and Explicit methods are used as „predictor‟
in predictor-corrector method. why?
Because the corresponding Local Truncation Error formula is smaller for implicit method on the
other hand the implicit methods has the inherent difficulty that extra processing is necessary to
evaluate implicit part.
REMARK

 Truncation Error of predictor is

 Local Truncation Error of Adam‟s Predictor is
 Truncation Error of Corrector is

Why Should one bother using the predictor corrector method When the Single step method
are of the comparable accuracy to the predictor corrector methods are of the same order?
A practical answer to that relies in the actual number of functional evaluations. For example, RK
- Method of order four, each step requires four evaluations where the Adams Moulton method of
the same order requires only as few as two evaluations. For this reason, predictor corrector
formulas are in General considerably more accurate and faster than single step methods.
REMEMBER: In predictor corrector method if values of against the values
of are given the we use symbol predictor corrector method and in this method
we use given values of
If Are not given against the values of then we first find
values of by using RK - method
OR By using formula

Visit us @ Youtube “learning with Usman Hamid”

 147

Visit us @ Youtube “learning with Usman Hamid”

 148

BASE (MAIN IDEA) OF PREDICTOR CORRECTOR METHOS

In predictor corrector methods a predictor formula is used to predict the value of at and
then a corrector formula is used to improve the value of
Following are predictor – corrector methods
1. Milne‟s Method
2. Adam – Moulton method

Visit us @ Youtube “learning with Usman Hamid”

 149

MILNE‟S METHOD

It‟s a multi-step method. In General, Milne‟s Predictor – Corrector pair can be written as

P:
C:

REMARK: Magnitude of truncation error in Milne‟s corrector formula is

and truncation error in Milne‟s predictor formula is
stable, convergent, efficient, accurate, compeer friendly.

ALGORITHM
 First predict the value of by above predictor formula.
Where derivatives are computed using the given differential equation itself.
 Using the predicted value we calculate the derivative from the given
differential Equation.
 Then use the corrector formula given above for corrected value of . Repeat this
process.

Visit us @ Youtube “learning with Usman Hamid”

 150

EXAMPLE: use Milne‟s method to solve and compute y(0.8)

SOLUTION:

Now by using Euler‟s method

Now For n=1

For n=2 For n=3

Visit us @ Youtube “learning with Usman Hamid”

 151

Now using Milne‟s Predictor formula

P:

Now using corrector formula

C:

Visit us @ Youtube “learning with Usman Hamid”

 152

Visit us @ Youtube “learning with Usman Hamid”

 153

ADAM‟S MOULTON METHOD

The predictor – corrector formulas for Adam‟s Moulton method are given as follows

REMARK

 The predictor Truncation Error is ( ) and Corrector Truncation Error

is( )
 Truncation Error in Adam‟s predictor is approximately 13-time more than that in the
corrector. OF course with Opposite Sign.
 In the predictor Corrector methods if is given another or not Then Using
Euler or RK or any other method we can find these. For example, by Euler method

For m=0

EXAMPLE: use Adam‟s Moulton method to solve at t = 1.0 taking

h=0.2 and compare it with analytic solution.
SOLUTION: in order to use Adam‟s Moulton method we require the solution of the given
Differential equation at the past four equispaced points for which we have RK-4th order method
which is self-starting.

Using RK method we get

where

Also in easy way we can find by using Euler‟s method with some error as follows
Now by using Euler‟s method

now as

Now using Adam‟s Predictor formula

Visit us @ Youtube “learning with Usman Hamid”

 154

Now using Adam‟s Corrector formula

[ ]
[ ]

Proceeding in similar way we can get

Now the analytic solution can be seen in the following steps

Then using integrating factor ∫

now using initial conditions we get c = -1 therefor analytic solution is

Visit us @ Youtube “learning with Usman Hamid”

 155

Visit us @ Youtube “learning with Usman Hamid”

 156

Visit us @ Youtube “learning with Usman Hamid”

 157

Visit us @ Youtube “learning with Usman Hamid”