Lectures on Computational Physics

Badis Ydri

Adel Bouchareb Rafik Chemam

Physics Department, Badji Mokhtar University, Annaba, Algeria

May 21, 2013

2 ydri et al, lectures on computational physics
 Contents

1 Introduction and References 1

2 Euler Algorithm 3
2.1 Euler Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2.2 First Example and Sample Code . . . . . . . . . . . . . . . . . . . . . . . 4
2.2.1 Radioactive Decay . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.2.2 A Sample Fortran Code . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3 More Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.3.1 Air Resistance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.3.2 Projectile Motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.4 Periodic Motions and Euler-Cromer and Verlet Algorithms . . . . . . . . . 11
2.4.1 Harmonic Oscillator . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.4.2 Euler Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.4.3 Euler-Cromer Algorithm . . . . . . . . . . . . . . . . . . . . . . . . 13
2.4.4 Verlet Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.5 Lab Problem 1: Euler Algorithm- Air Resistance . . . . . . . . . . . . . . 14
2.6 Lab Problem 2: Euler Algorithm- Projectile Motion . . . . . . . . . . . . 15
2.7 Lab Problem 3: Euler, Euler-Cromer and Verlet Algorithms . . . . . . . . 16

3 Numerical Integration 19
3.1 Rectangular Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3.2 Trapezoidal Approximation . . . . . . . . . . . . . . . . . . . . . . . . . . 20
3.3 Parabolic Approximation or Simpson’s Rule . . . . . . . . . . . . . . . . . 20
3.4 Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
3.5 Lab Problem 4: Numerical Integration . . . . . . . . . . . . . . . . . . . . 23

4 Newton-Raphson Algorithms and Interpolation 25

4.1 Bisection Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4.2 Newton-Raphson Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . 26
4.3 Hybrid Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.4 Lagrange Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
4.5 Cubic Spline Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
4.6 Lab Problem 5: Newton-Raphson Algorithm . . . . . . . . . . . . . . . . . 31
4 ydri et al, lectures on computational physics

5 The Solar System-The Runge-Kutta Methods 33

5.1 The Solar System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
5.1.1 Newton’s Second Law . . . . . . . . . . . . . . . . . . . . . . . . . 33
5.1.2 Astronomical Units and Initial Conditions . . . . . . . . . . . . . . 34
5.1.3 Kepler’s Laws . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
5.1.4 The inverse-Square Law and Stability of Orbits . . . . . . . . . . . 37
5.2 Euler-Cromer Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
5.3 The Runge-Kutta Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 38
5.3.1 The Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
5.3.2 Example 1: The Harmonic Oscillator . . . . . . . . . . . . . . . . . 40
5.3.3 Example 2: The Solar System . . . . . . . . . . . . . . . . . . . . . 40
5.4 Precession of the Perihelion of Mercury . . . . . . . . . . . . . . . . . . . . 42
5.5 Lab Problem 6: Runge-Kutta Algorithm- The Solar System . . . . . . . . 43
5.6 Lab Problem 7: Precession of the perihelion of Mercury . . . . . . . . . . 44

6 Chaos: Chaotic Pendulum 47

6.1 Equation of Motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
6.2 Numerical Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
6.2.1 Euler-Cromer Algorithm . . . . . . . . . . . . . . . . . . . . . . . . 50
6.2.2 Runge-Kutta Algorithm . . . . . . . . . . . . . . . . . . . . . . . . 50
6.3 Elements of Chaos . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
6.3.1 Butterfly Effect: Sensitivity to Initial Conditions . . . . . . . . . . 51
6.3.2 Poincare Section and Attractors . . . . . . . . . . . . . . . . . . . . 52
6.3.3 Period-Doubling Bifurcations . . . . . . . . . . . . . . . . . . . . . 52
6.3.4 Feigenbaum Ratio . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
6.3.5 Spontaneous Symmetry Breaking . . . . . . . . . . . . . . . . . . . 53
6.4 Lab Problem 8: The Butterfly Effect . . . . . . . . . . . . . . . . . . . . . 54
6.5 Lab Problem 9: Poincaré Sections . . . . . . . . . . . . . . . . . . . . . . . 55
6.6 Lab Problem 10: Period Doubling . . . . . . . . . . . . . . . . . . . . . . . 56
6.7 Lab Problem 11: Bifurcation Diagrams . . . . . . . . . . . . . . . . . . . . 57

7 Molecular Dynamics 59
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
7.2 The Lennard-Jones Potential . . . . . . . . . . . . . . . . . . . . . . . . . 60
7.3 Units, Boundary Conditions and Verlet Algorithm . . . . . . . . . . . . . 61
7.4 Some Physical Applications . . . . . . . . . . . . . . . . . . . . . . . . . . 63
7.4.1 Dilute Gas and Maxwell Distribution . . . . . . . . . . . . . . . . . 63
7.4.2 The Melting Transition . . . . . . . . . . . . . . . . . . . . . . . . 64
7.5 Lab Problem 12: Maxwell Distribution . . . . . . . . . . . . . . . . . . . . 64
7.6 Lab Problem 13: Melting Transition . . . . . . . . . . . . . . . . . . . . . 65
ydri et al, lectures computational physics 5

8 Pseudo Random Numbers and Random Walks 67

8.1 Random Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
8.1.1 Linear Congruent or Power Residue Method . . . . . . . . . . . . . 67
8.1.2 Statistical Tests of Randomness . . . . . . . . . . . . . . . . . . . . 68
8.2 Random Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
8.2.1 Random Walks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
8.2.2 Diffusion Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
8.3 The Random Number Generators RAN 0, 1, 2 . . . . . . . . . . . . . . . . 74
8.4 Lab Problem 14: Random Numbers . . . . . . . . . . . . . . . . . . . . . . 77
8.5 Lab Problem 15: Random Walks . . . . . . . . . . . . . . . . . . . . . . . 78

9 Monte Carlo Integration 79

9.1 Numerical Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
9.1.1 Rectangular Approximation Revisted . . . . . . . . . . . . . . . . . 79
9.1.2 Midpoint Approximation of Multidimensional Integrals . . . . . . . 80
9.1.3 Spheres and Balls in d Dimensions . . . . . . . . . . . . . . . . . . 82
9.2 Monte Carlo Integration: Simple Sampling . . . . . . . . . . . . . . . . . . 83
9.2.1 Sampling (Hit or Miss) Method . . . . . . . . . . . . . . . . . . . . 83
9.2.2 Sample Mean Method . . . . . . . . . . . . . . . . . . . . . . . . . 84
9.2.3 Sample Mean Method in Higher Dimensions . . . . . . . . . . . . . 84
9.3 The Central Limit Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . 85
9.4 Monte Carlo Errors and Standard Deviation . . . . . . . . . . . . . . . . . 86
9.5 Nonuniform Probability Distributions . . . . . . . . . . . . . . . . . . . . . 89
9.5.1 The Inverse Transform Method . . . . . . . . . . . . . . . . . . . . 89
9.5.2 The Acceptance-Rejection Method . . . . . . . . . . . . . . . . . . 91
9.6 Lab Problem 16: Midpoint and Monte Carlo Approximations . . . . . . . 91
9.7 Lab Problem 17: Nonuniform Probability Distributions . . . . . . . . . . . 92

10 Monte Carlo Importance Sampling, Metropolis Algorithm and Ising

Model 95
10.1 The Canonical Ensemble . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
10.2 Importance Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
10.3 The Ising Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
10.4 The Metropolis Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
10.5 The Heat-Bath Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
10.6 The Mean Field Approximation . . . . . . . . . . . . . . . . . . . . . . . . 101
10.6.1 Phase Diagram and Critical Temperature . . . . . . . . . . . . . . 101
10.6.2 Critical Exponents . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
10.7 Simulation of The Ising Model and Numerical Results . . . . . . . . . . . 105
10.7.1 The Fortran Code . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
10.7.2 Some Numerical Results . . . . . . . . . . . . . . . . . . . . . . . . 106
10.8 Lab Problem 18: The Metropolis Algorithm and The Ising Model . . . . . 109
10.9 Lab Problem 19: The Ferromagnetic Second Order Phase Transition . . . 110
10.10Lab Problem 20: The 2−Point Correlator . . . . . . . . . . . . . . . . . . 111
6 ydri et al, lectures on computational physics

10.11Lab Problem 21: Hysteresis and The First Order Phase Transition . . . . 111

Appendix 115

A Notes on Numerical Errors 115

A.1 Floating Point Representation . . . . . . . . . . . . . . . . . . . . . . . . . 115
A.2 Machine Precision and Roundoff Errors . . . . . . . . . . . . . . . . . . . 117
A.3 Systematic (Algorithmic) Errors . . . . . . . . . . . . . . . . . . . . . . . . 118

B The Jackknife Method and The Method of Least Squares 121

C Exercises 123
 Introduction and References
1
• Computational physics is a subfield of computational science/scientific computing.

• In computational physics we combine elements from physics (especially theoretical),

elements from mathematics (in particular applied mathematics such as numerical
analysis) and elements from computer science (programming) for the purpose of
solving a physics problem.

• In physics there are traditionally two approaches. 1) The experimental approach

and 2) The theoretical approach. Nowadays we consider “The computational ap-
proach” as a third approach in physics. It can even be argued that the computa-
tional approach is independent from the first two approaches and it is not just a
bridge between them.

• The most important use of computers in physics is simulation. Simulations are

suited for nonlinear problems which can not generally solved by analytical methods.
The starting point of a simulation is an idealized model of a physical system of
interest. We want to check whether or not the behaviour of this model is consistent
with observation. We specify an algorithm for the implementation of the model on
a computer. The execution of this implementation is a simulation. Simulations are
therefore virtual experiments. The comparison between simulations and laboratory
experiments goes as follows:
2 ydri et al, lectures on computational physics

Laboratory experiment Simulation

sample model
physical apparatus computer program (the
code)
calibration testing of code
measurement computation
data analysis data analysis

• A crucial tool in computational physics is programming languages. In simulations

as used by the majority of research physicists codes are written in a high-level
compiled language such as Fortran and C/C++. In such simulations we also use
calls to routine libraries such as LAPACK. In this course we follow a different path
by writing all our codes in a high-level compiled language and not call any libraries.

• The use of mathematical software packages such as MAPLE, MATHEMATICA

and MATLAB is only suited for relatively small calculations. These packages are
interpreted languages and thus the code they produce run generally far too slowly
compared to compiled languages.

• As our programming language we will use Fortran 77 under the Linux operating
system. We adopt the ubuntu distribution of Linux. We will use the Fortran
compilers f77 and gfortran. As an editor we will use emacs and for graphics we will
use gnuplot.

• The main references which we will follow in this course are:

– N.J.Giordano, Computational Physics.

– R.H.Landau, M.J.Paez, C.C.Bordeianu, Computational Physics.
– H.Gould, J.Tobochnick, D.Christian, An Introduction To Computer Simula-
tion Methods.
– R.Fitzpatrick, Computational Physics.
– M. Hjorth-Jensen,Computational Physics.
– Paul L.DeVries, A First Course in Computational Physics.
 Euler Algorithm
2
2.1 Euler Algorithm
It is a well appreciated fact that first order differential equations are commonplace in
all branches of physics. They appear virtually everywhere and some of the most funda-
mental problems of nature obey simple first order differential equations or second order
differential equations. It is so often possible to recast second order differential equations
as first order differential equations with a doubled number of unknown. From the numer-
ical standpoint the problem of solving first order differential equations is a conceptually
simple one as we will now explain.
We consider the general first order ordinary differential equation
′ dy
y = = f (x, y). (2.1)
dx
We impose the general initial-value boundary condition is
y(x0 ) = y0 . (2.2)
We solve for the function y = y(x) in the unit x−interval starting from x0 . We make the
x−interval discretization
xn = x0 + n∆x , n = 0, 1, ... (2.3)
The Euler algorithm is one of the oldest known numerical recipe. It consists in replacing
the function y(x) in the interval [xn , xn+1 ] by the straight line connecting the points
(xn , yn ) and (xn+1 , yn+1 ). This comes from the definition of the derivative at the point
x = xn given by
yn+1 − yn
= f (xn , yn ). (2.4)
xn+1 − xn
4 ydri et al, lectures on computational physics

This means that we replace the above first order differential equation by the finite dif-
ference equation

yn+1 ≃ yn + ∆xf (xn , yn ). (2.5)

This is only an approximation. The truncation error is given by the next term in the
Taylor’s expansion of the function y(x) which is given by
1 df (x, y)
yn+1 ≃ yn + ∆xf (xn , yn ) + ∆x2 |x=xn + .... (2.6)
2 dx
The error then reads
1 df (x, y)
(∆x)2 |x=xn . (2.7)
2 dx
The error per step is therefore proportional to (∆x)2 . In a unit interval we will perform
N = 1/∆x steps. The total systematic error is therefore proportional to
1
N (∆x)2 = . (2.8)
N

2.2 First Example and Sample Code

2.2.1 Radioactive Decay
It is an experimental fact that radioactive decay obeys a very simple first order differential
equation. In a spontaneous radioactive decay a particle with no external influence will
decay into other particles. A typical example is the nuclear isotope uranium 235. The
exact moment of decay of any one particle is random. This means that the number
−dN (t) = N (t) − N (t + dt) of nuclei which will decay during a time inetrval dt must be
proportional to dt and to the number N (t) of particles present at time t, i.e.

−dN (t) ∝ N (t)dt. (2.9)

In other words the probability of decay per unit time given by (−dN (t)/N (t))/dt is a
constant which we denote 1/τ . The minus sign is due to the fact that dN (t) is negative
since the number of particles decreases with time. We write
dN (t) N (t)
=− . (2.10)
dt τ
The solution of this first order differential equation is given by a simple exponential
function, viz

N (t) = N0 exp(−t/τ ). (2.11)

The number N0 is the number of particles at time t = 0. The time τ is called the mean
lifetime. It is the average time for decay. For the uranium 235 the mean lifetime is
around 109 years.
ydri et al, lectures computational physics 5

The goal now is to obtain an approximate numerical solution to the problem of

radioactivity using the Euler algorithm. In this particular case we can compare to an
exact solution given by the exponential decay law (2.11). We start evidently from the
Taylor’s expansion

dN 1 d2 N
N (t + ∆t) = N (t) + ∆t + (∆t)2 2 + ... (2.12)
dt 2 dt
We get in the limit ∆t −→ 0
dN N (t + ∆t) − N (t)
= Lim∆t−→0 . (2.13)
dt ∆t
We take ∆t small but non zero. In this case we obtain the approximation
dN N (t + ∆t) − N (t)
≃ . (2.14)
dt ∆t
Equivalently
dN
N (t + ∆t) ≃ N (t) + ∆t . (2.15)
dt
By using (2.10) we get

N (t)
N (t + ∆t) ≃ N (t) − ∆t . (2.16)
τ
We will start from the number of particles at time t = 0 given by N (0) = N0 which is
known. We substitute t = 0 in (2.16) to obtain N (∆t) = N (1) as a function of N (0).
Next the value N (1) can be used in equation (2.16) to get N (2∆t) = N (2), etc. We are
thus led to the time discretization

t ≡ t(i) = i∆t , i = 0, ..., N. (2.17)

In other words

N (t) = N (i). (2.18)

The integer N determine the total time interval T = N ∆t. The numerical solution (2.16)
can be rewritten as
N (i)
N (i + 1) = N (i) − ∆t , i = 0, ..., N. (2.19)
τ
This is Euler algorithm for radioactive decay. For convenience we shift the integer i so
that the above equation takes the form
N (i − 1)
N (i) = N (i − 1) − ∆t , i = 1, ..., N + 1. (2.20)
τ
6 ydri et al, lectures on computational physics

We introduce N̂ (i) = N (i − 1), i.e N̂ (1) = N (0) = N0 . We get

N̂ (i)
N̂ (i + 1) = N̂ (i) − ∆t , i = 1, ..., N + 1. (2.21)
τ
The corresponding times are
t̂(i + 1) = i∆t , i = 1, ..., N + 1. (2.22)
The initial number of particles at time t̂(1) = 0 is N̂ (1) = N0 . This approximate solution
should be compared with the exact solution (2.11).

2.2.2 A Sample Fortran Code

The goal in this section is to provide a sample Fortran code which implements the above
algorithm (2.21). The reasons behind choosing Fortran were explained in the introduc-
tion. Any Fortran program, like any other programing language, must start with some
program statement and conclude with an end statement. The program statement allows
us to give a name to the program. The end statement may be preceded by a return
statement. This looks like
program radioactivity

c Here is the code

return
end
We have chosen the name “radioactivity” for our program. The “c” in the second line
indicates that the sentence “here is the code” is only a comment and not a part of the
code.
After the program statement come the declaration statements. We state the variables
and their types which are used in the program. In Fortran we have the integer type for
integer variables and the double precision type for real variables. In the case of (2.21) the
variables N̂ (i), t̂(i), τ , ∆t, N0 are real numbers while the variables i and N are integer
numbers.
An array A of dimension K is an ordered list of K variables of a given type called
the elements of the array and denoted A(1), A(2),...,A(K). In our above example N̂ (i)
and t̂(i) are real arrays of dimension N + 1. We declare that N̂ (i) and t̂(i) are real for
all i = 1, ..., N + 1 by writing N̂ (1 : N + 1) and t̂(1 : N + 1).
Since an array is declared at the begining of the program it must have a fixed size. In
other words the upper limit must be a constant and not a variable. In Fortran a constant
is declared with a parameter statement. In our above case the upper limit is N + 1 and
hence N must be declared in parameter statement.
In the Fortran code we choose to use the notation A = N̂ , A0 = N̂0 , time = t̂, ∆ = ∆t
and tau = τ . By putting all declarations together we get the following preliminary lines
of code
ydri et al, lectures computational physics 7

program radioactivity
integer i,N
parameter (N=100)
doubleprecision A(1:N+1),A0,time(1:N+1),Delta,tau

c Here is the code

return
end

The input of the computation in our case are obviously given by the parameters N0 ,
τ , ∆t and N .
For the radioactivity problem the main part of the code consists of equations (2.21)
and (2.22). We start with the known quantities N̂ (1) = N0 at t̂(1) = 0 and generate via
the successive use of (2.21) and (2.22) N̂ (i) and t̂(i) for all i > 1. This will be coded
using a do loop. It begins with a do statement and ends with an enddo statement. We
may also indicate a step size.
The output of the computation can be saved to a file using a write statement inside
the do loop. In our case the output is the number of particles N̂ (i) and the time t̂(i).
The write statement reads explicitly

write(10, ∗) t̂(i), N̂ (i).

The data will then be saved to a file called fort.10.

By including the initialization, the do loop and the write statement we obtain the
complete code

program radioactivity
integer i,N
parameter (N=100)
doubleprecision A(1:N+1),A0,time(1:N+1),Delta,tau
parameter (A0=1000,Delta=0.01d0,tau=1.0d0)

A(1)=A0
time(1)=0
do i=1,N+1,1
A(i+1)=A(i)-Delta*A(i)/tau
time(i+1)=i*Delta
write(10,*) time(i+1),A(i+1)
enddo

return
end
8 ydri et al, lectures on computational physics

2.3 More Examples

2.3.1 Air Resistance
We consider an athlete riding a bicycle moving on a flat terrain. The goal is to determine
the velocity. Newton’s second law is given by
dv
m = F. (2.23)
dt
F is the force exerted by the athlete on the bicycle. It is clearly very difficult to write down
a precise expression for F . Formulating the problem in terms of the power generated
by the athlete will avoid the use of an explicit formula for F . Multiplying the above
equation by v we obtain
dE
= P. (2.24)
dt
E is the kinetic energy and P is the power, viz
1
E = mv 2 , P = F v. (2.25)
2
Experimentaly we find that the output of well trained athletes is around P = 400 watts
over periods of 1h. The above equation can also be rewritten as
dv 2 2P
= . (2.26)
dt m
For P constant we get the solution
2P
v2 = t + v02 . (2.27)
m
We remark the unphysical effect that v −→ ∞ as t −→ ∞. This is due to the absence of
the effect of friction and in particular air resistance.
The most important form of friction is air resistance. The force due to air resistance
(the drag force) is
Fdrag = −B1 v − B2 v 2 . (2.28)
At small velocities the first term dominates whereas at large velocities it is the second
term that dominates. For very small velocities the dependence on v given by Fdrag =
−B1 v is known as Stockes’ law. For reasonable velocities the drag force is dominated by
the second term, i.e. it is given for most objects by
Fdrag = −B2 v 2 . (2.29)
The coefficient B2 can be calculated as follows. As the bicycle-rider combination moves
with velocity v it pushes in a time dt a mass of air given by dmair = ρAvdt where ρ is
the air density and A is the frontal cross section. The corresponding kinetic energy is
dEair = dmair v 2 /2. (2.30)
ydri et al, lectures computational physics 9

This is equal to the work done by the drag force, i.e.

−Fdrag vdt = dEair . (2.31)

From this we get

B2 = CρA. (2.32)

The drag coefficient is C = 12 . The drag force becomes

Fdrag = −CρAv 2 . (2.33)

Taking into account the force due to air resistance we find that Newton’s law becomes

dv
m = F + Fdrag . (2.34)
dt
Equivalently

dv P CρAv 2
= − . (2.35)
dt mv m
It is not obvious that this equation can be solved exactly in any easy way. The Euler
algorithm gives the approximate solution

dv
v(i + 1) = v(i) + ∆t (i). (2.36)
dt
In other words

CρAv 2 (i)
 
P
v(i + 1) = v(i) + ∆t − , i = 0, ..., N. (2.37)
mv(i) m

This can also be put in the form (with v̂(i) = v(i − 1))

CρAv̂ 2 (i)
 
P
v̂(i + 1) = v̂(i) + ∆t − , i = 1, ..., N + 1. (2.38)
mv̂(i) m

The corresponding times are

t ≡ t̂(i + 1) = i∆t , i = 1, ..., N + 1. (2.39)

The initial velocity v̂(1) at time t(1) = 0 is known.

2.3.2 Projectile Motion

There are two forces acting on the projectile. The weight force and the drag force. The
drag force is opposite to the velocity. In this case Newton’s law is given by
10 ydri et al, lectures on computational physics

d~v
m = F~ + F~drag
dt
~v
= m~g − B2 v 2
v
= m~g − B2 v~v . (2.40)

The goal is to determine the position of the projectile and hence one must solve the two
equations

d~x
= ~v . (2.41)
dt

d~v
m = m~g − B2 v~v . (2.42)
dt
In components (the horizontal axis is x and the vertical axis is y) we have 4 equations
of motion given by

dx
= vx . (2.43)
dt

dvx
m = −B2 vvx . (2.44)
dt

dy
= vy . (2.45)
dt

dvy
m = −mg − B2 vvy . (2.46)
dt
We recall the constraint
q
v= vx2 + vy2 . (2.47)

The numerical approach we will employ in order to solve the 4 equations of motion (2.43)-
(2.46) together with (2.47) consists in using Euler algorithm. This yields the approximate
solution given by the equations

x(i + 1) = x(i) + ∆tvx (i). (2.48)

B2 v(i)vx (i)
vx (i + 1) = vx (i) − ∆t . (2.49)
m
ydri et al, lectures computational physics 11

y(i + 1) = y(i) + ∆tvy (i). (2.50)

B2 v(i)vy (i)
vy (i + 1) = vy (i) − ∆tg − ∆t . (2.51)
m
The constraint is
q
v(i) = vx (i)2 + vy (i)2 . (2.52)

In the above equations the index i is such that i = 0, ..., N . The initial position and
velocity are given, i.e. x(0), y(0), vx (0) and vy (0) are known.

2.4 Periodic Motions and Euler-Cromer and Verlet Algo-

rithms
As discussed above at each iteration using the Euler algorithm there is a systematic error
proportional to 1/N . Obviously this error will accumulate and may become so large that
it will alter the solution drastically at later times. In the particular case of periodic
motions, where the true nature of the motion can only become clear after few elapsed
periods, the large accumulated error can lead to diverging results. In this section we will
discuss simple variants of the Euler algorithm which perform much better than the plain
Euler algorithm for periodic motions.

2.4.1 Harmonic Oscillator

We consider a simple pendulum: a particle of mass m suspended by a massless string
from a rigid support. There are two forces acting on the particle. The weight and the
tension of the string. Newton’s second law reads
d2~s
m = m~g + T~ . (2.53)
dt
The parallel (with respect to the string) projection reads

0 = −mg cos θ + T. (2.54)

The perpendicular projection reads

d2 s
m = −mg sin θ. (2.55)
dt2
The θ is the angle that the string makes with the vertical. Clearly s = lθ. The force
mg sin θ is a restoring force which means that it is always directed toward the equilibrium
position (here θ = 0) opposite to the displacement and hence the minus sign in the above
equation. We get by using s = lθ the equation
d2 θ g
= − sin θ. (2.56)
dt2 l
12 ydri et al, lectures on computational physics

For small θ we have sin θ ≃ θ. We obtain

d2 θ g
= − θ. (2.57)
dt2 l
p
The solution is a sinusoidal function of time with frequency Ω = g/l. It is given by

θ(t) = θ0 sin(Ωt + φ). (2.58)

The constants θ0 and φ depend on the initial displacement and velocity of the pendulum.
The frequency is independent of the mass m and the amplitude of the motion and depends
only on the length l of the string.

2.4.2 Euler Algorithm

The numerical solution is based on Euler algorithm. It is found as follows. First we
replace the equation of motion (2.57) by the following two equations
dθ
= ω. (2.59)
dt

dω g
= − θ. (2.60)
dt l
We use the definition of a derivative of a function, viz
df f (t + ∆t) − f (t)
= , ∆t −→ 0. (2.61)
dt ∆t
We get for small but non zero ∆t the approximations

θ(t + ∆t) ≃ θ(t) + ω(t)∆t

g
ω(t + ∆t) ≃ ω(t) − θ(t)∆t. (2.62)
l
We consider the time discretization

t ≡ t(i) = i∆t , i = 0, ..., N. (2.63)

In other words

θ(t) = θ(i) , ω(t) = ω(i). (2.64)

The integer N determine the total time interval T = N ∆t. The above numerical solution
can be rewritten as

g
ω(i + 1) = ω(i) − θ(i)∆t
l
θ(i + 1) = θ(i) + ω(i)∆t. (2.65)
ydri et al, lectures computational physics 13

We shift the integer i such that it takes values in the range [1, N + 1]. We obtain
g
ω(i) = ω(i − 1) − θ(i − 1)∆t
l
θ(i) = θ(i − 1) + ω(i − 1)∆t. (2.66)

We introduce ω̂(i) = ω(i − 1) and θ̂(i) = θ(i − 1). We get with i = 1, ..., N + 1 the
equations
g
ω̂(i + 1) = ω̂(i) − θ̂(i)∆t
l
θ̂(i + 1) = θ̂(i) + ω̂(i)∆t. (2.67)

By using the values of θ and ω at time i we calculate the corresponding values at time
i + 1. The initial angle and angular velocity θ̂(1) = θ(0) and ω̂(1) = ω(0) are known.
This process will be repeated until the functions θ and ω are determined for all times.

2.4.3 Euler-Cromer Algorithm

As it turns out the above Euler algorithm does not conserve energy. In fact Euler’s
method is not good for all oscillatory systems. A simple modification of Euler’s algorithm
due to Cromer will solve this problem of energy non conservation. This goes as follows.
We use the values of the angle θ̂(i) and the angular velocity ω̂(i) at time step i to calculate
the angular velocity ω̂(i + 1) at time step i + 1. This step is the same as before. However
we use θ̂(i) and ω̂(i + 1) (and not ω̂(i)) to calculate θ̂(i + 1) at time step i + 1. This
procedure as shown by Cromer’s will conserve energy in oscillatory problems. In other
words equations (2.67) become

g
ω̂(i + 1) = ω̂(i) − θ̂(i)∆t
l
θ̂(i + 1) = θ̂(i) + ω̂(i + 1)∆t. (2.68)

The error can be computed as follows. From these two equations we get
g
θ̂(i + 1) = θ̂(i) + ω̂(i)∆t − θ̂(i)∆t2
l
d2 θ̂
= θ̂(i) + ω̂(i)∆t + |i ∆t2 . (2.69)
dt
In other words the error per step is still of the order of ∆t2 . However the Euler-Cromer
algorithm does better than Euler algorithm with periodic motion. Indeed at each step i
the energy conservation condition reads
g g
Ei+1 = Ei + (ωi2 − θi2 )∆t2 . (2.70)
2l l
The energy of the simple pendulum is of course by
1 g
Ei = ωi2 + θi2 . (2.71)
2 2l
14 ydri et al, lectures on computational physics

The error at each step is still proportional to ∆t2 as in the Euler algorithm. However
the coefficient is precisely equal to the difference between the values of the kinetic energy
and the potential energy at the step i. Thus the accumulated error which is obtained by
summing over all steps vanishes since the average kinetic energy is equal to the average
potential energy. In the Euler algorithm the coefficient is actually equal to the sum of
the kinetic and potential energies and as consequence no cancellation can occur.

2.4.4 Verlet Algorithm

Another method which is much more accurate and thus very suited to periodic motions
is due to Verlet. Let us consider the forward and backward Taylor expansions

dθ 1 d2 θ 1 d3 θ
θ(ti + ∆t) = θ(ti ) + ∆t |ti + (∆t)2 2 |ti + (∆t)3 3 |ti + ... (2.72)
dt 2 dt 6 dt

dθ 1 d2 θ 1 d3 θ
θ(ti − ∆t) = θ(ti ) − ∆t |ti + (∆t)2 2 |ti − (∆t)3 3 |ti + ... (2.73)
dt 2 dt 6 dt
Adding these expressions we get

d2 θ
θ(ti + ∆t) = 2θ(ti ) − θ(ti − ∆t) + (∆t)2 |t + O(∆4 ). (2.74)
dt2 i
We write this as
g
θi+1 = 2θi − θi−1 − (∆t)2 θi . (2.75)
l
This is the Verlet algorithm for the harmonic oscillator. First we remark that the error
is proportional to ∆t4 which is less than the errors in the Euler, Euler-Cromer (and even
less than the error in the second-order Runge-Kutta) methods so this method is much
more accurate. Secondly in this method we do not need to calculate the angular velocity
ω = dθ/dt. Thirdly this method is not self-starting. In other words given the initial
conditions θ1 and ω1 we need also to know θ2 for the algorithm to start. We can for
example determine θ2 using the Euler method, viz θ2 = θ1 + ∆t ω1 .

2.5 Lab Problem 1: Euler Algorithm- Air Resistance

The equation of motion of a cyclist exerting a force on his bicycle corresponding to a
constant power P and moving against the force of air resistance is given by

dv P CρAv 2
= − .
dt mv m
The numerical approximation of this first order differential equation which we will con-
sider in this problem is based on Euler algorithm.
ydri et al, lectures computational physics 15

(1) Calculate the speed v as a function of time in the case of zero air resistance and
then in the case of non-vanishing air resistance. What do you observe. We will
take P = 200 and C = 0.5. We also give the values

m = 70kg , A = 0.33m2 , ρ = 1.2kg/m3 , ∆t = 0.1s , T = 200s.

The initial speed is

v̂(1) = 4m/s , t̂(1) = 0.

(2) What do you observe if we change the drag coefficient and/or the power. What do
you observe if we decrease the time step.

2.6 Lab Problem 2: Euler Algorithm- Projectile Motion

The numerical approximation based on the Euler algorithm of the equations of motion
of a projectile moving under the effect of the forces of gravity and air resistance is given
by the equations

B2 v(i)vx (i)
vx (i + 1) = vx (i) − ∆t .
m
B2 v(i)vy (i)
vy (i + 1) = vy (i) − ∆tg − ∆t .
m
q
v(i + 1) = vx2 (i + 1) + vy2 (i + 1).

x(i + 1) = x(i) + ∆t vx (i).

y(i + 1) = y(i) + ∆t vy (i).

(1) Write a Fortran code which implements the above Euler algorithm.

(2) We take the values

B2
= 0.00004m−1 , g = 9.8m/s2 .
m
v(1) = 700m/s , θ = 30 degree.

vx (1) = v(1) cos θ , vy (1) = v(1) sin θ.

N = 105 , ∆t = 0.01s.
Calculate the trajectory of the projectile with and without air resistance. What do
you observe.
16 ydri et al, lectures on computational physics

(3) We can determine numerically the range of the projectile by means of the condi-
tional instruction if. This can be done by adding inside the do loop the following
condition
if (y(i + 1).le.0) exit
Determine the range of the projectile with and without air resistance.
(4) In the case where air resistance is absent we know that the range is maximal when
the initial angle is 45 degrees. Verify this fact numerically by considering several
angles. More precisely add a do loop over the initial angle in order to be able to
study the range as a function of the initial angle.
(5) In the case where air resistance is non zero calculate the angle for which the range
is maximal.

2.7 Lab Problem 3: Euler, Euler-Cromer and Verlet Algo-

rithms
We will consider the numerical solutions of the equation of motion of a simple harmonic
oscillator given by the Euler, Euler-Cromer and Verlet algorithms which take the form
g
ωi+1 = ωi − θi ∆t , θi+1 = θi + ωi ∆t , Euler.
l
g
ωi+1 = ωi − θi ∆t , θi+1 = θi + ωi+1 ∆t , Euler − Cromer.
l
g
θi+1 = 2θi − θi−1 − θi (∆t)2 , Verlet.
l
(1) Write a Fortran code which implements the Euler, Euler-Cromer and Verlet algo-
rithms for the harmonic oscillator problem.
(2) Calculate the angle, the angular velocity and the energy of the harmonic oscillator
as functions of time. The energy of the harmonic oscillator is given by
1 1g 2
E = ω2 + θ .
2 2l
We take the values
g = 9.8m/s2 , l = 1m .
We take the number of iterations N and the time step ∆t to be
N = 10000 , ∆t = 0.05s.
The initial angle and the angular velocity are given by
θ1 = 0.1 radian , ω1 = 0.
By using the conditional instruction if we can limit the total time of motion to be
equal to say 5 periods as follows
if (t(i + 1).ge.5 ∗ period) exit.
ydri et al, lectures computational physics 17

(3) Compare between the value of the energy calculated with the Euler method and
the value of the energy calculated with the Euler-Cromer method. What do you
observe and what do you conclude.

(4) Repeat the computation using the Verlet algorithm. Remark that this method can
not self-start from the initial values θ1 and ω1 only. We must also provide the angle
θ2 which can be calculated using for example Euler, viz

θ2 = θ1 + ω1 ∆t.

We also remark that the Verlet algorithm does not require the calculation of the
angular velocity. However in order to calculate the energy we need to evaluate the
angular velocity which can be obtained from the expression
θi+1 − θi−1
ωi = .
2∆t
18 ydri et al, lectures on computational physics
 Numerical Integration
3
3.1 Rectangular Approximation
We consider a generic one dimensional integral of the form

Z b
F = f (x)dx. (3.1)
a

In general this can not be done analytically. However this integral is straightforward to
do numerically. The starting point is Riemann definition of the integral F as the area
under the curve of the function f (x) from x = a to x = b. This is obtained as follows.
We discretize the x−interval so that we end up with N equal small intervals of lenght
∆x, viz

b−a
xn = x0 + n∆x , ∆x = (3.2)
N
Clearly x0 = a and xN = b. Riemann definition is then given by the following limit
 N
X −1 
F = lim
∆x f (xn ) . (3.3)
∆x−→0 , N −→∞ , b−a=fixed
n=0

The first approximation which can be made is to drop the limit. We get the so-called
rectangular approximation given by
N
X −1
FN = ∆x f (xn ). (3.4)
n=0
20 ydri et al, lectures on computational physics

General integration algorithms approximate the integral F by

N
X
FN = f (xn )wn . (3.5)
n=0

In other words we evaluate the function f (x) at N + 1 points in the interval [a, b] then
we sum the values f (xn ) with some corresponding weights wn . For example in the
rectangular approximation (3.4) the values f (xn ) are summed with equal weights wn =
∆x, n = 0, N − 1 and wN = 0. It is also clear that the estimation FN of the integral F
becomes exact only in the large N limit.

3.2 Trapezoidal Approximation

The trapezoid rule states that we can approximate the integral by a sum of trapezoids.
In the subinterval [xn , xn+1 ] we replace the function f (x) by a straight line connecting
the two points (xn , f (xn )) and (xn+1 , f (xn+1 )). The trapezoid has as vertical sides the
two straight lines x = xn and x = xn+1 . The base is the interval ∆x = xn+1 − xn . It is
not difficult to convince ourselves that the area of this trapezoid is

(f (xn+1 ) − f (xn ))∆x (f (xn+1 ) + f (xn ))∆x

+ f (xn )∆x = . (3.6)
2 2
The integral F computed using the trapezoid approximation is therefore given by sum-
ming the contributions from all the N subinterval, viz

N −1  N −1 
X (f (xn+1 ) + f (xn ))∆x 1 X 1
TN = = f (x0 ) + f (xn ) + f (xN ) ∆x. (3.7)
n=0
2 2 n=1
2

We remark that the weights here are given by w0 = ∆x/2, wn = ∆x, n = 1, ..., N − 1
and wN = ∆x/2.

3.3 Parabolic Approximation or Simpson’s Rule

In this case we approximate the function in the subinterval [xn , xn+1 ] by a parabola given
by

f (x) = αx2 + βx + γ. (3.8)

The area of the corresponding box is thus given by

xn+1 xn+1
αx3 βx2
Z 
2
dx(αx + βx + γ) = + + γx . (3.9)
xn 3 2 xn
ydri et al, lectures computational physics 21

Let us go back and consider the integral

Z 1
2α
dx(αx2 + βx + γ) = + 2γ. (3.10)
−1 3

We remark that

f (−1) = α − β + γ , f (0) = γ , f (1) = α + β + γ. (3.11)

Equivalently
f (1) + f (−1) f (1) − f (−1)
α= − f (0) , β = , γ = f (0). (3.12)
2 2
Thus
1
f (−1) 4f (0) f (1)
Z
dx(αx2 + βx + γ) = + + . (3.13)
−1 3 3 3

In other words we can express the integral of the function f (x) = αx2 + βx + γ over the
interval [−1, 1] in terms of the values of this function f (x) at x = −1, 0, 1. Similarly we
can express the integral of f (x) over the adjacent subintervals [xn−1 , xn ] and [xn , xn+1 ]
in terms of the values of f (x) at x = xn+1 , xn , xn−1 , viz

Z xn+1 Z xn+1
dx f (x) = dx(αx2 + βx + γ)
xn−1 xn−1
 
f (xn−1 ) 4f (xn ) f (xn+1 )
= ∆x + + . (3.14)
3 3 3
By adding the contributions from each pair of adjacent subintervals we get the full integral
N−2
2
X  f (x2p ) 
4f (x2p+1 ) f (x2p+2 )
SN = ∆x + + . (3.15)
3 3 3
p=0

Clearly we must have N (the number of subintervals) even. We compute

 
∆x
SN = f (x0 ) + 4f (x1 ) + 2f (x2 ) + 4f (x3 ) + 2f (x4 ) + ... + 2f (xN −2 ) + 4f (xN −1 ) + f (xN ) .
3
(3.16)

It is trivial to read from this expression the weights in this approximation.

Let us now recall the trapezoidal approximation given by

 N −1 
X ∆x
TN = f (x0 ) + 2 f (xn ) + f (xN ) . (3.17)
2
n=1
22 ydri et al, lectures on computational physics

Let us also recall that N ∆x = b − a is the length of the total interval which is always
kept fixed. Thus by doubling the number of subintervals we halve the width, viz
 2N −1 
X ∆x
4T2N = 2f (x̂0 ) + 4 f (x̂n ) + 2f (x̂2N )
2
n=1
 N −1 N −1 
X X ∆x
= 2f (x̂0 ) + 4 f (x̂2n ) + 4 f (x̂2n+1 ) + 2f (x̂2N )
n=1 n=0
2
 N −1 N −1 
X X ∆x
= 2f (x0 ) + 4 f (xn ) + 4 f (x̂2n+1 ) + 2f (xN ) . (3.18)
2
n=1 n=0

In above we have used the identification x̂2n = xn , n = 0, 1, ..., N − 1, N . Thus

 N
X −1 N
X −1 
4T2N − TN = f (x0 ) + 2 f (xn ) + 4 f (x̂2n+1 ) + f (xN ) ∆x̂
n=1 n=0
= 3SN . (3.19)

3.4 Errors
The error estimates for numerical integration are computed as follows. We start with the
Taylor expansion
1
f (x) = f (xn ) + (x − xn )f (1) (xn ) + (x − xn )2 f (2) (xn ) + ... (3.20)
2!
Thus
xn+1
1 (1) 1
Z
dx f (x) = f (xn )∆x + f (xn )(∆x)2 + f (2) (xn )(∆x)3 + ... (3.21)
xn 2! 3!

The error in the interval [xn , xn+1 ] in the rectangular approximation is

Z xn+1
1 1
dx f (x) − f (xn )∆x = f (1) (xn )(∆x)2 + f (2) (xn )(∆x)3 + ... (3.22)
xn 2! 3!

This is of order 1/N 2 . But we have N subintervals. Thus the total error is of order 1/N .
The error in the interval [xn , xn+1 ] in the trapezoidal approximation is
Z xn+1 Z xn+1
1
dx f (x) − (f (xn ) + f (xn+1 ))∆x = dx f (x)
xn 2 xn
1 1
− (2f (xn ) + ∆xf (1) (xn ) + (∆x)2 f (2) (xn ) + ...)∆x
2 2!
1 1 1 (2)
= ( − )f (xn )(∆x)3 + ... (3.23)
3! 2 2!
This is of order 1/N 3 and thus the total error is of order 1/N 2 .
ydri et al, lectures computational physics 23

In order to compute the error in the interval [xn−1 , xn+1 ] in the parabolic approxi-
mation we compute
Z xn Z xn+1
2 2
dx f (x) + dx f (x) = 2f (xn )∆x + (∆x)3 f (2) (xn ) + (∆x)5 f (4) (xn ) + ...
xn−1 xn 3! 5!
(3.24)

Also we compute
∆x 2 2
(f (xn+1 ) + f (xn−1 ) + 4f (xn )) = 2f (xn )∆x + (∆x)3 f (2) (xn ) + (∆x)5 f (4) (xn ) + ...
3 3! 3.4!
(3.25)

Hence the error in the interval [xn−1 , xn+1 ] in the parabolic approximation is
Z xn+1
∆x 2 2
dx f (x) − (f (xn+1 ) + f (xn−1 ) + 4f (xn )) = ( − )(∆x)5 f (4) (xn ) + ...
xn−1 3 5! 3.4!
(3.26)

This is of order 1/N 5 . The total error is therefore of order 1/N 4 .

3.5 Lab Problem 4: Numerical Integration

(1) We take the integral
Z 1
I= f (x)dx ; f (x) = 2x + 3x2 + 4x3 .
0

Calculate the value of this integral using the rectangular approximation. Compare
with the exact result.
Hint: You can code the function using either "subroutine" or "function".

(2) Calculate the numerical error as a function of N . Compare with the theory.

(3) Repeat the computation using the trapezoid method and the Simpson’s rule.

(4) Take now the integrals

π
e +1  
1 1 ǫ
Z Z Z
2
I= cos xdx , I = dx , I = lim dx.
0 1 x −1 ǫ−→0 π x2 + ǫ 2
24 ydri et al, lectures on computational physics
 Newton-Raphson Algorithms and Interpolation
4
4.1 Bisection Algorithm
Let f be some function. We are interested in the solutions (roots) of the equation

f (x) = 0. (4.1)

The bisection algorithm works as follows. We start with two values of x say x+ and x−
such that

f (x− ) < 0 , f (x+ ) > 0. (4.2)

In other words the function changes sign in the interval between x− and x+ and thus
there must exist a root between x− and x+ . If the function changes from positive to
negative as we increase x we conclude that x+ ≤ x− . We bisect the interval [x+ , x− ] at
x+ + x−
x= . (4.3)
2
If f (x)f (x+ ) > 0 then x+ will be changed to the point x otherwise x− will be changed
to the point x. We continue this process until the change in x becomes insignificant or
until the error becomes smaller than some tolerance. The relative error is defined by
x+ − x−
error = . (4.4)
x
Clearly the absolute error e = xi − xf is halved at each iteration and thus the rate of
convergence of the bisection rule is linear. This is slow.
26 ydri et al, lectures on computational physics

4.2 Newton-Raphson Algorithm

We start with a guess x0 . The new guess x is written as x0 plus some unknown correction
∆x, viz
x = x0 + ∆x. (4.5)
Next we expand the function f (x) around x0 , namely
df
f (x) = f (x0 ) + ∆x|x=x0 . (4.6)
dx
The correction ∆x is determined by finding the intersection point of this linear approxi-
mation of f (x) with the x axis. Thus
df f (x0 )
f (x0 ) + ∆x |x=x0 = 0 =⇒ ∆x = − . (4.7)
dx (df /dx)|x=x0
The derivative of the function f is required in this calculation. In complicated problems
it is much simpler to evaluate the derivative numerically than analytically. In these cases
the derivative may be given by the forward-difference approximation (with some δx not
necessarily equal to ∆x)
df f (x0 + δx) − f (x0 )
|x=x0 = . (4.8)
dx δx
In summary this method works by drawing the tangent to the function f (x) at the old
guess x0 and then use the intercept with the x axis as the new hopefully better guess x.
The process is repeated until the change in x becomes insignificant.
Next we compute the rate of convergence of the Newton-Raphson algorithm. Starting
from xi the next guess is xi+1 given by
f (xi )
xi+1 = xi − . (4.9)
f ′ (x)
The absolute error at step i is ǫi = x − xi while the absolute error at step i + 1 is
ǫi+1 = x − xi+1 where x is the actual root. Then
f (xi )
ǫi+1 = ǫi + . (4.10)
f ′ (x)
By using Taylor expansion we have
′ (x − xi )2 ′′
f (x) = 0 = f (xi ) + (x − xi )f (xi ) + f (xi ) + ... (4.11)
2!
In other words
′ ǫ2i ′′
f (xi ) = −ǫi f (xi ) − f (xi ) + ... (4.12)
2!
Therefore the error is given by
′′
ǫ2i f (xi )
ǫi+1 = − . (4.13)
2 f ′ (xi )
This is quadratic convergence. This is faster than the bisection rule.
ydri et al, lectures computational physics 27

4.3 Hybrid Method

We can combine the certainty of the bisection rule in finding a root with the fast con-
vergence of the Newton-Raphson algorithm into a hybrid algorithm as follows. First we
must know that the root is bounded in some interval [a, c]. We can use for example a
graphical method. Next we start from some initial guess b. We take a Newton-Raphson
step
′ f (b)
b =b− . (4.14)
f ′ (b)

We check whether or not this step is bounded in the interval [a, c]. In other words we
must check that
f (b) ′ ′
a≤b − ≤c ⇔ (b − c)f (b) − f (b)≤0≤(b − a)f (b) − f (b). (4.15)
f ′ (b)
Therefore if
  
′ ′
(b − c)f (b) − f (b) (b − a)f (b) − f (b) < 0 (4.16)

Then the Newton-Raphson step is accepted else we take instead a bisection step.

4.4 Lagrange Interpolation

Let us first recall that taylor expansion allows us to approximate a function at a point
x if the function and its derivatives are known in some neighbouring point x0 . The
lagrange interpolation tries to approximate a function at a point x if only the values of
the function in several other points are known. Thus this method does not require the
knowledge of the derivatives of the function. We start from taylor expansion
′ 1 ′′
f (y) = f (x) + (y − x)f (x) + (y − x)2 f (x) + .. (4.17)
2!
Let us assume that the function is known at three points x1 , x2 and x3 . In this case we
can approximate the function f (x) by some function p(x) and write
′ 1 ′′
f (y) = p(x) + (y − x)p (x) + (y − x)2 p (x). (4.18)
2!
We have
′1 ′′
f (x1 ) = p(x) + (x1 − x)p (x) + (x1 − x)2 p (x)
2!
′ 1 ′′
f (x2 ) = p(x) + (x2 − x)p (x) + (x2 − x)2 p (x)
2!
′ 1 ′′
f (x3 ) = p(x) + (x3 − x)p (x) + (x3 − x)2 p (x). (4.19)
2!
28 ydri et al, lectures on computational physics

We can immediately find

1 a2 a3
p(x) = f (x1 ) + f (x2 ) + f (x3 ). (4.20)
1 + a2 + a3 1 + a2 + a3 1 + a2 + a3
The coefficients a2 and a3 solve the equations
a2 (x2 − x)2 + a3 (x3 − x)2 = −(x1 − x)2
a2 (x2 − x) + a3 (x3 − x) = −(x1 − x). (4.21)
We find
(x1 − x)(x3 − x1 ) (x1 − x)(x2 − x1 )
a2 = , a3 = − . (4.22)
(x2 − x)(x2 − x3 ) (x3 − x)(x2 − x3 )
Thus
(x3 − x1 )(x2 − x1 )
1 + a2 + a3 = . (4.23)
(x2 − x)(x3 − x)
Therefore we get
(x − x2 )(x − x3 ) (x − x1 )(x − x3 ) (x − x1 )(x − x2 )
p(x) = f (x1 ) + f (x2 ) + f (x3 ).
(x1 − x2 )(x1 − x3 ) (x2 − x1 )(x2 − x3 ) (x3 − x1 )(x3 − x2 )
(4.24)
This is a quadratic polynomial.
Let x be some independent variable with tabulated values xi , i = 1, 2, ..., n.. The
dependent variable is a function f (x) with tabulated values fi = f (xi ). Let us then
assume that we can approximate f (x) by a polynomial of degree n − 1 , viz
p(x) = a0 + a1 x + a2 x2 + ... + an−1 xn−1 . (4.25)
A polynomial which goes through the n points (xi , fi = f (xi )) was given by Lagrange.
This is given by
p(x) = f1 λ1 (x) + f2 λ2 (x) + ... + fn λn (x). (4.26)
Yn x − xj
λi (x) = . (4.27)
j(6=i)=1 xi − xj

We remark
λi (xj ) = δij . (4.28)

n
X
λi (x) = 1. (4.29)
i=1

The Lagrange polynomial can be used to fit the entire table with n equal the number of
points in the table. But it is preferable to use the Lagrange polynomial to to fit only a
small region of the table with a small value of n. In other words use several polynomials
to cover the whole table and the fit considered here is local and not global.
ydri et al, lectures computational physics 29

4.5 Cubic Spline Interpolation

We consider n points (x1 , f (x1 )),(x2 , f (x2 )),...,(xn , f (xn )) in the plane. In every interval
xj ≤x≤xj+1 we approximate the function f (x) with a cubic polynomial of the form

p(x) = aj (x − xj )3 + bj (x − xj )2 + cj (x − xj ) + dj . (4.30)

We assume that

pj = p(xj ) = f (xj ). (4.31)

In other words the pj for all j = 1, 2, ..., n − 1 are known. From the above equation we
conclude that

dj = pj . (4.32)

We compute
′
p (x) = 3aj (x − xj )2 + 2bj (x − xj ) + cj . (4.33)

′′
p (x) = 6aj (x − xj ) + 2bj . (4.34)
′′
Thus we get by substituting x = xj into p (x) the result
′′
pj
bj = . (4.35)
2
′′
By substituting x = xj+1 into p (x) we get the result
′′ ′′
pj+1 − pj
aj = . (4.36)
6hj

By substituting x = xj+1 into p(x) we get

pj+1 = aj h3j + bj h2j + cj hj + pj . (4.37)

By using the values of aj and bj we obtain

pj+1 − pj hj ′′ ′′
cj = − (pj+1 + 2pj ). (4.38)
hj 6

Hence
′′ ′′ ′′
pj+1 − pj pj pj+1 − pj
 
hj ′′ ′′
p(x) = (x − xj )3 + (x − xj )2 + − (pj+1 + 2pj ) (x − xj ) + pj .
6hj 2 hj 6
(4.39)
30 ydri et al, lectures on computational physics

′′
In other words the polynomials are determined from pj and pj . The pj are known given
′′
by pj = f (xj ). It remains to determine pj . We take the derivative of the above equation

′′ ′′
pj+1 − pj pj+1 − pj
 
′ ′′ hj ′′ ′′
p (x) = (x − xj )2 + pj (x − xj ) + − (pj+1 + 2pj ) . (4.40)
2hj hj 6

This is the derivative in the interval [xj , xj+1 ]. We compute

pj+1 − pj
 
′ hj ′′ ′′
p (xj ) = − (pj+1 + 2pj ) . (4.41)
hj 6

The derivative in the interval [xj−1 , xj ] is

′′ ′′
pj − pj−1 pj − pj−1 hj−1 ′′
 
′ ′′ ′′
p (x) = (x − xj−1 )2 + pj−1 (x − xj−1 ) + − (pj + 2pj−1 ) (4.42)
.
2hj−1 hj−1 6

We compute
′′ ′′
pj − pj−1 pj − pj−1 hj−1 ′′
 
′ ′′ ′′
p (xj ) = hj−1 + pj−1 hj−1 + − (pj + 2pj−1 ) . (4.43)
2 hj−1 6
′
By matching the two expressions for p (xj ) we get

pj+1 − pj pj − pj−1
 
′′ ′′ ′′
hj−1 pj−1 + 2(hj + hj−1 )pj + hj pj+1 =6 − . (4.44)
hj hj−1
′′
These are n − 2 equations since j = 2, ..., n − 1 for n unknown pj . We need two more
′
equations. These are obtained by computing the first derivative p (x) at x = x1 and
x = xn . We obtain the two equations

′′ ′′ 6(p2 − p1 ) ′
h1 (p2 + 2p1 ) = − 6p1 . (4.45)
h1

′′ ′′ 6(pn − pn−1 ) ′
hn−1 (pn−1 + 2pn ) = − + 6pn . (4.46)
hn−1

The n equations (4.44), (4.45) and (4.46) correspond to a tridiagonal linear system. In
′ ′
general p1 and pn are not known. In this case we may use natural spline in which the
second derivative vanishes at the end points and hence

p2 − p1 ′ pn − pn−1 ′
− p1 = − pn = 0. (4.47)
h1 hn−1
ydri et al, lectures computational physics 31

4.6 Lab Problem 5: Newton-Raphson Algorithm

A particle of mass m moves inside a potential well of height V and length 2a centered
around 0. We are interested in the states of the system which have energies less than V ,
i.e. bound states. The states of the system can be even or odd. The energies associated
with the even wave functions are solutions of the transcendental equation

α tan αa = β.
r r
2mE 2m(V − E)
α= , β= .
~2 ~2
In the case of the infinite potential well we find the solutions
(n + 21 )2 π 2 ~2
En = , n = 0, 1....
2ma2
We choose (dropping units)
~ = 1 , a = 1 , 2m = 1.
In order to find numerically the energies En we will use the Newton-Raphson algorithm
which allows us to find the roots of the equation f (x) = 0 as follows. From an initial
guess x0 , the first approximation x1 to the solution is determined from the intersection
of the tangent to the function f (x) at x0 with the x−axis. This is given by

f (x0 )
x1 = x0 − .
f ′ (x0 )
Next by using x1 we repeat the same step in order to find the second approximation x2
to the solution. In general the approximation xi+1 to the desired solution in terms of the
approximation xi is given by the equation
f (xi )
xi+1 = xi − .
f ′ (xi )
(1) For V = 10, determine the solutions using the graphical method. Consider the two
functions r
β V
f (α) = tan αa , g(α) = = − 1.
α α2
(2) Find using the method of Newton-Raphson the two solutions with a tolerance equal
10−8 . For the first solution we take the initial guess α = π/a and for the second
solution we take the initial guess α = 2π/a.
(3) Repeat for V = 20.
(4) Find the 4 solutions for V = 100. Use the graphical method to determine the initial
step each time.
(5) Repeat the above questions using the bisection method.
32 ydri et al, lectures on computational physics
 The Solar System-The Runge-Kutta Methods
5
5.1 The Solar System
5.1.1 Newton’s Second Law
We consider the motion of the Earth around the Sun. Let r be the distance and Ms and
Me be the masses of the Sun and the Earth respectively. We neglect the effect of the
other planets and the motion of the Sun (i.e. we assume that Ms >> Me ). The goal
is to calculate the position of the Earth as a function of time. We start from Newton’s
second law of motion

d2~r GMe Ms
Me = − ~r
dt2 r3
GMe Ms ~
= − (xi + y~j). (5.1)
r3
We get the two equations
d2 x GMs
= − 3 x. (5.2)
dt2 r

d2 y GMs
= − 3 y. (5.3)
dt2 r
We replace these two second-order differential equations by the four first-order differential
equations
dx
= vx . (5.4)
dt
34 ydri et al, lectures on computational physics

dvx GMs
= − 3 x. (5.5)
dt r
dy
= vy . (5.6)
dt
dvy GMs
= − 3 y. (5.7)
dt r
We recall
p
r= x2 + y 2 . (5.8)

5.1.2 Astronomical Units and Initial Conditions

The distance will be measured in astronomical units (AU) whereas time will be measured
in years. One astronomical unit of lenght (1 AU) is equal to the average distance between
the earth and the sun, viz 1AU = 1.5 × 1011 m. The astronomical unit of mass can be
found as follows. Assuming a circular orbit we have
Me v 2 GMs Me
= . (5.9)
r r2
Equivalently
GMs = v 2 r. (5.10)
The radius is r = 1AU. The velocity of the earth is v = 2πr/yr = 2πAU/yr. Hence
GMs = 4π 2 AU3 /yr2 . (5.11)
For the numerical simulations it is important to determine the correct initial conditions.
The orbit of Mercury is known to be an ellipse with eccentricity e = 0.206 and radius
(semimajor axis) a = 0.39 AU with the Sun at one of the foci. The distance between
the Sun and the center is ea. The first initial condition is x0 = r1 , y0 = 0 where r1
is the maximum distance from Mercury to the Sun,i.e. r1 = (1 + e)a = 0.47 AU. The
second initial condition is the velocity (0, v1 ) which can be computed using conservation
of energy and angular momentum. For example by √ comparing with the point (0, b) on
the orbit where b is the semiminor axis, i.e b = a 1 − e2 the velocity (v2 , 0) there can
be obtained in terms of (0, v1 ) from conservation of angular momentum as follows
r1 v1
r1 v1 = bv2 ⇔ v2 = . (5.12)
b
Next conservation of energy yields
GMs Mm 1 GMs Mm 1
− + Mm v12 = − + Mm v22 . (5.13)
r1 2 r2 2
√
In above r2 = e2 a2 + b2 is the distance between the Sun and Mercury when at the
point (0, b). By substituting the value of v2 we get an equation for v1 . This is given by
r
GMs 1 − e
v1 = = 8.2 AU/yr. (5.14)
a 1+e
ydri et al, lectures computational physics 35

5.1.3 Kepler’s Laws

Kepler’s laws are given by the following three statements:

• The planets move in elliptical orbits around the sun. The sun resides at one focus.

• The line joining the sun with any planet sweeps out equal areas in equal times.

• Given an orbit with a period T and a semimajor axis a the ratio T 2 /a3 is a constant.

The derivation of these three laws proceeds as follows. We work in polar coordinates.
Newton’s second law reads

GMs Me
Me~r¨ = − r̂. (5.15)
r2
˙
We use r̂˙ = θ̇θ̂ and θ̂ = −θ̇r̂ to derive ~r˙ = ṙr̂ + r θ̇θ̂ and ~r¨ = (r̈ − r θ̇ 2 )r̂ + (r θ̈ + 2ṙ θ̇)θ̂.
Newton’s second law decomposes into the two equations

r θ̈ + 2ṙ θ̇ = 0. (5.16)

GMs
r̈ − r θ̇ 2 = − . (5.17)
r2

Let us recall that the angular momentum by unit mass is defined by ~l = ~r × ~r˙ = r 2 θ̇r̂ × θ̂.
Thus l = r 2 θ̇. Equation (5.16) is precisely the requirement that angular momentum is
conserved. Indeed we compute
dl
= r(r θ̈ + 2ṙ θ̇) = 0. (5.18)
dt
Now we remark that the area swept by the vector ~r in a time interval dt is dA = (r×rdθ)/2
where dθ is the angle traveled by ~r during dt. Clearly
dA 1
= l. (5.19)
dt 2
In other words the planet sweeps equal areas in equal times since l is conserved. This is
Kepler’s second law.
The second equation (5.17) becomes now

l2 GMs
r̈ = − 2 (5.20)
r3 r
By multiplying this equation with ṙ we obtain

d 1 l2 GMs
E = 0 , E = ṙ 2 + 2 − . (5.21)
dt 2 2r r
36 ydri et al, lectures on computational physics

This is precisely the statement of conservation of energy. E is the energy per unit mass.
Solving for dt in terms of dr we obtain

dr
dt = s   (5.22)
l2 GMs
2 E− 2r 2
+ r

However dt = (r 2 dθ)/l. Thus

ldr
dθ = s   (5.23)
2 GMs
r 2 2 E − 2rl 2 + r

By integrating this equation we obtain (with u = 1/r)

ldr
Z
θ = s  
l2 GMs
r2 2 E− 2r 2 + r

du
Z
= − q . (5.24)
2E 2GMs
l2 + l2 u − u2

This integral can be done explicitly. We get

s
u−C 2l2 E
 
′ GMs
θ = − arccos +θ , e= 1+ 2 2
, C= 2 . (5.25)
eC G Ms l

By inverting this equation we get an equation of ellipse with eccentricity e since E < 0,
viz
1 ′
= C(1 + e cos(θ − θ )). (5.26)
r
′
This is Kepler’s first law. The angle at which r is maximum is θ − θ = π. This distance
is precisely (1 + e)a where a is the semi-major axis of the ellipse since ea is the distance
between the Sun which is at one of the two foci and the center of the ellipse. Hence we
obtain the relation
1 l2
(1 − e2 )a = = . (5.27)
C GMs

From equation (5.19) we can derive Kepler’s third law. By integrating both sides of the
equation over a single period T and then taking the square we get
1
A2 = l2 T 2 . (5.28)
4
ydri et al, lectures computational physics 37

A is the area of the ellipse,

√ i.e. A = πab where the semi-minor axis b is related the
semi-major axis a by b = a 1 − e2 . Hence
1
π 2 a4 (1 − e2 ) = l2 T 2 . (5.29)
4
By using equation (5.27) we get the desired formula

T2 4π 2
= . (5.30)
a3 GMs

5.1.4 The inverse-Square Law and Stability of Orbits

Any object with mass generates a gravitational field and thus gravitational field lines
will emanate from the object and radiate outward to infinity. The number of field lines
N is proportional to the mass. The density of field lines crossing a sphere of radius r
surrounding this object is given by N/4πr 2 . This is the origin of the inverse-square law.
Therefore any other object placed in this gravitational field will experience a gravitational
force proportional to the number of field lines which intersect it. If the distance between
this second object and the source is increased the force on it will become weaker because
the number of field lines which intersect it will decrease as we are further away from the
source.

5.2 Euler-Cromer Algorithm

The time discretization is

t ≡ t(i) = i∆t , i = 0, ..., N. (5.31)

The total time interval is T = N ∆t. We define x(t) = x(i), vx (t) = vx (i), y(t) = y(i),
vy (t) = vy (i). Equations (5.4), (5.5), (5.6),(5.7) and (5.8) become (with i = 0, ..., N )

GMs
vx (i + 1) = vx (i) − x(i)∆t. (5.32)
(r(i))3

x(i + 1) = x(i) + vx (i)∆t. (5.33)

GMs
vy (i + 1) = vy (i) − y(i)∆t. (5.34)
(r(i))3

y(i + 1) = y(i) + vy (i)∆t. (5.35)

p
r(i) = x(i)2 + y(i)2 . (5.36)
38 ydri et al, lectures on computational physics

This is Euler algorithm. It can also be rewritten with x̂(i) = x(i − 1), ŷ(i) = y(i − 1),
v̂x (i) = vx (i − 1), v̂y (i) = vy (i − 1), r̂(i) = r(i − 1) and i = 1, ..., N + 1 as

GMs
v̂x (i + 1) = v̂x (i) − x̂(i)∆t. (5.37)
(r̂(i))3

x̂(i + 1) = x̂(i) + v̂x (i)∆t. (5.38)

GMs
v̂y (i + 1) = v̂y (i) − ŷ(i)∆t. (5.39)
(r̂(i))3

ŷ(i + 1) = ŷ(i) + v̂y (i)∆t. (5.40)

p
r̂(i) = x̂(i)2 + ŷ(i)2 . (5.41)

In order to maintain energy conservation we employ Euler-Cromer algorithm. We calcu-

late as in the Euler’s algorithm the velocity at time step i + 1 by using the position and
velocity at time step i. However we compute the position at time step i + 1 by using the
position at time step i and the velocity at time step i + 1, viz
GMs
v̂x (i + 1) = v̂x (i) − x̂(i)∆t. (5.42)
(r̂(i))3

x̂(i + 1) = x̂(i) + v̂x (i + 1)∆t. (5.43)

GMs
v̂y (i + 1) = v̂y (i) − ŷ(i)∆t. (5.44)
(r̂(i))3

ŷ(i + 1) = ŷ(i) + v̂y (i + 1)∆t. (5.45)

5.3 The Runge-Kutta Algorithm

5.3.1 The Method
The problem is still trying to solve the first order differential equation
dy
= f (x, y). (5.46)
dx
In the Euler’s method we approximate the function y = y(x) in each interval [xn , xn+1 ]
by the straight line

yn+1 = yn + ∆xf (xn , yn ). (5.47)

ydri et al, lectures computational physics 39

The slope f (xn , yn ) of this line is exactly given by the slope of the function y = y(x) at
the begining of the inetrval [xn , xn+1 ].
Given the value yn at xn we evaluate the value yn+1 at xn+1 using the method of
Runge-Kutta as follows. First the middle of the interval [xn , xn+1 ] which is at the value
xn + 12 ∆x corresponds to the y-value yn+1 calculated using the Euler’s method, viz
yn+1 = yn + 12 k1 where

k1 = ∆xf (xn , yn ). (5.48)

Second the slope at this middle point (xn + 21 ∆x, yn + 12 k1 ) which is given by

k2 1 1
= f (xn + ∆x, yn + k1 ) (5.49)
∆x 2 2

is the value of the slope which will be used to estimate the correct value of yn+1 at xn+1
using again Euler’s method, namely

yn+1 = yn + k2 . (5.50)

In summary the Runge-Kutta algorithm is given by

k1 = ∆xf (xn , yn )
1 1
k2 = ∆xf (xn + ∆x, yn + k1 )
2 2
yn+1 = yn + k2 . (5.51)

The error in this method is proportional to ∆x3 . This can be shown as follows. We have

dy 1 d2 y
y(x + ∆x) = y(x) + ∆x + (∆x)2 2 + ...
dx 2 dx
1 d
= y(x) + ∆xf (x, y) + (∆x)2 f (x, y) + ...
 2 dx 
1 ∂f 1 ∂f
= y(x) + ∆x f (x, y) + ∆x + ∆xf (x, y) + ...
2 ∂x 2 ∂y
1 1
= y(x) + ∆xf (x + ∆x, y + ∆xf (x, y)) + O(∆x3 )
2 2
1 1
= y(x) + ∆xf (x + ∆x, y + k1 ) + O(∆x3 )
2 2
3
= y(x) + k2 + O(∆x ). (5.52)

Let us finally note that the above Runge-Kutta method is strictly speaking the second-
order Runge-Kutta method. The first-order Runge-Kutta method is the Euler algorithm.
The higher-order Runge-Kutta methods will not be discussed here.
40 ydri et al, lectures on computational physics

5.3.2 Example 1: The Harmonic Oscillator

Let us apply this method to the problem of the harmonic oscillator. We have the differ-
ential equations

dθ
=ω
dt
dω g
= − θ. (5.53)
dt l
Euler’s equations read

θn+1 = θn + ∆tωn
g
ωn+1 = ωn − θn ∆t. (5.54)
l
First we consider the function θ = θ(t). The middle point is (tn + 21 ∆t, θn + 21 k1 ) where
k1 = ∆tωn . For the function ω = ω(t) the middle point is (tn + 12 ∆t, ωn + 21 k3 ) where
k3 = − gl ∆tθn . Therefore we have

k1 = ∆tωn
g
k3 = − ∆tθn . (5.55)
l
The slope of the function θ(t) at its middle point is
k2 1
= ωn + k3 . (5.56)
∆t 2
The slope of the function ω(t) at its middle point is
k4 g 1
= − (θn + k1 ). (5.57)
∆t l 2
The Runge-Kutta solution is then given by

θn+1 = θn + k2
ωn+1 = ωn + k4 . (5.58)

5.3.3 Example 2: The Solar System

Let us consider the equations
dx
= vx . (5.59)
dt

dvx GMs
= − 3 x. (5.60)
dt r
ydri et al, lectures computational physics 41

dy
= vy . (5.61)
dt

dvy GMs
= − 3 y. (5.62)
dt r
First we consider the function x = x(t). The middle point is (tn + 12 ∆t, xn + 12 k1 ) where
k1 = ∆t vxn . For the function vx = vx (t) the middle point is (tn + 21 ∆t, vxn + 21 k3 ) where
k3 = − GM
rn ∆t xn . Therefore we have
s

k1 = ∆t vxn
GMs
k3 = − 3 ∆t xn . (5.63)
rn

The slope of the function x(t) at the middle point is

k2 1
= vxn + k3 . (5.64)
∆t 2
The slope of the function vx (t) at the middle point is
k4 GMs 1
= − 3 (xn + k1 ). (5.65)
∆t Rn 2

Next we consider the function y = y(t). The middle point is (tn + 21 ∆t, yn + 21 k1 ) where
′

k1 = ∆t vyn . For the function vy = vy (t) the middle point is (tn + 12 ∆t, vyn + 12 k3 ) where
′ ′

k3 = − GM
′
rn ∆t yn . Therefore we have
s

′
k1 = ∆t vyn
′ GMs
k3 = − 3 ∆t yn . (5.66)
rn

The slope of the function y(t) at the middle point is

′
k2 1 ′
= vyn + k3 . (5.67)
∆t 2
The slope of the function vy (t) at the middle point is
′
k4 GMs 1 ′
= − 3 (yn + k1 ). (5.68)
∆t Rn 2
In the above equations
r
1 1 ′
Rn = (xn + k1 )2 + (yn + k1 )2 . (5.69)
2 2
42 ydri et al, lectures on computational physics

The Runge-Kutta solutions are then given by

xn+1 = xn + k2
vx(n+1) = vxn + k4
′
yn+1 = yn + k2
′
vy(n+1) = vyn + k4 . (5.70)

5.4 Precession of the Perihelion of Mercury

The orbit of Mercury is elliptic. The orientation of the axes of the ellipse rotate with
time. This is the precession of the perihelion (the point of the orbit nearest to the
Sun) of Mercury. Mercury’s perihelion makes one revolution every 23000 years. This is
approximately 566 arcseconds per century. The gravitational forces of the other planets
(in particular Jupiter) lead to a precession of 523 arcseconds per century. The remaining
43 arcseconds per century are accounted for by general relativity.
For objects too close together (like the Sun and Mercury) the force of gravity predicted
by general relativity deviates from the inverse-square law. This force is given by
GMs Mm α
F = 2
(1 + 2 ) , α = 1.1 × 10−8 AU2 . (5.71)
r r
We discuss here some of the numerical results obtained with the Runge-Kutta method
for different values of α. We take the time step and the number of iterations to be
N = 20000 and dt = 0.0001. The angle of the line joining the Sun and Mercury with the
horizontal axis when mercury is at the perihelion is found to change linearly with time.
We get the following rates of precession
dθ
α = 0.0008 , = 8.414 ± 0.019
dt
dθ
α = 0.001 , = 10.585 ± 0.018
dt
dθ
α = 0.002 , = 21.658 ± 0.019
dt
dθ
α = 0.004 , = 45.369 ± 0.017. (5.72)
dt
Thus
dθ
= aα , α = 11209.2 ± 147.2 degrees/(yr.α). (5.73)
dt
By extrapolating to the value provided by general relativity, viz α = 1.1 × 10−8 we get
dθ
= 44.4 ± 0.6 arcsec/century. (5.74)
dt
ydri et al, lectures computational physics 43

5.5 Lab Problem 6: Runge-Kutta Algorithm- The Solar

System
Part I We consider a solar system consisting of a single planet moving around the Sun.
We suppose that the Sun is very heavy compared to the planet that we can safely assume
that it is not moving at the center of the system. Newton’s second law gives the following
equations of motion

dx dvx GMs dy dvy GMs

vx = , = − 3 x , vy = , = − 3 y.
dt dt r dt dt r
We will use here the astronomical units defined by GMs = 4π 2 AU3 /yr2 .

(1) Write a Fortran code in which we implement the Runge-Kutta algorithm for the
problem of solving the equations of motion of the the solar system.

(2) Compute the trajectory, the velocity and the energy as functions of time. What do
you observe for the energy.

(3) According to Kepler’s first law the orbit of any planet is an ellipse with the Sun at
one of the two foci. In the following we will only consider planets which are known
to have circular orbits to a great accuracy. These planets are Venus, Earth, Mars,
Jupiter and Saturn. The radii in astronomical units are given by

avenus = 0.72 , aearth = 1 , amars = 1.52 , ajupiter = 5.2 , asaturn = 9.54.

Verify that Kepler’s first law indeed holds for these planets.
In order to answer questions 2 and 3 above we take the initial conditions

x(1) = a , y(1) = 0 , vx (1) = 0 , vy (1) = v.

The value chosen for the initial velocity is very important to get a correct orbit
and must be determined for example by assuming that the orbit is indeed circular
and as a consequence the
pcentrifugal force is balanced by the force of gravitational
attraction. We get v = GMs /a.
We take the step and the number of iterations ∆t = 0.01 yr , N = 103 − 104 .

Part II

(1) According to Kepler’s third law the square of the period of a planet is directly
proportional to the cube of the semi-major axis of its orbit. For circular orbits the
proportionality factor is equal 1 exactly. Verify this fact for the planets mentioned
above. We can measure the period of a planet by monitoring when the planet
returns to its farthest point from the sun.
44 ydri et al, lectures on computational physics

(2) By changing the initial velocity appropriately we can obtain an elliptical orbit.
Check this thing.
(3) The fundamental laws governing the motion of the solar system are Newton’s law
of universal attraction and Newton’s second law of motion. Newton’s law of uni-
versal attraction states that the force between the Sun and a planet is inversely
proportioanl to the square of the distance between them and it is directed from
the planet to the Sun. We will assume in the following that this force is inversely
proportional to a different power of the distance. Modify the code accordingly and
calculate the new orbits for powers between 1 and 3. What do you observe and
what do you conclude.

5.6 Lab Problem 7: Precession of the perihelion of Mercury

According to Kepler’s first law the orbits of all planets are ellipses with the Sun at one
of the two foci. This law can be obtained from applying Newton’s second law to the
system consisting of the Sun and a single planet. The effect of the other planets on the
motion will lead to a change of orientation of the orbital ellipse within the orbital plane
of the planet. Thus the point of closest approach (the perihelion) will precess, i.e. rotate
around the sun. All planets suffer from this effect but because they are all farther from
the sun and all have longer periods than Mercury the amount of precession observed for
them is smaller than that of Mercury.
However it was established earlier on that the precession of the perihelion of Mer-
cury due to Newtonian effects deviates from the observed precession by the amount
43 arcsecond/century. As it turns out this can only be explained within general rela-
tivity. The large mass of the Sun causes space and time around it to be curved which
is felt the most by Mercury because of its proximity. This spacetime curvature can be
approximated by the force law

GMs Mm α
F = 2
(1 + 2 ) , α = 1.1.10−8 AU 2 .
r r
(1) Include the above force in the code. The initial position and velocity of Mercury
are

x0 = (1 + e)a , y0 = 0.
r
GMs 1 − e
vx0 = 0 , vy0 = .
a 1+e
Thus initially Mercury is at its farthest point from the Sun since a is the semi-major
axis of Mercury (a = 0.39 AU) and e is its eccentricity (e = 0.206) and hence ea
is the distance between√the Sun and the center of the ellipse. The semi-minor
axis is defined by b = a 1 − e2 . The initial velocity was calculated from applying
the principles of conservation of angular momentum and conservation of energy
between the above initial point and the point (0, b).
ydri et al, lectures computational physics 45

(2) The amount of precession of the perihelion of Mercury is very small because α is
very small. In fact it can not be measured directly in any numerical simulation
with a limited amount of time. Therefore we will choose a larger value of α for
example α = 0.0008 AU2 . We also work with N = 20000 , dt = 0.0001. Compute
the orbit for these values. Compute the angle θ made between the vector position
of Mercury and the horizontal axis as a function of time. Compute also the distance
between Mercury and the sun and its derivative with respect to time given by

dr xvx + yvy
= .
dt r
This derivative will vanish each time Mercury reaches its farthest point from the
sun or its closest point from the sun (the perihelion). Plot the angle θp made
between the vector position of Mercury at its farthest point and the horizontal axis
as a function of time. What do you observe. Determine the slope dθp /dt which is
precisely the amount of precession of the perihelion of Mercury for the above value
of α.

(3) Repeat the above question for other values of α say α = 0.001, 0.002, 0.004. Each
time compute dθp /dt. Plot dθp /dt as a function of α. Determine the slope.
Deduce the amount of precession of the perihelion of Mercury for the value of
α = 1.1.10−8 AU2 .
46 ydri et al, lectures on computational physics
 Chaos: Chaotic Pendulum
6
6.1 Equation of Motion
We start from a simple pendulum. The equation of motion is given by

d2 θ
ml = −mg sin θ. (6.1)
dt2
We consider the effect of air resistance on the motion of the mass m. We will assume
that the force of air resistance is given by Stokes’ law. We get

d2 θ dθ
ml = −mg sin θ − mlq . (6.2)
dt2 dt
The air friction will drain all energy from the pendulum. In order to maintain the motion
against the damping effect of air resistance we will add a driving force. We will choose a
periodic force with amplitude mlFD and frequency ωD . This arise for example if we apply
a periodic electric field with amplitude ED and frequency ωD on the mass m which is
assumed to have an electric charge q, i.e mlFD = qED . It can also arise from the periodic
oscillations of the pendulum’s pivot point. By adding the driving force we get then the
equation of motion

d2 θ dθ
ml = −mg sin θ − mlq + mlFD cos ωD t. (6.3)
dt2 dt
The natural frequency of the oscillations is given by the frequency of the simple pendulum,
viz
r
g
ω0 = . (6.4)
l
48 ydri et al, lectures on computational physics

We will always take ω0 = 1, i.e. l = g. The equation of motion becomes

d2 θ 1 dθ
+ + sin θ = FD cos ωD t. (6.5)
dt2 Q dt
The coefficient Q = 1/q is known as the quality factor. It measures how many oscillations
the pendulum without driving force will make before its energy is drained. We will write
the above second order differential equation as two first order differential equations,
namely
dθ
=Ω
dt
dΩ 1
= − Ω − sin θ + FD cos ωD t. (6.6)
dt Q
This system of differential equations does not admit a simple analytic solution. The
linear approximation corresponds to small amplitude oscillations, viz

sin θ ≃ θ. (6.7)

The differential equations become linear given by

dθ
=Ω
dt
dΩ 1
= − Ω − θ + FD cos ωD t. (6.8)
dt Q
Or equivalently

dθ 2 1 dθ
2
=− − θ + FD cos ωD t. (6.9)
dt Q dt
For FD = 0 the solution is given by
  r
1 θ(0)
t
− 2Q 1
θt0 = θ(0) cos ω∗ t + Ω(0) + sin ω∗ t e , ω∗ = 1 − . (6.10)
ω∗ 2Q 4Q2
For FD 6= 0 a particular solution is given by

θ∞ = FD (a cos ωD t + b sin ωD t). (6.11)

We find
1 2 1 ωD
a= 2 (1 − ωD ) ,b = 2 . (6.12)
(1 − 2 )2
ωD +
ωD
(1 2 )2
− ωD +
ωD Q
Q2 Q2

For FD 6= 0 the general solution is given by

θ = θ∞ + θt . (6.13)
ydri et al, lectures computational physics 49

2)
FD (1 − ωD 2 ) 
1 FD (1 − 3ωD
   
1 θ(0) t
− 2Q
θt = θ(0) − 2 cos ω∗ t + Ω(0) + − 2 sin ω ∗ t e .
2 )2 +
(1 − ωD
ωD ω∗ 2Q 2Q (1 − ω 2 )2 + ωD
Q2 D Q2
(6.14)

The last two terms depend on the initial conditions and will vanish exponentially at very
large times t −→ ∞, i.e. they are transients. The asymptotic motion is given by θ∞ .
Thus for t −→ ∞ we get

θ = θ∞ = FD (a cos ωD t + b sin ωD t). (6.15)

Also for t −→ ∞ we get

dθ
Ω = = FD ωD (−a sin ωD t + b cos ωD t). (6.16)
dt
We compute in the limit of large times t −→ ∞

Ω2 FD2
θ2 + 2 = F̃D
2
= FD
2 2
(a + b 2
) = 2
ωD
. (6.17)
ωD 2 )2 +
(1 − ωD Q2

In other words the orbit of the system in phase space is an ellipse. The motion is periodic
with period equal to the period of the driving force. This ellipse is also called a periodic
attractor because regardless of the initial conditions the trajectory of the system will
tend at large times to this ellipse.
Let us also remark that the maximum angular displacement is F̃D . The function
F̃D = F̃D (ωD ) exhibits resonant behavior as the driving frequency approaches the natural
frequency which is equivalent to the limit ωD −→ 1. In this limit F̃D = QFD . The width
of the resonant window is proportional to 1/Q so for Q −→ ∞ we observe that F̃D −→ ∞
when ωD −→ 1 while for Q −→ 0 we observe that F̃D −→ 0 when ωD −→ 1.
In general the time-asymptotic response of any linear system to a periodic drive
is periodic with the same period as the driving force. Furthermore when the driving
frequency approaches one of the natural frequencies the response will exhibits resonant
behavior.
The basic ingredient in deriving the above results is the linearity of the dynamical
system. As we will see shortly periodic motion is not the only possible time-asymptotic
response of a dynamical system to a periodic driving force.

6.2 Numerical Algorithms

The equations of motion are
dθ
=Ω
dt
dΩ 1
= − Ω − sin θ + F (t). (6.18)
dt Q
50 ydri et al, lectures on computational physics

The external force is periodic and it will be given by one of the following expressions
F (t) = FD cos ωD t. (6.19)

F (t) = FD sin ωD t. (6.20)

6.2.1 Euler-Cromer Algorithm

Numerically we can employ the Euler-Cromer algorithm in order to solve this system of
differential equations. The solution goes as follows. First we choose the initial conditions.
For example
Ω(1) = 0
θ(1) = 0
t(1) = 0. (6.21)
For i = 1, ..., N + 1 we use
 
1
Ω(i + 1) = Ω(i) + ∆t − Ω(i) − sin θ(i) + F (i)
Q
θ(i + 1) = θ(i) + ∆t Ω(i + 1)
t(i + 1) = ∆t i. (6.22)

F (i) ≡ F (t(i)) = FD cos ωD ∆t(i − 1). (6.23)

F (i) ≡ F (t(i)) = FD sin ωD ∆t(i − 1). (6.24)

6.2.2 Runge-Kutta Algorithm

In order to achieve better precision we employ the Runge-Kutta algorithm. For i =
1, ..., N + 1 we use
k1 = ∆t Ω(i)
 
1
k3 = ∆t − Ω(i) − sin θ(i) + F (i)
Q
 
1
k2 = ∆t Ω(i) + k3
2
     
1 1 1 1
k4 = ∆t − Ω(i) + k3 − sin θ(i) + k1 + F (i + )
Q 2 2 2
(6.25)

θ(i + 1) = θ(i) + k2
Ω(i + 1) = Ω(i) + k4
t(i + 1) = ∆t i. (6.26)
ydri et al, lectures computational physics 51

F (i) ≡ F (t(i)) = FD cos ωD ∆t(i − 1). (6.27)

F (i) ≡ F (t(i)) = FD sin ωD ∆t(i − 1). (6.28)

1 1 1
F (i + ) ≡ F (t(i) + ∆t) = FD cos ωD ∆t(i − ). (6.29)
2 2 2

1 1 1
F (i + ) ≡ F (t(i) + ∆t) = FD sin ωD ∆t(i − ). (6.30)
2 2 2

6.3 Elements of Chaos

6.3.1 Butterfly Effect: Sensitivity to Initial Conditions
The solution in the linear regime (small amplitude) reads

θ = θ∞ + θt . (6.31)

The transient is of the form

θt = f (θ(0), Ω(0))e−t/2Q . (6.32)

This goes to zero at large times t. The time-asymptotic is thus given by

θ∞ = FD (a cos ωD t + b sin ωD t). (6.33)

The motion in the phase space is periodic with period equal to the period of the driving
force. The orbit in phase space is precisley an ellipse of the form

2 Ω2∞ 2 2 2
θ∞ + 2 = FD (a + b ). (6.34)
ωD

Let us consider a perturbation of the initial conditions. We can imagine that we have
two pendulums A and B with slightly different initial conditions. Then the difference
between the two trajectories is

δθ = δf (θ(0), Ω(0))e−t/2Q . (6.35)

This goes to zero at large times. If we plot ln δθ as a function of time we find a straight line
with a negative slope. The time-asymptotic motion is not sensitive to initial conditions.
It converges at large times to θ∞ no matter what the initial conditions are. The curve
θ∞ = θ∞ (Ω∞ ) is called a (periodic) attractor. This is because any perturbed trajectory
will decay exponentially in time to the attractor.
In order to see chaotic behavior we can for example increase Q keeping everything
else fixed. We observe that the slope of the line ln δθ = λt starts to decrease until at some
value of Q it becomes positive. At this value the variation between the two pendulums
52 ydri et al, lectures on computational physics

increases exponentially with time. This is the chaotic regime. The value λ = 0 is the
value where chaos happens. The coefficient λ is called Lyapunov exponent.
The chaotic pendulum is a deterministic system (since it obeys ordinary differential
equations) but it is not predictable in the sense that given two identical pendulums their
motions will diverge from each other in the chaotic regime if there is the slightest error in
determining their initial conditions. This high sensitivity to initial conditions is known
as the butterfly effect and could be taken as the definition of chaos itself.
However we should stress here that the motion of the chaotic pendulum is not random.
This can be seen by inspecting Poincare sections.

6.3.2 Poincare Section and Attractors

The periodic motion of the linear system with period equal to the period of the driving
force is called a period-1 motion. In this motion the trajectory repeats itself exactly
every one single period of the external driving force. This is the only possible motion in
the low amplitude limit.
Generally a period-N motion corresponds to an orbit of the dynamical system which
repeats itself every N periods of the external driving force. These orbits exist in the
non-linear regime of the pendulum.
The Poincare section is defined as follows. We plot in the θ-Ω phase space only one
point per period of the external driving force. We plot for example (θ, Ω) for

ωD t = φ + 2πn. (6.36)

The angle φ is called the Poincare phase and n is an integer. For period-1 motion the
Poincare section consists of one single point. For period-N motion the Poincare section
consists of N points.
Thus in the linear regime if we plot (θ, Ω) for ωD t = 2πn we get a single point since
the motion is periodic with period equal to that of the driving force. The single point we
get as a Poincare section is also an attractor since all pendulums with almost the same
initial conditions will converge onto it.
In the chaotic regime the Poincare section is an attractor known as strange attractor.
It is a complicated curve which could have fractal structure and all pendulums with
almost the same initial conditions will converge onto it.

6.3.3 Period-Doubling Bifurcations

In the case of the chaotic pendulum we encounter between the linear regime and the
emergence of chaos the so-called period doubling phenomena. In the linear regime the
Poincare section is a point P which corresponds to a period-1 motion with period equal
TD = 2π/ωD . The θ or Ω coordinate of this point P will trace a line as we increase
Q while keeping everything fixed. We will eventually reach a value Q1 of Q where this
line bifurcates into two lines. By close inspection we see that at Q1 the motion becomes
period-2 motion, i.e. the period becomes equal to 2TD .
ydri et al, lectures computational physics 53

In a motion where the period is TD (below Q1 ) we get the same value of θ each time
t = mTD and since we are plotting θ each time t = 2nπ/ωD = nTD we will get a single
point in the Poincare section. In a motion where the period is 2TD (at Q2 ) we get the
same value of θ each time t = 2mTD , i.e. the value of θ at times t = mTD is different
and hence we get two points in the Poincare section.
As we increase Q the motion becomes periodic with period equal 4TD , then with
period equal 8TD and so on. The motion with period 2N TD is called period-N motion.
The corresponding Poincare section consists of N distinct points.
The diagram of θ as a function of Q is called a bifurcation diagram. It has a fractal
structure. Let us point out here that normally in ordinary oscillations we get harmonics
with periods equal to the period of the driving force divided by 2N . In this case we
obtained in some sense subharmonics with periods equal to the period of the driving
force times 2N . This is very characteristic of chaos. In fact chaotic behavior corresponds
to the limit N −→ ∞. In other words chaos is period-∞ (bounded) motion which could
be taken as another definition of chaos.

6.3.4 Feigenbaum Ratio

Let QN be the critical value of Q above which the N th bifurcation is triggered. In other
words QN is the value where the transition to period-N motion happens. We define the
Feigenbaum ratio by

QN −1 − QN −2
FN = . (6.37)
QN − QN −1

It is shown that FN −→ F = 4.669 as N −→ ∞. This is a universal ratio called the

Feigenbaum ratio and it characterizes many chaotic systems which suffer a transition to
chaos via an infinite series of period-doubling bifurcations. The above equation can be
then rewritten as
N −2
X 1
QN = Q1 + (Q2 − Q1 ) (6.38)
Fj
j=0

Let us define the accumulation point by Q∞ then

F
Q∞ = Q1 + (Q2 − Q1 ) (6.39)
F −1
This is where chaos occur. In the bifurcation diagram the chaotic region is a solid black
region.

6.3.5 Spontaneous Symmetry Breaking

The bifurcation process is associated with a deep phenomenon known as spontaneous
symmetry breaking. The first period-doubling bifurcation corresponds to the breaking of
54 ydri et al, lectures on computational physics

the symmetry t −→ t + TD . The linear regime respects this symmetry. However period-2
motion and in general period-N motions with N > 2 do not respect this symmetry.
There is another kind of spontaneous symmetry breaking which occurs in the chaotic
pendulum and which is associated with a bifurcation diagram. This happens in the region
of period-1 motion and it is the breaking of spatial symmetry or parity θ −→ −θ. Indeed
there exists solutions of the equations of motion that are either left-favoring or right-
favoring. In other words the pendulums in such solutions spend much of its time in the
regions to the left of the pendulum’s vertical (θ < 0) or to the right of the pendulum’s
vertical (θ > 0). This breaking of left-right symmetry can be achieved by a gradual
increase of Q. We will then reach either the left-favoring solution or the right-favoring
solution starting from a left-right symmetric solution depending on the initial conditions.
The symmetry θ −→ −θ is also spontaneously broken in period-N motions.

6.4 Lab Problem 8: The Butterfly Effect

We consider a pendulum of a mass m and a length l moving under the influence of the
force of gravity, the force of air resistance and a driving periodic force. Newton’s second
law of motion reads
d2 θ g dθ
2
= − sin θ − q + FD sin 2πνD t.
dt l dt
p
We will always take the angular frequency g/l associated with simple oscillations of
the pendulum equal 1, i.e. l = g. The numerical solution we will consider here is based
on the Euler-Cromer algorithm.
The most important property of a large class of solutions of this differential equation
is hyper sensitivity to initial conditions known also as the butterfly effect which is the
defining characteristic of chaos. For this reason the driven non-linear pendulum is also
known as the chaotic pendulum.
The chaotic pendulum can have two distinct behaviors. In the linear regime the
motion (neglecting the initial transients) is periodic with a period equal to the period of
the external driving force. In the chaotic regime the motion never repeats and any error
even infinitesimal in determining the initial conditions will lead to a completely different
orbit in the phase space.

(1) Write a code which implements the Euler-Cromer algorithm for the chaotic pendu-
lum. The angle θ must always be taken between −π and π which can be maintained
as follows
if(θi .lt. ∓ π) θi = θi ± 2π.

(2) We take the values and initial conditions

2 1
dt = 0.04s , 2πνD = s−1 , q = s−1 , N = 1000 − 2000.
3 2
θ1 = 0.2 radian , Ω1 = 0 radian/s.
ydri et al, lectures computational physics 55

FD = 0 radian/s2 , FD = 0.1 radian/s2 , FD = 1.2 radian/s2 .

Plot θ as a function of time. What do you observe for the first value of FD . What
is the period of oscillation for small and large times for the second value of FD . Is
the motion periodic for the third value of FD .

6.5 Lab Problem 9: Poincaré Sections

In the chaotic regime the motion of the pendulum although deterministic is not pre-
dictable. This however does not mean that the motion of the pendulum is random which
can clearly be seen from Poincare sections.
A Poincare section is a curve in the phase space obtained by plotting one point of the
orbit per period of the external drive. Explicitly we plot points (θ, Ω) which corresponds
to times t = n/νD where n is an integer. In the linear regime of the pendulum Poincare
section consists of a single point. Poincare section in the chaotic regime is a curve which
does not depend on the initial conditions thus confirming that the motion is not random
and which may have a fractal structure. As a consequence this curve is called a strange
attractor.

(1) We consider two identical chaotic pendulums A and B with slightly different initial
conditions. For example we take

θ1A = 0.2 radian , θ1B = 0.201 radian.

The difference between the two motions can be measured by

∆θi = θiA − θiB .

Compute ln ∆θ as a function of time for

FD = 0.1 radian/s2 , FD = 1.2 radian/s2 .

What do you observe. Is the two motions identical. What happens for large times.
Is the motion of the pendulum predictable. For the second value of FD use

N = 10000 , dt = 0.01s.

(2) Compute the angular velocity Ω as a function of θ for

FD = 0.5 radian/s2 , FD = 1.2 radian/s2 .

What is the orbit in the phase space for small times and what does it represent.
What is the orbit for large times. Compare between the two pendulums A and B.
Does the orbit for large times depend on the initial conditions.
56 ydri et al, lectures on computational physics

(3) A Poincare section is obtained numerically by plotting the points (θ, Ω) of the orbit
at the times at which the function sin πνD t vanishes. These are the times at which
this function changes sign. This is implemented as follows

if(sin πνD ti sin πνD ti+1 .lt.0)then

write(∗, ∗)ti , θi , Ωi .

Verify that Poincare section in the linear regime is given by a single point in the
phase space. Take and use FD = 0.5 radian/s2 , N = 104 − 107 , dt = 0.001s.
Verify that Poincare section in the chaotic regime is also an attractor. Take and use
FD = 1.2 radian/s2 , N = 105 , dt = 0.04s. Compare between Poincare sections
of the pendulums A and B. What do you observe and what do you conclude.

6.6 Lab Problem 10: Period Doubling

Among the most important chaotic properties of the driven non-linear pendulum is the
phenomena of period doubling. The periodic orbit with period equal to the period of
the external driving force are called period-1 motion. There exist however other periodic
orbits with periods equal twice, four times and in general 2N times the period of the
external driving force. The orbit with period equal 2N times the period of the external
driving force is called period-N motion. The period doubling observed in the driven
non-linear pendulum is a new phenomena which belongs to the world of chaos. In the
standard phenomena of mixing the response of a non-linear system to a single frequency
external driving force will contain components with periods equal to the period of the
driving force divided by 2N . In other words we get "harmonics" as opposed to the
"subharmonics" we observe in the chaotic pendulum.
For period-N motion we expect that there are N different values of the angle θ for
every value of FD . The function θ = θ(FD ) is called a bifurcation diagram. Formally the
transition to chaos occurs at N −→ ∞. In other words chaos is defined as period-infinity
motion.

(1) We take the values and initial conditions

2 1
l = g , 2πνD = s−1 , q = s−1 , N = 3000 − 100000 , dt = 0.01s.
3 2
θ1 = 0.2 radian , Ω1 = 0 radian/s.
Determine the period of the motion for

FD = 1.35 radian/s2 , FD = 1.44 radian/s2 , FD = 1.465 radian/s2 .

What happens to the period when we increase FD . Does the two second values of
FD lie in the linear or chaotic regime of the chaotic pendulum.
ydri et al, lectures computational physics 57

(2) Compute the angle θ as a function of FD for the times t which satisfy the condition
2πνD t = 2nπ. We take FD in the interval

FD = (1.34 + 0.005k) radian/s2 , k = 1, ..., 30.

Determine the interval of the external driving force in which the orbits are period-1,
period-2 and period-4 motions.
In this problem it is very important to remove the initial transients before we start
measuring the bifurcation diagram. This can be done as follows. We calculate the
motion for 2N steps but then only consider the last N steps in the computation of
the Poincare section for every value of FD .

6.7 Lab Problem 11: Bifurcation Diagrams

Part I The chaotic pendulum is given by the equation

d2 θ 1 dθ
= − sin θ − + FD cos 2πνD t.
dt2 Q dt

In this simulation we take the values FD = 1.5 radian/s2 and 2πνD = 23 s−1 . In order to
achieve a better numerical precision we use the second-order Runge-Kutta algorithm.
In the linear regime the orbits are periodic with period equal to the period TD of the
external driving force and are symmetric under θ −→ −θ. There exists other solutions
which are periodic with period equal TD but are not symmetric under θ −→ −θ. In these
solutions the pendulum spends the majority of its time in the region to the left of its
vertical (θ < 0) or in the region to the right of its vertical (θ > 0).
These symmetry breaking solutions can be described by a bifurcation diagram Ω =
Ω(Q). For every value of the quality factor Q we calculate the Poincare section. We
observe that the Poincare section will bifurcate at some value Q∗ of Q. Below this value
we get one line whereas above this value we get two lines corresponding to the two
symmetry breaking solutions in which the pendulum spends the majority of its time in
the regions (θ > 0) and (θ < 0).

(1) Rewrite the code for the chaotic pendulum using Runge-Kutta algorithm.

(2) We take two different sets of initial conditions

θ = 0.0 radian , Ω = 0.0 radian/s.

θ = 0.0 radian , Ω = −3.0 radian/s .

Study the nature of the orbit for the values Q = 0.5s, Q = 1.24s and Q = 1.3s.
What do you observe.

(3) Plot the bifurcation diagram Ω = Ω(Q) for values of Q in the interval [1.2, 1.3].
What is the value Q∗ at which the symmetry θ −→ −θ is spontaneously broken.
58 ydri et al, lectures on computational physics

Part II As we have seen in the previous simulation period doubling can also be de-
scribed by a bifurcation diagram. This phenomena is also an example of a spontaneous
symmetry breaking. In this case the symmetry is t −→ t + TD . Clearly only orbits with
period TD are symmetric under this transformation.
Let QN be the value of Q at which the N th bifurcation occurs. In other words this
is the value at which the orbit goes from being a period-(N − 1) motion to a period-N
motion. The Feigenbaum ratio is defined by

QN −1 − QN −2
FN = .
QN − QN −1
As we approach the chaotic regime, i.e. as N −→ ∞ the ratio FN converges rapidly to the
constant value F = 4.669. This is a general result which holds for many chaotic systems.
Any dynamical system which can exhibit a transition to chaos via an infinite series of
period-doubling bifurcations is characterized by a Feigenbaum ratio which approaches
4.669 as N −→ ∞.

(1) Calculate the orbit and Poincare section for Q = 1.36s. What is the period of the
motion. Is the orbit symmetric under t −→ t + TD . Is the orbit symmetric under
θ −→ −θ.

(2) Plot the bifurcation diagram Ω = Ω(Q) for two different sets of initial conditions
for values of Q in the interval [1.3, 1.36]. What is the value Q at which the period
gets doubled. What is the value of Q at which the symmetry t −→ t + TD is
spontaneously broken.

(3) In this question we use the initial conditions

θ = 0.0 radian , Ω = 0.0 radian/s.

Calculate the orbit and Poincare section and plot the bifurcation diagram Ω = Ω(Q)
for values of Q in the interval [1.34, 1.38]. Determine from the bifurcation diagram
the values QN for N = 1, 2, 3, 4, 5. Calculate the Feigenbaum ratio. Calculate the
accumulation point Q∞ at which the transition to chaos occurs.
 Molecular Dynamics
7
7.1 Introduction
In the molecular dynamics approach we attempt to understand the behavior of a classical
many-particle system by simulating the trajectory of each particle in the system. In
practice this can be applied to systems containing 109 particles at most. The molecular
dynamics approach is complementary to the more powerful Monte Carlo method. The
Monte Carlo method deals with systems that are in thermal equilibrium with a heat
bath. The molecular dynamics approach on the other hand is useful in studying how fast
in real time a system moves from one microscopic state to another.
We consider a box containing a collection of atoms or molecules. We will use Newton’s
second law to calculate the positions and velocities of all the molecules as functions of
time. Some of the questions we can answer with the molecular dynamics approach are:
• The melting transition.
• The rate of equilibration.
• The rate of diffusion.
As state above molecular dynamics allows us to understand classical systems. A classical
treatment can be justified as follows. We consider the case of liquid argon as an example.
The energy required to excite an argon atom is of the order of 10eV while the typical
kinetic energy of the center of mass of an argon atom is 0.1eV. Thus a collision between
two argon atoms will not change the electron configuration of either atoms. Hence for
all practical purposes we can ignore the internal structure of argon atoms. Furthermore
the wavelength of an argon atom which is of the order of 10−7 A is much smaller than the
spacing between argon atoms typically of the order of 1A which again justifies a classical
treatment.
60 ydri et al, lectures on computational physics

7.2 The Lennard-Jones Potential

We consider a box containing N argon atoms. For simplicity we will assume that our
argon atoms move in two dimensions. The equations of motion of the ith atom which is
located at the position (xi , yi ) with velocity (vi,x , vi,y ) read

dvi,x dxi
= ax,i , = vi,x . (7.1)
dt dt

dvi,y dyi
= ay,i , = vi,y . (7.2)
dt dt
Each argon atom experience a force from all other argon atoms. In order to calculate
this force we need to determine the interaction potential. We assume that the interaction
potential between any pair of argon atoms depend only on the distance between them.
Let rij and u(rij ) be the distance and the interaction potential between atoms i and j.
The total potential is then given by

N
X −1 N
X
U= u(rij ). (7.3)
i=1 j=i+1

The precise form of u can be calculated from first principles, i.e. from quantum me-
chanics. However this calculation is very complicated and in most circumstances a phe-
nomenological form of u will be sufficient.
For large separations rij the potential u(rij ) must be weakly attractive given by
the Van der Walls force which arises from electrostatic interaction between the electric
dipole moments of the two argon atoms. In other words u(rij ) for large rij is attractive
due to the mutual polarization of the two atoms. The Van der Walls potential can be
computed from quantum mechanics where it is shown that it varies as 1/rij 6 . For small

separations rij the potential u(rij ) must become strongly repulsive due to the overlap of
the electron clouds of the two argon atoms. This repulsion known also as core repulsion
is a consequence of Pauli exclusion principle. It is a common practice to choose the
12 . The total potential takes
repulsive part of the potential u to be proportional to 1/rij
the form
 12  6 
σ σ
u(r) = 4ǫ − . (7.4)
r r

This is the Lennard-Jones potential. The parameter σ is of dimension length while ǫ is

of dimension energy. We observe that at r = σ the potential is 0 identically while for
r > 2.5σ the potential approaches zero rapidly. The minimum of the potential occurs at
r = 21/6 σ. The depth of the potential at the minimum is ǫ.
The force of atom k on atom i is
ydri et al, lectures computational physics 61

  12  6 
~ k,i u(rk,i ) = 24ǫ 2 σ
f~k,i = −∇ −
σ
r̂ki . (7.5)
rki rki rki

The acceleration of the ith atom is given by

1 X 1 X xi − xk
ax,i = fk,i cos θk,i = fk,i
m m rki
k6=i k6=i
  12  6 
24ǫ X xi − xk σ σ
= 2 2 − . (7.6)
m rki rki rki
k6=i

1 X 1 X yi − yk
ay,i = fk,i sin θk,i = fk,i
m m rki
k6=i k6=i
  12  6 
24ǫ X yi − yk σ σ
= 2 2 − . (7.7)
m rki rki rki
k6=i

7.3 Units, Boundary Conditions and Verlet Algorithm

Reduced Units We choose σ and ǫ as the units of distance and energy respectively.
We also choose the unit of mass to be the mass m of a single argon atom. Everything
else is measured in terms of σ, ǫ and m. For example velocity is measured in units of
(ǫ/m)1/2 and time in units of σ(ǫ/m)1/2 . The reduced units are given by

σ = ǫ = m = 1. (7.8)

For argon atoms we have the values

σ = 3.4 × 10−10 m , ǫ = 1.65 × 10−21 J = 120kB J , m = 6.69 × 10−26 kg. (7.9)

Thus
r
m
σ = 2.17 × 10−12 s. (7.10)
ǫ
Hence a molecular dynamics simulation which runs for 2000 steps with a reduced time
step ∆t = 0.01 corresponds to a total reduced time 2000 × 0.01 = 20 which is equivalent
to a real time 20σ(ǫ/m)1/2 = 4.34 × 10−11 s.

Periodic Boundary Conditions The total number of atoms in a real physical system
is huge of the order of 1023 . If the system is placed in a box the fraction of atoms of the
system near the walls of the box is negligible compared to the total number of atoms.
In typical simulations the total number of atoms is only of the order of 103 − 105 and in
62 ydri et al, lectures on computational physics

this case the fraction of atoms near the walls is considerable and their effect can not be
neglected.
In order to reduce edge effects we use periodic boundary conditions. In other words
the box is effectively a torus and there are no edges. Let Lx and Ly be the lengths of the
box in the x and y directions respectively. If an atom crosses the walls of the box in a
particular direction we add or subtract the length of the box in that direction as follows

if (x > Lx ) then x = x − Lx
if (x < 0) then x = x + Lx . (7.11)

if (y > Ly ) then y = y − Ly
if (y < 0) then y = y + Ly . (7.12)

The maximum separation in the x direction between any two particles is only Lx /2
whereas the maximum separation in the y direction between any two particles is only
Ly /2. This can be implemented as follows

if (xij > +Lx /2) then xij = xij − Lx

if (xij < −Lx /2) then xij = xij + Lx . (7.13)

if (yij > +Ly /2) then yij = yij − Ly

if (yij < −Ly /2) then yij = yij + Ly . (7.14)

Verlet Algorithm The numerical algorithm we will use is Verlet algorithm. Let us
consider the forward and backward Taylor expansions of a function f given by

df 1 d2 f 1 d3 f
f (tn + ∆t) = f (tn ) + ∆t |tn + (∆t)2 2 |tn + (∆t)3 3 |tn + ... (7.15)
dt 2 dt 6 dt

df 1 d2 f 1 d3 f
f (tn − ∆t) = f (tn ) − ∆t |tn + (∆t)2 2 |tn − (∆t)3 3 |tn + ... (7.16)
dt 2 dt 6 dt
Adding these expressions we get

d2 f
f (tn + ∆t) = 2f (tn ) − f (tn − ∆t) + (∆t)2 |t + O(∆t4 ). (7.17)
dt2 n
We remark that the error is proportional to ∆t4 which is less than the errors in the Euler,
Euler-Cromer and second-order Runge-Kutta methods so this method is more accurate.
We have therefore for the ith atom

xi,n+1 = 2xi,n − xi,n−1 + (∆t)2 ax,i,n . (7.18)

ydri et al, lectures computational physics 63

yi,n+1 = 2yi,n − yi,n−1 + (∆t)2 ay,i,n . (7.19)

The force and the acceleration are given by

  12  6 
24ǫ σ σ
fk,i,n = 2 − . (7.20)
rki,n rki,n rki,n

1 X xi,n − xk,n
ax,i,n = fk,i,n . (7.21)
m rki,n
k6=i

1 X yi,n − yk,n
ay,i,n = fk,i,n . (7.22)
m rki,n
k6=i

The separation rki,n between the two atoms k and i is given by

q
rki,n = (xi,n − xk,n )2 + (yi,n − yk,n ). (7.23)

In the Verlet method it is not necessary to calculate the components dxi,n /dt and dyi,n /dt
of the velocity. However since the velocity will be needed for other purposes we will also
compute it using the equations
xi,n+1 − xi,n−1
vx,i,n = . (7.24)
2∆t

yi,n+1 − yi,n−1
vy,i,n = . (7.25)
2∆t
Let us remark that the Verlet method is not self starting. In other words given the initial
conditions xi,1 , yi,1 , vx,i,1 and vy,i,1 we need also to know xi,2 , yi,2 , vx,i,2 and vy,i,2 for the
algorithm to start which can be determined using the Euler method.

7.4 Some Physical Applications

7.4.1 Dilute Gas and Maxwell Distribution
A gas in thermal equilibrium is characterized by a temperature T . Molecular dynamics
allows us to study how a dilute gas approaches equilibrium. The temperature of the gas
can be computed using the molecular dynamics simulations as follows. According to the
equipartition theorem the average thermal energy of each quadratic degree of freedom in
a gas in thermal equilibrium is equal kB T /2. In other words

1 1 1
kB T = < m~v 2 > . (7.26)
2 d 2
64 ydri et al, lectures on computational physics

The average <> can be understood in two different but equivalent ways. We can follow
the motion of a single atom and take the time average of its kinetic energy. The same
result can be obtained by taking the average of the kinetic energy over the different
atoms. In this latter case we write
N
1 1 X1
kB T = m~v 2 . (7.27)
2 dN 2 i
i=1

Another way of measuring the temperature T of a dilute gas is through a study of the
distribution of atom velocities. A classical gas in thermal equilibrium obeys Maxwell dis-
tribution. The speed and velocity distributions in two dimensions are given respectively
by
2
v mv
− 2k
P (v) = C e B T
. (7.28)
kB T

2 mvy 2
1 −
mvx
1 −
P (vx ) = Cx √ e 2kB T , P (vy ) = Cy √ e 2kB T . (7.29)
kB T kB T

Recall that the probability per unit v of finding an atom with speed v is equal P (v)
whereas the probability per unit vx,y of finding an atom with velocity vx,y is equal P (vx,y ).
The constants C and Cx,y are determined from the normalization conditions. There are
peaks in the distributions P (v) and P (vx,y ). Clearly the temperature is related to the
location of the peak which occurs in P (v). This is given by

2
kB T = mvpeak . (7.30)

7.4.2 The Melting Transition

This is a very important subject which we will discuss at great length in the second lab
problem of this chapter.

7.5 Lab Problem 12: Maxwell Distribution

We consider the motion in two dimensions of N argon atoms in an L × L box. The
interaction potential u between any two atoms in the gas separated by a distance r is
given by the Lennard-Jones potential. The numerical algorithm we will use is Verlet
algorithm.
In this problem we will always take L odd and N a perfect square. The lattice spacing
is defined by

L
a= √ .
N
ydri et al, lectures computational physics 65

Clearly there are N cells of area a × a. We choose L and N such that a > 2σ. For
simplicity we will use reduced units σ = ǫ = m = 1. In order to reduce edge effects we
use periodic boundary conditions. In other words the box is effectively a torus and there
are no edges. Thus the maximum separation in the x direction between any two particles
is only L/2 and similarly the maximum separation in the y direction between any two
particles is only L/2. √
The initial positions of the atoms are fixed as follows. The atom k = N (i−1)+j will
be placed at the center of the cell with corners (i, j), (i + 1, j), (i, j + 1) and (i + 1, j + 1).
Next we perturb in a random way these initial positions by adding random numbers in
the interval [−a/4, +a/4] to the x and y coordinates of the atoms. The initial velocities
can be chosen in random directions with a speed equal v0 for all atoms.

(1) Write a molecular dynamics code along the above lines. Take L = 15, N = 25,
∆t = 0.02, Time = 500 and v0 = 1. As a first test verify that the total energy is
conserved. Plot the trajectories of the atoms. What do you observe.

(2) As a second test we propose to measure the temperature by observing how the gas
approaches equilibrium. Use the equipartition theorem
N
m X 2 2
kB T = (vi,x + vi,y ).
2N
i=1

Plot T as a function of time. Take Time = 1000 − 1500. What is the temperature
of the gas at equilibrium.

(3) Compute the speed distribution of the argon atoms by constructing an appropriate
histogram as follows. We take the value Time = 2000. We consider the speeds of
all particles at all times. There are Time × N values of the speed in this sample.
Construct the histogram for this sample by 1) finding the maximum and minimum,
2) dividing the interval into bins, 3) determining the number of times a given value
of the speed falls in a bin and (4) properly normalizing the distribution. Compare
with the Mawell distribution

v 2 − 2kmv2T
PMaxwell (v) = C e B .
kB T
2
Deduce the temperature from the peak of the distribution given by kB T = mvpeak .
Compare with the value of the temperature obtained from the equipartition theo-
rem. What happens if we increase the initial speed.

7.6 Lab Problem 13: Melting Transition

We would like to study the melting transition. First we need to establish the correct
conditions for a solid phase. Clearly the temperature must be sufficiently low and the
density must be sufficiently high. To make the temperature as low as possible we will
66 ydri et al, lectures on computational physics

start with all particles at rest. In order to obatin maximum attraction between atoms we
choose a low density of approximately one particle per unit reduced area. In particular
we choose N = 16 and L = 4.

(1) Show that with these conditions you obtain a crystalline solid with a triangular
lattice.

(2) In order to observe melting we must heat up the system. This can be achieved by
increasing the kinetic energy of the atoms by hand. A convenient way of doing this
is to rescale the current and previous positions of the atoms periodically (say every
1000 steps) as follows

hh = int(n/1000)
if (hh ∗ 1000.eq.n) then
x(i, n) = x(i, n + 1) − R(x(i, n + 1) − x(i, n))
y(i, n) = y(i, n + 1) − R(y(i, n + 1) − y(i, n))
endif.

This procedure will rescale the velocity by the amount R. We choose R = 1.5.
Verify that we will indeed reach the melting transition by means of this method.
What happens to the energy and the temperature.
 Pseudo Random Numbers and Random Walks
8
8.1 Random Numbers
A sequence of numbers r1 , r2 ,... is called random if there are no correlations between
the numbers. The sequence is called uniform if all numbers have an equal probability
to occur. More precisely let the probability that a number ri in the sequence occurs
between r and r + dr be P (r)dr where P (r) is the probability distribution. A uniform
distribution corresponds P (r) = constant.
Most random number generators on computers generate uniform distributions be-
tween 0 and 1. These are sequences of pseudo random numbers since given ri and its
preceding elements we can compute ri+1 . Therefore these sequences are not really ran-
dom and correlations among the numbers of the sequence exist. True random numbers
can be found in tables of random numbers determined during say radioactive decay or
other naturally occurring random physical phenomena.

8.1.1 Linear Congruent or Power Residue Method

In this method we generate a set of k random numbers r1 ,r2 ,...,rk in the interval [0, M −1]
as follows. Given a random number ri−1 we generate the next random number ri by the
rule
 
ari−1 + c
ri = (ari−1 + c) mod M = remainder . (8.1)
M
The notation y = z mod M means that we subtract M from z until 0≤y≤M − 1. The
first random number r1 is supplied by the user and it is called the seed. Also supplied
are the multiplier a, the increment c and the modulus M . The remainder is a built-in
function in most computer languages. The largest possible integer number generated by
68 ydri et al, lectures on computational physics

the above rule is M − 1. Thus the maximum possible period is M , i.e k≤M . In general
the period k depends on a, c and M . To get a uniform sequence in the interval [0, 1] we
divide by M − 1.
Let us take the following example a = 4,c = 1 and M = 9 with seed r1 = 3. We get
a sequence of length 9 given by

3, 4, 8, 6, 7, 2, 0, 1, 5. (8.2)

After the last number 5 we get 3 and therefore the sequence will repeat. In this case the
period is M = 9.
It is clear that we need to choose the parameters a, c and M and the seed r1 with
care so that we get the longest sequence of pseudo random numbers. The maximum
possible period depends on the size of the computer word. A 32−bit machine may use
M = 231 = 2 × 109 . The numbers generated by (8.1) are random only in the sense that
they are evenly distributed over their range. Equation (8.1) is related to the logistic map
which is known to exhibit chaotic behaviour. Although chaos is deterministic it looks
random. In the same way although equation (8.1) is deterministic the numbers generated
by it look random. This is the reason why they are called pseudo random numbers.

8.1.2 Statistical Tests of Randomness

Period : The first obvious test is to verify that the random number generator has a
sufficiently long period for a given problem. We can use the random number generator to
plot the position of a random walker. Clearly the plot will repeat itself when the period
is reached.

Uniformity : The kth moment of the random number distribution is

N
1 X k
< xki >= xi . (8.3)
N
i=1

Let P (x) be the probability distribution of the random numbers. Then

1
1
Z
< xki >= dx xk P (x) + O( √ ). (8.4)
0 N
For a uniform distribution P (x) = 1 we must have

1 1
< xki >= + O( √ ). (8.5)
k+1 N
In the words

√
 N 
1 X k 1
N xi − = O(1). (8.6)
N k+1
i=1
ydri et al, lectures computational physics 69

This is a test of uniformity as well as of randomness. To be more precise if < xki >
is equal to 1/(k + 1) √
then we can infer that the distribution is uniform whereas if the
deviation varies as 1/ N then we can infer that the distribution is random.
A direct test of uniformity is to divide the unit interval into K equal subintevals
(bins) and place each random number in one of these bins. For a uniform distribution
we must obtain N/K numbers in each bin where N is the number of generated random
numbers.

Chi-Square Statistic : In the above test there will be statistical fluctuations about the
ideal value N/K for each bin. The question is whether or not these fluctuations are
consistent with the laws of statistics. The answer is based on the so-called chi-square
statistic defined by
K
X (Ni − nideal )2
χ2m = . (8.7)
nideal
i=1

In the above definition Ni is the number of random numbers which fall into bin i and
nideal is the expected number of random numbers in each bin.
The probability of finding any particular value χ2 which is less than χ2m is found to
be proportional to the incomplete gamma function γ(ν/2, χ2m /2) where ν is the number
of degrees of freedom given by ν = K − 1. We have
γ(ν/2, χ2m /2)
P (χ2 ≤ χ2m ) = ≡ P (ν/2, χ2m /2). (8.8)
Γ(ν/2)
The most likely value of χ2m , for some fixed number of degrees of freedom ν, corresponds
to the value P (ν/2, χ2m /2) = 0.5. In other words in half of the measurements (bin tests),
for some fixed number of degrees of freedom ν, the chi-square statistic predicts that we
must find a value of χ2m smaller than the maximum.

Randomness : Let r1 , r2 ,...,rN be a sequence of random numbers. A very effective test

of randomness is to make a scatterplot of (xi = r2i , yi = r2i+1 ) for many i. There must
be no regularity in the plot otherwise the sequence is not random.

Short-Term Correlations : Let us define the autocorrelation function

< xi xi+j > − < xi >< xi+j >
C(j) =
< xi xi > − < xi > 2
< xi xi+j > − < xi >2
= , j = 1, 2, ... (8.9)
< xi xi > − < xi > 2
In the above equation we have used the fact that < xi+j >=< xi > for a large sample,
i.e. the choice of the origin of the sequence is irrelevant in that case and
N −j
1 X
< xi xi+j >= xi xi+j . (8.10)
N −j
i=1
70 ydri et al, lectures on computational physics

Again if xi and xi+j are independent random numbers which are distributed with the
joint probability distribution P (xi , xi+j ) then
Z 1 Z 1
< xi xi+j >≃ dx dyxyP (x, y). (8.11)
0 0

We have clearly assumed that N is large. For a uniform distribution, viz P (x, y) = 1 we
get
1
< xi xi+j >≃ . (8.12)
4
√
For a random distrubution the deviation from this result is of order 1/ N . Hence in the
case that the random numbers are not correlated we have

C(j) = 0. (8.13)

8.2 Random Systems

Both quantum and statistical physics deal with systems that are random or stochastic.
These are non deterministic systems as opposed to classical systems. The dynamics of
a deterministic system is given by a unique solution to the equations of motion which
describes the physics of the system at all times.
We take the case of the diffusion of fluid molecules. For example the motion of dust
particles in the atmosphere, the motion of perfume molecules in the air or the motion of
milk molecules in a coffee. These are all cases of a Brownian motion.
In the case of a drop of milk in a coffee the white mass of the drop of milk will
slowly spread until the coffee takes on a uniform brown color. At the molecular level
each milk molecule collides with molecules in the coffee. Clearly it will change direction
so frequently that its motion will appear random. This trajectory can be described by a
random walk. This is a system in which each milk molecule moves one step at a time in
any direction with equal probability.
The trajectory of a dust, perfume or milk molecule is not really random since it can
in principle be computed by solving Newton’s equations of motion for all molecules which
then allows us to know the evolution of the system in time. Although this is possible
in principle it will not be feasible in practice. The random walk is thus effectively an
approximation. However the large number of molecules and collisions in the system
makes the random walk a very good approximation.

8.2.1 Random Walks

Let us consider a one dimensional random walk. It can take steps of lenght unity along
a line. It begins at s0 = 0 and the first step is chosen randomly to be either to the left
or to right with equal probabilities. In other words there is a 50 per cent chance that
the walker moves to the point s1 = +1 and a 50 per cent chance that it moves to the
ydri et al, lectures computational physics 71

point s1 = −1. Next the walker will again move either to the right or to the left from
the point s1 to the point s2 with equal probabilities. This process will be repeated N
times and we get the position of the walker xN as a function of the step number N . In
the motion of a molecule in a solution the time between steps is a constant and hence
the step number N is proportional to time. Therefore xN is the position of the walker
as a function of time.
In general a one-dimensional random walker can move to the right with probability p
and to the left with probability q = 1 − p with steps of equal lenght a. The direction of
each step is independent of the previous one. The displacement or position of the walker
after N steps is

N
X
xN = si . (8.14)
i=1

The walker for p = q = 1/2 can be generated by flipping a coin N times. The position
is increased by a for heads and decreased by a for tails.
Averaging over many walks each consisting of N steps we get

N
X
< xN >= < si >= N < s > . (8.15)
i=1

In above we have used the fact that the average over every step is the same given by

< si >=< s >= p(a) + q(−a) = (p − q)a. (8.16)

For p = q = 1/2 we get < xN >= 0. A better measure of the walk is given by

N
X 2
x2N = si . (8.17)
i=1

The mean square net displacement ∆x2 is defined by

∆x2 =< (xN − < xN >)2 >=< x2N > − < xN >2 . (8.18)

We compute

N X
X N
∆x2 = < (si − < s >)(sj − < s >) >
i=1 j=1
N
X N
X
= < (si − < s >)(sj − < s >) > + < (si − < s >)2 > . (8.19)
i6=j=1 i=1
72 ydri et al, lectures on computational physics

In the first term since i 6= j we have < (si − < s >)(sj − < s >) >=< (si − < s >) ><
(sj − < s >) >. But < (si − < s >) >= 0. Thus

N
X
∆x2 = < (si − < s >)2 >
i=1
= N (< s2i > − < s >2 >)
= N (a2 − (p − q)2 a2 )
= 4N pqa2 . (8.20)

For p = q = 1/2 and a = 1 we get

< x2N > = N. (8.21)

The main point is that since N is proportional to time we have < x2N >∝ t. This is an
example of a diffusive behaviour.

8.2.2 Diffusion Equation

The random walk is successful in simulating many physical systems because it is related
to the solutions of the diffusion equation. To see this we start from the probability
P (i, N ) that the random walker is at site si after N steps. This is given by
 
1
P (i, N ) = P (i + 1, N − 1) + P (i − 1, N − 1) . (8.22)
2

Let τ be the time between steps and a the lattice spacing. Then t = N τ and x = ia.
Also we define P (x, t) = P (i, N )/a. We get
 
1
P (x, t) = P (x + a, t − τ ) + P (x − a, t − τ ) . (8.23)
2

Let us rewrite this equation as

a2
   
1 1
P (x, t) − P (x, t − τ ) = P (x + a, t − τ ) − 2P (x, t − τ ) + P (x − a, t − τ ) 2 .
τ 2τ a
(8.24)

In the limit a −→ 0, τ −→ 0 with the ratio D = a2 /2τ kept fixed we obtain the equation

∂P (x, t) ∂ 2 P (x, t)
=D . (8.25)
∂t ∂x2
This is the diffusion equation. Generalization to 3−dimensions is

∂P (x, y, z, t)
= D∇2 P (x, y, z, t). (8.26)
∂t
ydri et al, lectures computational physics 73

A particular solution of (8.25) is given by

1 − x22 √
P (x, t) =e 2σ , σ = 2Dt. (8.27)
σ
In other words the spatial distribution of√the diffusing molecules is always a gaussian
with half-width σ increasing with time as t.
The average of any function f of x is given by
Z
< f (x, t) >= f (x)P (x, t)dx. (8.28)

Let us multiply both sides of (8.25) by f (x) and then integrate over x, viz
∂P (x, t) ∂ 2 P (x, t)
Z Z
f (x) dx = D f (x) dx. (8.29)
∂t ∂x2
Clearly
∂P (x, t) ∂ d d
Z Z Z


f (x) dx = f (x)P (x, t) dx = f (x)P (x, t)dx = < f (x) > (. 8.30)
∂t ∂t dt dt
Thus
d ∂ 2 P (x, t)
Z
< f (x) > = D f (x) dx
dt ∂x2
 
∂P (x, t) x=+∞ ∂f (x) ∂P (x, t)
Z
= D f (x) |x=−∞ − D dx. (8.31)
∂x ∂x ∂x
We have P (x = ±∞, t) = 0 and also all spatial derivatives are zero at x = ±∞. We then
get

d ∂f (x) ∂P (x, t)
Z
< f (x) > = −D dx. (8.32)
dt ∂x ∂x
Let us choose f (x) = x. Then
d ∂P (x, t)
Z
< x > = −D dx = 0. (8.33)
dt ∂x
In other words < x >= constant and since x = 0 at t = 0 we must have constant = 0.
Thus
< x >= 0. (8.34)
Let us next choose f (x) = x2 . Then
d ∂P (x, t)
Z
< x2 > = −2D x dx
dt ∂x
= 2D. (8.35)
Hence
< x2 > = 2Dt. (8.36)
This is the diffusive behaviour we have observed in the random walk problem.
74 ydri et al, lectures on computational physics

8.3 The Random Number Generators RAN 0, 1, 2

Linear congruential generators are of the form

ri = (ari−1 + c) mod M. (8.37)

For c > 0 the linear congruential generators are called mixed. They are denoted by
LCG(a, c, M ). The random numbers generated with LCG(a, c, M ) are in the range
[0, M − 1].
For c = 0 the linear congruential generators are called multiplicative. They are
denoted by MLCG(a, M ). The random numbers generated with MLCG(a, M ) are in the
range [1, M − 1].
In the case that a is a primitive root modulo M and M is a prime the period of the
generator is M − 1. A number a is a primitive root modulo M means that for any integer
n such that gcd(n, M ) = 1 there exists a k such that ak = n mod M .
An example of MLCG is RAN0 due to Park and Miller which is used extensively on
IBM computers. In this case

a = 16807 = 75 , M = 231 − 1. (8.38)

The period of this generator is not very long given by

period = 231 − 2 ≃ 2.15 × 109 . (8.39)

This generator can not be implemented directly in a high level language because of
integer overflow. Indeed the product of a and M − 1 exceeds the maximum value for
a 32−bit integer. Assemply language implementation using 64−bit product register is
straightforward but not portable.
A better solution is given by Schrage’s algorithm. This algorithm allows the multipli-
cation of two 32−bit integers without using any intermediate numbers which are larger
than 32 bits. To see how this works explicitly we factor M as

M = aq + r. (8.40)

M
r = M mod a , q = [ ]. (8.41)
r
In the above equation [ ] denotes integer part. Remark that

M
r = M mod a = M − [ ]a. (8.42)
a
Thus by definition r < a. We will also demand that r < q and hence
r
<< 1. (8.43)
qa
ydri et al, lectures computational physics 75

We have also

aXi
Xi+1 = aXi mod M = aXi − [ ]M
M
aXi
= aXi − [ ]M. (8.44)
aq + r

We compute

aXi Xi Xi 1
= =
aq + r q + ar r
q 1 + qa
Xi r
= (1 − )
q qa
Xi Xi r
= − . (8.45)
q aq q

Clearly

Xi Xi Xi
= ≃ < 1. (8.46)
aq M −r M

Hence

aXi Xi
[ ] = [ ], (8.47)
M q

if neglecting ǫ = (rXi )/(aq 2 ) does not affect the integer part of aXi /M and

aXi Xi
[ ] = [ ] − 1, (8.48)
M q

if neglecting ǫ does affect the integer part of aXi /M . Therefore we get

aXi
Xi+1 = aXi − [ ](aq + r)
M
aXi aXi
= a(Xi − [ ]q) − [ ]r (8.49)
M M
Xi Xi
= a(Xi − [ ]q) − [ ]r (8.50)
q q
Xi
= a(Xi mod q) − [ ]r, (8.51)
q

if

Xi
a(Xi mod q) − [ ]r ≥ 0. (8.52)
q
76 ydri et al, lectures on computational physics

Also
aXi
Xi+1 = aXi − [ ](aq + r)
M
aXi aXi
= a(Xi − [ ]q) − [ ]r (8.53)
M M
Xi Xi
= a(Xi − [ ]q + q) − [ ]r + r (8.54)
q q
Xi
= a(Xi mod q) − [ ]r + M, (8.55)
q
if
Xi
a(Xi mod q) − [ ]r < 0. (8.56)
q
The generator RAN0 contains serial correlations. For example D−dimensional vectors
(x1 , ..., xD ), (xD+1 , ..., x2D ),...which are obtained by successive calls of RAN0 will lie on
a small number of parallel (D − 1)−dimensional hyperplanes. Roughly there will be
M 1/D such hyperplanes. In particular successive points (xi , xi+1 ) when binned into a
2−dimensional plane for i = 1, ..., N will result in a distribution which fails the χ2 test
for N ≥ 107 which is much less than the period M − 1.
The RAN1 is devised so that the correlations found in RAN0 is removed using the
Bays-Durham algorithm. The Bays-Durham algorithm shuffles the sequence to remove
low-order serial correlations. In other words it changes the order of the numbers so
that the sequence is not dependent on order and a given number is not correlated with
previous numbers. More precisely the jth random number is output not on the jth call
but on a randomized later call which is on average the j + 32th call on .
The RAN2 is an improvement over RAN1 and RAN0 due to L’Ecuyer. It uses two
sequences with different periods so as to obtain a new sequence with a larger period
equal to the least common multiple of the two periods. In this algorithm we add the two
sequences modulo the modulus M of one of them. In order to avoid overflow we subtract
rather than add and if the result is negative we add M − 1 so as to wrap around into the
inetrval [0, M − 1]. L’Ecuyer uses the two sequences
M1 = 2147483563 , a1 = 40014 , q1 = 53668 , r1 = 12211. (8.57)

M2 = 2147483399 , a2 = 40692 , q2 = 52774 , r2 = 3791. (8.58)

The period is 2.3 × 1018 . Let us also point out that RAN2 uses Bays-Durham algorithm
in order to implement an additional shuffle.
We conclude this section by discussing another generator based on the linear congru-
ential method which is the famous random number generator RAND given by
RAND = LCG(69069, 1, 232 ). (8.59)
The period of this generator is 232 and lattice structure is present for higher dimensions
D ≥ 6.
ydri et al, lectures computational physics 77

8.4 Lab Problem 14: Random Numbers

Part I We consider a linear congruential pseudo-random number generator given by
 
ari + c
ri+1 = remainder .
M
We take the values

a = 899, c = 0, M = 32768, r1 = 12 ”good”

a = 57, c = 1, M = 256, r1 = 10 , ”bad”.

The function “remainder” is implemented in Fortran by

a
remainder = mod(a, b).
b
(1) Compute the sequence of the random numbers ri obtained using the above param-
eters. Plot ri as a function of i. Construct a scatterplot (xi = r2i , yi = r2i+1 ).
(2) Compute the average of the random numbers. What do you observe.
(3) Let N be the number of generated random numbers. Compute the correlation
functions defined by
N −k
1 X
sum1 (k) = xi xi+k .
N −k
i=1

sum1 (k)− < xi >2

sum2 = .
sum1 (0)− < xi >2
What is the behavior of these functions as a function of k.
(4) Compute the period of the above generators.

Part II We take N random numbers in the interval [0, 1] which we divide into K bins
of length δ = 1/K. Let Ni be the number of random numbers which fall in the ith bin.
For a uniform sequence of random numbers the number of random numbers in each bin
is nideal = N/K.
(1) Verify this result for the generator “rand” found in the standard Fortran library
with seed given by seed = 32768. We take K = 10 and N = 1000. Plot Ni as a
function of the position xi of the ith bin.
(2) The number of degrees of freedom is ν = K − 1. The most probable value of the
chi-square statistics χ2 is ν. Verify this result for a total number of bin tests equal
L = 1000 and K = 11. Each time calculate the number of times Li in the L = 1000
bin tests we get a specific value of χ2 . Plot Li as a function of χ2 . What do you
observe.
78 ydri et al, lectures on computational physics

8.5 Lab Problem 15: Random Walks

Part I We consider the motion of a random walker in one dimension. The walker can
move with a step si = a to the right with a probability p or with a step si = −a toP the
left with a probability q = 1 − p. After N steps the position of the walker is xN = i si .
We take
1
p = q = , a = 1.
2
In order to simulate the motion of a random walker we need a generator of random
numbers. In this problem we work with the generator “rand” found in the standard
Fortran library. We call this generator as follows

call srand(seed)
rand()

The motion of the random walker is implemented with the code

if (rand() < p) then

xN = xN + a
else
xN = xN − a
endif.

(1) Compute the positions xi of three different random walkers as functions of the step
number i. We take i = 1, 100. Plot the three trajectories.
(2) We consider now the motion of K = 500 random walkers. Compute the averages
K K
1 X (i) 1 X (i) 2
< xN >= xN , < x2N >= (xN ) .
K K
i=1 i=1

(i)
In the above equations xN is the position of the ith random walker after N steps.
Study the behavior of these averages as a function of N . Compare with the theo-
retical predictions.

Part II (optional) We consider next a random walker in two dimensions on an infinite

lattice of points. From any point (i, j) on the lattice the walker can reach one of the 4
possible nearest neighbor sites (i+1, j), (i−1, j), (i, j +1) and (i, j −1) with probabilities
px , qx , py and qy respectively such that px + qx + py + qy = 1. For simplicity we will
assume that px = qx = py = qy = 0.25.
(1) Compute the averages < ~rN > and < ~rN 2 > as function of the number of steps

N for a collection of L = 500 two dimensional random walkers. We consider the

values N = 10, ..., 1000.
 Monte Carlo Integration
9
9.1 Numerical Integration
9.1.1 Rectangular Approximation Revisted
As usual let us start with something simple. The approximation of one-dimensional
integrals by means of the rectangular approximation. This is a topic we have already
discussed before.
Let us then begin by recalling how the rectangular approximation of one dimensional
integrals works. We consider the integral

Z b
F = f (x)dx. (9.1)
a

We discretize the x−interval so that we end up with N equal small intervals of lenght
∆x, viz

b−a
xn = x0 + n∆x , ∆x = (9.2)
N
Clearly x0 = a and xN = b. Riemann definition of the integral is given by the following
limit
N
X −1
F = lim ∆x f (xn ) , ∆x −→ 0 , N −→ ∞ , b − a = fixed. (9.3)
n=0

The first approximation which can be made is to simply drop the limit. We get the
80 ydri et al, lectures on computational physics

so-called rectangular approximation given by

N
X −1
FN = ∆x f (xn ). (9.4)
n=0

The error can be computed as follows. We start with the Taylor expansion
1
f (x) = f (xn ) + (x − xn )f (1) (xn ) + (x − xn )2 f (2) (xn ) + ... (9.5)
2!
Thus
xn+1
1 (1) 1
Z
dx f (x) = f (xn )∆x + f (xn )(∆x)2 + f (2) (xn )(∆x)3 + ... (9.6)
xn 2! 3!

The error in the interval [xn , xn+1 ] is

Z xn+1
1 1
dx f (x) − f (xn )∆x = f (1) (xn )(∆x)2 + f (2) (xn )(∆x)3 + ... (9.7)
xn 2! 3!

This is of order 1/N 2 . But we have N subintervals. Thus the total error is of order 1/N .

9.1.2 Midpoint Approximation of Multidimensional Integrals

Let us start with the two dimensional integral
Z
F = dx dy f (x, y). (9.8)
R

R is the domain of integration. In order to give the midpoint approximation of this

integral we imagine a rectangle of sides xb − xa and yb − ya which encloses the region R
and we divide it into squares of lenght h. The points in the x/y direction are

1
xi = xa + (i − )h , i = 1, ..., nx . (9.9)
2

1
yi = ya + (i − )h , i = 1, ..., ny . (9.10)
2
The number of points in the x/y direction are

xb − xa yb − ya
nx = , ny = . (9.11)
h h
The number of cells is therefore
(xb − xa )(yb − ya )
n = nx ny = . (9.12)
h2
ydri et al, lectures computational physics 81

The integral is then approximated by

ny
nx X
X
F = h2 f (xi , yj )H(xi , yj ). (9.13)
i=1 j=1

The Heaviside function is defined by

H(xi , yj ) = 1 if (xi , yj ) ∈ R otherwise H(xi , yj ) = 0. (9.14)

The generalization to many dimensions is straightforward. We get

n1 nd
f (xi11 , ..., xidd )H(xi11 , ..., xidd ).
X X
F = hd ... (9.15)
i1 =1 id =1

The meaning of the different symbols is obvious.

The midpoint approximation is an improvement over the rectangular approximation.
To see this let us consider a one dimensional integral
Z
F = dx f (x). (9.16)
R

The midpoint approximation reads in this case as follows

nx
X nx
X
F =h f (xi )H(xi ) = h f (xi ). (9.17)
i=1 i=1

Let us say that we have nx intervals [xi , xi+1 ] with x0 = a and xi = xa + (i − 0.5)h,
i = 1, ..., nx − 1. The term hf (xi+1 ) is associated with the interval [xi , xi+1 ]. It is clear
that we can write this approximation as

x −1
nX
xi + xi+1
F =h f( ) , xi = xa + ih. (9.18)
2
i=0

The error in the interval [xi , xi+1 ] is given by

Z xi+1
xi + xi+1 1 ′′
f (x) dx − f ( )∆x = f (xi )(∆x)3 + ... (9.19)
xi 2 24

The total error is thereore 1/n2x as opposed to the 1/nx of the rectangular approximation.
Let us do this in two dimensions. We write the error as
Z xi+1 Z yj+1
xi + xi+1 yj + yj+1
f (x, y) dx dy − f ( , )∆x∆y (9.20)
xi yj 2 2

As usual we use Taylor series in the form

82 ydri et al, lectures on computational physics

′ ′ 1 ′′
f (x, y) = f (xi , yj ) + fx (xi , yj )(x − xi ) + fy (xi , yj )(y − yj ) + fx (xi , yj )(x − xi )2
2
1 ′′ ′′
+ f (xi , yj )(y − yj )2 + fxy (xi , yj )(x − xi )(y − yj ) + ... (9.21)
2 y
We find
Z xi+1 Z yj+1
xi + xi+1 yj + yj+1 1 ′′ 1 ′′
f (x, y) dx dy − f ( , )∆x∆y = fx (xi , yj )(∆x)3 ∆y + fy (xi , yj )∆x(∆y)3
xi yj 2 2 24 24
+ ... (9.22)

Since ∆x = ∆y = h. The individual error is proportional to h4 . The total error is

nh4 where n = nx ny . Since n is proportional to 1/h2 , the total error in dimension
two is proportional to h2 or equivalently to 1/n. As we have already seen the same
method led to an error proportional to 1/n2 in dimension one. Thus as we increase the
number of dimensions the error becomes worse. If in one dimension the error behaves
a
as 1/na then in dimension d it will behave as 1/n d . In other words classical numerical
integration methods become impractical at sufficiently higher dimensions (which is the
case of quantum mechanics and statistical mechanics).

9.1.3 Spheres and Balls in d Dimensions

The volume of a ball of radius R in d dimensions is given by
Z
Vd = dx1 ...dxd
x21 +...+x2d ≤R2
Z
= r d−1 dr dΩd−1
x21 +...+x2d ≤R2

Rd
Z
= dΩd−1
d
d
Rd 2π 2
= . (9.23)
d Γ( d2 )

The surface of a sphere of radius R in d dimensions is similarly given by

Z
Sd−1 = dx1 ...dxd
x21 +...+x2d =R2
d
2π 2
= Rd−1 . (9.24)
Γ( d2 )

Here are some properties of the gamma function

1 √
Γ(1) = 1 , Γ( ) = π , Γ(n + 1) = nΓ(n). (9.25)
2
ydri et al, lectures computational physics 83

In order to compute numerically the volume of the ball in any dimension d we need a
recursion formula which relates the volume of the ball in d dimensions to the volume of
the ball in d − 1 dimensions. The derivation goes as follows

Z +R Z
Vd = dxd dx1 ...dxd−1
−R x21 +...+x2d−1 ≤R2 −x2d
Z +R Z √R2 −x2 Z
d
= dxd r d−2 dr dΩd−2
−R 0
+R
Vd−1
Z
d−1
= dxd (R2 − x2d ) 2 . (9.26)
Rd−1 −R
At each dimension d we are thus required to compute only the remaining integral over
xd using, for instance, the midpoint approximation while the volume Vd−1 is determined
in the previous recursion step. The starting point of the recursion process, for example
the volume in d = 2, can be determined also using the midpoint approximation. As we
will see in the lab problems this numerical calculation is very demanding with significant
errors compared with the Monte Carlo method.

9.2 Monte Carlo Integration: Simple Sampling

Let us start with the one dimensional integral
Z b
F = dx f (x). (9.27)
a
A Monte Carlo method is any procedure which uses (pseudo) random numbers to com-
pute or estimate the above integral. In the following we will describe two very simple
Monte Carlo methods based on simple sampling which give an approximate value for
this integral. As we progress we will be able to give more sophisticated Monte Carlo
methods. First we start with the sampling (hit or miss) method then we go on to the
sample mean method.

9.2.1 Sampling (Hit or Miss) Method

This method consists of the following three main steps:
• We imagine a rectangle of width b − a and height h such that h is greater than the
maximum value of f (x), i.e the function is within the boundaries of the rectangle.
• To estimate the value F of the integral we choose n pairs of uniform random
numbers (xi , yi ) where a ≤ xi ≤ b and 0 ≤ yi ≤ h.
• Then we evaluate the function f at the points xi . Let nin be the number of random
points (xi , yi ) such that yi ≤ f (xi ). The value F of the integral is given by
nin
F =A , A = h(b − a). (9.28)
n
84 ydri et al, lectures on computational physics

9.2.2 Sample Mean Method

We start from the mean-value theorem of calculus, viz
Z b
F = dx f (x) = (b − a) < f > . (9.29)
a
< f > is the average value of the function f (x) in the range a ≤ x ≤ b. The sample
mean method estimates the average < f > as follows:
• We choose n random points xi from the interval [a, b] which are distributed uni-
formly.
• We compute the values of the function f (x) at these point.
• We take their average. In other words
n
1X
F = (b − a) f (xi ). (9.30)
n
i=1

This is formally the same as the rectangular approximation. The only difference is that
here the points xi are chosen randomly from the interval [a, b] whereas the points in
the rectangular approximation are chosen with equal spacing. For lower dimensional
integrals the rectangular approximation is more accurate whereas for higher dimensional
integrals the sample mean method becomes more accurate.

9.2.3 Sample Mean Method in Higher Dimensions

We start with the two dimensional integral
Z
F = dx dy f (x, y). (9.31)
R
Again we consider a rectangle of sides yb − ya and xb − xa which encloses the region R.
The Monte carlo sample mean method yields the approximation

n
1X
F =A f (xi , yi )H(xi , yi ). (9.32)
n
i=1
The points xi are random and uniformly distributed in the interval [xa , xb ] whereas the
points yi are random and uniformly distributed in the interval [ya , yb ]. A is the areas of
the rectangle, i.e A = (xb − xa )(yb − ya ). The Heaviside function is defined by
H(xi , yi ) = 1 if (xi , yi ) ∈ R otherwise H(xi , yi ) = 0. (9.33)
Generalization to higher dimensions is obvious. For example in three dimensions we
would have
n
1X
Z
F = dx dy dz f (x, y, z) −→ F = V f (xi , yi , zi )H(xi , yi , zi ). (9.34)
R n
i=1
V is the volume of the parallelepiped which encloses the three dimensional region R.
ydri et al, lectures computational physics 85

9.3 The Central Limit Theorem

Let p(x) be a probability distribution function. We generate (or measure) n values xi of
a certain variable x according to the probability distribution function p(x). The average
y1 =< xi > is given by

n
1X
y1 =< xi >= xi p(xi ). (9.35)
n
i=1

We repeat this measurement N times thus obtaining N averages y1 , y2 ,...,yN . The mean
z of the averages yi is
N
1 X
z= yi . (9.36)
N
i=1

The question we want to answer is: what is the probability distribution function of z.
Clearly the probability of obtaining a particular value z is the product of the prob-
abilities of obtaining the individual averages yi (which are assumed to be independent)
with the constraint that the average of yi is z.
Let p̃(y) be the probability distribution function of the average y and let P (z) be the
probability distribution of the average z of the averages. We can then write P (z) as
y1 + ... + yN
Z Z
P (z) = dy1 ... dyN p̃(y1 )...p̃(yN )δ(z − ). (9.37)
N
The delta function expresses the constraint that z is the average of yi . The delta function
can be written as
y1 + ... + yN 1
Z
y1 +...+yN
δ(z − )= dqeiq(z− N )
. (9.38)
N 2π
Let µ be the actual average of yi , i.e.
Z
µ =< yi >= dy p̃(y)y. (9.39)

We write
1
Z Z Z
iq iq
iq(z−µ) (µ−y1 )
P (z) = dqe dy1 p̃(y1 )e N ... dyN p̃(yN )e N (µ−yN )
2π
Z N
1
Z
iq
iq(z−µ) (µ−y)
= dqe dy p̃(y)e N . (9.40)
2π
But
q 2 (µ − y)2
 
iq
Z Z
iq
(µ−y)
dy p̃(y)e N = dy p̃(y) 1 + (µ − y) − + ...
N 2N 2
q2σ2
= 1− + ... (9.41)
2N 2
86 ydri et al, lectures on computational physics

We have used
Z
dy p̃(y)(µ − y)2 =< y 2 > − < y >2 = σ 2 . (9.42)

Hence
1
Z
q2 σ 2
P (z) = dqeiq(z−µ) e− 2N
2π
1 − N2 (z−µ)2
Z
σ2 iN 2
= e 2σ dqe− 2N (q− σ (z−µ))
2π
2
− (z−µ)
2
2σ
1 e N
= √ . (9.43)
2π σN

σ
σN = √ . (9.44)
N
This is the normal distribution. Clearly the result does not depend on the original
probability distribution functions p(x) and p̃(y).
The average z of N random numbers yi corresponding to a probability distribution
function p̃(y) is distributed according to the normal probability distribution function
with average√equal to the average value of p̃(y) and variance equal to the variance of p̃(y)
divided by N.

9.4 Monte Carlo Errors and Standard Deviation

√
In any Monte Carlo approximation method the error goes as 1/ N where N is the num-
ber of samples. This behaviour is independent of the integrand and is independent of
the number of dimensions. In contrast if the error in a classical numerical approximation
method goes as 1/N a in one dimension (where N is now the number of intervals) then
a
the error in the same approximation method will go as 1/N d in d dimensions. Thus as
we increase the number of dimensions the error becomes worse. In other words classi-
cal numerical integration methods become impractical at sufficiently higher dimensions.
This is the fundamental appeal of Monte Carlo methods in physics (quantum mechanics
and statistical mechanics) where we usually and so often encounter integrals of infinite
dimensionality.
Let us again consider for simplicity the one dimensional integral as an example. We
take
Z b
F = dx f (x). (9.45)
a

The Monte Carlo sample mean method gives the approximation

N
1 X
FN = (b − a) < f > , < f >= f (xi ). (9.46)
N
i=1
ydri et al, lectures computational physics 87

The error is by definition given by

∆ = F − FN . (9.47)

However in general we do not know the exact result F . The best we can do is to calculate
the probability that the approximate result FN is within a certain range centered around
the exact result F .
The starting point is the central limit theorem. This states that the average z of
N random numbers yα corresponding to a probability distribution function p̃(y) is dis-
tributed according to the normal probability distribution function. Here the variable y
is (we assume for simplicity that b − a = 1)
N
1 X
y= f (xi ). (9.48)
N
i=1

We make M measurements yα of y. We write

N
1 X
yα = f (xi,α). (9.49)
N
i=1

The mean z of the averages is given by

M
1 X
z= yα . (9.50)
M
α=1

According to the central limit theorem the mean z is distributed according to the normal
probability distribution function with average equal√to the average value < y > of yα
and variance equal to the variance of yα divided by M , viz

s
(z− < y >)2
 
M
2 exp −M 2 . (9.51)
2πσ̃M 2σ̃M

The σ̃M is the standard deviation of the mean given by the square root of the variance
M
2 1 X
σ̃M = (yα − < y >)2 . (9.52)
M − 1 α=1

The use of M − 1 instead of M is known as Bessel’s correction. The reason for this
correction is the fact that the computation of the mean < y > reduces the number of
independent data points yα by one. For very large M we can replace σ̃M with σM defined
by
M
2 2 1 X
σ̃M ∼ σM = (yα − < y >)2 =< y 2 > − < y >2 . (9.53)
M
α=1
88 ydri et al, lectures on computational physics

The standard deviation of the sample (one single measurement with N data points) is
given by the square root of the variance
N
2 1 X
σ̃ = (f (xi )− < f >)2 . (9.54)
N −1
i=1

Again since N is large we can replace σ̃ with σ defined by

N
1 X
σ2 = (f (xi )− < f >)2 =< f 2 > − < f >2 . (9.55)
N
i=1

N N
1 X 1 X
< f >= f (xi ) , < f 2 >= f (xi )2 . (9.56)
N N
i=1 i=1

The standard deviation of the mean σ̃M ∼ σM is given in terms of the standard deviation
of the sample σ̃ ∼ σ by the equation
σ
σM = √ . (9.57)
N
The proof goes as follows. We generalize equations (9.55) and (9.56) to the case of M
measurements each with N samples. The total number of samples is M N . We have
M N
1 XX
σ2 = (f (xi,α )− < f >)2 =< f 2 > − < f >2 . (9.58)
NM
α=1 i=1

M N M N
1 XX 1 XX
< f >= f (xi,α ) , < f 2 >= f (xi,α )2 . (9.59)
NM NM
α=1 i=1 α=1 i=1

The standard deviation of the mean σ̃M ∼ σM is given by

M
2 1 X
σM = (yα − < y >)2
M
α=1
M  N 2
1 X 1 X
= f (xi,α )− < f >
M N
α=1 i=1
M N N   
1 XXX
= f (xi,α )− < f > f (xi,α )− < f > . (9.60)
N 2 M α=1
i=1 j=1

In above we have used the fact that < y >=< f >. For every set α the sum over i and
j splits into two pieces. The first is the sum over the diagonal elements with i = j and
the second is the sum over the off diagonal elements with i 6= j. Clearly f (xi,α )− < f >
ydri et al, lectures computational physics 89

and f (xj,α)− < f > are on the average equally positive and negative and hence for large
numbers M and N the off diagonal terms will cancel and we end up with

M N  2
2 1 XX
σM = f (xi,α )− < f >
N 2M
α=1 i=1
σ2
= . (9.61)
N
The standard deviation of the mean σM can therefore be interpreted as the probable error
in the original N measurements since if we make M sets of measurements each with N
samples the standard deviation of the mean σM will estimate how much an average over
N measurements will deviate from the exact mean.
This means in particular that the original measurement FN of the integral F has a
68 per cent chance of being within one standard deviation σM of the true mean and a 95
per cent chance of being within 2σM and a 99.7 per cent chance of being within 3σM . In
general the proportion of data values within κσM standard deviations of the true mean
is defined by the error function

κ
<y>+κσM
(z− < y >)2 √
 
1 2 κ
Z Z
2
exp −x2 dx = erf( √ ).


q exp − 2 dz = √
<y>−κσM 2
2πσM 2σM π 0 2

(9.62)

9.5 Nonuniform Probability Distributions

9.5.1 The Inverse Transform Method
We consider two discrete events 1 and 2 which occur with probabilities p1 and p2 respec-
tively such that p1 + p2 = 1. The question is how can we choose the two events with
the correct probabilities using only a uniform probability distribution. The answer is as
follows. Let r be a uniform random number between 0 and 1. We choose the event 1 if
r < p1 else we choose the event 2.
Let us now consider three discrete events 1, 2 and 3 with probabilities p1 , p2 and p3
respectively such that p1 + p2 + p3 = 1. Again we choose a random number r between 0
and 1. If r < p1 then we choose event 1, if p1 < r < p1 + p2 we choose event 2 else we
choose event 3.
We consider now n discrete events with probabilities pi such that ni=1 pi = 1. Again
P
we choose a random number r between 0 and 1. We choose the event i if the random
number r satisfies the inequality
i−1
X i
X
pj ≤ r ≤ pj . (9.63)
j=1 j=1
90 ydri et al, lectures on computational physics

In the continuum limit we replace the probability pi with p(x)dx P

which is the probability
that the event x is found between x and x + dx. The condition ni=1 pi = 1 becomes
Z +∞
p(x) dx = 1. (9.64)
−∞

The inequality (9.63) becomes the identity

Z x
′ ′
P (x) ≡ p(x ) dx = r (9.65)
−∞

Thus r is equal to the cumulative probability distribution P (x), i.e the probability of
choosing a value less than or equal to x. This equation leads to the inverse transform
method which allows us to generate a nonuniform probability distribution p(x) from a
uniform probability distribution r. Clearly we must be able to 1) perform the integral
analytically to find P (x) then 2) invert the relation P (x) = r for x.
As a first example we consider the Poisson distribution
1 −x
p(x) = e λ , 0 ≤ x ≤ ∞. (9.66)
λ
We find
x
P (x) = 1 − e− λ = r. (9.67)

Hence

x = −λ ln(1 − r). (9.68)

Thus given the uniform random numbers r we can compute directly using the above for-
mula the random numbers x which are distributed according to the Poisson distribution
x
p(x) = λ1 e− λ .
The next example is the Gaussian distribution in two dimensions
1 − x2 +y2 2
p(x, y) = e 2σ . (9.69)
2πσ 2
We can immediately compute that
Z +∞ Z +∞ 2 2
Z 1 Z 1
1 − x +y
dx dy e 2σ =2 dw dv. (9.70)
2πσ 2 −∞ −∞ 0 0

x = r cos φ , y = r sin φ. (9.71)

r 2 = −2σ 2 ln v , φ = 2πw. (9.72)

The random numbers v and w are clearly uniformly distributed between 0 and 1. The
random numbers x (or y) are distributed according to the Gaussian distribution in one
dimension. This method is known as the Box-Muller method.
ydri et al, lectures computational physics 91

9.5.2 The Acceptance-Rejection Method

This was proposed by Von Neumann. The goal is to generate a sequence of random
numbers distributed according to some normalized probability density y = p(x). This
method consists of the following steps:

• We start by generating a uniform random number rx in the range of interest xmin ≤

rx ≤ xmax where [xmin , xmax ] is the interval in which y = p(x) does not vanish.

• We evaluate p(rx ).

• Then we generate another uniform random number ry in the range [0, ymax ] where
ymax is the maximum value of the distribution y = p(x).

• If ry < p(rx ) then we accept the random number rx else we reject it.

• We repeat this process a sufficient number of times.

It is not difficult to convince ourselves that the accepted random numbers rx will be
distributed according to y = p(x).

9.6 Lab Problem 16: Midpoint and Monte Carlo Approxi-

mations
Part I The volume of a ball of radius R in d dimensions is given by
Z
Vd = dx1 ...dxd
x21 +...+x2d ≤R2
Z q
= 2 dx1 ...dxd−1 R2 − x21 − ... − x2d−1
d
Rd 2π 2
= .
d Γ( d2 )

(1) Write a program that computes the three dimensional integral using the midpoint
approximation. We take the stepsize h = 2R/N , the radius R = 1 and the number
of steps in each direction to be N = Nx = Ny = 2p where p = 1, 15.

(2) Show that the error goes as 1/N . Plot the logarithm of the absolute value of the
absolute error versus the logarithm of N .

(3) Try out the two dimensional integral. Work in the positive quadrant and again
take the stepsize h = R/N where R = 1 and N = 2p , p = 1, 15. We know that
generically the theoretical error goes at least as 1/N 2 . What do you actually find?
Why do you find a discrepancy?
Hint: the second derivative of the integrand is singular at x = R which changes
the dependence from 1/N 2 to 1/N 1.5 .
92 ydri et al, lectures on computational physics

Part II In order to compute numerically the volume of the ball in any dimension d we
use the recursion formula

+R
Vd−1
Z
d−1
Vd = dxd (R2 − x2d ) 2 .
Rd−1 −R

(1) Find the volumes in d = 4, 5, 6, 7, 8, 9, 10, 11 dimensions. Compare with the exact
result given above.

Part III
(1) Use the Monte Carlo sampling (hit or miss) method to find the integrals in d =
2, 3, 4 and d = 10 dimensions. Is the Monte Carlo method easier to apply than the
midpoint approximation?
(2) Use the Monte Carlo sample mean value method to find the integrals in d = 2, 3, 4
and d = 10 dimensions. For every d we perform M measurements each with N
samples. We consider M = 1, 10, 100, p
√ 150 and N = 2 , p = 10, 19. Verify that the
exact error in this case goes like 1/ N .
Hint: Compare the exact error which √ is known in this case with the standard
deviation of the mean σM and with σ/ N where σ is the standard deviation of
the sample, i.e. of a single measurement. These three quantities must be identical.

Part IV
(1) The value of π can be given by the integral
Z
π= dx dy.
x2 +y 2 ≤R2

Use the Monte Carlo sampling (hit or miss) method to give an approximate value
of π.

(2) The above integral can also be put in the form

Z +1 p
π=2 dx 1 − x2 .
−1

Use the Monte Carlo sample mean value method to give another approximate value
of π.

9.7 Lab Problem 17: Nonuniform Probability Distributions

Part I The Gaussian distribution is given by
1 (x − µ)2
P (x) = √ exp − .
2πσ 2 2σ
ydri et al, lectures computational physics 93

The parameter µ is the mean and σ is the variance, i.e the square root of the standard
deviation. We choose µ = 0 and σ = 1.

(1) Write a program that computes a sequence of random numbers x distributed ac-
cording to P (x) using the inverse transform method (Box-Muller algorithm) given
by the equations

x = r cos φ.

r 2 = −2σ 2 ln v , φ = 2πw.

The v and w are uniform random numbers in the interval [0, 1].

(2) Draw a histogram of the random numbers obtained in the previous question. The
steps are as follows:

a- Determine the range of the points x.

b- We divide the interval into u bins. The lenght of each bin is h = interval/u.
We take for example u = 100.
c- We determine the location of every point x among the bins. We increase the
counter of the corresponding bin by a unit.
d- We plot the fraction of points as a function of x. The fraction of point is equal
to the number of random numbers in a given bin divided by hN where N is
the total number of random numbers. We take N = 10000.

(3) Draw the data on a logarithmic scale, i.e plot log(fraction) versus x2 . Find the fit
and compare with theory.

Part II
(1) Apply the acceptance-rejection method to the above problem.

(2) Apply the Fernandez-Criado algorithm to the above problem. The procedure is as
follows

a- Start with N points xi such that xi = σ.

b- Choose at random a pair (xi , xj ) from the sequence and make the following
change
xi + xj
xi −→ √
2
√
xj −→ −xi + 2xj .

c- Repeat step 2 until we reach equilibrium. For example try it M times where
M = 10, 100, ....
94 ydri et al, lectures on computational physics
 Monte Carlo Importance Sampling, Metropolis
10
Algorithm and Ising Model

10.1 The Canonical Ensemble

We consider physical systems which are in thermal contact with an environment. The
environment is usually much larger than the physical system of interest and as a conse-
quence energy exchange between the two of them will not change the temperature of the
environement. The environement is called heat bath or heat reservoir. When the system
reaches equilibrium with the heat bath its temperature will be given by the temperature
of the heat bath.
A system in equilibrium with a heat bath is described statistically by the canonical
ensemble in which the temperature is fixed. In contrast an isolated system is described
statistically by the microcanonical ensemble in which the energy is fixed. Most systems
in nature are not isolated but are in thermal contact with the environment. It is a
fundamental result of statistical mechanics that the probability of finding a system in
equilibrium with a heat bath at temperature T in a microstate s with energy Es is given
by the Boltzmann distribution

1 −βEs 1
Ps = e , β= . (10.1)
Z kB T

The normalization connstant Z is the partition function. It is defined by

X
Z= e−βEs . (10.2)
s

The sum is over all the microstates of the system with a fixed N and V . The Helmholtz
96 ydri et al, lectures on computational physics

free energy F of a system is given by

F = −kB T ln Z. (10.3)

In equilibrium the free energy is minimum. All other thermodynamical quantities can
be given by various derivatives of F . For example the internal energy U of the system
which is the expectation value of the energy can be expressed in terms of F as follows

1 X ∂ ∂
Es e−βEs = −
X
U =< E >= Es Ps = ln Z = (βF ). (10.4)
s
Z s ∂β ∂β

The specific heat is given by

∂
Cv = U. (10.5)
∂T

In the definition of the partition function (10.2) we have implicitly assumed that we
are dealing with a physical system with configurations (microstates) which have discrete
energies. This is certainly true for many quantum systems. However for many other
systems especially classical ones the energies are not discrete. For example the partition
function of a gas of N distinguishable classical particles is given by

N
d3 pi d3 qi −βH(~pi ,~qi )
Z Y
Z= e . (10.6)
h3
i=1

For quantum dynamical field systems (in Euclidean spacetimes) which are of fundamental
importance to elementary particles and their interactions the partition function is given
by the so-called path integral which is essentially of the same form as the previous
equation with the replacement of the Hamiltonian H(~ pi , ~qi ) by theQaction S[Φ] where Φ
stands for the field variables and the replacement of the measure N 3 3 3
i=1 (d pi d qi )/h by
the relevant (infinite dimensional) measure DΦ on the space of field configurations. We
obtain therefore
Z
Z = DΦ e−βS[Φ] . (10.7)

Similarly to what happens in statistical mechanics where all observables can be derived
from the partition function the observables of a quantum field theory can all be derived
from the path integral. The fundamental problem therefore is how to calculate the
partition function or the path integral for a given physical system. Normally an analytic
solution will be ideal. However finding such a solution is seldom possible and as a
consequence only the numerical approach remains available to us. The partition function
and the path integral are essentially given by multidimensional integrals and thus one
should seek numerical approaches to the problem of integration.
ydri et al, lectures computational physics 97

10.2 Importance Sampling

In any Monte Carlo integration the numerical error is proportional to the standard de-
viation of the integrand and is inversely proportional to the number of samples. Thus
in order to reduce the error we should either reduce the variance or increase the number
of samples. The first option is preferable since it does not require any extra computer
time. Importance sampling allows us to reduce the standard deviation of the integrand
and hence the error by sampling more often the important regions of the integral where
the integrand is largest. Importance sampling uses also in a crucial way nonuniform
probability distributions.
Let us again consider the one dimensional integral
Z b
F = dx f (x). (10.8)
a

We introduce the probability distribution p(x) such that

Z b
1= dx p(x). (10.9)
a

We write the integral as

b
f (x)
Z
F = dx p(x) . (10.10)
a p(x)

We evaluate this integral by sampling according to the probability distribution p(x). In

other words we find a set of N random numbers xi which are distributed according to
p(x) and then approximate the integral by the sum
N
1 X f (xi )
FN = . (10.11)
N p(xi )
i=1

The probability distribution p(x) is chosen such that the function f (x)/p(x) is slowly
varying which reduces the corresponding standard deviation.

10.3 The Ising Model

We consider a d−dimensional periodic lattice with n points in every direction so that
there are N = nd points in total in this lattice. In every point (lattice site) we put a spin
variable si (i = 1, ..., N ) which can take either the value +1 or −1. A configuration of
this system of N spins is therefore specified by a set of numbers {si }. In the Ising model
the energy of this system of N spins in the configuration {si } is given by

X N
X
EI {si } = − ǫij si sj − H si . (10.12)
<ij> i=1
98 ydri et al, lectures on computational physics

The parameter H is the external magnetic field. The symbol < ij > stands for nearest
neighbor spins. The sum over < ij > extends over γN /2 terms where γ is the number of
nearest neighbors. In 2, 3, 4 dimensions γ = 4, 6, 8. The parameter ǫij is the interaction
energy between the spins i and j. For isotropic interactions ǫij = ǫ. For ǫ > 0 we obtain
ferromagnetism while for ǫ < 0 we obtain antiferromagnetism. We consider only ǫ > 0.
The energy becomes with these simplifications given by

X N
X
EI {si } = −ǫ si sj − H si . (10.13)
<ij> i=1

The partition function is given by

XX X
Z= ... e−βEI {si } . (10.14)
s1 s2 sN

There are 2N terms in the sum and β = 1/kB T .

In d = 2 we have N = n2 spins in the square lattice. The configuration {si } can
be viewed as an n × n matrix. We impose periodic boundary condition as follows. We
consider (n + 1) × (n + 1) matrix where the (n + 1)th row is identified with the first
row and the (n + 1)th column is identified with the first column. The square lattice is
therefore a torus.

10.4 The Metropolis Algorithm

The internal energy U =< E > can be put into the form
−βEs
P
s Es e
< E >= P −βEs
. (10.15)
se

Generally given any physical quantity A its expectation value < A > can be computed
using a similar expression, viz
−βEs
P
s As e
< A >= P −βEs
. (10.16)
se

The number As is the value of A in the microstate s. In general the number of microstates
N is very large. In any Monte Carlo simulation we can only generate a very small number
n of the total number N of the microstates. In other words < E > and < A > will be
approximated with
Pn −βEs
s=1 Es e
< E > ≃ < E >n = P n −βEs
. (10.17)
s=1 e

Pn −βEs
s=1 As e
< A > ≃ < A >n = P n −βEs
. (10.18)
s=1 e
ydri et al, lectures computational physics 99

The calculation of < E >n and < A >n proceeds therefore by 1) choosing at random
a microstate s, 2) computing Es , As and e−βEs then 3) evaluating the contribution of
this microstate to the expectation values < E >n and < A >n . This general Monte
Carlo procedure is however highly inefficient since the microstate s is very improbable
and therefore its contribution to the expectation values is negligible. We need to use
importance sampling. To this end we introduce a probability distribution ps and rewrite
the expectation value < A > as
P As −βEs
s p e ps
< A >= P 1s −βE . (10.19)
s ps e
sp
s

Now we generate the microstates s with probabilities ps and approximate < A > with
< A >n given by
Pn As −βEs
s=1 p e
< A >n = Pn 1s −βE . (10.20)
s=1 ps e
s

This is importantce sampling. The Metropolis algorithm is importance sampling with ps

given by the Boltzmann distribution, i.e.

e−βEs
p s = Pn −βEs
. (10.21)
s=1 e

We get then the arithmetic average

n
1X
< A >n = As . (10.22)
n s=1

The Metropolis algorithm in the case of spin systems such as the Ising model can be
summarized as follows:
(1) Choose an initial microstate.

(2) Choose a spin at random and flip it.

(3) Compute ∆E = Etrial − Eold . This is the change in the energy of the system due
to the trial flip.

(4) Check if ∆E ≤ 0. In this case the trial microstate is accepted.

(5) Check if ∆E > 0. In this case compute the ratio of probabilities w = e−β∆E .

(6) Choose a uniform random number r in the inetrval [0, 1].

(7) Verify if r ≤ w. In this case the trial microstate is accepted, otherwise it is rejected.

(8) Repeat steps 2) through 7) until all spins of the system are tested. This sweep
counts as one unit of Monte Carlo time.
100 ydri et al, lectures on computational physics

(9) Repeat setps 2) through 8) a sufficient number of times until thermalization, i.e.
equilibrium is reached.

(10) Compute the physical quantities of interest in n thermalized microstates. This can
be done periodically in order to reduce correlation between the data points.

(11) Compute averages.

The proof that this algorithm leads indeed to a sequence of states which are distributed
according to the Boltzmann distribution goes as follows.
It is clear that the steps 2) through 7) corresponds to a transition probability between
the microstates {si } and {sj } given by

W (i −→ j) = min(1, e−β∆E ) , ∆E = Ej − Ei . (10.23)

Since only the ratio of probabilities w = e−β∆E is needed it is not necessary to normalize
the Boltzmann probability distribution. It is clear that this probability function satisfies
the detailed balance condition

W (i −→ j) e−βEi = W (j −→ i) e−βEj . (10.24)

Any other probability function W which satisfies this condition will generate a sequence
of states which are distributed according to the Boltzmann distribution.
P This can be
shown by summing over the index j in the above equation and using j W (i −→ j) = 1.
We get

e−βEi = W (j −→ i) e−βEj .
X
(10.25)
j

The Boltzmann distribution is an eigenvector of W . In other words W leaves the equilib-

rium ensemble in equilibrium. As it turns out this equation is also a sufficient condition
for any ensemble to approach equilibrium.

10.5 The Heat-Bath Algorithm

The heat-bath algorithm is generally a less efficient algorithm than the Metropolis algo-
rithm. The acceptance probability is given by

1
W (i −→ j) = min(1, ) , ∆E = Ej − Ei . (10.26)
1 + eβ∆E
This acceptance probability satisfies also detailed balance for the Boltzmann probability
distribution. In other words the detailed balance condition which is sufficient but not
necessary for an ensemble to reach equilibrium does not have a unique solution.
ydri et al, lectures computational physics 101

10.6 The Mean Field Approximation

10.6.1 Phase Diagram and Critical Temperature
We consider N = L2 spins on a square lattice where L is the number of lattice sites
in each direction. Each spin can take only two possible values si = +1 (spin up) and
si = −1 (spin down). Each spin interacts only with its 4 neigbhors and also with a
magnetic field H. The Ising model in 2 dimensions is given by the energy
X X
E{s} = −J si sj − H si . (10.27)
<ij> i

The system is assumed to be in equilibrium with a heat bath with temperature T . Ther-
mal equilibrium of the Ising model is described by the canonical ensemble. The prob-
ability of finding the Ising model in a configuration {s1 , ..., s2N } is given by Boltzmann
distribution

e−βE{s}
P {s} = . (10.28)
Z
The partition function is given by

e−βE{s} = e−βE{s} .
X X X
Z= ... (10.29)
{s} s1 s2N

The magnetization M in a configuration {s1 , ..., s2N } is the order parameter of the system.
It is defined by
X
M= si . (10.30)
i

The average of M is given by

X
< M >= < si >= N < s > . (10.31)
i

In above < si >=< s > since all spins are equivalent. We have

1 ∂ log Z ∂F
< M >= =− . (10.32)
β ∂H ∂H

In order to compute < M > we need to compute Z. In this section we use the mean
field approximation. First we rewrite the energy E{s} in the form
X X
E{s} = (−J sj )si − H si
<ij> i
X X
i
= Heff si −H si . (10.33)
i i
102 ydri et al, lectures on computational physics

i is given by
The effective magnetic field Heff
X
i
Heff = −J sj(i) . (10.34)
j(i)

The index j(i) runs over the four nearest neighbors of the spin i. In the mean field
approximation we replace the spins sj(i) by their thermal average < s >. We obtain
i
Heff = −Jγ < s > , γ = 4. (10.35)

In other words
X X
E{s} = −(H + Jγ < s >) si = Heff si (10.36)
i i

The partition function becomes

X N
Z = e−βHeff si
s1
 N
−βHeff βHeff
= e +e (10.37)
 N
= 2 cosh βHeff . (10.38)

The free energy and magnetization are then given by

 
F = −kT ln Z = −kT N ln 2 cosh βHeff . (10.39)

< M >= N < s >= N tanh βHeff . (10.40)

Thus for zero magnetic field we get from the second equation the constraint

< s >= tanh γβJ < s > . (10.41)

Clearly < s >= 0 is always a solution. This is the high temperature paramagnetic phase.
For small temperature we have also a solution < s >6= 0. This is the ferromagnetic phase.
There must exist a critical temperature Tc which separates the two phases. We expect
< s > to approach < s >= 0 as T goes to Tc from below. In other words near Tc we can
treat < s > as small and as a consequence we can use the expansion tanh x = x − 13 x3 .
We obtain
1
3
< s >= γβJ < s > − γβJ < s > . (10.42)
3
Equivalently
 
23 1 γJ

<s> <s> − −T = 0. (10.43)
T (γβJ)3 kB
ydri et al, lectures computational physics 103

We get the two solutions

< s >= 0 , paramagnetic phase

s
3 1
< s >= ± (Tc − T )β , ferromagnetic phase. (10.44)
T (γβJ)3

The critical temperature Tc and the critical exponent β are given by

γJ 1
Tc = , β= . (10.45)
kB 2
The ferromagnetic solution can only exist for T < Tc .

10.6.2 Critical Exponents

The free energy for zero magnetic field is

 
F = −kT N ln 2 cosh γβJ < s > . (10.46)

We see that for T < Tc the ferromagnetic solution has a lower free energy than the
paramagnetic solution < s >= 0. The phase T < Tc is indeed ferromagnetic. The
transition at T = Tc is second order. The free energy is continuous at T = Tc , i.e.
there is no latent heat while the specific heat is logarithmically divergent. The mean
field theory yields the correct value 0 for the critical exponent α although it does not
reproduce the logarithmic divergence. The susceptibility diverges at T = Tc with critical
exponent γ = 1. These latter statements can be seen as follows.
The specific heat is given by

 
∂ 2 ∂
Cv = − kB T (βF )
∂T ∂T
∂ ∂2
= −2kB T (βF ) − kB T 2 2 (βF ). (10.47)
∂T ∂T
Next we use the expression βF = −N ln(ex + e−x ) where x = γβJ < s >. We find

Cv ∂x ∂2x 1 ∂x 2
= 2kB T tanh x + kB T 2 tanh2 x 2 + kB T 2 2 ( ∂T ) . (10.48)
N ∂T ∂T cosh x
We compute
s s s
3kB 1 ∂x 1 3kB 1 ∂ 2x 1 3kB 3
x=± (Tc − T ) 2 , =∓ (Tc − T )− 2 , 2
=∓ (Tc − T )− 2 .
γJ ∂T 2 γJ ∂T 4 γJ
(10.49)
104 ydri et al, lectures on computational physics

It is not difficult to show that the divergent terms cancel and as a consequence

Cv
∼ (Tc − T )−α , α = 0. (10.50)
N

The susceptibility is given by

∂
χ= <M >. (10.51)
∂H

To compute the behavior of χ near T = Tc we consider the equation

< s >= tanh(γβJ < s > +βH). (10.52)

For small magnetic field we can still assume that γβJ < s > +βH is small near T = Tc
and as a consequence we can expand the above equation as

1
< s >= (γβJ < s > +βH) − (γβJ < s > +βH)3 . (10.53)
3

Taking the derivative with respect to H of both sides of this equation we obtain

χ̂ = (γβJ χ̂ + β) − (γβJ χ̂ + β)(γβJ < s > +βH)2 . (10.54)

∂
χ̂ = <s>. (10.55)
∂H

Setting the magnetic field to zero we get

χ̂ = (γβJ χ̂ + β) − (γβJ χ̂ + β)(γβJ < s >)2 . (10.56)

In other words
 
2
1 − γβJ + γβJ(γβJ < s >) χ̂ = β − β(γβJ < s >)2 . (10.57)

Tc − T 1
2 χ̂ = (1 − (γβJ < s >)2 ). (10.58)
T kB T

Hence

1
χ̂ = (Tc − T )−γ , γ = 1. (10.59)
2kB
ydri et al, lectures computational physics 105

10.7 Simulation of The Ising Model and Numerical Results

10.7.1 The Fortran Code
We choose to write our code in Fortran. The reason is simplicity and straightforwardness.
A person who is not well versed in programming languages, who has a strong background
in physics and maths, and who wants to get up and running quickly with the coding so
that she starts doing physics (almost) immediately the choice of Fortran for her is ideal
and thus it is only natural. The potential superior features which may be found in C are
peripheral to our purposes here.
The spin found in the intersection point of the ith row and jth column of the lattice
will be represented with the matrix element φ(i, j). The energy will then read (with
N = n2 and n ≡ L)
n    
X J
E=− φ(i, j) φ(i + 1, j) + φ(i − 1, j) + φ(i, j + 1) + φ(i, j − 1) + Hφ(i, j) .
2
i,j=1
(10.60)

We impose periodic boundary condition in order to reduce edge and boundary effects.
This can be done as follows. We consider (n + 1) × (n + 1) matrix where the (n + 1)th
row is identified with the first row and the (n + 1)th column is identified with the first
column. The square lattice is therefore a torus. The toroidal boundary condition will
read explicitly as follows

φ(0, j) = φ(n, j) , φ(n + 1, j) = φ(1, j) , φ(i, 0) = φ(i, n) , φ(i, n + 1) = φ(i, 1).

The variation of the energy due to the flipping of the spin φ(i, j) is an essential ingredient
in the Metropolis algorithm. This variation is explicitly given by

∆E = 2Jφ(i, j) φ(i + 1, j) + φ(i − 1, j) + φ(i, j + 1) + φ(i, j − 1) + 2Hφ(i, j).(10.61)

The Fortran code contains the following pieces:

• A subroutine which generates pseudo random numbers. We prefer to work with

well established suboutines such as the RAN 2 or the RANLUX.

• A subroutine which implements the Metropolis algorithm for the Ising model. This
main part will read (with some change of notation such as J = exch)

do i=1,L
ip(i)=i+1
im(i)=i-1
enddo
ip(L)=1
106 ydri et al, lectures on computational physics

im(1)=L

do i=1,L
do j=1,L
deltaE=2.0d0*exch*phi(i,j)*(phi(ip(i),j)+phi(im(i),j)+phi(i,ip(j))+phi(i,im(j)))
deltaE=deltaE + 2.0d0*H*phi(i,j)
if (deltaE.ge.0.0d0)then
probability=dexp(-beta*deltaE)
call ranlux(rvec,len)
r=rvec(1)
if (r.le.probability)then
phi(i,j)=-phi(i,j)
endif
else
phi(i,j)=-phi(i,j)
endif
enddo
enddo

• We compute the energy < E > and the magnetization < M > of the Ising model
in a separate subroutine.

• We compute the errors using for example the Jackknife method in a separate sub-
routine.

• We fix the parameters of the model such as L, J, β = 1/T and H.

• We choose an initial configuration. We consider both cold and hot starts which are
given respectively by

φ(i, j) = +1. (10.62)

φ(i, j) = random signs. (10.63)

• We run the Metropolis algorithm for a given thermalization time and study the
history of the energy and the magnetization for different values of the temperature.

• We add a Monte Carlo evolution with a reasonably large number of steps and
compute the averages of E and M .

• We compute the specific heat and the susceptibility of the system.

10.7.2 Some Numerical Results

Energy: The energy is continuous through the transition point and as a consequence
there is no latent heat. This indicates a second order behavior.
ydri et al, lectures computational physics 107

Specific Heat: The critical exponent associated with the specific heat is given by
α = 0. However the specific heat diverges logarithmically at T = Tc . This translates into
the fact that the peak grows with n logarithmically, namely
Cv
∼ log n. (10.64)
n2

Magnetization: The magnetization near but below the critical temperature in the
two-dimensional Ising model scales as
<M >
∼ (Tc − T )−β , β = 1/8. (10.65)
n2

Susceptibility: The susceptibility near the critical temperature in the two-dimensional

Ising model scales as
χ
∼ |T − Tc |−γ , γ = 7/4. (10.66)
n2

Critical Temperature: From the behavior of the above observable we can measure
the critical temperature, which marks the point where the second order ferromagnetic
phase transition occurs, to be given approximately by
2J
kB Tc = √ . (10.67)
ln( 2 + 1)

Critical Exponents and 2−Point Correlation Function: The 2−point correlation

function of the two-dimensional Ising model is defined by the expression

f (x) = < s0 sx >

 
1 X
= < 2 φ(i, j) φ(i + x, j) + φ(i − x, j) + φ(i, j + x) + φ(i, j − x) > .
4n
i,j
(10.68)

We can verify numerically the following statements:

• At T = Tc the behaviour of f (x) is given by
1
f (x) ≃ , η = 1/4. (10.69)
xη
• At T less than Tc the behavior of f (x) is given by

f (x) =< M >2 . (10.70)

• At T larger than Tc the behaviour of f (x) is given by

1 − xξ
f (x) ≃ a e . (10.71)
xη
108 ydri et al, lectures on computational physics

• Near Tc the correlation lenght diverges as

1
ξ≃ , ν = 1. (10.72)
|T − Tc |ν

Note that near-neighbor lattice sites which are a distance x away in a given direc-
tion from a given index i are given by

do x=1,nn
if (i+x .le. n) then
ipn(i,x)=i+x
else
ipn(i,x)=(i+x)-n
endif
if ((i-x).ge.1)then
imn(i,x)=i-x
else
imn(i,x)=i-x+n
endif
enddo

For simplicity we consider only odd lattices, viz n = 2nn + 1. Clearly because
of the toroidal boundary conditions the possible values of the distance x are x =
1, 2, ..., nn.

First Order Transition and Hysteresis: We can also consider the effect of a mag-
netic field H on the physics of the Ising model. We observe a first order phase transition
at H = 0 or H near 0 and a phenomena of hysteresis. We observe the following:

• For T < Tc we can observe a first order phase transition. Indeed we observe a
discontinuity in the energy and the magnetization which happens at a non-zero
value of H due to hysteresis. The jumps in the energy and the magnetization are
typical signal for a first order phase transition.

• For T > Tc the magnetization becomes a smooth function of H near H = 0 which

means that above Tc there is no distinction between the ferromagnetic states with
M ≥ 0 and M ≤ 0.

• We recompute the magnetization as a function of H for a range of H back and

fourth. We observe the following:

– A hysteresis loop.
– The hysteresis window shrinks with increasing temperature or accumulating
more Monte Carlo time.
ydri et al, lectures computational physics 109

– The hysteresis effect is independent of the size of the lattice.

The phenomena of hysteresis indicates that the behaviour of the system depends
on its initial state and history. Equivalently we say that the system is trapped in
a metastable state.

10.8 Lab Problem 18: The Metropolis Algorithm and The

Ising Model
Part I We consider N = L2 spins on a square lattice where L is the number of lattice
sites in each direction. Each spin can take only two possible values si = +1 (spin up)
and si = −1 (spin down). Each spin interacts only with its 4 neigbhors and also with a
magnetic field H. The Ising model in 2 dimensions is given by the energy
X X
E = −J si sj − H si .
<ij> i

We will impose toroidal boundary condition. The system is assumed to be in equilib-

rium with a heat bath with temperature T . Thermal fluctuations of the system will be
simulated using the Metropolis algorithm.

(1) Write a subroutine that computes the energy E and the magnetization M of the
Ising model in a configuration φ. The magnetization is the order parameter of the
system. It is defined by
X
M= si . (10.73)
i

(2) Write a subroutine that implements the Metropolis algorithm for this system. You
will need for this the variation of the energy due to flipping the spin φ(i, j).

(3) We choose L = 10, H = 0, J = 1, β = 1/T . We consider both a cold start and a

hot start.
Run the Metropolis algorithm for a thermalization time TTH = 26 and study the
history of the energy and the magnetization for different values of the temperature.
The energy and magnetization should approach the values E = 0 and M = 0 when
T −→ ∞ and the values E = −2JN and M = +1 when T −→ 0.

(4) Add a Monte Carlo evolution with TTM = 210 and compute the averages of E and
M.

(5) Compute the specific heat and the susceptibility of the system. These are defined
by
∂ β ∂
Cv = < E >= (< E 2 > − < E >2 ) , χ = < M >= β(< M 2 > − < M >2 ).
∂β T ∂H
110 ydri et al, lectures on computational physics

(6) Determine the critical point. Compare with the theoretical exact result
2J
kB Tc = √ .
ln( 2 + 1)
Part II Add to the code a separate subroutine which implements the Jackknife method
for any set of data points. Compute the errors in the energy, magnetization, specific heat
and susceptibility of the Ising model using the Jackknife method.

10.9 Lab Problem 19: The Ferromagnetic Second Order

Phase Transition
Part I The critical exponent associated with the specific heat is given by α = 0, viz
Cv
∼ (Tc − T )−α , α = 0.
L2
However the specific heat diverges logarithmically at T = Tc . This translates into the
fact that the peak grows with L logarithmically, namely
Cv
∼ log L.
L2
Verify this behaviour numerically. To this end we take lattices between L = 10 − 30 with
TTH = 210 , TMC = 213 . The temperature is taken in the range
T = Tc − 10−2 × step , step = −50, 50.
Plot the maximum of Cv /L2 versus ln L.

Part II The magnetization near but below the critical temperature in 2D Ising model
scales as
<M > 1
2
∼ (Tc − T )−β , β = .
L 8
We propose to study the magnetization near Tc in order to determine the value of β
numerically. Towards this end we plot | < M > | versus Tc − T where T is taken in the
the range
T = Tc − 10−4 × step , step = 0, 5000.
We take large lattices say L = 30 − 50 with TTH = TMC = 210 .

Part III The susceptibility near the critical temperature in 2D Ising model scales as
χ 7
∼ |T − Tc |−γ , γ = .
L2 4
Determine γ numerically. Use TTH = 2 , TMC = 213 , L = 50 with the two ranges
10

T = Tc − 5 × 10−4 × step , step = 0, 100.

T = Tc − 0.05 − 4.5 × 10−3 step , step = 0, 100.

ydri et al, lectures computational physics 111

10.10 Lab Problem 20: The 2−Point Correlator

In this exercise we will continue our study of the ferromagnetic second order phase
transition. In particular we will calculate the 2−point correlator defined by the expression
 
1 X
f (n) =< s0 sn >=< φ(i, j) φ(i + n, j) + φ(i − n, j) + φ(i, j + n) + φ(i, j − n) > .
4L2
i,j

(1) Verify that at T = Tc the behaviour of f (n) is given by

1 1
f (n) ≃ , η= .
nη 4

(2) Verify that at T less than Tc the behaviour of f (n) is given by

f (n) =< M >2 .

(3) Verify that at T larger than Tc the behaviour of f (n) is given by

1 − nξ
f (n) ≃ a e .
nη
In all the above questions we take odd lattices say L = 2LL + 1 with LL = 20 − 50.
We also consider the parameters TTH = 210 , TTC = 213 .
(4) Near Tc the correlation lenght diverges as
1
ξ≃ , ν = 1.
|T − Tc |ν

In the above question we take LL = 20. We also consider the parameters TTH =
210 , TTC = 215 and the temperatures

T = Tc + 0.1 × step , step = 0, 10.

10.11 Lab Problem 21: Hysteresis and The First Order Phase
Transition
In this exercise we consider the effect of the magnetic field on the physics of the Ising
model. We will observe a first order phase transition at H = 0 or H near 0 and a
phenomena of hysteresis .
(1) We will compute the magnetization and the energy as functions of H for a range of
temperatures T . The initialization will be done once for all H. The thermalization
will be performed once for the first value of the magnetic field H say H = −5.
After we compute the magnetization for H = −5, we start slowly (adiabatically)
changing the magnetic field with small steps so we do not loose the thermalization
of the Ising system of spins. We try out the range H = −5, 5 with step equal 0.25.
112 ydri et al, lectures on computational physics

a- For T < Tc say T = 0.5 and 1.5 determine the first order transition point from
the discontinuity in the energy and the magnetization. The transition should
happen at a non-zero value of H due to hysteresis. The jump in the energy
is associated with a non-zero latent heat. The jumps in the energy and the
magnetization are the typical signal for a first order phase transition.
b- For T > Tc say T = 3 and 5 the magnetization becomes a smooth function of
H near H = 0 which means that above Tc there is no distinction between the
ferromagnetic states with M ≥ 0 and M ≤ 0.

(2) We recompute the magnetization as a function of H for a range of H from −5 to

5 and back. You should observe a hysteresis loop.

a- Verify that the hysteresis window shrinks with increasing temperature or ac-
cumulating more Monte Carlo time.
b- Verify what happens if we increase the size of the lattice.

The phenomena of hysteresis indicates that the behaviour of the system depends
on its initial state and history or equivalently the system is trapped in metastable
states.
Appendix
 Notes on Numerical Errors
A
A.1 Floating Point Representation
Any real number x can be put in the binary form
x = ±m × 2e−bias , 1≤m < 2 , m = b0 .b1 b2 b3 ... (A.1)
We consider a 32−bit computer. Since 1≤m < 2 we must have b0 = 1. This binary
expansion is called normalized. For single precision floating-point numbers (singles or
floats) we use a 32−bit word with one bit for the sign, 8 bits for the exponent e and
23 bits for the significand m. Since only 8 bits are used to store the exponent we must
have e in the range 0≤e≤255. The bias is chosen bias = 127 so that the actual exponent
is in the range −127≤e − bias≤128. This way we can have very small numbers while
the stored exponent is always positive. Since the first bit of the significand is 1 the
stored bits of the significand are only b1 b2 ...b23 . If b24 , b25 , .. are not all zero the floating
point representation is not exact. Strictly speaking a floating point number is a number
for which b24 = b25 = ..0. The floating point representation of a non-zero real number
is unique because of the condition 1≤m < 2. In summary the above real number is
represented on the computer by
xnormal float = (−1)s 1.f × 2e−127 , 0 < e < 255. (A.2)
These are normal numbers. The terminology floating point is now clear. The binary
point can be moved (floated) to any position in the bitstring by choosing the appropriate
exponent.
The smallest normalized number is 2−126 . The subnormal numbers are represented
by
xsubnormal float = (−1)s 0.f × 2−126 . (A.3)
116 ydri et al, lectures on computational physics

These are not normalized numbers. In fact the space between 0 and the smallest positive
normalized number is filled by the subnormal numbers.
Explicitly

s e f
Bit Position 31 30-23 22-0

Because only a finite number of bits is used the set of machine numbers (the numbers
that the computer can store exactly or approximately) is much smaller than the set of
real numbers. There is a maximum and a minimum. Exceeding the maximum we get the
error condition known as overflow. Falling below the minimum we get the error condition
known as underflow.
The largest number corresponds to the normal floating number with s = 0, e = 254
and 1.f = 1.111..1 (with 23 1s after the binary point). We compute 1.f = 1+0.5+0.25+
0.125 + ... = 2. Hence xnormal float max = 2 × 2127 ≃ 3.4 × 1038 . The smallest number
corresponds to the subnormal floating number with s = 0 and 0.f = 0.00...1 = 2−23 .
Hence xsubnormal float min = 2−149 ≃ 1.4 × 10−45 . We get for single precision floats the
range

1.4 × 10−45 ≤ single precision ≤3.4 × 1038 . (A.4)

We remark that

2−23 ≃ 10−6.9 . (A.5)

Thus single precision numbers have 6 − 7 decimal places of significance.

There are special cases. The zero can not be normalized. It is represented by two
floats ±0. Also ±∞ are special numbers. Finally NaN (not a number) is also a special
case. Explicitly we have

±0 = (−1)s 0.0...0 × 2−126 . (A.6)

±∞ = (−1)s 1.0...0 × 2127 . (A.7)

NaN = (−1)s 1.f × 2127 , f 6= 0. (A.8)

The double precision floating point numbers (doubles) occupy 64 bits. The first bit
is for the sign, 11 bits for the exponent and 52 bits for the significand. They are stored
as two 32−bist words. Explicitly

s e f f
Bit Position 63 62-52 51-32 31-0
ydri et al, lectures computational physics 117

In this case the bias is bias = 1023. They correspond approximately to 16 decimal places
of precision. They are in the range

4.9 × 10−324 ≤ double precision ≤1.8 × 10308 . (A.9)

The above description corresponds to the IEEE 754 standard adopted in 1987 by
the Institute of Electrical and Electronics Engineers (IEEE) and American National
Standards Institute (ANSI).

A.2 Machine Precision and Roundoff Errors

The gap ǫ between the number 1 and the next largest number is called the machine
precision. For single precision we get ǫ = 2−23 . For double precision we get ǫ = 2−52 .
Alternatively the machine precision ǫm is the largest positive number which if added
to the number stored as 1 will not change this stored 1, viz

1c + ǫm = 1c . (A.10)

Clearly ǫm < ǫ. The number xc is the computer representation of of the number x. The
relative error ǫx in xc is therefore such that
xc − x
|ǫx | = | |≤ǫm . (A.11)
x
All single precision numbers contain an error in their 6th decimal place and all double
precision numbers contain an error in their 15th decimal place.
An operation on the computer will therefore only approximate the analytic answer
since numbers are stored approximately. For example the difference a = b − c is on the
computer ac = bc − cc . We compute
ac b c
= 1 + ǫb − ǫc . (A.12)
a a a
In particular the subtraction of two very large nearly equal numbers b and c may lead to
a very large error in the answer ac . Indeed we get the error
b
ǫa ≃ (ǫb − ǫc ). (A.13)
a
In other words the large number b/a can magnify the error considerably. This is called
subtractive cancellation.
Let us next consider the operation of multiplication of two numbers b and c to produce
a number a, viz a = b × c. This operation is represented on the computer by ac = bc × cc .
We get the error

ǫa = ǫb + ǫc . (A.14)
118 ydri et al, lectures on computational physics

Let us now consider an operation involving a large number N of steps. The question
we want to ask is how does the roundoff error accumulate.
The main observation is that roundoff errors grow slowly and randomly with N . They
diverge as N gets very large. By assuming that the roundoff errors in the individual steps
of the operation are not correlated we can view the accumulation of error as a random
walk problem with step size equal to the machine precison ǫm . We know from the study
of the random walk
√ problem in statistical mechanics that the total roundoff error will be
proportional to N , namely
√
ǫro = N ǫm . (A.15)

This is the most conservative estimation of the roundoff errors. The roundoff errors are
analogous the uncertainty in the measurement of a physical quantity.

A.3 Systematic (Algorithmic) Errors

This type of errors arise from the use of approximate numerical solutions. In general the
algorithmic (systematic) error is inversely proportional to some power of the number of
steps N ,i.e
α
ǫsys = . (A.16)
Nβ
The total error is obtained by adding the roundoff error, viz
α √
ǫtot = ǫsys + ǫro = β + N ǫm . (A.17)
N
There is a competition between the two types of errors. For small N it is the systematic
error which dominates while for large N the roundoff error dominates. This is very
interesting because it means that by trying to decrease the systematic error (by increasing
N ) we will increase the roundoff error. The best algorithm is the algorithm which gives
an acceptable approximation in a small number of steps so that there will be no time for
roundoff errors to grow large.
As an example let us consider the case β = 2 and α = 1. The total error is
1 √
ǫtot = 2
+ N ǫm . (A.18)
N
This error is minimum when
dǫtot
= 0. (A.19)
dN
For single precision calculation (ǫm = 10−7 ) we get N = 1099. Hence ǫtot = 4 × 10−6 .
Most of the error is roundoff.
In order to decrease the roundoff error and hence the total error in the above example
we need to decrease the number of steps. Furthermore in order for the systematic error
ydri et al, lectures computational physics 119

to not increase when we decrease the number of steps we must find another algorithm
which converges faster with N . For an algorithm with α = 2 and β = 4 the total error is
2 √
ǫtot = + N ǫm . (A.20)
N4
This error is minimum now at N = 67 for which ǫtot = 9 × 10−7 . We have only 1/16 as
many steps with an error smaller by a factor of 4.
120 ydri et al, lectures on computational physics
 The Jackknife Method and The Method of Least
B
Squares

The Jackknife Method: Any set of data points in a typical simulation will generally
tend to contain correlation between the different points. In other words the data points
will not be statistically independent and as a consequence one can not use the usual
formula to compute the standard deviation of the mean (i.e the probable error). The
aim of the Jackknife method is to estimate the error in a set of data points which contain
correlation. This method works as follows.
1) We start with a sample of N measurements (data points) {X1 , ..., XN }. We com-
pute the mean
N
1 X
< X >= Xi . (B.1)
N
i=1

2) We throw out the data point Xj . We get a sample of N − 1 measurements

{X1 , ..., Xj−1 , Xj+1 , ..., XN }. This sample is called a bin. Since j = 1, ..., N we
have N bins.We compute the mean
N
1 X
< X >j = ( Xi − Xj ). (B.2)
N −1
i=1

3) The standard deviation of the mean will be estimated using the formula
N
N −1X
σ12 = (< X >j − < X >)2 . (B.3)
N
j=1
122 ydri et al, lectures on computational physics

The Jackknife error is σ1 .

It is not difficult to show that
< X > −Xj
< X >j − < X >= . (B.4)
N −1
Thus
N
1 X
σ12 = (Xj − < X >)2 = σmean
2
. (B.5)
N (N − 1)
j=1

However in general this will not be true and the Jackknife estimate of the error is
more robust.

4) This can be generalized by throwing out z data points from the set {X1 , ...., XN }.
We end up with n = N/z bins. We compute the means < X >j over the bins in
an obvious way. The corresponding standard deviation will be given by
n
n−1X
σz2 = (< X >j − < X >)2 . (B.6)
n
j=1

5) The z takes the values z = 1, ..., N − 1. The error is the maximum of σz as a

function of z.

The Method of Least Squares: We assume that we have N data points (x(i), y(i)).
We want to fit this data to some curve say a straight line yfit = mx + b. To this end we
define the function
N
X N
X
∆= (y(i) − yfit (i))2 = (y(i) − mx(i) − b)2 . (B.7)
i=1 i=1

The goal is to minimize this function with respect to b and m. We have

∂∆ ∂∆
=0, = 0. (B.8)
∂m ∂b
We get the solution

x(j)y(j) − i x(i)2 j y(j)

P P P P
i x(i) j
b= . (B.9)
( i x(i))2 − N i x2i
P P

P P P
i x(i) j y(j) − N x(i)y(i)
m= P 2
Pi 2 . (B.10)
( i x(i)) − N i xi
 C
Exercises

Exercise 1: We give the differential equations

dx
= v. (C.1)
dt

dv
= a − bv. (C.2)
dt
• Write down the exact solutions.

• Write down the numerical solutions of these differential equations using Euler and
Verlet methods and determine the corresponding errors.

Exercise 2: The equation of motion of the solar system in polar coordinates is

d2 r l2 GM
2
= 3
− 2 . (C.3)
dt r r
Solve this equation using Euler, Euler-Cromer and Verlet methods.

Exercise 3: The equation of motion of a free falling object is

d2 z
= −g. (C.4)
dt2
• Write down the exact solution.

• Give a solution of this problem in terms of Euler method and determine the error.
124 ydri et al, lectures on computational physics

• We choose the initial conditions z = 0, v = 0 at t = 0. Determine the position and

the velocity between t = 0 and t = 1 for N = 4. Compare with the exact solution
and compute the error in each step. Express the result in terms of l = g∆t2 .

• Give a solution of this problem in terms of Euler-Cromer and Verlet methods and
determine the corresponding errors.

Exercise 4: The equation governing population growth is

dN
= aN − bN 2 . (C.5)
dt
The linear term represents the rate of birth while the quadratic term represents the rate
of death. Give a solution of this problem in terms of the Euler and Verlet methods and
determine the corresponding errors.

Exercise 5: Using the Runge-Kutta method solve the following differential equations

d2 r l2 GM
= − 2 . (C.6)
dt2 r3 r

d2 z
= −g. (C.7)
dt2

dN
= aN − bN 2 . (C.8)
dt

Exercise 6: The Lorenz model is a chaotic system given by three coupled first order
differential equations
dx
= σ(y − x)
dt
dy
= −xz + rx − y
dt
dz
= xy − bz. (C.9)
dt
This system is a simplified version of the system of Navier-Stokes equations of fluid me-
chanics which are relevant for the Rayleigh-Bénard problem. Write down the numercial
solution of these equations according to Runge-Kutta method.