Mathematics For Quant

Finance
Introduction to Mathematical Methods
Riaz Ahmad

This is a revision course designed to act as a mathematics

refresher. The volume of work covered is signi…cantly
large so the emphasis is on working through the notes
and problem sheets. The four topics covered in detail are

Calculus

Linear Algebra

Probability

Di¤erential Equations
[email protected]
1
1. INTRODUCTION TO CALCULUS

1 Introduction to Calculus

1.1 Basic Terminology

Mathematics is not actually that complex. Historically

books on the subject tend to be written in a fairly in-
comprehensible manner by mathematicians who want to
look clever. In addition the basic notation used can put
many o¤. Once familiar with a particular style of termi-
nology, another text or lecturer uses a di¤erent style of
nomenclature. Quant Finance is no exception. We start
by presenting some standard mathematical shorthand:

Natural Numbers N = f0; 1; 2; 3; :::::g

Integers ( N) Z = f0; 1; 2; 3; :::::g

n o
Rationals Q = 1; 0:76; 2:25; 0:3333333::::
2

np o
Irrationals 2; 0:01001000100001:::; ; e
2
1. INTRODUCTION TO CALCULUS

Reals R all the above

n p o
Complex numbers C = x + iy : i = 1

9 there exists ! which gives

equivalent
8 for all s.t such that
similar
) therefore !x a unique x
2 an element of
* because i¤ if and only if

3
1. INTRODUCTION TO CALCULUS

1.2 Functions

A function f (x) of a single variable x is a rule that

assigns each element of a set X (written x 2 X ) to
exactly one element y of a set Y (y 2 Y ) : A function
is denoted by the form y = f (x) or:

x 7! f (x) :
We can also write f : X !Y

For example,if f (x) = x3; then f ( 1) = 13 = 1

30

20

10

0
-4 -3 -2 -1 0 1 2 3 4

-10

-20

-30

We often write y = f (x) where y is the dependent

variable and x is the independent variable.

4
1. INTRODUCTION TO CALCULUS

domain range/image

-1 1
f

0 0

2 4

input output

The set X is called the domain of f and the set Y is

called the image (or range), written Dom f and Im f;
in turn.

For a given value of x there should be at most one value

of y . The de…nition of a function requires this property.

The example above is a 1 1 function.

f (x) = x2 is a many to one function.

p
f (x) = x is a one to many mapping. Note we do not
use the term function here.

5
1. INTRODUCTION TO CALCULUS

The inverse function f 1 (x) is de…ned so that

f f 1 (x) = x and f 1 (f (x)) = x:

p
Thus x and x2 are inverse mappings (for x 0).

Further Notation:

(a; b) = a < x < b open interval

[a; b] = a x b closed interval

(a; b] = a < x b semi-open/closed interval

[a; b) = a x<b semi-open/closed interval

Example 1: What is the inverse of y = 2x2 1:

i.e. we want y 1: This can be done by rearranging the

function as
s
y+1
x=
2
6
1. INTRODUCTION TO CALCULUS

q
therefore y 1 ( x) = x+1 :
2

Check:
0s 12
x + 1A
yy 1 (x) = 2 @ 1 = x = y 1 y ( x)
2

Example 2: Consider f (x) = 1=x; therefore f 1 (x) =

1=x

Domf = ( 1; 0) [ (0; 1) or R f 0g

Returning to the earlier example

y = 2 x2 1
clearly Domf = R (clearly)

and for
s
x+1
y 1 ( x)
=
2
to exist we require the term inside the square root sign to
be non-negative, i.e. x+12 0 =) x 1; therefore
Domf = f[ 1; 1)g :
7
1. INTRODUCTION TO CALCULUS

An even function is one which has the property

f ( x) = f ( x)
e.g. f (x) = x2:

The function we plotted earlier f (x) = x3 is an example

of an odd function because

f ( x) = f ( x) :

8
1. INTRODUCTION TO CALCULUS

1.2.1 Polynomials

These are functions which involve powers of x;

f (x) = a0 + a1x + a2x2 + :::::

:: + an 1xn 1 + anxn:
The highest power is called the degree of the polynomial
- so f (x) is an nth degree polynomial. We can express
this more compactly as
n
X
f ( x) = a k xk
k=0
where the coe¢ cients of x are constants.

k = 1; 2 gives a linear and quadratic in turn. The most

general form of quadratic equation is

ax2 + bx + c = 0
which can be solved for x using the formula
p
b b2 4ac
x=
2a

There are three cases to consider:

9
1. INTRODUCTION TO CALCULUS

(1) b2 4ac > 0 ! x1 6= x2 2 R : 2 distinct real

roots

b
(2) b2 4ac = 0 ! x = x1 = x2 = 2 R : one
2a
two fold root

(3) b2 4ac < 0 ! x1 6= x2 2 C - Complex conjugate

pair

10
1. INTRODUCTION TO CALCULUS

1.2.2 Explicit/Implicit Representation

When we express a function as y = f (x) ; then we can

obtain y corresponding to a (known) value of x: We say
y is an explicit function. All known terms are on the right
hand side (rhs) and unknown on the left hand side (lhs).
For example

y = 2 x2 + 4 x 16 = 0

Occasionally we may write a function in an implicit form

f (x; y ) = 0; although in general there is no guarantee
that for each x there is a unique y .

A trivial example is

y x2 = 0 ;
which in its current form is implicit. Simple rearranging
gives y = x2 which is explicit:

A more complex example is

11
1. INTRODUCTION TO CALCULUS

4y 4 2 y 2 x2 yx2 + x2 + 3 = 0:

So we see all known and unknown variables are bundled

together. An implicit form which does not give rise to a
function is

y 2 + x2 16 = 0:

This can be written as

q
y= 16 x2 :
and e.g. for x = 0 we can have either y = 4 or
y = 4; i.e. one to many.

12
1. INTRODUCTION TO CALCULUS

1.2.3 The Modulus Function

Sometimes we wish to obtain the absolute value of a

number, i.e. positive part. For example the absolute
value of 3:9 is 3:9: In maths there is a function which
gives us the absolute value of a variable x called the
modulus function, written jxj and de…ned as
(
x x 0
y = jxj =
x x<0

modulus function

3.5

2.5

1.5

0.5

-4 -3 -2 -1 0 1 2 3 4

13
1. INTRODUCTION TO CALCULUS

1.2.4 The exponential and log functions

The logarithm (or simply log) was introduced to solve

equations of the form

ap = N
and we say p is log of N to base a: That is we take logs
of both sides (loga)

loga ap = loga N
which gives
p = loga N:
By de…nition loga a = 1 (important).

We will often need the exponential function ex and the

(natural) logarithm loge x or (ln x) : Here

e = 2:718281828 : : : :
which is the approximation to
1 n
1+
n
14
1. INTRODUCTION TO CALCULUS

when n is very large. Similarly the exponential function

can be approximated from
x n
1+
n

ln x and ex are mutual inverses:

log (ex) = elog x = x:

Also

1 x:
= e
ex
Here we have used the property (xa)b = xab; which
allowed us to write e1x = (ex) 1 = e x:

Their graphs look like this:

15
1. INTRODUCTION TO CALCULUS

Exponential Functions

7
exp(x) and (exp(-x) 6

0
-2.5 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 2.5
x

logx and lnx

1.5
1

0.5

0
-0.5 0 1 2 3 4 5

-1

-1.5
-2

-2.5
x

We see that log10 x grows faster than natural logarithm.

16
1. INTRODUCTION TO CALCULUS

Note that ex is always strictly positive. It tends to zero

as x becomes very large and negative, and to in…nity
as x becomes large and positive. To get an idea of how
quickly ex grows, note the approximation e5 t 150:

2
Later we will also see e x =2; which is particularly useful
in probability: This function decays particularly rapidly as
jxj increases. The term x2=2 is called the exponent,
i.e. the "bit" above the exponential. When there is a
large exponent (many terms) then it is common to write

exp (:::::::)

Note:

exey = ex+y ; e0 = 1

(recall xa:xb = xa+b) and

log (xy ) = log x+log y; log (1=x) = log x; log 1 = 0:

17
1. INTRODUCTION TO CALCULUS

!
x
log = log x log y:
y

Dom (ex) = R
Im (ex) = (0; 1)

Dom (ln x) = (0; 1)

Im (ln x) = R

18
1. INTRODUCTION TO CALCULUS

1.2.5 Trigonometric Functions

sinx and cosx

1.5

0.5

0
-8 -6 -4 -2 0 2 4 6 8

-0.5

-1

-1.5

sin x is an odd function, i.e. sin ( x) = sin x:

It is periodic with period 2 : sin (x + 2 ) = sin x. This

means that after every 360 it repeats itself.

sin x = 0 () x = n 8n 2 Z

Dom =R and Im = [ 1; 1]

cos x is an even function, i.e. cos ( x) = cos x:

It is periodic with period 2 : cos (x + 2 ) = cos x.

19
1. INTRODUCTION TO CALCULUS

cos x = 0 () x = (2n + 1) 2 8n 2 Z

Dom =R and Im = [ 1; 1]

Trigonometric Identities:

cos2 x + sin2 x = 1
sin (x y ) = sin x cos y cos x sin y
cos (x y ) = cos x cos y sin x sin y
sin x + = cos x
2
cos x = sin x
2

sin x
tan x =
cos x

This is an odd function: tan ( x) = tan x

Periodic: tan (x + ) = tan x

20
1. INTRODUCTION TO CALCULUS

n o
Dom = fx : cos x 6= 0g = x : x 6= (2n + 1) 2 ; n 2 Z =
n o
R (2n + 1) 2 ; n 2 Z

The inverse trigonometric functions are de…ned by

1 1 1
sec x = ; csc x = ; cot x =
cos x sin x tan x

21
1. INTRODUCTION TO CALCULUS

1.2.6 Hyperbolic Functions

1 x
sinh x = e e x
2

Odd function: sinh ( x) = sinh x

Dom =R

Im = R

1 x
cosh x = e +e x
2

Even function: cosh ( x) = cosh x

22
1. INTRODUCTION TO CALCULUS

Dom =R

Im = [1; 1)

Identities:

cosh2 x sinh2 x = 1
sinh (x + y ) = sinh x cosh y + cosh x sinh y
cosh (x + y ) = cosh x cosh y + sinh x sinh y

sinh x
tanh x =
cosh x

Dom =R

Im = ( 1; 1)
23
1. INTRODUCTION TO CALCULUS

Inverse Hyperbolic Functions

exp y exp( y)
y = sinh 1 x ! x = sinh y = 2 ;

2x = exp y exp ( y )

multiply both sides by exp y to obtain 2xey = e2y 1

which can be written as

(ey )2 2x (ey ) 1 = 0:

This gives us a quadratic in ey therefore

p q
2x 4 x2 + 4
ey = =x x2 + 1
2

p p
Now x2
+ 1 > x =) x x2 + 1 <p0 and we know
that ey > 0 therefore we have ey = x + x2 + 1: Hence
24
1. INTRODUCTION TO CALCULUS

taking logs of both sides gives us

q
sinh 1 x = ln x + x2 + 1

Dom sinh 1 x =R

Im sinh 1 x = R

exp y+exp( y)
Similarly y = cosh 1 x ! x = cosh y = 2 ;

2x = exp y + exp ( y ) and again multiply both sides by

exp y to obtain
(ey )2 2x (ey ) + 1 = 0:

and
q
ey = x + x2 1
25
1. INTRODUCTION TO CALCULUS

We take the positive root (not both) to ensure this is a

function.
q
cosh 1 x = ln x + x2 1

Dom cosh 1x =[1; 1)

Im cosh 1 x = [0; 1)

We …nish o¤ by obtaining an expression for tanh 1 x:

Put y = tanh 1 x !
exp y exp ( y )
x = tanh y = ;
exp y + exp ( y )

x exp y + x exp ( y ) = exp y exp ( y )

26
1. INTRODUCTION TO CALCULUS

and as before multiply through by ey

x exp 2y + x = exp 2y 1
1+x
exp 2y (1 x) = 1 + x ! exp 2y =
1 x

taking logs gives

1+x
2y = ln
1 x

hence
1+x
tanh 1 x = 12 ln
1 x

Dom tanh 1x = ( 1; 1)

Im tanh 1 x = R

27
1. INTRODUCTION TO CALCULUS

1.3 Limits

Choose a point x0 and function f (x) : Suppose we

are interested in this function near the point x = x0:
The function need not be de…ned at x = x0: We write
f (x) ! l as x ! x0; "if f (x) gets closer and closer
to l as x gets close to x0". Mathematically we write this
as

lim f (x) ! l;
x!x0

if 9 a number l such that

Whenever x is close to x0
28
1. INTRODUCTION TO CALCULUS

f (x) is close to l:

Let us have a look at a few basic examples and corre-

sponding "tricks" to evaluate them

Example 1:

lim x2 + 2x + 3 ! 0 + 0 + 3 ! 3;
x!0

Example 2:

lim e x ! 0; lim ex ! 1; lim ex ! e0 = 1:

x!1 x!1 x!0

Example 3:
x2 + 2x 2
x2 + 2x + 2 x2 x2
+ x2
lim = lim =
x!1 3 x2 + 4 x!1 3x2 + 4
x2 x2
1 + x2 + x22 1
lim ! :
x!1 3 + x42 3
29
1. INTRODUCTION TO CALCULUS

Example 4:
x2 9 (x + 3) (x 3)
lim = lim = lim (x + 3) ! 6
x!3 x 3 x!3 (x 3) x!3

The limit only exists if

f ( x) ! l as x ! x0
f ( x) ! l as x ! x+
0

More Examples:

lim sin x ! 0
x!0

sin x
lim !1
x!0 x

lim jxj ! 0
x!0

30
1. INTRODUCTION TO CALCULUS

jxj
What about lim ?
x!0 x

jxj
lim = 1
x!0 x
+
jxj
lim = 1
x!0 x
jxj
therefore does not tend to a limit as x ! 0:
x

31
1. INTRODUCTION TO CALCULUS

1.4 Continuity

A function f (x) is continuous at x0 if

lim f (x) = f (x0) :

x!x0

That is, ’we can draw its graph without taking the pen
o¤ the paper’.

32
1. INTRODUCTION TO CALCULUS

1.5 Di¤erentiation

How fast does a function f (x) change with x? The

gradient or derivative of f (x) ,written

df
f 0 ( x) or
dx

is de…ned for each x as

f (x + x) f ( x)
f 0 (x) = lim
x!0 x

assuming the limit exists (it may not). Di¤erentiability

implies continuity (but converse does not always hold).

33
1. INTRODUCTION TO CALCULUS

The earlier form of the derivative given is also called a

forward derivative. Other possible de…nitions of the deriv-
ative are

1
f 0 ( x) = (f (x) f (x h)) backward
lim
h!0 h
1
f 0 (x) = lim (f (x + h) f (x h)) centred
h!0 2h

Examples:

Di¤erentiating x2 from …rst principles:

f ( x) = x2
f (x + h) = (x + h)2 = x2 + 2xh + h2
f ( x + h ) f ( x) 2hx + h2
=
h h
= 2x + h
! 2x as h ! 0;

34
1. INTRODUCTION TO CALCULUS

d n
x = nxn 1;
dx

d x d ax
e = ex; e = aeax;
dx dx

d 1
log x =
dx x
d
cos x = sin x
dx
d
sin x = cos x
dx
d
tan x = sec2 x
dx

and so on. Take these as de…ned (standard results).

The inverse trigonometric functions are de…ned by

1 1 1
sec x = ; csc x = ; cot x =
cos x sin x tan x

35
1. INTRODUCTION TO CALCULUS

Examples:

f ( x ) = x 5 ! f 0 ( x ) = 5x 4
g (x) = e3x ! g 0 (x) = 3e3x = 3g (x)

1.5.1 Rules For Di¤erentiation

Linearity

If and are constants and y = f (x) + g (x) then

dy d
= ( f (x) + g (x)) = f 0 (x) + g 0 (x) :
dx dx

Thus if y = 3x2 6e 2x then

dy=dx = 6x + 12e 2x:

36
1. INTRODUCTION TO CALCULUS

1.5.2 Product Rule

If y = f (x) g (x) then

dy
= f 0 ( x) g ( x) + f ( x) g 0 ( x) :
dx

Thus if y = x3e3x then

dy=dx = 3x2e3x + x3 3e3x = 3x2 (1 + x) e3x:

We can derive this rule as follows. Put h (x) = f (x) g (x) :

If dh
dx exists, we can de…ne
h (x + x) h (x)
h 0 ( x)= lim
x!0 x
f (x + x) g (x + x) f (x) g (x)
= lim
x!0 x
To evaluate the limit, we perform a small trick, subtract
and add f (x + x) g (x) from the numerator
37
1. INTRODUCTION TO CALCULUS

f (x+ x)g(x+ x) f (x+ x)g(x)+f (x+ x)g(x) f (x)g(x)

x
which can be written as
g(x+ x) g(x) f (x+ x) f (x)
lim f (x + x) : x + g ( x) : x
x!0
so
g (x + x) g (x)
h 0 ( x) = lim f (x + x) : lim
x!0 x!0 x
f (x + x) f (x)
+ lim g (x) : lim
x!0 x!0 x
g (x + x) g (x)
= f (x) lim +
x!0 x
f (x + x) f (x)
g (x) : lim
x!0 x
= f ( x) g 0 ( x) + g ( x) f 0 ( x)

38
1. INTRODUCTION TO CALCULUS

1.5.3 Function of a Function Rule

Di¤erentiation is often a matter of breaking a complicated

problem up into simpler components.

The function of a function rule is one of the main ways

of doing this. If

y = f (g (x)) then

dy
= f 0 (g (x)) g 0 (x) :
dx

4x 2
Thus if y = e then

4x 2 4x 2
dy=dx = e 4 :2 x = 8xe :

So di¤erentiate the whole function, then multiply by the

derivative of the "inside" (g (x)) :

Another way to think of this is in terms of the chain rule.

39
1. INTRODUCTION TO CALCULUS

Write y = f (g (x)) as

y = f (u) ; u = g ( x) :

Then

dy d du d
= f ( u) = f ( u ) = g 0 ( x) f 0 ( u )
dx dx dx du
0 0
= g (x) f (g (x)) :

Symbolically, we write this as

dy du dy
=
dx dx du

provided u is a function of x alone.

2
Thus for y = e4x ; write u = 4x2; y = eu: Then
40
1. INTRODUCTION TO CALCULUS

dy du dy 4x 2
= = 8xe :
dx dx du

Further examples:

y = sin x3

y = sin u; where u = x3
y 0 = cos u:3x2 ! y 0 = 3x2 cos x3

y = tan2 x : this is how we write (tan x)2 so put

y = u2 where u = tan x
y 0 = 2u: sec2 x ! y 0 = 2 tan x sec2 x

y = ln sin x: Put u = sin x ! y = ln u

dy 1 du
= ; = cos x
du u dx
hence y 0 = cot x:

41
1. INTRODUCTION TO CALCULUS

1.5.4 Quotient Rule

f ( x)
If y = then
g ( x)

dy g ( x ) f 0 ( x ) f ( x) g 0 ( x )
= 2
:
dx (g (x))

Thus if y = e3x=x2;

dy x23e3x 2xe3x 3x 2 3x
= = e :
dx x4 x3

This is a combination of the product rule and the function

of a function (or chain) rule. It is very simple to derive:

f ( x)
Staring with y = and writing as y = f (x) (g (x)) 1
g ( x)
we apply the product rule

42
1. INTRODUCTION TO CALCULUS

dy df 1 d
= (g (x)) + f (x) (g (x)) 1
dx dx dx

Now use the chain rule on (g (x)) 1 ; i.e. write u =

g (x) so

d 1 du d
(g (x)) = u 1 = g 0 ( x) u 2
dx dx du
g 0 ( x)
= 2
:
g ( x)

Then

dy 1 df g 0 ( x) f 0 ( x) f ( x) g 0 ( x)
= f ( x) 2
= 2
:
dx g (x) dx g ( x) g ( x) g ( x)

To simplify we note that the common denominator is

g (x)2 hence

43
1. INTRODUCTION TO CALCULUS

dy g ( x ) f 0 ( x ) f ( x) g 0 ( x )
= 2
:
dx g ( x)

Examples:

d x d x x d
(xe ) = x (e ) + e ( x)
dx dx dx
= xex + ex = ex (x + 1) ;

d x x (ex)0 ex (x)0 xex ex

(e =x) = 2
=
dx ( x) x2
ex
= 2 (x 1) ;
x

d 2 d u
e x = (e ) where u = x2 ) du = 2xdx
dx dx
2
= ( 2 x) e x :

44
1. INTRODUCTION TO CALCULUS

1.5.5 Implicit Di¤erentiation

Consider the function

y = ax
where a is a constant. If we take natural log of both
sides
ln y = x ln a
and now di¤erentiate both sides by applying the chain
rule to the left hand side
1 dy
= ln a
y dx
dy
= y ln a
dx
and replace y by ax to give
dy
= ax ln a:
dx
This is an example of implicit di¤erentiation.

We could have obtained the same solution by initially

45
1. INTRODUCTION TO CALCULUS

writing ax as a combination of a log and exp

y = exp (ln ax) = exp (x ln a)

d d
y0 = ex ln a = ex ln a (x ln a)
dx dx
x
= a ln a:
Consider the earlier implicit function given by

4y 4 2 y 2 x2 yx2 + x2 + 3 = 0:
The resulting derivative will also be an implicit function.
Di¤erentiating gives

16y 3y 0 2 2yy 0x2 + 2y 2x y 0x2 + 2xy = 2x

16y 3 4yx2 x2 y 0 = 2x + 4y 2x + 2xy

2x + 4y 2x + 2xy
y0 =
16y 3 4yx2 x2

46
1. INTRODUCTION TO CALCULUS

1.5.6 Alternative Proof of the Product Rule

The proof of this rule can be fairly rigorous. However

we can present a fairly simple working using the log of
a function to obtain the proof of the product rule: Start
with
y = f ( x ) g ( x)
and now take log of both sides
log y = log f (x) + log g (x)
di¤erentiating implicity gives
1 dy 1 df 1 dg
= +
y dx f dx g dx
gdf + f dg
=
f gdx
taking y = f (x) g (x) across gives
!
dy gdf + f dg
= fg
dx f gdx
gdf + f dg
=
dx
df dg
= g +f :
dx dx

47
1. INTRODUCTION TO CALCULUS

1.5.7 Higher Derivatives

These are de…ned recursively;

d2f d df
f 00 (x)
= =
dx2 dx dx !
d 3f d d2f
000
f ( x) = =
dx3 dx dx2

and so on. For example:

f ( x) = 4x 3
0
f (x) = 12x2 ! f 00 (x) = 24x
f 000 (x) = 24 ! f (iv) (x) = 0:
so for any nth degree polynomial

f (x) = anxn + an 1xn 1 + ::::::: + a1x + a0

we have f (n+1) (x) = 0:

48
1. INTRODUCTION TO CALCULUS

Consider another example

f (x) = ex
f 0 (x) = ex ! f 00 (x) = ex
...
f (n) (x) = ex = f (x)

f (x) = log x
f 0 (x) = 1=x
f 00 (x) = 1=x2
f 000 (x) = 2=x3:

Warning

Not all functions are di¤erentiable everywhere. For ex-

ample, 1=x has the derivative 1=x2 but only for
x 6= 0:

49
1. INTRODUCTION TO CALCULUS

1
Easy way is to "look for a hole", e.g. f (x) =
x 2
does not exist at x = 2:

x = 2 is called a singularity for this function. We say

f (x) is singular at the point x = 2:

50
1. INTRODUCTION TO CALCULUS

1.5.8 Further Limits

This will be an application of di¤erentiation. Consider

the limiting case
f ( x) 0
lim
x!a g (x) 0
This is called an indeterminate form. Then L’Hospitals
rule states
f ( x) f 0 ( x) f (r) (x)
lim = lim 0 = ::::::: = lim (r)
x!a g (x) x!a g (x) x!a g ( x)
for r such that we have the indeterminate form 0=0: If
for r + 1 we have
f (r+1) (x)
lim !A
x!a g (r+1) (x)

where A is not of the form 0=0 then

f ( x) f (r+1) (x)
lim lim :
x!a g (x) x!a g (r+1) (x)

Note: Very important to verify quotient has this indeter-

minate form before using L’Hospitals rule. Else we end
up with an incorrect solution. We can also use this rule
1
for the form :
1
51
1. INTRODUCTION TO CALCULUS

Examples:

1.
cos x + 2x 1 0
lim
x!0 3x 0
So di¤erentiate both numerator and denominator !
d (cos x + 2x 1) sin x + 2 0 2
lim dx d (3x)
= lim 6= !
x!0 x!0 3 0 3
dx

ex + e x 2
2. lim ; quotient has form 0=0: By L’
x!0 1 cos 2x
ex e x
Hospital’s rule we have lim ; which has
x!0 2 sin 2x
indeterminate form 0=0 again for 2nd time, so we
apply L’Hospital’s rule again
ex + e x 1
lim = :
x!0 4 cos 2x 2

x2 1 2x
3. lim ) use L’Hospital , so lim !
x!1 ln x 1 x!1 1=x
1
52
1. INTRODUCTION TO CALCULUS

e3x 1
4. lim ) lim 3xe3x ! 1
x!1 ln x 1 x!1

5. lim x2e 3x 0:1; so we convert to form 1=1

x!1
x2
by writing lim 3x ; and now use L’Hospital (di¤er-
x!1 e
2
entiate twice), which gives lim !0
x!1 9e3x

sin x
6. lim lim cos x 1
x!0 x x!0

What is example 6: saying?

When x is very close to 0 then sin x x: That is sin x

can be approximated with the function x for small values.

53
1. INTRODUCTION TO CALCULUS

1.6 Taylor Series

Many functions are so complicated that it is not easy to

see what they look like. If we only want to know what a
function looks like locally , we can approximate it by sim-
pler functions: polynomials. The crudest approximation
is by a constant: if f (x) is continuous at x0;

f ( x) t f ( x0 )

for x near x0:

Before we consider this in a more formal manner we start

by looking at a simple motivating example:

Consider f (x) = ex:

Suppose we wish to approximate this function for very

small values of x (i.e. x ! 0). We know at x = 0;
df
dx = 1: So this is the gradient at x = 0: We can …nd the
54
1. INTRODUCTION TO CALCULUS

equation of the line that passes through a point (x0; y0)

using
y y0 = m (x x0 ) :
df
Here m = dx = 1; x0 = 0; y0 = 1; so y = 1 + x; is a
polynomial. What information have we ascertained from
this?

If x ! 0 then the point (x; 1 + x) on the tangent is

close to the point (x; ex) on the graph f (x) and hence

ex 1+x

55
1. INTRODUCTION TO CALCULUS

25

20

15

10

0
-4 -3 -2 -1 0 1 2 3 4

-5

56
1. INTRODUCTION TO CALCULUS

Suppose now that we are not that close to 0: We look

for a second degree polynomial (i.e. quadratic)

g (x) = ax2 + bx + c ! g 0 = 2ax + b ! g 00 = 2a

If we want this parabola g (x) to have

(i) same y intercept as f :

g (0) = f (0) =) c = 1

(ii) same tangent as f

g 0 (0) = f 0 (0) =) b = 1

(iii) same curvature as f

g 00 (0) = f 00 (0) =) 2a = 1

This gives
1
ex g ( x ) = x2 + x + 1
2

57
1. INTRODUCTION TO CALCULUS

25

20

15

10

0
-4 -3 -2 -1 0 1 2 3 4

58
1. INTRODUCTION TO CALCULUS

Moving further away we would look at a third order poly-

nomial h (x) which gives
1 3 1
ex h ( x) = x + x2 + x + 1
3! 2!

25

20

15

10

0
-4 -3 -2 -1 0 1 2 3 4

-5

59
1. INTRODUCTION TO CALCULUS

and so on.

Better is to approximate by the tangent at x0: This

makes the approximation and its derivative agree with
the function:

f (x) t f (x0) + (x x0 ) f 0 ( x0 ) :

Better still is by the best …t parabola (quadratic), which

makes the …rst two derivatives agree:

1
f (x) t f (x0)+(x 0
x0) f (x0)+ (x x0)2 f 00 (x0) :
2

This process can be continued inde…nitely as long as f

can be di¤erentiated often enough.

The nth term is

60
1. INTRODUCTION TO CALCULUS

1 (n)
f ( x0 ) ( x x0 ) n ;
n!

where f (n) means the nth derivative of f and n! =

n: (n 1) : : : 2:1 is the factorial.

x0 = 0 is the special case, called Maclaurin Series.

Examples:

Expanding about the origin x0 = 0;

x2 x3 xn
ex =1+x+ + + ::: +
2! 3! n!

Near 0; the logarithm looks like

x2 x3 x4 n x
n+1
log (1 + x) = x + + ::: + ( 1)
2 3 4 (n + 1)!

61
1. INTRODUCTION TO CALCULUS

How can we obtain this? Put f (x) = log (1 + x) ; then

f (0) = 0

1
f 0 (x) = 1+x f 0 (0) = 1
f 00 (x) = 1 f 00 (0) = 1
(1+x)2
f 000 (x) = 2 f 000 (0) = 2
(1+x)3
f (4) (x) = 6 f (4) (0) = 6
(1+x)4

Thus
1 (n)
X f (0) n
f ( x) = x
n=0 n!
1 ( 1) 2 1 ( 6) 4
= 0+ x+ x + : 2 x3 + x + :::::
1! 2! 3! 4!
x2 x3 x4
= x + + :::
2 3 4

62
1. INTRODUCTION TO CALCULUS

Taylor’s theorem, in general, is this : If f (x) and its

…rst n derivatives exist (and are continuous) on some
interval containing the point x0 then

f (x) = f (x0) + 1! 1 f 0 (x ) ( x x0 ) +
0
1 f 00 (x ) (x 2
2! 0 x 0 ) + :::
+ (n 1 1)! f (n 1) (x0) (x x0 ) n 1 + R n ( x )

where Rn (x) = (1=n!) f (n) ( ) (x x0)n ; is some

(usually unknown) number between x0 and x and f (n)
is the nth derivative of f .

We can expand about any point x = a; and shift this

point to the origin, i.e. x x0 0 and we express in
powers of (x x0)n :

So for f (x) = sin x about x = =4 we will have

1 f (n)
X 4
f ( x) = (x =4)n
n=0 n!
63
1. INTRODUCTION TO CALCULUS

where f (n) 4 is the nth derivative of sin x at x0 =

=4:

As another example suppose we wish to expand log (1 + x)

about x0 = 2; i.e. x 2 = 0 then
1
X 1
f ( x) = f (n) (2) (x 2)n
n=0 n!

where f (n) (2) is the nth derivative of log (1 + x) eval-

uated at the point x = 2:

Note that log (1 + x) does not exist for x = 1:

64
1. INTRODUCTION TO CALCULUS

1.6.1 The Binomial Expansion

The Binomial Expansion is the Taylor expansion of

(1 + x)n where n is a positive integer. It reads:

n n (n 1) 2 n (n 1) (n 2) 3
(1 + x) = 1+nx+ x + x +::: :
2! 3!

We can extend this to expressions of the form

n(n 1) 2 n(n 1)(n 2) 3

(1 + ax)n = 1+n (ax)+ 2! ( ax ) + 3! ( ax ) +:::

" !#n " ! #

n a a
(p + ax) = p 1 + x = pn 1+n x + ::::::::
p p

The binomial coe¢ cients are found in Pascal’s triangle:

65
1. INTRODUCTION TO CALCULUS

1 (n=0) (1 + x)0

1 1 (n=1) (1 + x)1

1 2 1 (n=2) (1 + x)2

1 3 3 1 (n=3) (1 + x)3

1 4 6 4 1 (n=4) (1 + x)4

1 5 10 10 5 1 (n=5) (1 + x)5

and so on ...

66
1. INTRODUCTION TO CALCULUS

As an example consider:

(1 + x)3 n = 3 ) 1 3 3 1 ) (1 + x)3 =
1 + 3x + 3 x 2 + x 3

(1 + x)5 n = 5 ! (1 + x)5 = 1 + 5x + 10x2 +

10x3 + 5x4 + x5:

If n is not an integer the theorem still holds but the

coe¢ cients are no longer integers. For example,

(1 + x) 1 = 1 x + x2 x3 + ::: :

and

1=2 1 1 1 x2
(1 + x) =1+ x+ ::: :
2 2 2 2!

67
1. INTRODUCTION TO CALCULUS

sin x
Example: We looked at lim ! 1 (by L’Hospital).
x!0 x
We can also do this using Taylor series:
sin x x3=3! + x5=5! + ::::
x
lim lim
x!0 x x!0 x
lim 1 x2=3! + x4=5! + ::::
x!0
! 1:

68
1. INTRODUCTION TO CALCULUS

1.7 Integration

1.7.1 The Inde…nite Integral

The inde…nite integral of f (x) ;

Z
f (x) dx;

is any function F (x) whose derivative equals f (x).

Thus if

Z
dF
F ( x) = f (x) dx then ( x) = f ( x ) :
dx

Since the derivative of a constant, C; is zero (dC=dx = 0) ;

the inde…nite integral of f (x) is only determined up to
an arbitrary constant;

69
1. INTRODUCTION TO CALCULUS

dF
if = f (x) then
dx

d dF dC dF
( F ( x) + C ) = ( x) + = ( x) = f ( x) :
dx dx dx dx

Thus we must always include an arbitrary constant of

integration in an inde…nite integral.

Simple examples are

Z
1
xndx = xn+1 + C (n 6= 1) ;
n+1
Z
dx
= log (x) + C;
Z x
ax 1 ax
e dx = e + C (a 6= 0) ;
Z a
1
cos axdx = sin ax + C
Z a
1
sin axdx = cos ax + C
a

70
1. INTRODUCTION TO CALCULUS

Linearity

Integration is linear:

Z Z Z
( f (x) + g (x)) dx = f (x) dx + g (x) dx

for constants A and B: Thus, for example

Z Z Z
Ax2 + Bx3 dx = A x2dx + B x3dx
A 3 B 4
= x + x + C;
3 4

Z Z Z
dx
(3ex + 2=x) dx = 3 exdx+2 = 3ex+2 log (x)+C;
x

and so forth.

71
1. INTRODUCTION TO CALCULUS

1.7.2 The De…nite Integral

The de…nite integral,

Z b
f (x) dx;
a

is the area under the graph of f (x) ; between x = a

and x = b; with positive values of f (x) giving posi-
tive area and negative values of f (x) contributing neg-
ative area. It can be computed if the inde…nite integral
is known. For example

Z 3
1 4 3 1 4
x3dx = x = 3 14 = 20;
1 4 1 4

Z 1
exdx = [ex]1 1 = e 1=e:
1

72
1. INTRODUCTION TO CALCULUS

Note that the de…nite integral is also linear in the sense

that

Z b Z b Z b
(Af (x) + Bg (x)) dx = A f (x) dx+B g (x) dx:
a a a

Note also that a de…nite integral

Z b
f (x) dx
a

does not depend on the variable of integration, x in the

above, it only depends on the function f and the limits
of integration (a and b in this case); the area under a
curve does not depend on what we choose to call the
horizontal axis.

So

Z b Z b Z b
f (x) dx = f (y ) dy = f (z ) dz:
a a a
73
1. INTRODUCTION TO CALCULUS

We should never confuse the variable of integration with

the limits of integration; a de…nite integral of the form

Z x
f (x) dx
a

is at best potentially confusing and at worst meaningless.

74
1. INTRODUCTION TO CALCULUS

Consider the following

Z x
2 s 2
erf (x) = p e ds;
0
Z 1
2 s 2
erf c (x) = p e ds;
x
Z x
1 2
N ( x) = p e s =2ds:
2 1

75
1. INTRODUCTION TO CALCULUS

Note that, by de…nition, if a < b < c then

Z c Z b Z c
f (x) dx = f (x) dx + f (x) dx:
a a b

By convention (which is not unreasonable if we think of

a de…nite integral in terms of an area)

Z a Z c
f (x) dx = f (x) dx:
c a

With this convention we …nd that for any a; b and c

Z c Z b Z c
f (x) dx = f (x) dx + f (x) dx:
a a b

76
1. INTRODUCTION TO CALCULUS

1.7.3 Integration by Substitution

This involves the change of variable and used to evaluate

integrals of the form

Z
g (f (x)) f 0 (x) dx;

and can be evaluated by writing z = f (x) so that

dz=dx = f 0 (x) or dz = f 0 (x) dx: Then the integral
becomes

Z
g (z ) dz:

For example:

Z Z
x 1 dz
dx =
1 + x2 2 z
1 1
= log (z ) + C = log 1 + x2 + C
2 q 2
= log 1 + x2 + C
77
1. INTRODUCTION TO CALCULUS

if we put z = 1 + x2 so dz = 2xdx:

Similarly:

Z Z
x2 1
xe dx = ez dz
2
1 z 1 x2
= e +C = e +C
2 2

this time with z = x2 so dz = 2xdx;

Z Z
1 1
log (x) dx = z dz = z 2 + C
x 2
1
= (log (x))2 + C
2

with z = log (x) so dz = dx=x and

Z Z Z
ex+ex dx = ex
exe dx = ez dz
x
= ez +C = ee +C
78
1. INTRODUCTION TO CALCULUS

with z = ex so dz = exdx:

The method can be used for de…nite integrals too. In this

case it is usually more convenient to change the limits of
integration at the same time as changing the variable;
this is not strictly necessary, but it can save a lot of time.

For example, consider

Z 2
x 2
e 2xdx:
1

Write z = x2; so dz = 2xdx: Now consider the limits

of integration; when x = 2; z = x2 = 4 and when
x = 1; z = x2 = 1: Thus

Z x=2 Z z=4
x2
e 2xdx = ez dz
x=1 z=1
= [ez ]z=4
z=1 = e4 e1:

79
1. INTRODUCTION TO CALCULUS

Further examples: consider

Z x=2
2xdx
2
:
x=1 1 + x

In this case we could write z = 1 + x2; so dz = 2xdx

and x = 1 corresponds to z = 2, x = 2 corresponds
to z = 5; and

Z x=2 z=5 dz Z
2x
dx =
x=1 1 + x2 z=2 z
= [ln (z )]z=5
z=2 = log (5) ln (2)
= ln (5=2)
We can solve the same problem without change of limit,
i.e.
n ox=2
ln 1 + x2 ! ln 5 ln 2 = ln 5=2:
x=1

80
1. INTRODUCTION TO CALCULUS

Or consider

Z x=e
log (x)
2 dx
x=1 x

in which case we should choose z = log (x) so dz =

dx=x and x = 1 gives z = 0; x = e gives z = 1
and so

Z x=e Z z=1 h iz=1

log (x) 2
2 dx = 2zdz = z = 1:
x=1 x z=0 z=0

81
1. INTRODUCTION TO CALCULUS

When we make a substitution like z = f (x) we are

implicitly assuming that dz=dx = f 0 (x) is neither
in…nite nor zero. It is important to remember this implicit
assumption.

Consider the integral

Z 1
1 h 3ix=1 1 2
x2dx = x = (1 ( 1)) = :
1 3 x= 1 3 3

p
Now put z = x2 so dz = 2xdx or dz = 2 z dx
and when x = 1; z = x2 = 1 and when x = 1;
z = x2 = 1; so

Z x=1 Z
2 1 z=1 dz
x dx = p =0
x= 1 2 z=1 z

p
as the area under the curve 1= z between z = 1 and
z = 1 is obviously zero.

82
1. INTRODUCTION TO CALCULUS

It is clear that x2 > 0 except at x = 0 and therefore

that

Z 1
2
x2dx =
1 3

must be the correct answer. The substitution z = x2

gave

Z x=1 Z
2 1 z=1 dz
x dx = p =0
x= 1 2 z=1 z

which is obviously wrong. So why did the substitution

fail?

It failed because f 0 (x) = dz=dx = 2x changed signs

between x = 1 and x = 1: In particular, dz=dx = 0
at x = 0; the function z = x2 is not invertible for
1 x 1:

83
1. INTRODUCTION TO CALCULUS

Moral: when making a substitution make sure that dz=dx 6=

0:

Earlier we saw the de…nition of the CDF for the Normal

Distribution
Z x
1 2
N ( x) = p e s =2ds
2 1
If x ! 1 then we know (by the fact that the area
under a PDF has to sum to unity) that
1 Z
1 s 2 =2
p e ds = 1:
2 1
This can be used to obtain an important result. First we
p p
make the substitution x = s= 2 to give dx = ds= 2;
hence the integral becomes
p Z1 x 2 p
2 e dx = 2
1
and hence we obtain
Z 1
2 p
e x dx = =)
1
Z 1 p
2
e x dx = :
0 2

84
1. INTRODUCTION TO CALCULUS

1.7.4 Integration by Parts

This is based on the product rule. In usual notation, if

y = u (x) v (x) then

dy du dv
= v+u
dx dx dx

so that

du dy dv
v= u
dx dx dx

and hence integrating

Z Z Z Z
du dy dv dv
vdx = dx u dx = y (x) u dx+C
dx dx dx dx

or
Z Z
du dv
vdx = u (x) v (x) u ( x) dx + C
dx dx
85
1. INTRODUCTION TO CALCULUS

i.e.
Z Z
u0vdx = uv uv 0dx + C

This is useful, for instance, if v (x) is a polynomial and

u (x) is an exponential.

How can we use this formula? Consider the example

Z
xexdx
Put
v = x u0 = ex
v 0 = 1 u = ex
hence
Z Z
dv
xexdx = uv u dx
Z dx
= xex ex:1dx = ex (x 1) + C
The formula we are using is the same as
Z Z
vdu = uv udv + C
R
Now using the same example xexdx
v = x du = exdx
dv = dx u = ex
86
1. INTRODUCTION TO CALCULUS

and
Z Z Z
vdu = uv udv = xex exdx
= ex (x 1) + C

Another example

Z Z
x 2 e2x dx = 1 x2 e2x xe 2x dx + C
|{z} |{z} | {z }
v(x) u0 |2 {z } uv 0
uv

and using integration by parts again

Z Z
2x 1 2x 1 1
xe dx = xe e2xdx = (2x 1) e2x + D
2 2 4

so

Z
1
x2e2xdx = 2 x2 2x + 1 e2x + E:
4
87
1. INTRODUCTION TO CALCULUS

Important Example:
Z
ex cos xdx
R x
so set I = e cos xdx: Now put
v = ex u0 = cos x
v 0 = ex u = sin x
which gives
Z
I = ex sin x ex sin xdx
R x
need to obtain e sin xdx for a second time by parts so
put
v = ex u0 = sin x
v 0 = ex u = cos x
and we have
Z Z
ex sin xdx = ex cos x + ex cos xdx
| {z }
I
so putting together with the earlier integral
I = ex sin x ( ex cos x + I )
2I = ex (sin x + cos x)
hence
Z
ex
ex cos xdx = (sin x + cos x) + C
2
88
1. INTRODUCTION TO CALCULUS

1.7.5 Other Results

Z
f 0 ( x)
dx = ln jf (x)j + C
f ( x)
e.g.
Z
3
dx = ln j1 + 3xj + C
1 + 3x

Z Z
1 1 7 1
dx = dx = ln j2 + 7xj + C
2 + 7x 7 2 + 7x 7
This allows us to state a standard result
Z
1 1
dx = ln ja + bxj + C
a + bx b

How can we re-do the earlier example

Z
x
2
dx;
1+x
which was initially treated by substitution? We note that
we can write this integral as
Z
1 2x 1 2 +C
dx = ln 1 + x
2 1 + x2 2r
= ln 1 + x2 + C
89
1. INTRODUCTION TO CALCULUS

1.7.6 Partial Fractions

Consider a fraction where both numerator and denomi-

nator are polynomial functions, i.e.
P
N
a n xn
f ( x) n=0
h ( x) =
g ( x) P
M
bnxn
n=0

where deg f (x) < deg g (x) , i.e. N < M: Then h (x)
is called a partial fraction. Suppose
c A B
+
(x + a) (x + b) (x + a) (x + b)
then writing

c = A (x + b) + B (x + a)
and solving for A and B allows us to obtain partial
fractions.

90
1. INTRODUCTION TO CALCULUS

The simplest way to achieve this is by setting x = b to

obtain the value of B; then putting x = a yields A:

1
Example: : Now write
(x 2) (x + 3)
1 A B
+
(x 2) (x + 3) x 2 x+3
which becomes

1 = A (x + 3) + B (x 2)

Setting x = 3!B= 1=5; x = 2 ! A = 1=5:

So
1 1 1
:
(x 2) (x + 3) 5 (x 2) 5 (x + 3)

91
1. INTRODUCTION TO CALCULUS

There is another quicker and simpler method to obtain

partial fractions, called the "cover-up" rule. As an ex-
ample consider
x A B
+ :
(x 2) (x + 3) x 2 x+3

A
Firstly, look at the term : The denominator van-
x 2
ishes for x = 2; so take the expression on the LHS and
"cover-up" (x 2) : Now evaluate the remaining ex-
x
pression, i.e. for x = 2; which gives 2=5: So
(x + 3)
A = 2 =5:

B
Now repeat this, by noting that does not exist at
x+3
x = 3: So cover up (x + 3) on the LHS and evaluate
x
for x = 3; which gives B = 3=5:
(x 2)

92
1. INTRODUCTION TO CALCULUS

f ( x)
Any rational expression (with degree of f (x) <
g ( x)
degree of g (x)) such as above can be written
f ( x)
F1 + F2 + :::::::: + Fk
g ( x)
where each Fi has form
A Cx + D
or n
(px + q )m ax2 + bx + c
A
where is written as
(px + q )m
A1 A2 A
+ + :::::: +
(px + q ) (px + q )2 (px + q )m
Cx + D
and n becomes
ax2 + bx + c

C1x + D1 Cnx + Dn
+ :::::: + n
ax2 + bx + c 2
ax + bx + c

93
1. INTRODUCTION TO CALCULUS

Examples:

3x 2 A B C
+ + +
(4x 3) (2x + 5)3 4x 3 2x + 5 (2x + 5)2
D
(2x + 5)3

4x2 + 13x 9 A B C
+ +
x (x + 3) (x 1) x x + 3 (x 1)

3 x3 18x2 + 29x 4 A B C
+ + +
(x + 1) (x 2)3 x + 1 x 2 (x 2) 2
D
(x 2)3

5 x2 x+2 Ax + B Cx + D
2 2
+ 2
+
x2 + 2x + 4 (x 1) x + 2 x + 4 x2 + 2 x + 4
E
x 1

x2 x 21 Ax + B Cx + D E
2
+ 2
+
x2 + 4 (2x 1) x2 + 4 x2 + 4 2x 1
94
1. INTRODUCTION TO CALCULUS

1.8 Complex Numbers

A complex number z is de…ned by z = x + iy where

p
x; y 2 R and i = 1: It follows that i2 = 1:

A complex number z may also be expressed in polar

co-ordinate form as

z = r (cos + i sin )
where r is always positive and counter-clockwise from
Ox: So x = r cos ; y = r sin

x z = x+iy

r
y

So
q
y
x = r cos ; y = r sin ; r = + x2 + y 2; = arctan
x
95
1. INTRODUCTION TO CALCULUS

We call the x axis the real line and the y axis the
imaginary line.

The set of all complex numbers is denoted C; and for

any complex number z we write z 2 C: We can think
of R C:

96
1. INTRODUCTION TO CALCULUS

1.8.1 Arithmetic

Given any two complex numbers z1 = a + ib; z2 =

c + id the following de…nitions hold:

Addition & Subtraction z1 z2 = ( a c)+i (b d)

Multiplication z1 z2 = (ac bd) + i (ad + bc)

z1 a + ib (ac + bd) + i (bc ad)

Division = =
z2 c + id c2 + d 2

c id
here we have simply multiplied by and note that
c id
2
(c + id) (c id) = c + d 2

Examples

z1 = 1 + 2i; z2 = 3 i

z1 + z2 = (1 + 3) + i (2 1) = 4 + i ; z1 + z2 =
(1 3) + i (2 ( 1)) = 2 + 3i
97
1. INTRODUCTION TO CALCULUS

z1 z2 = (1:3 2: 1) + i (1: 1 + 2:3) = 5 + 5i

z1 1 + 2i 3 + i 1 + 7i
= : =
z2 3 i 3+i 10

1.8.2 Modulus and Argument

Given z = x + iy; the

q modulus of z denoted jzj is
de…ned jzj = r = + x2 + y 2 (as given earlier). So
we are using Pythagoras to calculate the length of point
joining the origin to the point z (x; y ) : As an example,
consider the complex number z = 3 + 5i; which is
represented in the x y plane (…rst quadrant) by
the point (3; 5) : The modulus can be calculated from
p p
2 2
j3 + 5ij = 3 + 5 ; to give r = 34:

98
1. INTRODUCTION TO CALCULUS

(3, 5)

1.8.3 Complex Conjugate

_
We de…ne complex conjugate of z by z where
_
z =x iy:
z is the re‡ection of z in the real line. So for example
if z = 1 2i; then z = 1 + 2i

_
1. z = z

_
2. (z1 + z2) = z1 + z 2

_ _
3. (z1z2) = z1z2

99
1. INTRODUCTION TO CALCULUS

_
_ z+z
4. z + z = 2x = 2 Re z ) Re z =
2

_
_ z z
5. z z = 2iy = 2i Im z ) Im z =
2i

_
6. z: z = (x + iy ) (x iy ) = jzj2

7. jzj2 = z (z ) = zz = jzj2 ) jzj = jzj

z1 z1 z 2 z1 z 2
8. = : =
z2 z2 z 2 jz 2j2

9. jz1z2j2 = jz1j2 jz2j2

1.8.4 Polar Form:

We return to the polar form representation of complex

numbers. We now introduce a new notation. If z 2 C;
then
z = r (cos + i sin ) = rei :
100
1. INTRODUCTION TO CALCULUS

Hence
ei = cos + i sin ;
which is a special relationship called Eulers Identity.
Knowing sin is an odd function gives e i = cos
i sin : Referring to the earlier …gure, we have:

jzj = r; arg z =
If
z1 = r1ei 1 and z2 = r2ei 2

then

z1z2 = r1r2ei( 1+ 2) ) jz1z2j = r1r2 = jz1j jz2j

arg (z1z2) = 1 + 2 = arg (z1) + arg (z2) :
If z2 6= 0 then
z1 r1ei 1 r1 i( 1 2)
= = e
z2 r2ei 2 r2
and hence
z1 jz1j r
= = 1
z2 jz2j r2
!
z1
arg = 1 2 = arg (z1 ) arg (z2)
z2
101
1. INTRODUCTION TO CALCULUS

Eulers Formula: Let be any angle, then

exp (i ) = cos + i sin :

We can prove this by considering the Taylor series for
exp (x) ; sin x; cos x

x2 x3 xn
ex =1+x+ + + ::::::::::::: + (a)
2! 3! n!

x3 x5 x2n+1 n
sin x = x + ::::::::::::: +( 1) (b)
3! 5! (2n + 1)!

x2 x4 n x
2n
cos x = 1 + ::::::::::::: + ( 1) (c)
2! 4! (2n)!
Replacing x by the purely imaginary quantity i in
(a), we obtain

(i )2 (i )3 (i )n
ei = 1+i + + + ::::::::::::: +
2! 3! ! n!
2 4 6
= 1 + + :::::::::::: +
2! 4! 6!
!
3 5
i + :::::::::
3! 5!
= cos + i sin

102
1. INTRODUCTION TO CALCULUS

Note: When = then exp i = 1 and = =2

gives exp (i =2) = i:

103
1. INTRODUCTION TO CALCULUS

1.9 Functions of Several Variables

A function can depend on more than one variable. For

example, the value of an option depends on the underly-
ing asset price S (for ’spot’or ’share’) and time t: We
can write its value as V (S; t) :

The value also depends on other parameters such as the

exercise price E; interest rate r and so on. Although
we could write V (S; t; E; r; :::) ; it is usually clearer to
leave these other variables out.

Depending on the application, the independent variables

may be x and t for space and time, or two space
variables x and y; or S and t for price and time, and
so on.

104
1. INTRODUCTION TO CALCULUS

1.9.1 Partial Derivatives

Consider a function z = f (x; y ) ; which can be thought

of as a surface in x; y; z space. We can think of
x and y as positions on a two dimensional grid (or as
spacial variables) and z as the height of a surface above
the (x; y ) grid.

How do we di¤erentiate a function f (x; y ) of two vari-

ables? What if there are more independent variables?

The partial derivative of f (x; y ) with respect to x is

written

@f
@x

(note @ and not d ). It is the x derivative of f with

y held …xed:

@f f (x + x; y ) f (x; y )
= lim :
@x x!0 x
105
1. INTRODUCTION TO CALCULUS

The other partial derivative, @f =@y; is de…ned similarly

but now x is held …xed:

@f f (x; y + y ) f (x; y )
= lim :
@y y!0 y

@f @f
and
@x @y

are sometimes written as fx and fy :

Examples

If
2
f (x; t) = x + t2 + xe t

then
@f t2
= fx = 1 + 0 + 1 e
@x

106
1. INTRODUCTION TO CALCULUS

@f t2
= ft = 0 + 2t + x ( 2t) e :
@t

The convention is, treat the other variable like a constant.

Let z = x3y 2 + sin xy then

zx = 3x2y 2 + y cos xy; zy = 2x3y + x cos xy

107
1. INTRODUCTION TO CALCULUS

1.9.2 Higher Derivatives

Like ordinary derivatives, these are de…ned recursively:

@ 2f @ @f
= fxx = ;
@x2 @x @x
@ 2f @ @f
= fxy = ;
@x@y @y @x
!
@ 2f @ @f
= fyx = ;
@y@x @x @y

and

!
@ 2f @ @f
2
= fyy = :
@y @y @y

108
1. INTRODUCTION TO CALCULUS

If f is well-behaved, the ’mixed’partial derivatives are

equal:

fxy = fyx:

i.e. the second order derivatives exist and are continuous.

In the previous example we have

zxx = 6xy 2 y 2 sin xy; zyy = 2x3 x2 sin xy;

zxy = 6x2y + cos xy xy sin xy = zyx:

109
1. INTRODUCTION TO CALCULUS

Examples:
2
With f (x; t) = x + t2 + xe t as above,

t2
fx = 1+e

so

2
fxx = 0; fxt = 2te t

Also

t2
ft = 2 t 2xte

so

t2 t2 2 t2
ftx = 2te ; ftt = 2 2xe + 4xt e

Note that fxt = ftx:

110
1. INTRODUCTION TO CALCULUS

1.9.3 The Chain Rule I

Suppose that x = x (s) and y = y (s) and F (s) =

f (x (s) ; y (s)) : Then

dF dx @f dy @f
(s ) = (s ) (x (s) ; y (s))+ (s) (x (s) ; y (s))
ds ds @x ds @y

Thus if f (x; y ) = x2+y 2 and x (s) = cos (s) ; y (s) =

sin (s) we …nd that F (s) = f (x (s) ; y (s)) has deriv-
ative

dF
= sin (s) 2 cos (s) + cos (s) 2 sin (s) = 0
ds

which is what it should be, since F (s) = cos2 (s) +

sin2 (s) = 1;

i.e. a constant.

111
1. INTRODUCTION TO CALCULUS

dz
Example: Calculate at t = =2 where
dt
z = exp xy 2 x = t cos t; y = t sin t:
Chain rule gives
dz @z dx @z dy
= +
dt @x dt @y dt
= y 2 exp xy 2 ( t sin t + cos t) +
2xy exp xy 2 (sin t + t cos t) :

dz 3
At t = =2 x = 0; y = =2 ) = :
dt t= =2 8

112
1. INTRODUCTION TO CALCULUS

1.9.4 The Chain Rule II

Suppose that x = x (u; v ) ; y = y (u; v ) and that

F (u; v ) = f (x (u; v ) ; y (u; v )) : Then

@F @x @f @y @f @F @x @f @y @f
= + and = + :
@u @u @x @u @y @v @v @x @v @y

This is sometimes written as

@ @x @ @y @ @ @x @ @y @
= + ; = + :
@u @u @x @u @y @v @v @x @v @y

so is essentially a di¤erential operator.

113
1. INTRODUCTION TO CALCULUS

Example:

T = x3 xy + y 3 where x = r cos ; y = r sin

@T @T @x @T @y
= + = cos 3x2 y + sin 3y 2 x
@r @x @r @y @r
= cos 3r2 cos2 r sin +
sin 3r2 sin2 r cos
= 3r2 cos3 + sin3 2r cos sin
= 3r2 cos3 + sin3 r sin 2 :

@T @T @x @T @y
= +
@ @x @ @y @
= r sin 3x2 y + r cos 3y 2 x
= r sin 3r2 cos2 r sin +
r cos 3r2 sin2 r cos
= 3r3 cos sin (sin cos ) +
r2 sin2 cos2 :
= r2 (sin cos ) (3r cos sin + sin + cos )

114
1. INTRODUCTION TO CALCULUS

1.9.5 Extensions

If x = x (u; v; w) ; y = y (u; v; w) and F (u; v; w) =

f (x (u; v; w) ; y (u; v; w)) then

@F @x @f @y @f
= +
@u @u @x @u @y
@F @x @f @y @f
= +
@v @v @x @v @y
@F @x @f @y @f
= +
@w @w @x @w @y

If x = x (u; v ) ; y = y (u; v ) ; z = z (u; v ) and

F (u; v ) = f (x (u; v ) ; y (u; v ) ; z (u; v )) then

@F @x @f @y @f @z @f
= + +
@u @u @x @u @y @u @z
@F @x @f @y @f @z @f
= + +
@v @v @x @v @y @v @z
So we can generalise this to obtain a chain rule for
115
1. INTRODUCTION TO CALCULUS

F (x1; x2; :::::xm) =

!
X1 (x1; x2; :::::xm) ; X2 (x1; x2; :::::xm) ; :::::
f
Xn (x1; x2; :::::xm)
by the result
@F @X1 @f @X2 @f @Xn @f
= + + ::::::::: +
@x1 @x1 @X1 @x1 @X2 @x1 @Xn
@F @X1 @f @X2 @f @Xn @f
= + + ::::::::: +
@x2 @x2 @X1 @x2 @X2 @x2 @Xn
...
...
@F @X1 @f @X2 @f @Xn @f
= + + ::::::::: + :
@xm @xm @X1 @xm @X2 @xm @Xn
Naturally this can be written in a more compact (and
pedantic) way
Xn @X
@F j @f
= ; i = 1; :::; m
@xi j=1 @xi @Xj

116
1. INTRODUCTION TO CALCULUS

1.9.6 Taylor for two Variables

Assuming that a function f (x; t) is di¤erentiable enough,

near x = x0; t = t0;

f (x; t) = f (x0; t0) + (x x0) fx (x0; t0) +

(t t ) ft (x0; t0)
20 2 3
(x x0) fxx (x0; t0)
16 7
+ 4 +2 (x x0) (t t0) fxt (x0; t0) 5 + ::::
2
+ (t t0)2 ftt (x0; t0)

That is,

f (x; t) = constant + linear + quadratic

+::::

The error in truncating this series after the second or-

der terms tends to zero faster than the included terms.
This result is particularly important for Itô’s lemma in
Stochastic Calculus.
117
1. INTRODUCTION TO CALCULUS

Suppose a function f = f (x; y ) and both x; y change by

a small amount, so x ! x + x and y ! y + y; then
we can examine the change in f using a two dimensional
form of Taylor

f (x + x; y + y ) = f (x; y ) + fx x + fy y +
1 1
fxx x2 + fyy y 2 +
2 2
fxy x y + O x3; y 3 :

By taking f (x; y ) to the lhs, writing

df = f (x + x; y + y ) f (x; y )
and considering only linear terms, i.e.
@f @f
df =
x+ y
@x @y
we obtain a formula for the di¤erential or total change
in f:

118
1. INTRODUCTION TO CALCULUS

1.10 Special Functions

1.10.1 The Gamma Function

The Gamma Function (x) is de…ned as

Z 1
( x) = e ttx 1dt (x > 0)
0

Z 1
Note e tdt = 1
0
Z 1
Integration by parts gives us e ttxdt = (x + 1) =
0
Z 1 Z 1
x e ttx 1dt = x (x 1) e ttx 2dt (1)
0 0
= :::::::: = x!
Important results:

(n + 1) = n! (n 0)
(1) = 1

119
1. INTRODUCTION TO CALCULUS

and also from (1)

(x + 1) = x (x) :

Z 1
If we make the substitution t = u2 in ( x) = e ttx 1dt
0
we obtain
Z 1
2
( x) = 2 e u u2x 1du
0
and put x = 1=2 so that
Z 1
1 2
2 =2 e u du
0
Z 1
2
and we know from the error function that e u du =
p 0
=2; hence
1 =p :
2

Examples:

(4) 3! 1
1. (4) = 3! = 6; = = ;
(5) 4! 4

120
1. INTRODUCTION TO CALCULUS

5
2. use (x + 1) = x (x) with x = 3=2
2
5 3 3 3 1 1 3 1 p
= = =
2 2 2 2 2 2 2 2
3p
=
4

3 (x + 1)
3. now use ( x) =
2 x
3
3 +1 2 1
= 2 =
2 3= 2 3 2
2 3
1
6 7 p
= 26 2 7= 2 2 = 4p
3 6 1=2 7
4 5 3 3

121
2. INTRODUCTION TO LINEAR ALGEBRA

2 Introduction to Linear Algebra

2.1 Properties of Vectors

We consider real n dimensional vectors belonging to

the set Rn. An n tuple

v = (v1; v2; :::::::::; vn) 2 Rn

is a vector of dimension n: The elements vi (i = 1; ::::; n)

are called components of v:

Any pair u; v 2 Rn are equal i¤

1. the corresponding components ui’s and vi’s are equal

2. dimensions of both vectors are the same and we write

u = v:

122
2. INTRODUCTION TO LINEAR ALGEBRA

Examples:

p
u1 = (1; 0) ; u2 = 1; e; 3; 6 ; u3 = (3; 4) ; u4 =
( ; ln 3; 2; 1)

1. u1 ; u3 2 R2 and u2 ; u4 2 R4

2. (x + y; x z; 2z 1) = (3; 2; 5) :For equality

to hold corresponding components are equal, so
9
x+y =3 >
=
x z = 2 ) x = 1; y = 2; z = 3
>
2z 1 = 5 ;

123
2. INTRODUCTION TO LINEAR ALGEBRA

2.1.1 Vector Arithmetic

Let u; v 2 Rn: Then vector addition is de…ned as

u + v = (u1 + v1; u2 + v2; :::::::::::; un + vn)

If k 2 R is any scalar then

ku = (ku1; ku2; :::::::::::; kun)

Note: vector addition only holds if the dimensions of
each are identical.

Examples:

u = (3; 1; 2; 0) ; v = (5; 5; 1; 2) ; w = (0; 5; 3; 1)

1. u+v = (3 + 5; 1 5; 2 + 1; 0 + 2) = (8; 4; 1; 2)

2. 2w = (2:0; 2: ( 5) ; 2:3; 2:1) = (0; 10; 6; 2)

3. u + v 2w = (8; 4; 1; 2) (0; 10; 6; 2) =

(8; 6; 7; 0)
124
2. INTRODUCTION TO LINEAR ALGEBRA

1 2 Rn is given by (1; 1; :::::; 1) :

Similarly 0 = (0; 0; :::::; 0) is the zero vector.

Vectors can also be multiplied together using the dot

product . If u; v 2 Rn then the dot product denoted
by u:v is

u:v = u1v1 + u2v2 + :::::::::::: + unvn 2 R

which is clearly a scalar quantity. The operation is com-

mutative , i.e.
u:v = v:u
If a pair of vectors have a scalar product which is zero,

they are said to be orthogonal.

Geometrically this means that the two vectors are per-

pendicular to each other.

125
2. INTRODUCTION TO LINEAR ALGEBRA

2.1.2 Concept of Length in Rn

Recall in 2-D u = (x1; y1)

y

y1

x1 x

The length or magnitude of u; written juj is given by

Pythagoras
q
juj = ( x1 ) 2 + ( y 1 ) 2
and the angle the vector makes with the horizontal is

y
= arctan 1 :
x1
Any vector u can be expressed as
b
u = juj u
126
2. INTRODUCTION TO LINEAR ALGEBRA

b is the unit vector because juj

where u b = 1:

Given any two vectors u; v 2 R2; we can calculate the

distance between them
y

uv

jv uj = j(v1 ; v2) (u1 ; u2)j

q
= (v 1 u1 ) 2 + ( v 2 u2 )2

127
2. INTRODUCTION TO LINEAR ALGEBRA

In 3D (or R3) a vector v = (x1; y1; z1) has length/magnitude

q
jvj = ( x1 ) 2 + ( y 1 ) 2 + ( z 1 ) 2 :

To extend this to Rn; is similar.

Consider v = (v1; v2; :::::::::; vn) 2 Rn: The length

of v is called the norm and denoted kvk ; where
q
kvk = (v1)2 + (v2)2 + :::::::: + (vn)2

If u; v 2 Rn then the distance between u and v is

can be obtained in a similar fashion
q
kv uk = (v 1 u1 ) 2 + ( v 2 u2)2 + :::::::: + (vn un ) 2

We mentioned earlier that two vectors u and v in two

dimension are orthogonal if u:v = 0:

The idea comes from the de…nition

u:v = juj : jvj cos :

128
2. INTRODUCTION TO LINEAR ALGEBRA

Re-arranging gives the angle between the two vectors.

Note when = =2 u:v = 0:

If u; v 2 Rn we write

u:v = j juj j:j jvj j cos

Examples: Consider the following vectors

u = (2; 1; 0; 3) ; v = (1; 1; 1; 3) ;
w = (1; 3; 2; 2)

q p
2 2 2 2
kuk = (2) + ( 1) + (0) + ( 3) = 14

Distance between v & w = kw vk =

q
(1 1)2 + (3 ( 1))2 + ( 2 ( 1))2 + (2 3)2
p
= 3 2

129
2. INTRODUCTION TO LINEAR ALGEBRA

The angle between u & w can be obtained from

u:v
cos = :
j juj j j jvj j
Hence
s
(2; 1; 0; 3) : (1; 1; 1; 3) 3
cos = p p = !
2 3 14 14
q
= cos 1 3
14

130
2. INTRODUCTION TO LINEAR ALGEBRA

2.2 Matrices

A matrix is a rectangular array A = ai j for i =

1; :::; m ; j = 1; :::; n written
0 1
a11
a12 :: :: :: a1n
B C
B a21
a22 :: :: :: a2n C
B C
B : : C
A=B
B :
C
B : CC
B
@ ::: :: :: :: : CA
am1 am2 :: :: :: amn
and is an (m n) matrix, i.e. m rows and n
columns.

If m = n the matrix is called square. The product

mn gives the number of elements in the matrix.

A vector which we have already seen is simply a case of

a (m 1) matrix, i.e.
0 1
x1
B C
B x2 C
B C
B : C
X=B
B : C
C
B C
B C
@ : A
xm
131
2. INTRODUCTION TO LINEAR ALGEBRA

2.2.1 Matrix Arithmetic

Let A; B 2 mRn

A+B =
0 1 0 1
a11 a12 :::: a1n b11 b12 :: :: b1n
B C B C
B a21 a22 :::: a2n C B b21 b22 :: :: b2n C
B C B C
B : : : : C B C
B C+B : : : : : C
B : : : : C B C
B C B : : : : : C
B
@ : :: :::: : C B
A @ : :: :: :: : C
A
am1 am2 :::: amn bm1 bm2 :: :: bmn
and the corresponding elements are added to give
0 1
a11 + b11 a12 + b12 :::: a1n + b1n
B C
B a21 + b21 a22 + +b22 :::: a2n + b2n C
B C
B : : : : C
B C = B +A
B : : : : C
B C
B C
@ : :: :::: : A
am1 + bm1 am2 + bm2 :::: amn + bmn

132
2. INTRODUCTION TO LINEAR ALGEBRA

Matrices can only added if they are of the same form.

Examples:
! !
1 1 2 4 0 3
A = ; B= ;
0 3 4 1 2 3
0 1 0 1
2 3 1 1 0 0
B C B C
C = @ 5 1 2 A; D=@ 0 1 0 A
1 0 3 0 0 1
0 1
! 3 3 1
5 1 1 B C
A+B = ; C +D = @ 5 0 2 A
1 1 7
1 0 4
We cannot perform any other combination of addition as
A and B are (2 3) and C and D are (3 3) :

133
2. INTRODUCTION TO LINEAR ALGEBRA

2.2.2 Matrix Multiplication

To multiply two square matrices A and B; so that

C = AB; the elements of C are found from the recipe

N
X
Cij = Aik Bkj :
k=1

That is, the i th row of A is dotted with the j th

column of B: For example,

! ! !
a b e f ae + bg af + bh
= :
c d g h ce + dg cf + dh

Note that in general AB 6= BA: The general rule for

multiplication is

Apn Bnm ! Cpm

134
2. INTRODUCTION TO LINEAR ALGEBRA

Example:
0 1
! 1 2
2 1 0 B C
@ 0 3 A
2 0 2
1 2
!
2 : 1 + 1: 0 + 0: 1 2 : 2 + 1: 3 + 0 : 2
=
2 : 1 + 0: 0 + 2: 1 2 : 2 + 0: 3 + 2 : 2
!
2 7
=
4 8

135
2. INTRODUCTION TO LINEAR ALGEBRA

2.2.3 Transpose

The transpose of a matrix with entries Aij is the

matrix with entries Aji; the entries are ’re‡ected’across
the leading diagonal, i.e. rows become columns. The
transpose of A is written AT: If A = AT then A
is symmetric. For example, of the matrices

! ! !
1 2 1 3 1 2
A= ; B= ; C= ;
3 4 2 4 2 1

we have B = AT and C = CT : Note that for any

matrix A and B

(i) (A + B )T = AT + B T

T T
(ii) A =A

(iii) (kA)T = kAT ; k is a scalar

136
2. INTRODUCTION TO LINEAR ALGEBRA

(iv) (AB )T = B T AT

Example:
0 1
2 1 !
B C 2 1 2
A = @ 1 2 A ! AT =
1 2 2
2 2

137
2. INTRODUCTION TO LINEAR ALGEBRA

2.2.4 Matrix Representation of Linear Equations

We begin by considering a two-by-two set of equations

for the unknowns x and y :

ax + by = p
cx + dy = q

The solution is easily found. To get x; multiply the

…rst equation by d; the second by b; and subtract to
eliminate y :

(ad bc) x = dp bq:

Then …nd y :

(ad bc) y = aq cp:

138
2. INTRODUCTION TO LINEAR ALGEBRA

This works and gives a unique solution as long as ad

bc 6= 0:

If ad bc = 0; the situation is more complicated: there

may be no solution at all, or there may be many.

Examples:
Here is a system with a unique solution:

x y = 0
x+y = 2

The solution is x = y = 1:

Now try

x y = 0
2x 2y = 2

Obviously there is no solution: from the …rst equation

x = y; and putting this into the second gives 0 = 2:
Here ad bc = 1 ( 2) (1 ) 2 = 0:
139
2. INTRODUCTION TO LINEAR ALGEBRA

Also note what is being said:

)
x=y
Impossible.
x=1+y

Lastly try

x y = 1
2x 2y = 2:

The second equation is twice the …rst so gives no new

information. Any x and y satisfying the …rst equation
satisfy the second. This system has many solutions.

Note: If we have one equation for two unknowns the sys-

tem is undetermined and has many solutions. If we have
three equations for two unknowns, it is over-determined
and in general has no solutions at all.

Then the general (2 2) system is written

140
2. INTRODUCTION TO LINEAR ALGEBRA

! ! !
a b x p
=
c d y q

or

Ax = p:

The equations can be solved if the matrix A is invert-

ible. This is the same as saying that its determinant

a b
= ad bc
c d

is not zero.

These concepts generalise to systems of N equations in

N unknowns. Now the matrix A is N N and the
vectors x and p have N entries.

141
2. INTRODUCTION TO LINEAR ALGEBRA

Here are two special forms for A: One is the identity

matrix, which has its own inverse

0 1
1 0 0 ::: 0
B C
B0 1 0 ::: C
B C
I=B
B0 0 1 : : : ... C
C:
B ... ... 0C
@ A
0 ::: 0 1

and for any x; Ix = x: The other is the tridiago-

nal form. This is common in …nite di¤erence numerical
schemes.

0 1
0 0
B ... ... ... ... C
B C
B . . . ... C
B 0 ... ... ... C
A=B
B ... . . . ... ... ... 0
C
C
B C
B ... ... ... ... C
@ A
0 0

142
2. INTRODUCTION TO LINEAR ALGEBRA

There is a main diagonal, and one above called the super

diagonal and one below called the sub-diagonal.

A symmetric matrix A has the property

AT = A
aij = aji
so the leading diagonal acts as a "mirror". Clearly a
symmetric matrix is square. For example
0 1
a b c
B a d e C
B C
B C
@ b d f A
c e f
is symmetric.

143
2. INTRODUCTION TO LINEAR ALGEBRA

To conclude:
System of Linear Equations

Inconsistent Consistent

No Solution

Unique Solution Many Solutions

E>n

E=n E<n
free variable

where E = number of equations and n = unknowns.

The theory and numerical analysis of linear systems ac-

counts for quite a large branch of mathematics.

144
2. INTRODUCTION TO LINEAR ALGEBRA

2.3 Using Matrix Notation For Solving Lin-

ear Systems

The usual notation for systems of linear equations is that

of matrices and vectors. Consider the system

ax + by + cz = p ( )
dx + ey + f z = q
gx + hy + iz = r

for the unknown variables x; y; z . We gather the

unknowns x; y and z and the given p ; q and r
into vectors:

0 1 0 1
x p
B C B C
@ y A; @ q A
z r

and put the coe¢ cients into a matrix

145
2. INTRODUCTION TO LINEAR ALGEBRA

0 1
a b c
B C
A = @ d e f A:
g h i

A is called the coe¢ cient matrix of the linear system

( ) and the special matrix formed by
0 1
a b c p
B C
@ d e f q A
g h i r
is called the augmented matrix.

146
2. INTRODUCTION TO LINEAR ALGEBRA

Now consider a general linear system consisting of m

equations in n unknowns which can be written in aug-
mented form as

0 1
a11 a12 :: :: :: a1n b1
B C
B a21 a22 :: :: :: a2n b2 C
B C
B : : : C
B C:
B : : : C
B C
B C
@ : :: :: :: :: : : A
am1 am2 amn bm

We can perform a series of row operations on this matrix

and reduce it to a simpli…ed matrix of the form
0 1
a11 a12 :: :: :: a1n b1
B C
B 0 a22 :: :: :: a2n b2 C
B C
B 0 0 : : C
B C:
B 0 0 0 : : C
B C
B C
@ : :: :: :: :: : : A
0 0 0 amn bm

Such a matrix is said to be of echelon form if the number

of zeros preceding the …rst nonzero entry of each row
increases row by row.
147
2. INTRODUCTION TO LINEAR ALGEBRA

A matrix A is said to be row equivalent to a matrix

B; written A B if B can be obtained from A
from a …nite sequence of operations called elementary
row operations of the form:

[ER1]: Interchange the i th and j th rows: Ri $ Rj

[ER2]:Replace the i th row by itself multiplied by a

nonzero constant k : Ri ! kRi

[ER3]:Replace the i th row by itself plus k times the

j th row: Ri ! Ri + kRj

These have no a¤ect on the solution of the of the linear

system which gives the augmented matrix.

148
2. INTRODUCTION TO LINEAR ALGEBRA

Examples:

Solve the following linear systems

1.
9
2x + y 2z = 10 >
=
3x + 2 y + 2 z = 1 Ax = b with
>
5x + 4 y + 3 z = 4 ;
0 1 0 1
2 1 2 10
B C B C
A=@ 3 2 2 A and b = @ 1 A
5 4 3 4

The
0 augmented matrix
1 for this system
0 is 1
2 1 2 10 2 1 2 10
B C R !2R 3R 1 B C
@ 3 2 2 1 A 2 2
@ 0 1 10 28 A
R3 !2R3 5R1
5 4 3 4 0 3 16 42
0 1
2 0 12 38
R3 !R3 3R2 B C
@ 0 1 10 28 A
R1 !R1 R2
0 0 14 42

14z = 42 ! z = 3
y + 10z = 28 ! y = 28 + 30 = 2
x 6z = 19 ! x = 19 18 = 1
149
2. INTRODUCTION TO LINEAR ALGEBRA

Therefore solution is unique with

0 1
1
B C
x=@ 2 A
3

150
2. INTRODUCTION TO LINEAR ALGEBRA

2.
9
x + 2 y 3z = 6 >
=
2x y + 4 z = 2
>
4x + 3y 2z = 14 ;

0 1
1 2 3 6
B C R2!R2 2R1
@ 2 1 4 2 A
4 3 2 14 R3!R3 4R1
0 1
1 2 3 6
B C R !R3 R2
@ 0 5 10 10 A 3
R2 !0:5R2
0 5 10 10
0 1
1 2 3 6
B C
@ 0 1 2 2 A
0 0 0 0
Number of equations is less than number of un-
knowns.
y 2z = 2 so z = a is a free variable) y =
2 (1 + a)
x + 2 y 3z = 6 ! x = 6 2y + 3 z = 2 a
) x = 2 a; y = 2 (1 + a) ; z = a

151
2. INTRODUCTION TO LINEAR ALGEBRA

Therefore there are many solutions

0 1
2 a
B C
x = @ 2 (1 + a) A
a

152
2. INTRODUCTION TO LINEAR ALGEBRA

3.
9
x + 2 y 3z = 1 >
=
3x y + 2 z = 7
>
5x + 3 y 4z = 2 ;

0 1
1 2 3 1
B C R2!R2 3R1
@ 3 1 2 7 A
R3 !R3 5R1
5 3 4 2
0 1
1 2 3 1
B C R3!R3 R2
@ 0 7 11 10 A
0 7 11 7
0 1
1 2 3 1
B C
@ 0 7 11 10 A
0 0 0 3

The last line reads 0 = 3: Also middle itera-

tion shows that the second and third equations are
inconsistent.

Hence no solution exists.

153
2. INTRODUCTION TO LINEAR ALGEBRA

2.4 Matrix Inverse

The inverse of a matrix A, written A 1; satis…es

AA 1 = A 1 A = I:

It may not always exist, but if it does, the solution of the

system

Ax = p

is

x=A 1 p:

154
2. INTRODUCTION TO LINEAR ALGEBRA

The inverse of the matrix for the special case of a 2 2

matrix

!
a b
c d

is

!
1 d b
ad bc c a

provided that ad bc 6= 0:

The inverse of any n n matrix A is de…ned as

1
A 1 = adj A
jAj

h iT
i+j
where adj A = ( 1) Mij is the adjoint, i.e. we
form the matrix of A’s cofactors and transpose it.
155
2. INTRODUCTION TO LINEAR ALGEBRA

Mij is the square sub-matrix obtained by "covering the

ith row and jth column", and its determinant is called the
Minor of the element aij. The term Aij = ( 1)i+j Mij
is then called the cofactor of aij:

Consider the following example with

0 1
1 1 0
B C
A=@ 1 2 1 A
0 1 3

So the determinant is given by jAj =

( 1)1+1 A11 jM11j+( 1)1+2 A12 jM12j+( 1)1+3 A13 jM13j

2 1 1 1 1 2
= 1 1 +0
1 3 0 3 0 1
= (2 3 1 1) (1 3 1 0) + 0 = 5 3
= 2

Here we have expanded about the 1st row - we can do

this about any row. If we expand about the 2nd row - we
should still get jAj = 2:
156
2. INTRODUCTION TO LINEAR ALGEBRA

We now calculate the adjoint:

2 1 1 1
( 1)1+1 M11 = + ( 1)1+2 M12 =
1 3 0 3

1 2
( 1)1+3 M13 = +
0 1

1 0 1 0
( 1)2+1 M21 = ( 1)2+2 M22 = +
1 3 0 3

1 1
( 1)2+3 M23 =
0 1

1 0 1 0
( 1)3+1 M31 = + ( 1)3+2 M32 =
2 1 1 1

1 1
( 1)3+3 M33 = +
1 2

0 1T
5 3 1
adj A = B
@ 3 3
C
1 A
1 1 1
157
2. INTRODUCTION TO LINEAR ALGEBRA

We can now write the inverse of A (which is symmetric)

0 1
5 3 1
1 B C
A 1= @ 3 3 1 A
2 1 1 1

Elementary row operations (as mentioned above) can be

used to simplify a determinant, as increased numbers of
zero entries present, requires less calculation. There are
two important points, however. Suppose the value of the
determinant is jAj ; then:

[ER1]: Ri $ Rj ) jAj ! jAj

[ER2]: Ri ! kRi ) jAj ! k jAj

158
2. INTRODUCTION TO LINEAR ALGEBRA

2.5 Orthogonal Matrices

A matrix P is orthogonal if

PPT = PTP =I:

This means that the rows and columns of P are orthog-

onal and have unit length. It also means that

P 1 = PT:

In two dimensions, orthogonal matrices have the form

! !
cos sin cos sin
or
sin cos sin cos

159
2. INTRODUCTION TO LINEAR ALGEBRA

for some angle and they correspond to rotations or

re‡ections.

So rows and columns being orthogonal means row i row

j = 0; i.e. they are perpendicular to each other.

(cos ; sin ) ( sin ; cos ) =

cos sin + sin cos = 0
(cos ; sin ) (sin ; cos ) =
cos sin sin cos = 0

v = (cos ; sin )T ! jvj = cos2 + ( sin )2 = 1

!
cos sin
Finally, if P = then
sin cos
!
1 1 cos sin
P = = PT:
cos2 sin2 sin cos
| {z }
=1

160
2. INTRODUCTION TO LINEAR ALGEBRA

2.6 Eigenvalues and Eigenvectors

If A is a square matrix, v is an eigenvector of A

with eigenvalue if

Av = v or (A I) v= 0:

An N N matrix has exactly N eigenvalues, not

all necessarily real or distinct; they are the roots of the
characteristic equation

det (A I) = 0:

and each solution has a corresponding eigenvector v:

A I is the characteristic polynomial.

The eigenvectors are in some sense special directions for

the matrix A: In complete generality this is a vast topic.
Many Boundary-Value
161
2. INTRODUCTION TO LINEAR ALGEBRA

Problems can be reduced to eigenvalue problems.

We will just look at real symmetric matrices for which

A = AT. For these matrices

The eigenvalues are real;

The eigenvectors corresponding to distinct eigenval-

ues are orthogonal;

The matrix can be diagonalised: that is, there is

an orthogonal matrix P such that

A = P DP T or P TAP = D
where D is diagonal, that is only the entries on the
leading diagonal are nonzero, and these are equal to the

162
2. INTRODUCTION TO LINEAR ALGEBRA

eigenvalues of A:
0 1
0 0 0 0
B 1 ... ... ... C
B 0 0 C
B ... ... ... C
D=B
B 0 0 C
C
B 0 ... ... ... 0 C
@ A
0 0 0 0 n

Example:

0 1
3 3 3
B C
A= @ 3 1 1 A
3 1 1

then

3 3 3
det (A I) = 3 1 1
3 1 1
= 3 + 2 + 24 + 36 = 0
= ( + 3) ( + 2) ( 6)
163
2. INTRODUCTION TO LINEAR ALGEBRA

so that the eigenvalues, i.e. the roots of this equation,

are 1 = 3; 2 = 2 and 3 = 6:

Eigenvectors are now obtained from

0 1 0 1
3 i 3 3 0
B C B C
@ 3 1 i 1 A v i = @ 0 A i = 1 ; 2; 3
3 1 1 i 0

0 10 1 0 1
6 3 3 x 0
B CB C B C
1 = 3: @ 3 2 1 A@ y A = @ 0 A
3 1 2 z 0

0 1
2 1 1 0
B C
Upon row reduction we have @ 0 1 1 0 A!y=
0 0 0 0
z;0so put
1 z = a and 2x = y z!x= ) v1 =
1
B C
@ 1A
1

Similarly

164
2. INTRODUCTION TO LINEAR ALGEBRA
0 1 0 1
0 2
B C B C
2 = 2 : v2 = @ 1 A; 3 = 6 : v3 = @1 A
1 1

If we take = = = 1 the corresponding eigenvec-

tors are

0 1 0 1 0 1
1 0 2
v1 = B C
@ 1A ; v2 = B C
@ 1 A; v3 = B C
@1A
1 1 1

b = v= jvj for
Now normalise these, i.e. jvj = 1: Use v
normalised eigenvectors
0 1 0 1 0 1
1 0 2
1 B C 1 B C 1 B C
b 1 = p @ 1A ;
v b2 = p @ 1 A ;
v b 3 = p @1 A
v
3 1 2 1 6 1

Hence

0 1 2 1 0 1 1
p 0 p p p1 p1
B 3 6C B 3 3 3C
B 1 p C
P = B p3 p1
2
1
6A
C! PT = B
B 0 p1
2
p C
1
C
2A
@ @
p1 p1 p1 p2 p1 p1
3 2 6 6 6 6
165
2. INTRODUCTION TO LINEAR ALGEBRA

so that

0 1
3 0 0
PTAP = B
@ 0
C
2 0A
0 0 6
= D:

166
2. INTRODUCTION TO LINEAR ALGEBRA

2.7 Criteria for invertibility

A system of linear equations is uniquely solvable if and

only if the matrix A is invertible. This in turn is true if
any of the following is:

1. If and only if the determinant is nonzero;

2. If and only if all the eigenvalues are nonzero;

3. If (but not only if) it is strictly diagonally domi-

nant.

In practise it takes far too long to work out the determi-

nant. The second criterion is often useful though, and
there are quite quick methods for working out the eigen-
values. The third method is explained on the next page.

(Note: there are many other criteria for invertibility.)

167
2. INTRODUCTION TO LINEAR ALGEBRA

A matrix A with entries Aij is strictly diagonally

dominant if

X
jAiij > Aij :
j6=i

That is, the diagonal element in each row is bigger in

modulus that the sum of the moduli of the o¤-diagonal
elements in that row.

Examples:

0 1
2 0 1
B C
@ 1 4 2 A is s.d.d. and so invertible;
1 3 6

0 1
1 0 2
B C
@ 2 5 1 A is not s.d.d. but still invertible;
3 2 13

168
2. INTRODUCTION TO LINEAR ALGEBRA

!
1 1
is neither s.d.d. nor invertible.
1 1

169
3. INTRODUCTION TO PROBABILITY

3 Introduction to Probability

3.1 Preliminaries

A set of all possible outcomes of some given experi-

ment is called the sample space.

A particular outcome ! 2 is called a sample point,

or sample path for a stochastic process.

An event is a set of outcomes, i.e. .

Example 1

Experiment: A dice is rolled and the number appear-

ing on top is observed. The sample space consists
of the 6 possible numbers:

= f1; 2; 3; 4; 5; 6g

If the number 4 appears then ! = 4 is a sample point,

clearly 4 2 .
170
3. INTRODUCTION TO PROBABILITY

Let 1, 2, 3 = events that an even, odd, prime num-

ber occurs respectively.

So

1 = f2; 4; 6g ; 2 = f1; 3; 5g ; 3 = f2; 3; 5g

1[ = f 2; 3; 4; 5; 6g
3 event that an even or
prime number occurs.

2\ 3 = f 3; 5g event that odd and prime number

occurs.

c = f1; 4; 6g event that prime number does not

3
occur (complement of event).

171
3. INTRODUCTION TO PROBABILITY

Example 2 Experiment:

Toss a coin twice and observe the sequence of heads (H)

and tails (T) that appears. Sample space

= fHH, TT, HT, THg

Let 1 be event that at least one head appears, and 2

be event that both tosses are the same:

1 = fHH, HT, THg , 2 = fHH, TTg

1\ 2 = fHHg

Events are subsets of , but not all subsets of are

events.

172
3. INTRODUCTION TO PROBABILITY

3.1.1 Random Variables

Outcomes of experiments are not always numbers, e.g. 2

heads appearing; picking an ace from a deck of cards. We
need some way of assigning real numbers to each random
event. Random variables assign numbers to events.

Thus a random variable (RV) X is a function which maps

from the sample space to the set of real numbers

X:!2 ! R;

i.e. it associates a number X (! ) with each outcome !:

Consider the example of tossing a coin and suppose we

are paid £ 1 for each head and we lose £ 1 each time a
1
tail appears. We know that P (H) = P (T) = : So now
2
we can assign the following outcomes
1
P (1) =
2
1
P ( 1) =
2
173
3. INTRODUCTION TO PROBABILITY

Mathematically, if our random variable is X; then

(
+1 if H
X=
1 if T
or using the notation above X : ! 2 fH,Tg ! f 1; 1g :

The probability that the RV takes on each possible value

is called the probability distribution.

If X is a RV then

P (X = a) = P (f! 2 : X (! ) = ag)
is the probability that a occurs (or X maps onto a).

P (a X b) = probability that X lies in the interval

[a; b] =

P (f! 2 :a X (! ) bg)

X: ! R
Domain Range (…nite)

X ( ) = fx1; ::::; xng = fxig1 i n

174
3. INTRODUCTION TO PROBABILITY

P [xi] = P [X = xi] = f (xi) 8 i:

So the earlier coin tossing example gives

1 1
P (X = 1) = ; P (X = 1) =
2 2

f (xi) is the probability distribution of X:

This is called a discrete probability distribution.

xi x1 x2 :::::::::::: xn
f ( xi ) f ( x1 ) f ( x2 ) :::::::::::: f ( xn )

There are two properties of the distribution f (xi)

(i) f (xi) 0 8 i 2 [1; n]

n
P
(ii) f (xi) = 1; i.e. sum of all probabilities is one.
i=1

175
3. INTRODUCTION TO PROBABILITY

3.1.2 Mean/Expectation

The mean measures the centre (average) of the dis-

tribution
n
P
= E [X ] = x i f ( xi )
i=1
= x1f (x1) + x2f (x2) + ::::: + xnf (xn)
which is equal to the weighted average of all possible
values of X together with associated probabilities.

This is also called the …rst moment.

Example:

xi 2 3 8
f ( xi ) 1 1 1
4 2 4

3
P 1 1 1
= E [X ] = xi f ( x i ) = 2 +3 +8
i=1 4 2 4
= 4

176
3. INTRODUCTION TO PROBABILITY

3.1.3 Variance/Standard Deviation

This measures the spread (dispersion) of X about the

mean.

Variance V [X ] =

h i h i n
P
2
E (X ) =E X2 2 = x2i f (xi) 2 = 2
i=1

h i
2
E (X ) is also called the second moment about
the mean.

From the previous example we have = 4; therefore

1 1 1
V [X ] = + 32
22 + 82 16
4 2 4
= 5:5 = 2 ! = 2:34

177
3. INTRODUCTION TO PROBABILITY

3.1.4 Rules for Manipulating Expectations

Suppose X; Y are random variables and ; ; 2R

are constant scalar quantities. Then

E [ X ] = E [X ]

E [X + Y ] = E [X ] + E [Y ] ; (linearity)

V[ X + ] = 2 V [X ]

E [XY ] = E [X ] E [Y ] ;

V [X + Y ] = V [X ] + V [Y ]

The last two are provided X; Y are independent.

178
3. INTRODUCTION TO PROBABILITY

3.1.5 Continuous Random Variables

As the number of discrete events becomes very large,

individual probabilities f (xi) ! 0: Now look at the
continuous case.

Instead of f (xi) we now have p (x) which is a con-

tinuous distribution called as probability density function,
PDF.

Z b
P (a X b) = p (x) dx
a

The cumulative distribution function F (x) of a RV X

is
Z x
F ( x) = P ( X x) = p (x) dx
1
F (x) is related to the PDF by
dF
p ( x) =
dx
(fundamental theorem of calculus) provided F (x) is
di¤erentiable. However unlike F (x) ; p (x) may have
singularities (and may be unbounded).
179
3. INTRODUCTION TO PROBABILITY

3.1.6 Special Expectations:

Given any PDF p (x) of X:

Z
Mean = E [X ] = xp (x) dx:
R

h i Z
Variance 2 = V [X ] = E (X )2 = x2p (x) dx
R
2

(2nd moment about the mean).

The nth moment about zero is de…ned as

= E [ X n]
n Z
= xnp (x) dx:
R

180
3. INTRODUCTION TO PROBABILITY

In general, for any function h

Z
E [h (X )] = h (x) p (x) dx:
R
where X is a RV following the distribution given by
p ( x) :

Moments about the mean are given by

E [(X )n] ; n = 2; 3; :::

The special case n = 2 gives the variance 2:

181
3. INTRODUCTION TO PROBABILITY

3.1.7 Skewness and Kurtosis

Having looked at the variance as being the second mo-

ment about the mean, we now discuss two further mo-
ments centred about ; that provide further important
information about the probability distribution.

Skewness is a measure of the asymmetry of a distribution

(i.e. lack of symmetry) about its mean. A distribution
that is identical to the left and right about a centre point
is symmetric.

The third central moment, i.e. third moment about the

mean scaled with 3
h i
3
E (X )
3
is called the skew and is a measure of the skewness (a
non-symmetric distribution is called skewed).

Any distribution which is symmetric about the mean has

a skew of zero.
182
3. INTRODUCTION TO PROBABILITY

Negative values for the skewness indicate data that are

skewed left and positive values for the skewness indicate
data that are skewed right.

By skewed left, we mean that the left tail is long relative

to the right tail. Similarly, skewed right means that the
right tail is long relative to the left tail.

The fourth centred moment scaled by the variance, called

the kurtosis is de…ned
h i
4
E (X )
4
:

This is a measure of how much of the distribution is out

in the tails at large negative and positive values of X:

183
3. INTRODUCTION TO PROBABILITY

Example:

Consider a continuous PDF

(
k 1 x2 jxj 1
p ( x) =
0 otherwise

Probability Density Function

0.8
0.7
0.6
0.5
p(x)

0.4
0.3
0.2
0.1
0
-1.5 -1 -0.5 0 0.5 1 1.5
x

184
3. INTRODUCTION TO PROBABILITY

i) Calculate k :

We know
Z 1 Z 1
p (x) dx = 1 ) k 1 x2 dx = 1
1 1

k x 1 x3 1 ! k = 34
3 1

Z Z 1
ii) E [X ] = xp (x) dx = 34 x x3 dx
R 1

If
Z f (x) is an odd function, i.e. f ( x) = f (x) then
a
f (x) dx = 0 ) = E [X ] = 0:
a

185
3. INTRODUCTION TO PROBABILITY

Z Z
iii) V [X ] = x2p (x) dx 2 = x2p (x) dx
R R
Z 1
= 43 x2 x4 dx:
1
If f Z(x) is an even function,
Z i.e. f ( x) = f (x)
a a
then f (x) dx = 2 f (x) dx )
a 0
Z 1
V [X ] = 3 x2 x4 dx = 3 1 x3 1 x5 1
2 0 2 3 5 0
= 1 2! standard deviation
5 = 0:45

iv) Calculate the probability that a random variable X

which follows this distribution, lies in the interval 1; 1 .
3 2
So
Z 1=2
P 1 X 1 = p (x) dx
3 2 1=3
Z 1=2
= 34 1 x2 dx 0:58
1=3

186
3. INTRODUCTION TO PROBABILITY

3.2 Normal Distribution

The normal (or Gaussian) distribution N ; 2 with

mean and standard deviation and 2 in turn is de…ned
in terms of its density function
2!
1 (x )
p (x) = p exp 2
:
2 2
For the special case = 0 and = 1 it is called the
standard normal distribution N (0; 1) :

This is also veri…ed by making the substitution

x
=

in p (x) which gives

1 1 2
( ) = p exp
2 2
and clearly has zero mean and unit variance:

187
3. INTRODUCTION TO PROBABILITY

X 1
E = E [X ] = 0;

X X
V =V

Now V [ X + ] = 2V [X ] (standard result), hence

1 1 2=1
2
V [X ] = 2
:

Its cumulative distribution function is

Z x
1 1 2
F ( x) = p e 2 d =P( 1 X x) :
2 1

The skewness of N (0; 1) is zero and its kurtosis is 3:

188
3. INTRODUCTION TO PROBABILITY

3.3 Moment Generating Function

The moment generating function of X; denoted MX ( )

is given by
h i Z
MX ( ) = E e x = e xp (x) dx
R
provided the expectation exists. We can expand as a

power series to obtain

1 n
X E (X n )
MX ( ) =
n=0 n!

so the nth moment is the coe¢ cient of n=n!; or the

nth derivative evaluated at zero.

How do we arrive at this result?

We use theZ Taylor series expansion for the exponential

function: e xp (x) dx =
R
Z 2 3 !
( x) ( x)
1+ x+ + + :::::: p (x) dx
R 2! 3!
189
3. INTRODUCTION TO PROBABILITY

Z Z Z
2
= p (x) dx + xp (x) dx + 2! x2p (x) dx +
| R {z } | R {z } |R {z }
1 E(X) E(X 2)
Z
3 3 p (x) dx + ::::
3! R x
| {z }
E(X 3)
2 3
= 1 + E (X ) + 2! E X 2 + 3! E X 3 + ::::
1 n
X E (X n )
= :
n=0 n!

190
3. INTRODUCTION TO PROBABILITY

Calculating Moments

The kth moment mk of the random variable X can now

be obtained by di¤erentiating, i.e.
(k)
mk = MX (0) ; k = 0; 1; 2; :::
(k) dk
MX (0) = k
MX ( )
d =0
A useful result in …nance is the MGF for the normal distri-
bution. If X N ; 2 , then we can construct a stan-
X
dard normal N (0; 1) by setting = =)
X= + :

The MGF is
h i h i
M (X ) = E e x =E e ( + )
h i
= e E e

So the MGF of X is therefore equal to the MGF of but

with replaced by :This is much nicer than trying to

191
3. INTRODUCTION TO PROBABILITY

calculate the MGF of X N ; 2 :

h i Z 1
1 x x 2 =2
E e = p e e dx
2 1
Z 1
1 2
= p e x x =2dx
2 1
Z 1
1 1 x2 2 x+ 2
( 2)
= p e 2 dx
2 1
Z 1
1 1 (x ) 2 1 2
+2
= p e 2 dx
2 1
Z 1
1 2 1 1 (x ) 2
= e p
2 e 2 dx
2 1
Now do a change of variable - put u = x
h i Z 1
1 2 1 1 u2
E e = e2 p e 2 du
2 1
1 2
= e2
Thus
h i
M (X ) = e E e
+ 12 2 2
= e
To get the simpler formula for a standard normal distrib-
1 2
ution put = 0; = 1 to get M (X ) = e2 :

192
3. INTRODUCTION TO PROBABILITY

We can now obtain the …rst four moments for a standard

normal
d 1 2
m1 = e2
d =0
1 2
= e2 =0
=0

d2 1 2
m2 = 2
e2
d =0
1 2
= 2 +1 e2 =1
=0

d3 1 2
m3 = 3
e2
d =0
1 2
= 3 +3 e2 =0
=0

d4 1 2
m4 = 4
e2
d =0
= 4 +6 2 + 3 e 12 2 =3
=0
The latter two are particularly useful in calculating the
skew and kurtosis.

193
3. INTRODUCTION TO PROBABILITY

3.4 Correlation

The covariance is useful in studying the statsitical depen-

dence between two random. If X; Y are RV’s, then their
covariance is de…ned as:
20 10 13
6B CB C7
Cov (X; Y ) = E 4@X E (X )A @Y E ( Y ) A5
| {z } | {z }
= x = y
= E [XY ] x y

which we denote as XY : Note:

h i
2
Cov (X; X ) = E (X x) = 2:

X; Y are correlated if
h i
E (X x) Y y 6= 0:

We can then de…ne an important dimensionless quantity

(used in …nance) called the correlation coe¢ cient and
denoted as XY (X; Y ) where
Cov (X; Y )
XY = :
x y
194
3. INTRODUCTION TO PROBABILITY

The correlation can be thought of as a normalised covari-

ance, as j XY j 1; for which the following conditions
are properties:

i. (X; Y ) = (Y; X )

ii. (X; X) = 1

iii. 1 1

XY = 1 ) perfect negative correlation

XY = 1 )perfect correlation

XY = 0 ) X; Y uncorrelated

Why is the correlation coe¢ cient bounded by 1? Justi…-

cation of this requires a result called the Cauchy-Schwartz
inequality. This is a theorem which most students en-
counter for the …rst time in linear algebra (although we
195
3. INTRODUCTION TO PROBABILITY

have not discussed this). Let’s start o¤ with the version

for random variables (RVs) X and Y , then the Cauchy-
Schwartz inequality is
h i h i
2
[E [XY ]] E X2 E 2
Y :
We know that the covariance of X; Y is

XY = E [(X X ) (Y Y )]
If we put
h i
2 2
V [X ] = X = E (X X)
h i
2 2
V [Y ] = Y = E (Y Y) :
From Cauchy-Schwartz we have
h i h i
2 2 2
(E [(X X ) (Y Y )]) E (X X) E (Y Y)
or we can write
2 2 2
XY X Y
Divide through by 2 2
X Y
2
XY 1
2 2
X Y
and we know that the left hand side above is 2XY , hence
2
2 = XY 1
XY 2 2
X Y
196
3. INTRODUCTION TO PROBABILITY

and since XY is a real number, this implies j XY j 1

which is the same as

1 XY +1:

197
4. DIFFERENTIAL EQUATIONS

4 Di¤erential Equations

4.1 Introduction

2 Types of Di¤erential Equation (D.E)

(i) Ordinary Di¤erential Equation (O.D.E)

Equation involving (ordinary) derivatives

dy d 2y d ny
x; y; ; 2
; ::::::::; (some …xed n)
dx dx dxn

y is some unknown function of x together with its deriv-

atives, i.e.

F x; y; y 0; y 00; ::::::; y (n) = 0 (1)

198
4. DIFFERENTIAL EQUATIONS

Note y 4 =
6 y (4)

Also if y = y (t), where t is time, then we often write

dy d 2y d 4y
y= , y= 2
, ......, y =
dt dt dt4

199
4. DIFFERENTIAL EQUATIONS

(ii) Partial Di¤erential Equation (PDE)

Involve partial derivatives, i.e. unknown function depen-

dent on two or more variables,

e.g.
@u @ 2u @u
+ + u=0
@t @x@y @z

More complicated to solve - better for modelling real-life

situations, e.g. …nance, engineering & science.

200
4. DIFFERENTIAL EQUATIONS

Order of the highest derivative is the order of the DE

d ny
An ode is of degree r if n
(where n is the order of
dx
the derivative) appears with power r

r Z+ the de…nition of n and r is distinct. Assume

that any ode has the property that each
!r !r
d`y `
dy n
d y
appears in the form ! order n
dx` dx` dxn
and degree r.

Examples:

DE order degree
(1) y 0 = 3y 1 1
(2) y 0 3 + 4 sin y = x3 1 3
2 5
(3) y (4) +x y 2 (2) + y0 6 + y =0 4 2
p
(4) 00
y = y0 + y + x 2 2
(5) y 00 + x y 0 3 xy = 0 2 1

Note - example (4) above can be written as y 00 2 =

y0 + y + x
201
4. DIFFERENTIAL EQUATIONS

We will consider ODE’s of degree one, and of the form

dny dn 1 y dy
an (x) dxn + a n 1 ( x ) dxn 1
+ :::: + a 1 ( x ) dx +a0 (x) y = g (x)

n
X
ai (x) y (i) (x) = g (x) (more pedantic)
i=0

Note: y (0) (x) - zeroth derivative, i.e. y (x):

This is a Linear ODE of order n, i.e. r = 1 8 (for

all) terms. Linear also because ai (x) not a function of
y (i) (x) - else equation is Non-linear.

202
4. DIFFERENTIAL EQUATIONS

Examples:

DE Nature of DE
(1) 2xy 00 + x2y 0 (sin x) y = x2 Linear
(2) yy 00 + xy 0 + y = 2 a2 = y ) Non-Linear
p 1
(3) y 00 + y0
+y = x2 Non-Linear * y0 2
d4y 4=0
(4) + y Non-Linear - y 4
dx4

Our aim is to solve our ODE either explicitly or by …nding

the most general y (x) satisfying it or implicitly by …nd-
ing the function y implicitly in terms of x, via the most
general function g s.t g (x; y ) = 0.

Suppose that y is given in terms of x and n arbitrary

constants of integration c1, c2, ......., cn.

So ge (x; c1; c2; :::::::; cn) = 0. Di¤erentiating ge, n

times to get (n + 1) equations involving

c1; c2; :::::::; cn; x; y; y 0; y 00; ::::::; y (n).

203
4. DIFFERENTIAL EQUATIONS

Eliminating c1; c2; :::::::; cn we get an ODE

fe x; y; y 0; y 00; ::::::; y (n) = 0

Examples:

(1) y = x3 + ce 3x (so 1 constant c)

dy
) = 3x2 3ce 3x, so eliminate c by taking 3y +
dx
y 0 = 3 x3 + 3 x 2

i.e.
3x2 (x + 1) + 3y + y 0 = 0

(2) y = c1e x + c2e2x (2 constant’s so di¤erentiate

twice)

y0 = c1e x + 2c2e2x ) y 00 = c1e x + 4c2e2x

204
4. DIFFERENTIAL EQUATIONS

Now
)
y + y 0 = 3c2e2x (a)
y 0 + y 00 = 6c2e2x (b)

and 2(a)=(b) ) 2 y + y 0 = y + y 00 !

y 00 2y 0 y = 0.

Conversely it can be shown (under suitable conditions)

that the general solution of an nth order ode will involve
n arbitrary constants. If we specify values (i.e. boundary
values) of

y; y 0; :::::::::::; y (n)

for values of x, then the constants involved may be de-

termined.

205
4. DIFFERENTIAL EQUATIONS

A solution y = y (x) of (0.1) is a function that produces

zero upon substitution into the lhs of (1).

Example:

y 00 3y 0 + 2y = 0 is a 2nd order equation and y = ex is

a solution.

y = y 0 = y 00 = ex - substituting in equation gives

ex 3ex + 2ex = 0. So we can verify that a function is
the solution of a DE simply by substitution.

Exercise:

(1) Is y (x) = c1 sin 2x + c2 cos 2x (c1,c2 arbitrary con-

stants) a solution of y 00 + 4y = 0

(2) Determine whether y = x2 1 is a solution of

dy 4
+ y2 = 1
dx

206
4. DIFFERENTIAL EQUATIONS

4.1.1 Initial & Boundary Value Problems

A DE together with conditions, an unknown function

y (x) and its derivatives, all given at the same value of in-
dependent variable x is called an Initial Value Problem
(IVP).

e.g. y 00 + 2y 0 = ex; y ( ) = 1, y 0 ( ) = 2 is an
IVP because both conditions are given at the same value
x= .

A Boundary Value Problem (BVP) is a DE together

with conditions given at di¤erent values of x, i.e. y 00 +
2y 0 = ex; y (0) = 1, y (1) = 1.

Here conditions are de…ned at di¤erent values x = 0

and x = 1.

A solution to an IVP or BVP is a function y (x) that both

solves the DE and satis…es all given initial or boundary
conditions.

207
4. DIFFERENTIAL EQUATIONS

Exercise: Determine whether any of the following func-

tions

(a) y1 = sin 2x (b) y2 = x (c) y3 = 21 sin 2x is

a solution of the IVP

y 00 + 4y = 0; y (0) = 0; y 0 (0) = 1

208
4. DIFFERENTIAL EQUATIONS

4.2 First Order Ordinary Di¤erential Equa-

tions

Standard form for a …rst order DE (in the unknown func-

tion y (x)) is
y 0 = f (x; y ) (2)

so given a 1st order ode

F x; y; y 0 = 0

can often be rearranged in the form (2), e.g.

y 2x
xy 0 + 2xy y=0) y0 =
x

209
4. DIFFERENTIAL EQUATIONS

4.2.1 One Variable Missing

This is the simplest case

y missing:
Z
y 0 = f ( x) solution is y = f (x)dx

x missing:
Z
1
y 0 = f (y ) solution is x = dy
f (y )

Example:

y 0 = cos2 y , y = when x = 2
4

Z Z
1
)x= 2
dy = sec2 y dy ) x = tan y + c ,
cos y
210
4. DIFFERENTIAL EQUATIONS

c is a constant of integration.

This is the general solution. To obtain a particular solu-

tion use

y (2) = ! 2 = tan +c)c=1

4 4

so rearranging gives

y = arctan (x 1)

211
4. DIFFERENTIAL EQUATIONS

4.2.2 Variable Separable

y 0 = g ( x) h ( y ) (3)

So f (x; y ) = g (x) h (y ) where g and h are functions of

x only and y only in turn. So
Z Z
dy dy
= g ( x) h ( y ) ! = g (x) dx + c
dx h (y )

c arbitrary constant

Examples:

dy x2 + 2
1. =
dx y
Z Z
y2 x3
y dy = x2 +2 dx ! = + 2x + c
2 3

212
4. DIFFERENTIAL EQUATIONS

dy
2. = y ln x subject to y = 1 at x = e ( y (e) = 1)
dx

Z Z Z
dy
= ln x dx Recall: ln x dx = x (ln x 1)
y

ln y = x (ln x 1) + c ! y = A exp (x ln x x)

A arb. constant

now putting x = e, y = 1 gives A = 1. So solution

becomes
xx x x
y= exp (ln xx) exp ( x) ! y = x )y=
e e

213
4. DIFFERENTIAL EQUATIONS

4.2.3 Linear Equations

These are equations of the form

y 0 + P ( x) y = Q ( x) (4)

which are similar to (3), but the presence of Q (x) renders

this no longer separable. We look for a function R(x),
called an Integrating Factor (I.F) so that

d
R ( x ) y 0 + R ( x ) P ( x) y = ( R ( x) y ) (5)
dx

So upon multiplying the lhs of (4), it becomes a derivative

of R(x)y , i.e.

Ry 0 + RP y = Ry 0 + R0y

from (4).

214
4. DIFFERENTIAL EQUATIONS

dR
This gives RP y = R0y ) R(x)P (x) = , which is
dx
a DE for R which is separable, hence

Z Z Z
dR
= P dx + c ! ln R = P dx + c
R

215
4. DIFFERENTIAL EQUATIONS

R
So R(x) = K exp ( P dx), hence there exists a func-
tion R(x) with the required property.

Multiply (4) through by R(x)

R(x) y 0 + P (x)y = R(x)Q(x)

| {z }
d (R(x)y)
= dx

Z
d
(Ry ) = R(x)Q(x) ! R(x)y = R(x)Q(x)dx+B
dx

B arb. constant.

We also know the form of R(x) !

Z Z Z
yK exp P dx = K exp P dx Q(x)dx + B

216
4. DIFFERENTIAL EQUATIONS

divide through by K to give

Z Z Z
y exp P dx = exp P dx Q(x)dx+ constant.

So we can take K = 1 in the expression for R(x).

R
To solve y 0+P (x) y = Q (x) calculate R(x) = exp ( P dx),
which is the I.F

Examples:

1. Solve xy 0 y = x3

This is currently not in standard form. However, dividing

through by x gives
1
y0 y = x2
x
1
Now comparing with (4) gives P (x) & Q(x)
x
x2, therefore

217
4. DIFFERENTIAL EQUATIONS

R 1 1
I.F R(x) = exp dx = exp ( ln x) = .
x x

1
Multiply DE by !
x

Z
1 0 1 d y
y y = x) =x! d x 1y
x x Z dx x
= xdx + c

y x2 x3
) = + c ) GS is y = + cx
x 2 2

218
4. DIFFERENTIAL EQUATIONS

dx
2. Obtain the general solution of (1 + yex) = ex
dy

dy
= (1 + yex) e x = e x + y )
dx

dy
y=e x
dx

Which is a linear equation, with P = 1; Q = e x

Z
I.F R(y ) = exp dx = e x

so multiplying DE by I.F

2x d
e x y0 y = e ! ye x = e 2x )
Z Z dx
d ye x = e 2xdx

219
4. DIFFERENTIAL EQUATIONS

1 2x
ye x = e +c
2

1 x
) y = cex e is the GS
2

220
4. DIFFERENTIAL EQUATIONS

4.3 Second Order ODE’s

Typical second order ODE (degree 1) is

y 00 = f x; y; y 0

solution involves two arbitrary constants.

4.3.1 Simplest Cases

A y 0, y missing, so y 00 = f (x)

R R
Integrate wrt x (twice): y = ( f (x) dx) dx

Example: y 00 = 4x

Z Z Z h i 2 x3
GS y = 4x dx dx = 2 x2 + C dx = +Cx+D
3
221
4. DIFFERENTIAL EQUATIONS

B y missing, so y 00 = f y 0; x

dP
Put P = y 0 ! y 00
= = f (P; x), i.e. P 0 =
dx
f (P; x) - …rst order ode

Solve once ! P (x)

Solve again ! y (x)

d 2y dy
Example: Solve x 2 + 2 = x3
dx dx

Note: A is a special case of B

C y 0 and x missing, so y 00 = f (y )

Put p = y 0, then

d 2y dp dp dy dp
= = = p
dx2 dx dy dx dy
= f (y )
222
4. DIFFERENTIAL EQUATIONS

So solve 1st order ode

dp
p = f (y )
dy

which is separable, so

Z Z
p dp = f ( y ) dy !

Z
1 2
p = f (y ) dy + const.
2

Example: Solve y 3y 00 = 4

4 d2y dp 4
) y 00 0
= 3 . Put p = y ! 2 = p = 3
y dx dy y
Z
R 4 4
) p dp = dy ) p2 = +D ) p =
q y3 y 2

Dy 2 4
, so from our de…nition of p,
y
223
4. DIFFERENTIAL EQUATIONS

q
Dy 2 4 Z Z
dy y
= ) dx = q dy
dx y Dy 2 4

Integrate rhs by substitution (i.e. u = Dy 2 4) to give

q
Dy 2 4 h
2
i
x= + E ! D (x E) = Dy 2 4
D

) GS is Dy 2 D 2 (x E )2 = 4

D x missing: y 00 = f y 0; y

d2y dP st order
Put P = y 0, so = P = f ( P; y ) - 1
dx2 dy
ODE

224
4. DIFFERENTIAL EQUATIONS

4.3.2 Linear ODE’s of Order at least 2

General nth order linear ode is of form:

an (x) y (n)+an 1 (x) y (n 1)+:::::::+a1 (x) y 0+a0 (x) y =

g ( x)

Use symbolic notation:

d dr d ry
D ; Dr so Dr y
dx dx r dx r

dr
) ar D r a r ( x) r
so
dx

d ry
ar D r y = a r ( x)
dx r

Now introduce

225
4. DIFFERENTIAL EQUATIONS

L = anDn+an 1Dn 1+an 2Dn 2+::::::::::::+a1D+a0

so we can write a linear ode in the form

Ly=g

L Linear Di¤erential Operator of order n and its de…-

nition will be used throughout.

If g (x) = 0 8x, then L y = 0 is said to be HOMOGE-

NEOUS.

L y = 0 is said to be the homogeneous part of L y = g:

L is a linear operator because as is trivially veri…ed:

(1) L (y1 + y2) = L (y1) + L(y2)

(2) L (cy ) = cL (y ) c2R

226
4. DIFFERENTIAL EQUATIONS

GS of Ly = g is given by

y = yc + yp

where yc Complimentary Function & yp Particular

Integral (or Particular Solution)

)
yc is solution of Ly = 0
) GS y = yc + yp
yp is solution of Ly = g

Look at homogeneous case Ly = 0. Put s = all so-

lutions of Ly = 0. Then s forms a vector space of
dimension n. Functions y1 (x) ; :::::::::::; yn(x) are LIN-
EARLY DEPENDENT if 9 1; :::::::::; n 2 R (not all
zero) s.t

1 y 1 ( x) + 2 y 2 ( x) + ::::::::::: + n y n ( x) =0

227
4. DIFFERENTIAL EQUATIONS

Otherwise yi’s (i = 1; :::::; n) are said to be LINEARLY

INDEPENDENT (Lin. Indep.) )

whenever

1 y 1 ( x) + 2 y 2 ( x) + ::::::::::: + n y n ( x) = 0 8x

then 1 = 2 = ::::::::: = n = 0:

FACT:

(1) L nth order linear operator, then 9 n Lin. Indep.

solutions y1; :::::; yn of Ly = 0 s.t GS of Ly = 0 is
given by

y = 1y1 + 2y2 + ::::::::::: + n yn i2 R.

1 i n

(2) Any n Lin. Indep. solutions of Ly = 0 have this

property.

To solve Ly = 0 we need only …nd by "hook or by crook"

n Lin. Indep. solutions.
228
4. DIFFERENTIAL EQUATIONS

4.3.3 Linear ODE’s with Constant Coe¢ cients

Consider Homogeneous case: Ly = 0 .

All basic features appear for the case n = 2, so we

analyse this.

d2y dy
L y =a 2 +b + cy = 0 a; b; c 2 R
dx dx

Try a solution of the form y = exp ( x)

L e x = aD2 + bD + c e x

hence a 2 + b + c = 0 and so is a root of the quadratic

equation

a 2 +b +c = 0 AUXILLIARY EQUATION (A.E)

229
4. DIFFERENTIAL EQUATIONS

There are three cases to consider:

(1) b2 4ac > 0

So 1 6= 2 2 R, so GS is
y = c1 exp ( 1x) + c2 exp ( 2x)

c1 , c2 arb. const.

(2) b2 4ac = 0

b
So = 1= 2=
2a

Clearly e x is a solution of L y = 0 - but theory tells us

there exist two solutions for a 2nd

order ode. So now try y = x exp ( x)

L xe x = aD2 + bD + c xe x
= a 2+b +c xe x + (2a + b) e x
| {z } | {z }
=0 =0
= 0
230
4. DIFFERENTIAL EQUATIONS

This gives a 2nd solution ) GS is y = c1 exp ( x) +

c2x exp ( x), hence

y = (c1 + c2x) exp ( x)

(3) b2 4ac < 0

So 1 6= 2 2 C - Complex conjugate pair =p iq

where

r
b 1
p= , q= b2 4ac (6= 0)
2a 2a

Hence

y = c1 exp (p + iq ) x + c2 exp (p iq ) x
= c1epxeiqx + c2epxe iqx = epx c1eiqx + c2e iqx

Eulers identity gives exp ( i ) = cos i sin

231
4. DIFFERENTIAL EQUATIONS

Simplifying (using Euler) then gives the GS

y (x) = epx (A cos qx + B sin qx)

Examples:

(1) y 00 3y 0 4y = 0

Put y = e x to obtain A.E

A.E: 2 3 4=0!( 4) ( + 1) = 0 )
=4& 1 - 2 distinct R roots

GS y (x) = Ae4x + Be x

232
4. DIFFERENTIAL EQUATIONS

(2) y 00 8y 0 + 16y = 0

A.E 2 8 + 16 = 0 ! ( 4)2 = 0 ) =4
, 4 (2 fold root)

’go up one’, i.e. instead of y = e x, take y = xe x

GS y (x) = (C + Dx) e4x

(3) y 00 3y 0 + 4 y = 0
p
2 3 9 16
A.E 3 +4 = 0 ! = =
p 2
3 i 7
2
p p
3+i 7 3 i 7
1= , 2 = p iq
2 2
p !
3 7
p= , q=
2 2

233
4. DIFFERENTIAL EQUATIONS

p p !
3x 7 7
y= e2 a cos x + b sin x
2 2

234
4. DIFFERENTIAL EQUATIONS

4.4 General nth Order Equation

Consider

L y = any (n) + an 1y (n 1) + :::::::::: + a1y 0 + a0y = 0

then

L b n 1Dn 1 + a
Dn + a b n 2D n 2 + ::::::::::
b 1D + a
+a b0
b i 2 R (0
a i n 1)

ai
bi =
we have divided through by an, i.e. a so L y =
an
0

A.E becomes n b n 1 n 1 + ::::::::::::: + a

+a b1 + a
b0 = 0

235
4. DIFFERENTIAL EQUATIONS

Case 1 (Basic)

n distinct roots 1; :::::::::; n then e 1x, e 2x, ........,

e nx are n Lin. Indep. solutions giving a GS

y = 1e 1x + 2e 2x + :::::::: + ne nx

i arb.

Case 2

If is a real r fold root of the A.E then e x, xe x,

x2e x,.........., xr 1e x are r Lin. Indep. solutions of
Ly = 0, i.e.

y=e x 1 + 2x + 3x2:::::::: + r xr 1

i arb.

236
4. DIFFERENTIAL EQUATIONS

Case 3

If = p + iq is a r - fold root of the A.E then so is p iq

)
epx cos qx, xepx cos qx, ..........,xr 1epx cos qx
epx sin qx, xepx sin qx, ............,xr 1epx sin qx
! 2r Lin. Indep. solutions of L y = 0

GS y = epx c1 + c2x + c3x2 + :::::::::::: cos qx +

epx C1 + C2x + C3x2 + :::::::::::: sin qx

237
4. DIFFERENTIAL EQUATIONS

Examples: Find the GS of each ODE

(1) y (4) 5y 00 + 6y = 0

A.E: 4 5 2+6=0 ! 2 2 2 3 =0

p p
So = 2, = 3 - four distinct roots
p p p p
) GS y = Ae 2x +Be 2x +Ce 3x +De 3x (Case
1)

d 6y d 4y
(2) 5 4 =0
dx6 dx

6 4
p
A.E: 5 =0 roots: 0; 0; 0; 0; 5
p p
GS y = Ae 5x + Be 5x + C + Dx + Ex2 + F x3
(* exp(0) = 1)

d 4y d 2y
(3) 4
+2 2 +y =0
dx dx

238
4. DIFFERENTIAL EQUATIONS

2
A.E: 4 + 2 2 + 1 = 2 +1 =0 = i is a 2
fold root.

Example of Case (3)

y = A cos x + Bx cos x + C sin x + Dx sin x

239
4. DIFFERENTIAL EQUATIONS

4.5 Non-Homogeneous Case - Method of

Undetermined Coe¢ cients

GS y = C.F + P.I

C.F comes from the roots of the A.E

There are three methods for …nding P.I

(a) "Guesswork" - which we are interested in

(b) Annihilator

(c) D-operator Method

(a) Guesswork Method

If the rhs of the ode g (x) is of a certain type, we can

guess the form of P.S. We then try it out and determine
the numerical coe¢ cients.

240
4. DIFFERENTIAL EQUATIONS

The method will work when g (x) has the following forms

i. Polynomial in x g (x) = p0 + p1x + p2x2 + :::::::::: +

pmxm.

ii. An exponential g (x) = Cekx (Provided k is not

a root of A.E).

iii. Trigonometric terms, g (x) has the form sin ax, cos ax
(Provided ia is not a root of A.E).

iv. g (x) is a combination of i. , ii. , iii. provided g (x)

does not contain part of the C.F (in which case use other
methods).

241
4. DIFFERENTIAL EQUATIONS

Examples:

(1) y 00 + 3y 0 + 2y = x2

GS y = C.F + P.I = yc + yp

C.F: A.E gives

2+3 +2 = 0 ) = 1; 2 ) yc = ae x + be 2x

P.I Now g (x) = x2,

so try yp = p0 + p1x + p2x2 ! yp0 = p1 +

2p2x ! yp00 = 2p2

Now substitute these in to the DE, ie

2p2 + 3 (p1 + 2p2x) + 2 p0 + p1x + p2x2 = x2 and

equate coe¢ cients of xn

O x2 : 2p2 = 1 ) p2 = 12

242
4. DIFFERENTIAL EQUATIONS

O ( x) : 6p2 + 2p1 = 0 ) p1 = 3
2

O x0 : 2p2 + 3p1 + 2p0 = 0 ) p0 = 74

7 3 1
) GS y = ae x + be 2x + x + x2
4 2 2

243
4. DIFFERENTIAL EQUATIONS

(2) y 00 + 3y 0 + 2y = 3e5x

The homogeneous part is the same as in (1), so yc =

Ae x + Be 2x. For the non-homog. part we note that
g (x) has the form ekx, so try yp = Ce5x, and k = 5 is
not a solution of the A.E.

Substituting yp into the DE gives

1
C 52 + 15 + 2 e5x = 3e5x ! C =
14

1
) y = Ae x + Be 2x + e5x
14

244
4. DIFFERENTIAL EQUATIONS

(3) y 00 5y 0 6y = cos 3x

A.E: 2 6=0) = 1, 6 ) yc = e x+ e6x

Guided by the rhs, i.e. g (x) is a trigonometric term, we

try

yp = A cos 3x + B sin 3x

! yp0 = 3A sin 3x+3B cos 3x ! yp00 = 9A cos 3x

9B sin 3x and substitute into DE.

Collecting coe¢ cients of sin 3x and cos 3x gives:

O (cos 3x) : 9A 15B 6A = 1

O (sin 3x) : 9B + 15A 6B = 0

A = 30 1 = B ! y = 1 (cos 3x + sin 3x), so

p 30
general solution becomes

1
y = e x + e6x (cos 3x + sin 3x)
30

245
4. DIFFERENTIAL EQUATIONS

4.5.1 Failure Case

Consider the DE y 00 5y 0 + 6y = e2x, which has a CF

given by y (x) = e2x + e3x. To …nd a

PS, if we try yp = Ae2x, we have upon substitution

Ae2x [4 10 + 6] = e2x

so when k (= 2) is also a solution of the C.F , then

the trial solution yp = Aekx fails, so we must seek the
existence of an alternative solution.

The methods given should be used in such cases.

Statement

a) Ly = y 00 + ay 0 + b = ekx - trial function is normally

yp = Cekx:

246
4. DIFFERENTIAL EQUATIONS

If k is a root of the A.E then L Cekx = 0 so this

substitution does not work. In this case, we try yp =
Cxekx - so ’go one up’.

This works provided k is not a repeated root of the A.E,

if so try yp = Cx2ekx, and so forth ....

b) Ly = g where g (x) has the form ( sin mx + cos mx) epx

try

yp = (c1 sin mx + c2 cos mx) epx

provided p + im is not a root of the A.E. If p + im is

a root then ’go one up’so try

yp = (c1 sin mx + c2 cos mx) xepx , etc.

c) Finally, if g (x) = g1 (x) + g2 (x) + g3 (x) where

247
4. DIFFERENTIAL EQUATIONS

m
X
g1 (x) = gk xk , g2 (x) = Cekx,
k=0
g3 (x) = ( sin mx + cos mx) epx

Then try

yp = y p (x) + yep (x) + ybp (x)

where

y p (x) = p0 + p1x + p2x2 + :::::::::: + pmxm

yep (x) = Cekx
ybp (x) = (c1 sin mx + c2 cos mx) epx

248
4. DIFFERENTIAL EQUATIONS

4.6 Linear ODE’s with Variable Coe¢ cients

- Euler Equation

In the previous sections we have looked at various second

order DE’s with constant coe¢ cients. We now introduce
a 2nd order equation in which the coe¢ cients are variable
in x. An equation of the form

2 d2y dy
Ly= ax + x + cy = g (x)
dx2 dx

is called a Cauchy-Euler equation. Note the relation-

ship between the coe¢ cient and corresponding derivative
d ny
term, ie an (x) = axn and n
, i.e. both power and
dx
order of derivative are n.

The equation is still linear. To solve the homogeneous

part, we look for a solution of the form

y=x
249
4. DIFFERENTIAL EQUATIONS

So y 0 = x 1 ! y 00 = ( 1) x 2 , which upon
substitution yields the quadratic, A.E.

a 2+b +c=0

[where b = ( a)] which can be solved in the usual

way - there are 3 cases to consider, depending upon the
nature of b2 4ac.

250
4. DIFFERENTIAL EQUATIONS

Case 1: b2 4ac > 0 ! 1, 2 2 R - 2 real

distinct roots

GS y = Ax 1 + Bx 2

Case 2: b2 4ac = 0 ! = 1 = 2 2 R-1

real (double fold) root

GS y = x (A + B ln x)

Case 3: b2 4ac < 0 ! = i 2 C - pair of

complex conjugate roots

GS y = x (A cos ( ln x) + B sin ( ln x))

Example 1 Solve x2y 00 2xy 0 4y = 0

251
4. DIFFERENTIAL EQUATIONS

Put y = x ) y 0 = x 1 ) y 00 = ( 1) x 2
and substitute in DE to obtain (upon simpli…cation) the
A.E. 2 3 4=0!( 4) ( + 1) = 0

) =4& 1 : 2 distinct R roots. So GS is

y (x) = Ax4 + Bx 1

Example 2 Solve x2y 00 7xy 0 + 16y = 0

So assume y = x

A.E 2 8 + 16 = 0 ) = 4 , 4 (2 fold root)

’go up one’, i.e. instead of y = x , take y = x ln x to

give
y (x) = x4 (A + B ln x)

252
4. DIFFERENTIAL EQUATIONS

Example 3 Solve x2y 00 3xy 0 + 13y = 0

Assume existence of solution of the form y = x

p
2 4 16 52
A.E becomes 4 +13 = 0 ! = =
2
4 6i
2

1 = 2+3i, 2 = 2 3i i ( = 2, = 3)

y = x2 (A cos (3 ln x) + B sin (3 ln x))

253
4. DIFFERENTIAL EQUATIONS

4.6.1 Reduction to constant coe¢ cient

The Euler equation considered above can be reduced to

the constant coe¢ cient problem discussed earlier by use
of a suitable transform. To illustrate this simple technique
we use a speci…c example.

Solve x2y 00 xy 0 + y = ln x

Use the substitution x = et i.e. t = ln x. We now

rewrite the the equation in terms of the variable t, so
require new expressions for the derivatives (chain rule):

dy dy dt 1 dy
= =
dx dt dx x dt

d 2y d dy d 1 dy 1 d dy 1 dy
2
= = =
dx dx dx dx x dt x dx dt x2 dt
1 dt d dy 1 dy 1 d2y 1 dy
= =
x dx dt dt x2 dt x2 dt2 x2 dt

254
4. DIFFERENTIAL EQUATIONS

) the Euler equation becomes

!
1 d2y 1 dy 1 dy
x2 x +y =t !
x2 dt2 x2 dt x dt

y 00 (t) 2y 0 (t) + y = t

The solution of the homogeneous part , ie C.F. is yc =

et (A + Bt) :

The particular solution (P.S.) is obtained by using yp =

p0 + p1t to give yp = 2 + t

The GS of this equation becomes

y (t) = et (A + Bt) + 2 + t

which is a function of t . The original problem was y =

y (x), so we use our transformation t = ln x to get the
GS

y = x (A + B ln x) + 2 + ln x.

255
4. DIFFERENTIAL EQUATIONS

4.7 Partial Di¤erential Equations

4.7.1 Introduction

The formation (and solution) of PDE’s forms the ba-

sis of a large number of mathematical models used to
study physical situations arising in science, engineering
and medicine.

More recently their use has extended to the modelling of

problems in …nance and economics.

We now look at the second type of DE, i.e. PDE’s. These

have partial derivatives instead of ordinary derivatives.

One of the underlying equations in …nance, the Black-

Scholes equation for the price of an option V (S; t) is
an example of a linear PDE

@V 1 2 2 @ 2V @V
+ S 2
+ (r D) S rV = 0
@t 2 @S @S

providing ; D; r are not functions of V or any of

its derivatives.
256
4. DIFFERENTIAL EQUATIONS

4.7.2 Similarity Reduction

Model equation is
@p 2 @ 2p
=c
@t @y 2

for the unknown function p = p (y; t) which is a proba-

bility density function.

We assume a solution of the following form exists:

y
p (y; t) = t f
t

where ; are constants to be determined. So put

y
=
t

which allows us to obtain the following derivatives

@ 1 @ 1
= ; = yt
@y t @t

257
4. DIFFERENTIAL EQUATIONS

we can now say

p (y; t) = t f ( )

therefore
@p @p @ 0 1
= = t f ( ): = t f0 ( )
@y @ @y t
!
@ 2p @ @p @
2
= = t f0 ( )
@y @y @y @y
@ @
= t f0 ( )
@y @
1 @ 0 2 f 00 (
= t f ( )=t )
t @

@p @ 1f
=t f( )+ t ( )
@t @t

we can use the chain rule to write

@ @f @ 1f 0 (
f( )= : = yt )
@t @ @t

258
4. DIFFERENTIAL EQUATIONS

so we have
@p 1f 1f 0 (
= t ( ) yt )
@t

and then substituting these expressions in to the pde gives

t 1f ( ) yt 1f 0 ( ) = c2 t 2 f 00 :

We know from that

y=t

hence the equation above becomes

t 1f ( ) t 1f 0 ( ) = c2 t 2 f 00 :

For the similarity solution to exist we require the equation

to be independent of t; i.e. 1= 2 =) = 1=2;
therefore
1 0
f f = c2f 00
2

thus we have so far

p = t f py
t
259
4. DIFFERENTIAL EQUATIONS

which gives us a whole family of solutions dependent upon

the choice of :

We know that p represents a pdf, hence

Z Z
p (y; t) dy = 1 = t f py dy
R R t

p p
change of variables u = y= t ! du = dy= t so the
integral becomes
Z 1
t +1=2 f (u) du = 1
1

which we need to normalise independent of time t: This

is only possible if = 1=2:

So the D.E becomes

1
f + f 0 = c2f 00:
2

d
We have an exact derivative on the lhs, i.e. ( f) =
d
f + f 0, hence
1d
( f ) = c2f 00
2d
260
4. DIFFERENTIAL EQUATIONS

and we can integrate once to get

1
( f ) = c2f 0 + K:
2

We set K = 0 in order to get the correct solution, i.e.

1
( f ) = c2 f 0
2

which can be solved as a simple …rst order variable sepa-

rable equation:

f ( ) = A exp 1 2
4c2

and returning to

p (y; t) = t 1=2f ( )

becomes
!
A y2
p (y; t) = p exp :
t 4tc2

261
4. DIFFERENTIAL EQUATIONS

This is a pdf for a variable y that is normally distributed

p
with mean zero and standard deviation c 2t; which we
ascertained by the following comparison:

1 y2 1 (x )2
:
2 2tc2 2 2

i.e. 0 and 2 2tc2:

262