Jim Lambers

MAT 415/515
Fall Semester 2013-14
Lecture 3 Notes
These notes correspond to Section 5.2 in the text.
Gram-Schmidt Orthogonalization
We have seen that it can be very convenient to have an orthonormal basis for a given vector
space, in order to compute expansions of arbitrary vectors within that space. Therefore, given a
non-orthonormal basis, it is desirable to have a process for obtaining an orthonormal basis from it.
Fortunately, we have such a process, known as Gram-Schmidt orthogonalization. Suppose that
we have a linearly independent, but not orthonormal, set of functions {
1
,
2
, . . .} that span a given
vector space V . To construct an orthonormal set {
1
,
2
, . . .} from this set, we proceed as follows.
First, to obtain
1
, we simply normalize
1
:

1
=

1

.
Next, to obtain
2
, we need to ensure that it is orthogonal to
1
, and then normalize it.
As an intermediate step, we seek a function
2
of the form

2
=
2
+c
12

1
such that
1
|
2
= 0. Then, we can set
2
=
2
/
2
. Taking the scalar product of both sides of
the above equation with
1
, we obtain
0 =
1
|
2
=
1
|
2
+c
12

1
|
1
.
Because the
j
are orthonormal, it follows that
c
12
=
1
|
2
.
We conclude that
2
can be obtained as follows:

2
=
2

1
|
2

2
=

2

.
We now have a set of two functions that is orthonormal.
Now, to obtain
3
, we must ensure that it is orthogonal to
1
and
2
, and then normalized.
To that end, we seek a function
3
of the form

3
=
3
+c
13

1
+c
23

2
such that
1
|
3
=
2
|
3
= 0. Then, we can set
3
=
3
/
3
. Taking the scalar product of
both sides of the above equation with
1
, and then separately,
2
, we obtain
0 =
1
|
3
=
1
|
3
+c
13

1
|
1
+c
23

1
|
2

0 =
2
|
3
=
2
|
3
+c
13

2
|
1
+c
23

2
|
2
.
1
Because the
j
are orthonormal, it follows that
c
13
=
1
|
3
, c
23
=
2
|
3
.
We conclude that
3
can be obtained as follows:

3
=
3

1
|
3

1

2
|
3

3
=

3

.
We now have a set of three functions that are orthonormal.
Continuing this process, we see that we can obtain each function
j
as follows:

j
=
j

j1

k=0

k
|
j

j
=

j

.
This yields a set of functions {
1
,
2
, . . .} that is an orthonormal basis of the space spanned by
{
1
,
2
, . . .}, with respect to the scalar product that is used.
Example We wish to obtain a set of orthonormal polynomials with respect to the scalar product
f|g =
_
1
1
f

(s)g(s) ds.
This will be accomplished by applying Gram-Schmidt orthogonalization to the set {1, x, x
2
, x
3
, . . .}.
Setting
j
(x) = x
j
for j = 0, 1, 2, . . . , our orthogonal set {
j
}, j = 0, 1, 2, . . . ,, is obtained as follows:

0
(x) =
0
(x)
= 1,

0
(x) =

0
(x)

=
1
1|1
1/2
=
1
_
_
1
1
1 ds
_
1/2
=
1

2
,

1
(x) =
1
(x)
0
|
1

0
(x)
= x
_
1

x
_
1

2
= x
1
2
_
1
1
s ds
= x
1
2
0
= x,
2

1
(x) =

1
(x)

=
x
x|x
1/2
=
x
_
_
1
1
s
2
ds
_
1/2
=
_
3
2
x,

2
(x) =
2
(x)
0
|
2

0
(x)
1
|
2

1
(x)
= x
2

_
1

x
2
_
1

_
_
3
2
x

x
2
_
_
3
2
x
= x
2

1
2
_
1
1
s
2
ds
3
2
x
_
1
1
s
3
ds
= x
2

1
2
2
3

3
2
0
= x
2

1
3
,

2
(x) =

2
(x)

=
x
2

1
3

x
2

1
3

x
2

1
3
_
1/2
=
x
2

1
3
_
_
1
1
_
x
1
3
_
2
ds
_
1/2
=
_
5
2
_
3
2
x
2

1
2
_
.
Continuing this process, we obtain

3
(x) =
_
7
2
_
5
2
x
3

3
2
x
_
.
In general,

n
(x) =
_
2n + 1
2
P
n
(x),
where P
n
(x) is the Legendre polynomial of nth degree. We will learn more about these orthogonal
(but not orthonormal) polynomials later in this course. 2
While Gram-Schmidt orthogonalization can be applied to the monomial bais {1, x, x
2
, x
3
, . . .}
to obtain an orthonormal sequence of polynomials, it can be quite cumbersome, as can be seen from
the preceding example. However, a modication of this procedure can yield a much more ecient
approach.
Suppose that we have already generated a sequence of n orthonormal polynomials
0
,
1
,
2
, . . . ,
n1
with respect to some scalar product
f|g =
_
b
a
f

(x)g(x)w(x) dx,
3
where
j
is of degree j for j = 0, 1, 2, . . . , n 1. Then, to obtain
n
, which of degree n, we
orthogonalize x
n1
(x), which is of degree n, against
0
,
1
, . . . ,
n1
using the same approach as
in Gram-Schmidt orthogonalization. That is, we compute

n
(x) = x
n1
(x)
n1

j=0

j
|x
n1

j
(x),

n
(x) =

n
(x)

.
Now, consider the scalar product
j
|x
n1
. Using the properties of the scalar product, we
have

j
|x
n1
= x
j
|
n1
.
However, because
0
,
1
, . . . ,
n1
are orthonormal, p|
n1
= 0 if p(x) is any polynomial of
degree less than n 1. Because x
j
(x) is of degree j + 1, it follows that
j
|x
n1
= 0 whenever
j + 1 < n 1, or j < n 2. Therefore, our orthogonalization procedure simplies to

n
(x) = x
n1
(x)
n2
|x
n1

n2
(x)
n1
|x
n1

n1
(x),

n
(x) =

n
(x)

.
That is, any family of orthogonal polynomials satises a three-term recurrence relation, in which
each polynomial depends on the previous two. Table lists several families of orthogonal polynomials
that can be generated from such a recurrence relation; we will see some of these families later in
the course.
Polynomials Scalar Product
Legendre
_
1
1
P
n
(x)P
m
(x) dx = 2
mn
/(2n + 1)
Shifted Legendre
_
1
0
P

n
(x)P

m
(x) dx =
mn
/(2n + 1)
Chebyshev, rst kind
_
1
1
T
n
(x)T
m
(x)(1 x
2
)
1/2
dx =
mn
/(2
n0
)
Shifted Chebyshev, rst kind
_
1
0
T

n
(x)T

m
(x)[x(1 x)]
1/2
dx =
mn
/(2
n0
)
Chebyshev, second kind
_
1
1
U
n
(x)U
m
(x)(1 x
2
)
1/2
dx =
mn
/2
Leguerre
_

0
L
n
(x)L
m
(x)e
x
dx =
mn
Associated Laguerre
_

0
L
k
n
(x)L
k
m
(x)e
x
dx =
mn
(n +k)!/n!
Hermite
_

H
n
(x)H
m
(x)e
x
2
dx = 2
n

mn

n!
As can be seen in the following example, Gram-Schmidt orthogonalization can be applied to
vectors in any inner product space, such as vectors in R
n
.
Example Given the vectors in R
3
,
|a
1
=
_
_
1
1
2
_
_
, |a
2
=
_
_
1
2
3
_
_
, |a
3
=
_
_
0
1
1
_
_
,
we will use Gram-Schmidt orthogonalization to obtain an orthonormal set of vectors, {|b
1
, |b
2
, |b
3
}.
We have
|b
1
=
|a
1

a
1
|a
1

1/2
4
=
1
6
1/2
|a
1

=
1

6
_
_
1
1
2
_
_
,
|b

2
= |a
2
b
1
|a
2
|b
1

=
_
_
1
2
3
_
_

6
1

6
_
_
1
1
2
_
_
=
_
_
1/2
1/2
0
_
_
,
|b
2
=
|b

2
|b

1/2
=
1
_
1
2
_
1/2
|b

=
1

2
_
_
1
1
0
_
_
,
|b

3
= |a
3
b
1
|a
3
|b
1
b
2
|a
3
|b
2

=
_
_
0
1
1
_
_

6
_
1

6
_
_
1
1
2
_
_

2
1

2
_
_
1
1
0
_
_
=
_
_
0
1
1
_
_
+
1
6
_
_
1
1
2
_
_

1
2
_
_
1
1
0
_
_
=
2
3
_
_
1
1
1
_
_
,
|b
3
=
|b

3
|b

1/2
=
1
_
4
3
_
1/2
|b

=
1

3
_
_
1
1
1
_
_
.
We conclude that our orthonormal set of vectors is
|b
1
=
1

6
_
_
1
1
2
_
_
, |b
2
=
1

2
_
_
1
1
0
_
_
, |b
3
=
1

3
_
_
1
1
1
_
_
.
2
5