INNER PRODUCT SPACES AND FOURIER SERIES

STEPHANIE YOUNGMI OH

Abstract. The aim of this paper is to discuss the fundamentals of Fourier

analysis with a focus on relevant inner product spaces. We begin with a brief
introduction to inner product spaces. We then explore the space of periodic
complex integrable functions and conclude with some results about the con-
vergence of Fourier series.

Contents
1. Inner product spaces 1
2. Fourier series 4
Appendix 10
Acknowledgments 10
References 10

Fourier analysis is the study of how general functions can be represented as

weighted sums of trigonometric functions. Specifically, sine and cosine functions
form a basis for the space of periodic integrable functions. Linear algebra thus
plays an important role in Fourier analysis. This paper briefly introduces inner
product spaces and Fourier series before culminating in some results that make
clear the connection between these two areas.

1. Inner product spaces

In this section, we present the subject of inner product spaces. This will be
applied to Fourier analysis in the following section.
Definition 1.1. Let V be a complex vector space. A complex inner product
on V is a binary operation < ·, · >: V × V → V which satisfies the following for all
x, y, z ∈ V and a, b ∈ C:
(1) conjugate symmetry: hx, yi = hy, xi
(2) linearity in the first term: hax + by, zi = ahx, zi + bhy, zi
(3) non-negativity: hx, xi ≥ 0
(4) non-degeneracy: hx, xi = 0 if and only if x = 0.
Inner products on real vector spaces are defined in a similar way. Going forward,
“inner product” will usually mean “complex inner product.”
Definition 1.2. A complex vector space with an inner product is called a (complex)
inner product space.

Date: October 20, 2020.

1
2 STEPHANIE YOUNGMI OH

Throughout this section, V will denote an arbtirary inner product space. The
following example gives valuable intuition for studying any V .
ExampleP1.3. If x, y are in the vector space Cn , define their inner product to be
n
hx, yi = k=1 xk yk . This is called the (complex) standard inner product.
In fact, inner product spaces generalize the notion of the standard inner product
on Rn or Cn . We can therefore think of the inner product of two vectors as encoding
some sense of “overlap” between the vectors.
p
Definition 1.4. Let x ∈ V . The norm of x is defined by kxk = hx, xi.
Thus, just as the norm of a vector in Rn is its magnitude, the norm of a vector
in a general inner product space is essentially its “length”. Moreover, we can think
of the norm of the difference of two vectors (e.g. kx − yk) as being the distance
between the vectors.
Definition 1.5. Let x, y ∈ V . If hx, yi = 0, then we say that x and y are orthog-
onal.
Definition 1.6. Let {x1 , x2 , ..., xn } be a subset of V . If every pair of distinct xj , xk
is orthogonal, then the set is called orthogonal. If, in addition, kxk k = 1 for all
k, the set is called orthonormal.
Example 1.7. The standard basis for the complex vector space Cn is
{e1 = (1, 0, ..., 0), e2 = (0, 1, ..., 0), ..., en = (0, 0, ..., 1)}.
This is an orthonormal set of vectors.
We will use the following familiar results in the next section.
Lemma 1.8. Let x, y ∈ V . Then hx + y, x + yi = hx, xi + hy, yi + hx, yi + hy, xi.

Proof. hx + y, x + yi = hx + y, x + yi
= hx, x + yi + hy, x + yi
= hx, x + yi + hy, x + yi
= hx + y, xi + hx + y, yi
= hx, xi + hy, yi + hx, yi + hy, xi.


Theorem 1.9. (Pythagorean theorem) Suppose x, y ∈ V are orthogonal. Then

kxk2 + kyk2 = kx + yk2 .
Proof. kxk2 + kyk2 = hx, xi + hy, yi
= hx, xi + hy, yi + hx, yi + hy, xi (by orthogonality of x, y)
= hx + y, x + yi (by Lemma 1.8)
= kx + yk2 .


Lemma 1.10. (Cauchy-Schwarz) Let x, y ∈ V . Then |hx, yi| ≤ kxkkyk.

 INNER PRODUCT SPACES AND FOURIER SERIES 3

Proof. If y = 0, we have |hx, yi| = |hx, 0i| = |h0, xi| = |hx, xi + h−x, xi = 0
= kxkkyk.
hx,yi
If y 6= 0, let c = hy,yi . Notice that
hx − cy, yi = hx, yi − chy, yi
hx, yi
= hx, yi − hy, yi
hy, yi
= hx, yi − hx, yi
= 0.
Then x − cy and y are orthogonal, and we can apply Theorem 1.9 to get
kxk2 = kx − cy + cyk2 = kx − cyk2 + kcyk2
≥ kcyk2 = hcy, cyi = chy, cyi = chcy, yi = c2 hy, yi = |c|2 hy, yi = |c|2 kyk2
hx, yi2
= kyk2 .
hy, yi2
2
This implies that kxk2 hy, yi ≥ hx,yi 2 2 2
hy,yi hy, yi. Thus, we have kxk kyk ≥ hx, yi .
Taking square roots gives the desired inequality. 

Theorem 1.11. (Triangle inequality) Let x, y ∈ V . Then kx + yk ≤ kxk + kyk.

Proof. (kxk + kyk)2 = kxk2 + kyk2 + 2kxkkyk
= hx, xi + hy, yi + 2kxkkyk
≥ hx, xi + hy, yi + 2|hx, yi| (by Lemma 1.11)
= hx, xi + hy, yi + |hx, yi| + |hy, xi|
≥ hx, xi + hy, yi + hx, yi + hy, xi
= hx + y, x + yi (by Lemma 1.8)
= kx + yk2 .
Taking square roots gives the desired inequality. 

We now introduce another important object.

Definition 1.12. Let W be a subspace of V . Let x ∈ V . A projection of x onto
W is a vector y ∈ W where x − y is orthogonal to every vector in W .
If W is finite-dimensional, then a projection of a vector onto W exists and is
unique. We do not give full justification here, but the proof of this fact follows from
the next two propositions.
Proposition 1.13. Let E = {e1 , e2 , ..., eN } be an orthonormal subset of V and let
PN
x ∈ V . Define cn = hx, en i and s = n=1 cn en . Then s is a projection of x onto
the span of E.
Proof. Let ek ∈ E be arbitrary. We will start by showing that x − s is orthogonal
PN
to ek . Since x − s = x + n=1 −cn en , we have by linearity of the inner product
that
hx − s, ek i = hx, ek i − c1 he1 , ek i − ... − ck hek , ek i − ... − cN heN , eN i.
4 STEPHANIE YOUNGMI OH

By the orthonormality of E, this is equal to hx, ek i − ck . Thus, we have that

hx − s, ek i = ck − ck = 0, so x − s and ek are orthogonal.
Now let y be an arbitrary vector in the span of E. We can represent y as a linear
PN
combination of vectors in E, y = n=1 bn en . Then
hx − s, yi = hy, x − si
= hb1 e1 + b2 e2 + ... + bN eN , x − si
= b1 he1 , x − si + b2 he2 , x − si + ... + bN heN , x − si
= b1 hx − s, e1 i + b2 hx − s, es i + ... + bN hx − s, eN i
= 0.
Since y was arbitrary, we have shown that s is the projection of x onto the span
of E. 

Proposition 1.14. Let E = {e1 , e2 , ..., en } be an orthonormal subset of V and let

PN
x ∈ V . Define cn = hx, en i and s = n=1 cn en . Then for any y in the span of E,
we have kx − sk ≤ kx − yk.
Proof. Fix an arbitrary y in the span of E. By the previous proposition, we have
hx − s, y − si = hx − s, yi + hx − s, −si = 0.
Then we can apply Theorem 1.9 to get
kx − sk2 + ky − sk2 = kx − sk2 + ks − yk2
= kx − s + s − yk2
= kx − yk2 .
This implies that kx − sk2 ≤ kx − sk2 + ky − sk2 = kx − yk2 . 

This proposition says that the projection of x onto the span of E gives a better
approximation to x than any other vector in the span of E.

2. Fourier series
In this section, we will discuss the space of complex periodic integrable functions
and their representation by weighted sums of trigonometric functions. We begin by
describing this space before giving a definition of Fourier series. We then present
some results about Fourier series in the context of the theory of inner product spaces
constructed in the previous section. Bessel’s inequality and Parseval’s identity
provide a relation between norms of functions and their Fourier coefficients. The
best approximation proposition recalls a conclusion from the previous section about
the distance between vectors and their projections. The most important proof we
give is that of the mean square convergence theorem, which describes one sense in
which the Fourier series of functions converge.
Proposition 2.1. Let R denote the vector space of 2π-periodic complex-valued
Riemann integrable functions. Then R is an inner product space with the inner
product defined by
Z 2π
1
hf, gi = f (x)g(x)
2π 0
 INNER PRODUCT SPACES AND FOURIER SERIES 5

and norm defined by

Z 2π
2 1
kf k = hf, f i = |f (x)|2 dx
2π 0
for f, g ∈ R.
Proof. Let f, g, h ∈ R, with f (x) = a(x) + ib(x) and g(x) = c(x) + id(x).
First, we need to check that our operation is conjugate symmetric. We have
Z 2π
1
hf, gi = f (x)g(x)dx
2π 0
Z 2π
1
= [(a(x) + ib(x))(c(x) − id(x))]dx
2π 0
Z 2π Z 2π
1 i
= [a(x)c(x) − b(x)d(x)]dx − [a(x)d(x) − b(x)c(x)]dx
2π 0 2π 0
Z 2π
1
= [(c(x) + id(x)(a(x) − b(x))]dx
2π 0
Z 2π
1
= g(x)f (x)dx
2π 0
= hg, f i.
Next, we need to show that the operation is linear. Let α, β ∈ C. Then
Z 2π
1
hαf + βg, hi = [αf (x) + βg(x)]h(x)dx
2π 0
Z 2π
1
= [αf (x)h(x) + βg(x)h(x)]dx
2π 0
Z 2π Z 2π
α β
= f (x)h(x)dx + g(x)h(x)dx
2π 0 2π 0
= αhf, hi + βhg, hi.
Finally, we need to show that the operation is non-negative and non-degenerate.
Since |f (x)|2 is always non-negative, we must also always have
Z 2π
1
hf, f i = |f (x)|2 dx ≥ 0.
2π 0
If f = 0, then
Z 2π
1
hf, f i = 0dx = 0.
2π 0
On the other hand, if
Z 2π
1
hf, f i = |f (x)|2 = 0,
2π 0
the function must be 0 everywhere, since the integrand never takes on negative
values. Thus, hf, f i = 0 if and only if f = 0. 
We next introduce an orthonormal subset of R that is essential to Fourier anal-
ysis.
Proposition 2.2. Let en (x) = einx . Then the set {en }n∈Z is orthonormal.
6 STEPHANIE YOUNGMI OH

Proof. Let j, k ∈ Z. Then

Z 2π
1
hej , ek i = eijx eikx dx
2π 0
Z 2π
1
= eijx e−ikx dx
2π 0
Z 2pi
1
= e(j−k)ix dx
2π 0
1
R 2π 0
If j = k, this expression equals 2π 0
e dx = 1.
1
2π
If j 6= k, this expression equals 2π(j−k)i e(j−k)ix 0 = 0.
In summary, we have (
1 if j = k
hej , ek i =
0 if j 6= k
and so {en }n∈Z is an orthonormal set.

Definition 2.3. Let f ∈ R and n ∈ Z. The nth Fourier coefficient of f is
Z 2π
1
an = f (x)e−inx dx.
2π 0
The Fourier series of f is
∞
X
an einx .
n=−∞

The Nth partial sum of the Fourier series of f is

N
X
sN (f, x) = an einx .
n=−N

The central idea of Fourier series is that the set {en }n∈Z is a basis for R; that
is, every periodic complex integrable function can be uniquely represented as a
linear combination of sines and cosines. The following results are a step towards
demonstrating this concept.
Proposition 2.4. Let f ∈ R andP suppose {on }n∈Z is an orthonormal subset of R.
Define an = hf, on i and tN (f ) = |n|≤N an on . Then tN (f ) is the projection of f
onto the span of {on }|n|≤N .
Proof. Let ok ∈ {on }|n|≤N be arbitrary. Then
X
hf − tN (f ), ok i = hf − an on , ok i
|n|≤N
X
= hf, ok i − an hon , ok i
|n|≤N

= hf, ok i − ak
= ak − ak
= 0.
 INNER PRODUCT SPACES AND FOURIER SERIES 7

P
Now let y = |n|≤N cn on with cn ∈ C be some vector in the span of {on }|n|≤N .
Then we have
X
hf − tN (f ), yi = h cn on , f − tN (f )i
|n|≤N

= hc−N o−N , f − tN (f )i + hc−N +1 o−N +1 , f − tN (f )i + ... + hcN oN , f − tN (f )i

= c−N hf − tN (f ), o−N i + c−N +1 hf − tN (f ), o−N +1 i + ... + cN hf − tN (f ), oN i
= 0.
.
Thus, f − tN (f ) is orthogonal to the span of {on }|n|≤N . 
Proposition 2.5. (Bessel’s inequality) Let f ∈ R and suppose {on }n∈Z is an
orthonormal subset of R. Let an = hf, on i. Then
∞
X
|an |2 ≤ kf k2 .
n=−∞
P
Proof. Let tN (f ) = |n|≤N an on as before. Since tN (f ) is in the span of {on }|n|≤N ,
it is orthogonal to f − tN (f ) by Proposition 2.4. Thus, we can apply Theorem 1.9
to get
kf k2 = kf − tN (f ) + tN (f )k2
= kf − tN (f )k2 + ktN (f )k2 .
Notice that X
ktN (f )k2 = k an on k2
|n|≤N
X X
= han on , an on i
|n|≤N |n|≤N
X X
= an hon , a n on i
|n|≤N |n|≤N
X X
= an h an on , on i
|n|≤N |n|≤N
X X
= an han on , on i
|n|≤N |n|≤N
X X
= an an hon , on i
|n|≤N |n|≤N
X
= |an |2 .
|n|≤N
Then we have X
kf k2 = kf − tN (f )k2 + |an |2
|n|≤N
.
Thus, X X
|an |2 ≤ kf − tN (f )k2 + |an |2 = kf k2 .
|n|≤N |n|≤N

8 STEPHANIE YOUNGMI OH

Proposition 2.6. (Best approximation) Let f ∈ R with Fourier coefficients an .

Then X
kf − sN (f )k ≤ kf − cn en k
|n|≤N

for any cn ∈ C.
Proof. First, notice that
X X X
cn en = an en − (an en − cn en )
|n|≤N |n|≤N |n|≤N
X
= sN (f ) − (an − cn )en .
|n|≤N

This gives us that

X X
kf − cn en k2 = kf − sN (f ) + (an − cn )en k2 .
|n|≤N |n|≤N
P
Since f − sN (f ) is orthogonal to |n|≤N (an − cn )en by Proposition 2.4, we can
use Theorem 1.9 to show that the right hand side of this equation equals
X
kf − sN (f )k2 + k (an − cn )en k2 .
|n|≤N

But this expression is greater than or equal to kf − sN (f )k2 .

Thus,
X
kf − sN (f )k2 ≤ kf − cn en k2
|n|≤N

and the desired inequality follows immediately. 

In words, this proposition says that the partial sum of the Fourier series of f gives
a better approximation to f than any linear combination of vectors in {en }n∈Z with
non-Fourier coefficients. Notice that this is actually a special case of Proposition
1.14.
We will use this result to prove the following theorem.
Theorem 2.7. (Mean square convergence) Suppose f ∈ R. Then
lim kf − sN (f )k = 0.
N →∞

Proof. Let f ∈ R. If f is continuous on [0, 2π], then we can use Lemma 3.1. Fix
PM
 > 0. There exists a function P (x) = −M cn einx such that |f (x) − P (x)| <  for
all x. Then we have
|f (x) − P (x)|2 < 2
Z 2π Z 2π
1 2 1
|f (x) − P (x)| dx < 2 dx
2π 0 2π 0
kf − P k < .
The previous proposition tells us that for all N ≥ M , we have
kf − sN (f )k ≤ kf − P k < 
so the limit is 0 when f is continuous.
 INNER PRODUCT SPACES AND FOURIER SERIES 9

If f is not continuous, then we need to use Lemma 3.2. Let B denote the least
upper bound of |f |. Then there exists a continuous function g such that
sup |g(x)| ≤ B
x∈[0,2π]

and
Z 2π
lim |f (x) − g(x)|dx < 2 .
k→∞ 0
This gives us that
Z 2π
1
kf − gk2 = |f (x) − g(x)|2 dx
2π 0
2B 2π
Z
≤ |f (x) − g(x)|dx
2π 0
≤ C2 ,
where C is a constant. So we have
√
kf − gk ≤ C.
PM
Using Lemma 3.1 again, there exists a function Q(x) = −M dn einx with dn ∈ C
such that |g(x) − Q(x)| < . Following the same logic as before, this implies that
kg − Qk < .
Applying the triangle inequality, we have
√ √
kf − Qk ≤ kf − gk + kg − Qk < C +  = ( C + 1).
Finally, the previous proposition gives us that
√
kf − sN (f )k ≤ kf − Qk < ( C + 1).


This theorem says that the partial sum of the Fourier series of f approximates
f with increasing accuracy as the degree of the sum approaches infinity. This is
one sense in which the Fourier series of a function converges to the function. It
implies a special case of Bessel’s inequality that highlights a connection between
the respective norms in Cn and R.
Corollary 2.8. (Parseval’s identity) If f ∈ R, then
∞
X 2
|an | = kf k2 .
n=−∞

Proof. From the proof of Proposition 2.4, we have the equality

X
kf − sN (f )k2 + |an |2 = kf k2 .
|n|≤N

The identity follows from taking the limit of both sides as N approaches infinity
and applying the previous theorem. 
10 STEPHANIE YOUNGMI OH

Appendix
The proofs of the following lemmas can be found in [1] or [3].
Lemma 2.9. Suppose f is a continuous 2π-periodic function. Then for all  > 0,
PM inx
there exists a function of the form P (x) = −M cn e with cn ∈ C such that
|f (x) − P (x)| <  for all x.
Lemma 2.10. Let f ∈ R. Suppose f is bounded by B. Then there exists a sequence
{fk }∞
k=1 of continuous 2π-periodic functions such that
sup |fk (x)| ≤ B
x∈[0,2π]

for all positive k ∈ N and

Z 2π
lim |f (x) − fk (x)|dx = 0.
k→∞ 0

Acknowledgments
Firstly, I would like to thank Dr. Brian Lawrence for working with me this
summer, and for being the best mentor anyone could ask for.
Thank you to Professor Anthony Cheung for introducing me to the subject
of Fourier analysis via Grisey’s Partiels, and for directing me to various related
resources.
My thanks as well to Evan Mata for helping me through multiple dot product
crises, and to the AGNT tea group for the distracting conversation.
Finally, thank you to Professor Peter May and all of the graduate students who
made this program possible, especially given the online format. I am very grateful
to have had the opportunity to participate.

References
[1] Elias M. Stein and Rami Shakarchi. Fourier Analysis: An Introduction. Princeton University
Press. 2003.
[2] Sergei Treil. Linear Algebra Done Wrong. https://www.math.brown.edu/~treil/papers/
LADW/LADW_2017-09-04.pdf
[3] Walter Rudin. Principles of Mathematical Analysis. McGraw-Hill, Inc. 1976.