Mathematical Principles of

Signal Processing

Springer-Verlag Berlin Heidelberg GmbH

Pierre Bremaud

Mathematical Principles of
Signal Processing
Fourier and Wavelet Analysis

Springer

Pierre Bremaud
Ecole Polytechnique Federale de Lausanne
Switzerland
and
INRIAJEcole Normale Superieure
France
[email protected]

Library of Congress Cataloging in Publication Data

Bremaud, Pierre.
Matbematical principles of signal processing / Pierre Bremaud.
p. cm.
Includes bibliographical references and index.
ISBN 978-1-4419-2956-3
ISBN 978-1-4757-3669-4 (eBook)
DOI 10.1007/978-1-4757-3669-4
1. Signal processing-Matbematics. I. TitIe.
TK5102.9.B72 2001
621.382'2'OI51-dc21
2001042957
Printed on acid-free paper.

2002 Springer Science+Business Media New York

Originally published by Springer Science+Business Media New York, Inc in 2002
Softcover reprint of tbe hardcover I st edition 2002
All rights reserved. This work may not be translated or copied in whole or in part witbout tbe written
permission oftbe publisher (Springer-Verlag New York, Inc., 175 Fifth Avenue, New York, NY 10010,
USA), except for brief excerpts in connection witb reviews or scholarly analysis. U se in connection
witb any form of information storage and retrievaI, electronic adaptation, computer software, or by
similar or dissimilar metbodology now known or hereafter developed is forbidden.
The use of general descriptive names, trade names, trademarks, etc., in tbis publication, even if tbe
former are not especially identified, is not to be taken as a sign tbat such names, as understood by tbe
Trade Marks and Merchandise Marks Act, may accordingly be used freely by anyone.
Production managed by Allan Abrams; manufacturing supervised by Joe Quatela.
Typeset by The BartIett Press, Inc., Marietta, GA.

987 6 5 4 3 2 1
SPIN 10845428

ToMarion

Contents

Preface

xi

A Fourier Analysis in LI

Introduction

Al Fourier Transforms of Stable Signals

A 11 Fourier Transform in L I
Al2 Inversion Formula . . . . . . . .

7
7
16

A2 Fourier Series of Locally Stable Periodic Signals

A21 Fourier Series in L}oc . . . . . . . .
A22 Inversion Formula . . . . . . . . . .

23

A3 Pointwise Convergence of Fourier Series

A31 Dini's and Jordan's Theorems.
A32 F6jer's Theorem . . .
A33 The Poisson Formula
References . . . . . . .

31

46

B Signal Processing

49

Introduction

51

BI Filtering
B 11 Impulse Response and Frequency Response
Bl2 Band-Pass Signals . . . . . . . . . . . . . .

55
55
68

23
26
31
39
43

viii

Contents

B2 Sampling
B21 Reconstruction and Aliasing .
B22 Another Approach to Sampling
B23 Intersymbol Interference .
B2-4 The Dirac Formalism . . . . .

75
75
82
84
88

B3 Digital Signal Processing

B31 The DFf and the FFf Algorithm
B32 The Z-Transform . . . . . . . . .
B33 All-Pass and Spectral Factorization

95
95
100
109

B4 Subband Coding
B41 Band Splitting with Perfect Reconstruction .
B42 FIR Subband Filters
References. . . . . . . . . . .

115
115
120
126

Fourier Analysis in L 2

127

Introduction

129

Cl Hilbert Spaces
C11 Basic Definitions.
C12 Continuity Properties
C13 Projection Theorem .

133

C2 Complete Orthonormal Systems

C21 Orthonormal Expansions .
C22 Two Important Hilbert Bases

145
145

C3 Fourier Transforms of Finite-Energy Signals

C31 Fourier Transform in L 2
C32 Inversion Formula in L 2

155
155
159

C4 Fourier Series of Finite-Power Periodic Signals

C41 Fourier Series in LToc . . . . . . . . . . .
C42 Orthonormal Systems of Shifted Functions
References. . . . . . . . . . . . . . . . . . . . .

161

Wavelet Analysis

133
136
139

150

161
163
166

167

Introduction

169

D1 The Windowed Fourier Transform

D11 The Uncertainty Principle . . . . . . . . .
D12 The WFf and Gabor's Inversion Formula .

175
175

D2 The Wavelet Transform

D21 Time-Frequency Resolution of Wavelet Transforms
D22 The Wavelet Inversion Formula . . . . . . . . . . .

185

178

185

187

Contents

03 Wavelet Orthonormal Expansions

D31 Mother Wavelet . . . . . . .
D32 Mother Wavelet in the Fourier Domain
D33 Mallat's Algorithm. . . . . . . . . .

ix

195
195
202
211

04 Construction of an MRA
D41 MRA from an Orthonormal System .
D42 MRA from a Riesz Basis ..
D43 Spline Wavelets . . . . . . . . . . .

217

05 Smooth MuItiresolution Analysis

D51 Autoreproducing Property of the Resolution Spaces
D52 Pointwise Convergence Theorem . . .
D53 Regularity Properties ofWavelet Bases
References . . . . . . . . . . . . . . . . . . .

229

217
220
223
229
231
234
237

Appendix

239

The Lebesgue Integral

References. . . .

241

Glossary of Symbols

263

Index

267

261

Preface

Fourier theory is one of the most useful tools in many applied sciences, particularly, in physics, economics, and electrical engineering. Fourier analysis is a
well-established discipline with a long history of successful applications, and the
recent advent of wavelets is the proof that it is still very alive. This book is an
introduction to Fourier and wavelet theory illustrated by applications in communications. It gives the mathematical principles of signal processing in such a way
that physicists and electrical engineers can recognize the familiar concepts of their
trade.
The material given in this textbook establishes on firm mathematical ground the
field of signal analysis. It is usually scattered in books with different goals, levels,
and styles, and one of the purposes of this textbook is to make these prerequisites
available in a single volume and presented in a unified manner.
Because Fourier analysis covers a large part of analysis and finds applications
in many different domains, the choice of topics is very important if one wants
to devise a text that is both of reasonable size and of meaningful content. The
coloration of this book is given by its potential domain of applications-signal
processing. In particular, I have included topics that are usually absent from the
table of contents of mathematics texts, for instance, the z-transform and the discrete
Fourier transform among others.
The interplay between Fourier series and Fourier transforms is at the heart of
signal processing, for instance in the sampling theory at large (including multiresolution analysis). In the classical Fourier theory, the formula at the intersection of the
Fourier transform and the Fourier series is the Poisson formula. In mathematically
oriented texts, it appears as a corollary or as an exercise and in most cases receives
little attention, whereas in engineering texts, it appears under its avatar, the formula

xii

Preface

giving the Fourier transfonn of the Dirac combo For obscure reasons, it is believed
that the Poisson sum fonnula, which belongs to classic analysis, is too difficult,
and students are gratified with a result of distributions theory that requires from
them a higher degree of mathematical sophistication. Surprisingly, in the applied
literature, whereas distribution theory is implicitly assumed to be innate, the basic
properties of the Lebesgue integral, such as the dorninated convergence and the
Fubini theorem, are never stated precisely and seldom used, although these tools
are easy to understand and would certainly answer many of the questions that alert
students are bound to ask. In order to correct this unfortunate tradition, which has
a demoralizing effect on good students, I have insisted on the fact that the c1assical
Poisson fonnula is all that is needed in signal processing to justify the Dirac
symbolism, and I have devoted some time and space to introduce the Lebesgue
integral in a concise appendix, giving the precise statements of the indispensable
tools.
The contents are organized in four chapters. Part A contains the Fourier theory
in LI up to the c1assical results on pointwise convergence and the Poisson sum
fonnula. Part B is devoted to the mathematical foundations of signal processing.
Part C gives the Fourier theory in L 2 . Finally, Part D is concemed with the timefrequency issue, inc1uding the Gabor transfonn, wavelets, and multiresolution
analysis. The mathematical prerequisites consist of a working knowledge of the
Lebesgue integral, and they are reviewed in the appendix.
Although the book is oriented toward the applications of Fourier analysis, the
mathematical treatment is rigorous, and I have aimed at maintaining a balance
between practical relevance and mathematical content.

Acknowledgments
Michael Cole translated and typed this book from a French manuscript, and Claudio Favi did the figures. Jean-Christophe Pesquet and Martin Vetterli encouraged
me with stimulating discussions and provided the illustrations of wavelet analysis. They also checked and corrected parts of the manuscript, together with Guy
Demoment and Emre Telatar. Sebastien Allam and Jean-Fran~ois Giovanelli were
always there when TEX tried to take advantage of my incompetence. To all of them,
I wish to express my gratitude, as well as to Tom von Foerster, who showed infinite
patience with my prornises to deliver the manuscript on time.
Gif sur Yvette, France
May 2,2001

Pierre Bremaud

Part

Fourier Analysis in L1

Introduction

In 1807 Joseph Fourier (1768-1830) presented a solution ofthe heat equation l

ae

a2e

-=K-,

at

a2x

where e(x, t) is the temperature at time t and at loeation x of an infinite rod, and
is the heat eonduetanee. The initial temperature distribution at time 0 is given:

e(x,O) = f(x).

(The solution of the heat equation is derived in Seetion A 11.)

In fact, Fourier eonsidered a cireular rod of length, say, 21T, whieh amounts to
imposing that the funetions x --+ f(x) and x --+ e(x, t) are 21T-periodie. He gave
the solution when the initial temperature distribution is a trigonometrie series
f(t)

= L cne int .
neZ

Fourier claimed that his solution was general beeause he was eonvineed that alI21Tperiodie funetions ean be expressed as a trigonometrie series with the eoefficients
Cn

= cn(f) = 21T

2lT

f(t)e -int dt.

lThe definitive form of his work was published in Theorie Analytique de la Chaleur,
Finnin Didot ed., Paris, 1822.

Part A Fourier Analysis in L I

Special cases of trigonometric developments were known, for instance, by

Leonhard Euler (1707-1783), who gave the formula
1

1 .

1 .

"2 x = sm(x) - "2 sm(2x) + 3" sm(3x) -

"',

true for - l ( < x < +l(. But the mathematicians of that time were skeptical about
Fourier's general conjecture. Nevertheless, when the propagation of heat in solids
was set as the topic for the 1811 annual prize of the French Academy of Sciences,
they surmounted their doubts and attributed the prize to Fourier's memoir, with the
explicit mention, however, that it lacked rigor. Fourier's results that were in any case
true for an initial temperature distribution that is a finite trigonometric sum, and be
it only for this, Fourier fully deserved the prize, because his proof uses the general
tricks (for instance, the differentiation rule and the convolution-multiplication
rule) that constitute the powerful toolkit of Fourier analysis.
Nevertheless, the mathematical problem that Fourier raised was still pending,
and it took a few years before Peter Gustav Dirichlet2 could prove rigorously,
in 1829, the validity of Fourier's development for a large class of periodic functions. Since then, perhaps the main guideline of research in analysis has been the
consolidation of Fourier's ingenious intuition.
The classical era of Fourier series and Fourier transforms is the time when the
mathematicians addressed the basic question, namely, what are the functions adrnitting a representation as a Fourier series? In 1873 Paul Dubois-Reymond exhibited
a continuous periodic function whose Fourier series diverges at O. For almost one
century the threat of painful negative results had been looming above the theory.
Of course, there were important positive results: Ulisse Dini3 showed in 1880 that
if the function is locally Lipschitz, for instance differentiable, the Fourier series
represents the function. In 1881, Carnille Jordan4 proved that this is also true for
functions of locally bounded variation. Finally, in 1904 Leopold Fejeii showed
that one could reconstruct any continuous periodic function from its Fourier coefficients. These results are reassuring, and for the purpose of applications to signal
processing, they are sufficient.
However, for a pure mathematician, the itch persisted. There were more and
more examples of periodic continuous functions with a Fourier series that diverges
at at least one point. On the other hand, Fejer had proven that if convergence is
taken in the Cesaro sense, the Fourier series of such continuous periodic function
converges to the function at all points.

2Sur la convergence des series trigonometriques qui servent arepresenter une fonetion
arbitraire entre des limites donnees, J. reine und angewan. Math., 4,157-169.
3 Serie di Fourier e altre rappresentazioni analitiche delle funzioni di une variabile reale,
Pisa, Nistri, vi + 329 p.
4Sur la serie de Fourier, CRAS Paris, 92, 228-230; See also Cours d'Analyse de l'lfcole
Polytechnique, I, 2nd ed., 1893, p. 99.
5Untersuchungen ber Fouriersehe Reihen, Math. Ann., 51-69.

Introduction

Outside continuity, the hope for a reasonable theory seemed to be completely

destroyed by Nikola'i Kolmogorov, 6 who proved in 1926 the existence of a periodic
locally Lebesgue-integrable function whose Fourier series diverges at alt points!
It was feared that even continuity could foster the worst pathologies. In 1966 JeanPierre Kahane and yitzhak Katznelson7 showed that given any set of null Lebesgue
measure, there exists a continuous periodie function whose Fourier series diverges
at all points of this preselected set.
The case of continuous functions was far from being elucidated when Lennart
Carleson8 published in the same year an unexpected result: Every periodic locally
square-integrable function has an almost-everywhere convergent Fourier series.
This is far more general than what the optirnistic party expected, since the periodic
continuous functions are, in particular, locally square-integrable. This, together
with the Kahane-Katznelson result, completely settled the case of continuous
periodic functions, and the situation finally tumed out to be not as bad as the 1873
result of Dubois-Reymond seemed to forecast.
In this book, the reader will not have to make her or his way through a jungle
of subtle and difficult results. Indeed, for the traveler with practical interests, there
is a path through mathematics leading directly to applications. One of the most
beautiful sights along this road may be Simeon Denis Poisson's9 sum formula

L J(n) = L
nEZ

jen),

nEZ

where J is an integrable function (satisfying some additional conditions to be

made precise in the main text) and where
j(v)

J(t)e-2irrvt dt

is its Fourier transform, where ~ is the set of real numbers. This striking formula
found very nice applications in the theory of series and is one of the theoretical
results founding signal analysis. The Poisson sum formula is the culrninating result
of Part A, which is devoted to the classical Fourier theory.

6Une serie de Fourier-Lebesgue divergente partout, CRAS Paris, 183, 1327-1328.

7S ur les ensembles de divergence des series trigonometriques, Studia Mathematica, 26,

305-306.
8Convergence and growth of partial sums of Fourier series, Acta Math., 116, 135-157.
9S ur la distribution de la chaleur dans les corps solides, J. Ecole Polytechnique, 1geme
Cahier, XII, 1-144, 145-162.

Al
Fourier Transforms of Stable Signals

A 11

Fourier Transform in LI

This first chapter gives the definition and elementary properties of the Fourier
transform of integrable functions, which constitute the specific calculus mentioned
in the introduction. Besides linearity, the toolbox of this calculus contains the
differentiation rule and the convolution-multiplication rule. The general problem
of recovering a function from its Fourier transform then receives a partial answer
that will be completed by the results on pointwise convergence of Chapter A3.
We first introduce the notation: N, Z, Q, ~, C are the sets of, respectively,
integers, relative integers, rationals, real numbers, complex numbers; N+ and ~+
are the sets of positive integers and nonnegative real numbers.
In signal theory, functions from ~ to C are called (complex) signals. We shall
use both terminologies (function, or signal), depending on whether the context is
theoretical or applied.
We denote by L~(~) (and sometimes, for short, LI) the set offunctions f(t) 10
from ~ into C such that

If(t)1 dt <

00.

In analysis, such functions are called integrable. In systems theory, they are called
stable signals.
IOWe shall often use this kind of loose notation, where a phrase such as "the function
f(t)" means "the function f : lR ~ c." We shall also use the notation "I" or "fO" with
a mute argument. For instance, "f( - a)" is the function t --+ f(t - a).

P. Brmaud, Mathematical Principles of Signal Processing

Springer Science+Business Media New York 2002

Al.

Fourier Transforms of Stable Signals

Let A be a subset of IR. The indicator function of Ais the function lA : IR

{O, 1} defined by

1o
1

lA(t) =

1-+

if tE A,
ift

A.

The function I(t) is called locally integrable if for any closed bounded interval
[a, b] C IR, the function I(t)l[a,bj(t) is integrable. We shall then write

I
or, for short, I E Lloe'
The set of functions

L~ loe(lR)

I (t) from IR into C such that

I/(t)1 2 <

00

is denoted by L~(IR). It is the set of square-integrable functions. A signal I(t) in

this set is said to have a finite energy
E =

I/(t)1 2 dt.

The function I(t) is called locally square-integrable if for any closed bounded
interval [a, b] C IR, the function I(t)l[a,bj(t) is square-integrable. We shall then
write

L~,loe(lR)

or, for short, I E L toe'

We recall that in L~(IR) or L~(IR) two functions are not distinguished if they are
equal almost everywhere with respect to the Lebesgue measure.
EXERCISE AI.I. Give an example 01 a function that is integrable but not 01finite
energy. Give an example 01 a function that is 01finite energy but not integrable 01
finite energy. Show that

A function I : IR 1-+ C is said to have bounded support if there exists a bounded

interval [a, b] c IR such that I(t) = 0 whenever t ~ [a, b].
If the function I(t) is n times continuously differentiable (that is, it has derivatives up to order n, and these derivatives are continuous), we say that it is in Cn .
If it is in Cn for all n E N, it is said to belong to COO The kth derivative of the
function I (t), if it exists, is denoted jCk)(t). The Oth derivative is the function itself:
I(O)(t) = I(t); in particular, CO is the collection of continuous functions from IR
to C. The set of continuous functions with bounded support is denoted by C~.

AII Fourier Transfonn in LI

Fourier Transform
We can now give the basic definition.
DEFINITION AI.I. Let s(t) be a stable complex signal. The Fourier transform (FT)
ofs(t) is thefunctionfrom:IR into C:

s(v) =

s(t) e-2i1rvI dt.

(1)

(Note that the argument of the exponential in the integrand is -2i7rvt.) The
mapping from the function to its Fourier transform will be denoted by
s(t) ~ s(v)

or

:F: s(t) --* s(v).

Table A 1.1 gives the immediate properties of the Fourier transform.

Table AI.I. Elementary Properties of Fourier Transfonns
Delay
Modulation
Doppler

s(t - to)

~ e- 2ill'vtos(v)

e2i1rvot s(t)

s(v - vo)

s(at)

Fr
--+

-IsA{V}
lai
a

AISI (t)

+ A2S2(t)

~ AISI(V) + A2 S2(V)
~

s*(t)

s(-v)*

EXERCISE

A1.2. Prove the assertions in Table Al.I.

EXERCISE

AI.3. Prove the modulation theorem:

Fr 1 A
s(t) cos(27rvot) --+
2(s(v - Vo ) + sA( v

+ Vo )).

(2)

(See Fig. AI.I.)

EXERCISE

AI.4. Show that the FT of a real signal is Hermitian even, that is,
s(- v) = s(v)*.

Show that the FT of an odd (resp., even; resp., real and even) signal is odd (resp.,
even; resp., real and even).
EXERCISE

A1.S. Defining the rectangular pulse

recT(t) = I I- t .+ t1 (t)

s(v)

Hs(v

-vo

+ vo) + s(v -

Figure Al.l. Modulation theorem

vo)}

+vo

10

Al.

Fourier Transfonns of Stable Signals

and the cardinal sine

sinc (x)

sin(Jrx)

= --Jrx

show that (see Fig. Al.2)

recT(t)

~ Tsinc (vT).

(3)
T

-T/2

+T/2

Tsinc(vT) = recT(v)

reCT(t)

Figure A 1.2. Fourier transfonn of the rectangle function

We will show that the Gaussian pulse is its own Fr, that is,

(4)
In order to compute the corresponding Fourier integral, we use contour integration
in the complex plane. First, we observe that it is enough to compute the Fr s( v)
for v 2: 0, since this Fr is even (see Exercise A1.4). Take a 2: v (eventually, a
will tend to 00).
Consider the rectangular contour Y in the complex plane (see Fig. A1.3),
Y

= Yl + Y2 + Y3 + Y4,

where the Yi 's are the oriented line segments

Yl : (-a, 0) -+ (+a, 0),

Y2 : (+a, 0) -+ (+a, v),
Y3 : (+a, v) -+ (-a, v),
Y4: (-a, v) -+ (-a, 0).

v
')'4

-a

+a

Figure A1.3. The integration path in the proof of (4)

A 11 Fourier Transform in L I

We denote by
Wehave

-Yi

11

the oriented segment whose orientation is opposite that of Yi.

e- rrz2

+ /z + h + 14,

dz = 11

where li is the integral of e- rr Z2 along Yi. Since the latter integrand is a holomorphic
function, by Cauchy's theorem (see, for instance, Theorem 2.5.2, p. 83, of [Al],
or Theorem 2.2, p. 101, of [A6]),

e- rrz2

= 0,

dz

and therefore,

+ /z + h + 14 =

O.

We now show that

lim /z

a~oo

= a-+oo
!im 14 = o.

For /Z, for instance, if we parameterize Y2 as folIows,

Y2

then

/z

i dt

e-rr(a+itf

Therefore, since v .:::: a,

l/zl .::::
=

= {a + it;O,:::: t .:::: v},

e-rr(a-t)(a+t)

e- rra

21o

errat

e-rr(a2-t2)e-2irrat

dt .::::

e-rra(a-t)

1
dt = -(1
_

dt

e- rra 2 ),

Jra

where the last quantity tends to 0 as a tends to

for 14, with sirnilar computations. Therefore,
!im (h

a-HXl

i dt.

+ h) =

+00. A sirnilar conclusion holds

0,

that is,
(5)

Using for YI the obvious parameterization

!im {
a--+oo

JYl

lim
a--+oo

j+a e- rrt2

dt

= { e- rrt2 dt = 1.

-a

JJ!I!.

Parameterizing -Y3 as folIows,

-Y3 = {iv

+ t; -a .:::: t

.:::: +a},

12

Al.

Fourier Transforms of Stahle Signals

wehave

= l+ a e-n(iv+tf dt
-a

-)'3

Therefore,

Going back to (5), we obtain

which gives the announced result.

EXERCISE

A1.6. Deduce Jrom (4) that, Jor all Cl > 0,

The Fr of the Gaussian pulse can be obtained by other means (see Exercise
A 1.16). However, in other cases, contour integration is often necessary.
Using contour integration in the complex plane, we show that, for a > 0,

= e-at IlR+(t)

s(t)

Fr

s(v)
A

First observe that

s(v)

00

.
e- 2znvt
e- at dt

1
2mv + a

= .
=

1
2inv + a

1
1

00

1
.
a+2mv

(6)

.
e-2znvt-a\2inv
+ a) dt

e- Z dz.

(The reader is refered to Fig. A1.4 for the definition ofthe lines y, YJ. Y2, and Y3.)
Therefore, it suffices to show that

By Cauchy's theorem,

e-Z dz

Yl

e- z dz

e- Zdz

Y2

= 1.
+

e- Zdz

)'3

= 0.

A 11 Fourier Transform in LI

13

2i7f1/

Figure A1.4. The integration path in the proof of (6)

The limit as A t 00 of I Yl e- Z dz is I y e- Z dz, and that of I Y3 e- Z dz = IoA e- t dt

is 1. It therefore remains to show that the limit as A t 00 of J,Y2 e- Z dz is 0, and
this foIlows from the bound

I{

e-zdzl ::: e- A IY21,

where IY21 = K x A is the length of Y2.

EXERCISE

AI.7. Deducefrom (6) that

Convolution-Multiplication Rule
THEOREM

AI.I. Let h(t) and x(t) be two stable signals. Then the right-hand side

oJ
y(t)

=1

(7)

h(t - s)x(s) ds

is defined Jor almost all t and defines almost everywhere a stable signal whose FT
is y(v) = h(v)x(v).

Proof"

By ToneIli's theorem and the integrability assumptions,

f1xIR Ih(t - s)llx(s)1 dt ds = (l'h(t)' dt) (l'X(t)' dt) <

00.

This implies that, for almost aIl t,

l'h(t - s)x(s)1 ds <

00.

The integral IIR h(t - s )x(s) ds is therefore weIl defined for almost aIl t. Also,
1,y(t)' dt

111

h(t - s)x(s)

dsl dt

::: 11'h(t - s)x(s)1 dt

ds <

00.

14

Al.

Fourier Transfonns of Stable Signals

Therefore, y(t) is stable. By Fubini's theorem,

L(Lh(t - S)X(S)dS) e-2irrvt dt

=L

Lh(t - s)e-2irrv(t-s)x(s)e-2irrvs ds dt

= h(v)x(v).

The funetion y(t), the convolution of h(t) with x(t), is denoted by

y(t) = (h

* x)(t).

We therefore have the convolution-multiplication rule,

(h

Fr
* x)(t) --+
h(v)x(v).
A

(8)

EXAMPLE AI.L

The convolution of the rectangular pulse reeT (t) with itself is the
triangular pulse of base [- T, + T] and height T,
TriT(t) = (T - Itl)1[-T,+T](t).

By the convolution-multiplication rule,

TriT(t)

~ (Tsine (vT)f

(9)

(see Fig. Al.5).

EXERCISE ALS.

Let x(t) be a stable complex signal. Show that its autoeorrelation

funetion
c(t)

=L

x(s

+ t)x*(s) ds

is well defined and integrable and that its FT is

Ix(v)1 2
T2

-T

&
I

I
I

+T

C'>~
I
~C'>
I
. 1
j
I

-~ -~ -~

TriT(t)

Figure A 1.5. FT of the triangle funetion

All Fourier Transform in LI

15

EXERCISEAl.9. Showthatthenthconvolutionpoweroff(t) = e-atlt~o(t), where

a > 0, is
fM(t)

(/*3

= f * f * f,

t n- 1

= (n -

I)!

e- at I t>o(t).
-

etc.) Deducefrom this the FTofs(t)

= tne-atlt~o(t).

Riemann-Lebesgue Lemma

The Riemann-Lebesgue lemmal! is one of the most important technical tools

in Fourier analysis, and we shall use it several times, especially in the study of
pointwise convergence of Fourier series (Chapter A3).
THEOREM

Al.2. The FT of a stable complex signal s(t) satisfies

lim Is(v)1

Ivl-+oo

= O.

(10)

Proof' The Fr of a rectangular pulse s(t) satisfies Is(v)1 ::; K/lvl [see Eq. (3)].
Hence every signal s(t) that is a finite linear combination of indicator functions
of intervals satisfies the same property. Such finite combinations are dense in
Lb(l~) (Theorem 28 of the appendix), and therefore there exists a sequence sn(t)
of integrable functions such that

lim (Isn(t) - s(t)1 dt

n-+oo

JIR

=0

and

Kn

ISn(v)1 ::; ~'

for finite numbers Kn From the inequality

Is(v) - sn(v)1 ::;

Is(t) - sn(t)1 dt,

we deduce that

::; -K n +
lvi

IR

Is(t) - sn(t)1 dt,

from which the conclusion follows easily.

The following uniform version of the Riemann-Lebesgue lemma will be needed

in the sequel.
11 Riemann, B., (1896), Sur la possibilite de representer une fonetion par une serie
trigonometrique, Oeuvre Math., p. 258.

16

Al.

Fourier Transfonns of Stable Signals

Al.3. Let f(t) be a 2:rr-periodic locally integrable function, and let

CbeinC I , where [a,b] ~ [-:rr, +:rr]. Then

THEOREM

g: [a,b]

f-*

lim l

....... 00

b
a

fex - u)g(u) sin(Au) du = 0

uniformly in x.
Proof

For arbitrary E: > 0, choose a 2:rr-periodic function h(t) in Cl such that

r:rr

If(x) - h(x)1 dx < E:

(Theorem 29 of the appendix). Integrating by parts yields

/(A)

=l

=-

COS(AU) Ib
hex - u)g(u) - A
a

hex - u)g(u) sin(Au) du

+l

[hex - u)g(u)]

COS(AU)
du.
A

Since h E Cl and is periodic, h and h' are uniformly bounded. The same is true of
g, g' (g is in Cl). Therefore,
lim /(A)

....... 00

=0

uniformly in x .

Now,

11

fex - U)g(U)Sin(AU)dUI
:S I/(A)I

+l

:S I/(A)I

+ a~~ Ig(U)ll b Ih(x -

:S I/(A)I

Ih(x - u) - fex - u)llg(u)1 sin(Au) du

u) - fex - u)1 sin(Au) du

max Ig(u)IE:.

a:'Ou:'Ob

The conc1usion then follows because E: is arbitrary.

AI 2

Inversion Formula

Al.lO. Show that the FT of a stable signal is uniformly bounded and

uniformly continuous.

EXERCISE

Despite the fact that the FT of an integrable signal is uniformly bounded and
uniformly continuous, it is not necessarily integrable. For instance, the FT of the
rectangular pulse is the cardinal sine, a non-integrable function. When its FT is
integrable, a signal admits a Fourier decomposition.

AI 2 Inversion Fonnula

17

THEOREM At.4. Let set) be an integrable complex signal with the Fourier
transform s(v). Under the additional condition

Is(v)1 dv <

the inversion formula

set)

(11)

00,

s(v)e+2iJrvt dv

(12)

holdsfor almost all t.lf set) is, in addition to the above assumptions, continuous,
equality in (12) holds for all t.

(Note that the exponent ofthe exponential ofthe integrand is +2irrvt.)

EXERCISE

At.ll. Check that the above result is true for the signal
(a E lR, a > 0, a E C).

Proof' We now proceed to the proof of the inversion formula. (lt is rather technical and can be skipped in a first reading.) Let set) be a stable signal and consider
the Gaussian density function

with the Fr

We first show that the inversion formula is true for the convolution (s
Indeed,
(s

* h u )(t).

* hu)(t) = JR{ s(u)hu(u)e...L ';;2 (t)du,

u

2u 2

(13)

and the Fr ofthis signal is, by the convolution-multiplication formula, s(v)hu(v).

Computing this Fr directly from the right-hand side of (13), we obtain
s(v)hu(v)

= (

J~

s(u)hu(u) ( { e

J~ ~;;r

(t)e-2iJrvt dt) du

= J~{ s(u)hu(u)e ~,;;r (v) du.

I

Therefore, using the result ofExercise A1.11,

{ s(v)hu(v)e2iJrvt dv

J~

= {( { s(u)hu(u)e
J~ J~

,U

;;r

(v) dU) e2iJrvt dv

= JR
{ s(u)hu(u)e Zc;2' -'<-(t)du
I

(12

= (s

* h u )(t).

18

Al.

Fourier Transforms of Stahle Signals

Thus, we have
(s

* hu)(t) =

(14)

s(v)hu(v)e2i1rvt dv,

and this is the inversion formula for (s

* h u )(t).

Since for all v E IR, limu-l-o v t hu(v) = 1, it follows from Lebesgue's dominated convergence theorem that when u ..I- 0 the right-hand side of (14) tends
to

s(v)e2i1rvt dv

for all t E IR. If we can prove that when u ..I- 0 the function on the left-hand side of
(14) converges in L~(IR) to the function s(t), then, for almost all t E IR, we have
the announced equality (Theorem 25 of the appendix).
To prove convergence in L~(IR), we observe that

L*
I(s

hu)(t) - s(t)1 dt =

(using the fact frr~. hu(u)du

s(t)1 dt,

L*
Is

LIL

(s(t - u) - S(t))hu(U)dul dt

= 1), and therefore, defining

hu(t) - s(t)1 dt :::::

f(u) =

iJR Is(t

- u)-

f(u)hu(u) du.

Now, If(u)1 is bounded (by 2 iJR Is(t)1 dt). Therefore, iflimu-l-o f(u)
dominated convergence,
lim [ f(u)hu(u) = lim [ f(uu)ht(u)du =

uwk

(15)

uwk

= 0, then, by

o.

(16)

Toprovethatlimu-l-O f(u) = 0, we begin with thecasewheres(t)iscontinuous with

compact support. In particular, it is uniformly bounded. Since we are interested in
a limit as u tends to 0, we may suppose that u is in a bounded interval around 0,
and in particular, the function t -+ Is(t - u) - s(t)1 is bounded uniformly in u by
an integrable function. It follows from the dominated convergence theorem that
limu-l-o f(u) = o.
Now, let s(t) be only integrable. Let {snO}n>t be a sequence of continuous
functions with compact support that converges i;' L~(IR) to sO (Theorem 27 of
the appendix). Writing
f(u) ::::: d(s( - u), sn(' - u)) +

where
d(s( - u), sn(' - u))

the result easily follows.

L
=L

ISn(t - u) - sn(t)1 dt

+ d(s(), snO),

Is(t - u) - sn(t - u)1 dt,

Al2 Inversion Fonnula

19

Suppose that, in addition, set) is continuous. The right-hand side of (12) defines
a continuous function because s(v) is integrable. The everywhere equality in (12)
follows from the fact that two continuous functions that are almost everywhere
equal are necessarily everywhere equal (Theorem 8).

The Fourier transform characterizes a stable signal:

COROLLARY ALL lf two stable signals SI (t) and S2(t) have the same Fourier
transform, then they are equal almost everywhere.

Proof' The signal set) = Sl(t) - S2(t) has the FT s(v)

and thus by (12), set) = for almost all t.

= 0, which is integrable,

EXERCISE AI.12. Give the FT of set) = 1/ A(a 2 + t 2). Deduce from this the value
of the integral

f ~dU,
t > 0.
J.R.t+u
EXERCISE Al.13. Deduce from the Fourier inversion formula that
l(t)=

L(Si~(t) Y

dt = Jr.

Exercise 1.14 is very important. It shows that for signals that cannot be called
pathological, the version of the Fourier inversion theorem that we have in this
chapter is not applicable, and therefore we shall need finer resuIts, which are given
in Chapter A3.
EXERCISE AI.14. Let set) be a stable right-continuous signal, with a limit from
the left at all times. Show that if s(t) is discontinuous at some time to, its FT cannot
be integrable.

Regularization Lemma
In the course of the proof of Theorem A1.4, we have used a special case of the
regularization lemma below, which is very useful in many circumstances.

DEFINITION AI.2. A regularizing function is a nonnegative function h a : IR ---+ IR

depending on a parameter a > and such that

lim
a'\-O

LEMMA

L
+a

-a

ha(u) du = 1,

forall a > 0,

= 1,
limha(u) = 1,
a'\-O

forall a > 0,

ha(u) du

forall u

IR.

ALl. Let h a : IR ---+ IR be a regularizing function. Let set) be in Lb(IR).

Then
lim
a'\-O

JIR

I(s

* h a )(t) -

s(t)1 dt

= 0.

20

Al.

Fourier Transforms of Stable Signals

Proof" We ean use the proof of Theorem A1.4, starting from (15). The only
plaee where the speeifie form of h" (a Gaussian density) is used is (16). We must
therefore prove that
lim ( J(u)h,,(u)
"-1-0 JIR

=0

independently. Fix e > O. Sinee limuto J(u) = 0, there exists a = aCe) such that

h,,(u) du :s -.
1-a~ J(u)h,,(u) du :s -2el~
-a
2

Sinee J(u) is bounded (say, by M),

(
J(u)h,,(u) du :s M (
h,,(u) du.
JIR\[ -a.+a]
JIR\[ -a.+a]

The last integral is, for suffieiently small a, less than e12M. Therefore, for
suffieiently small a,

JIR J(u)h,,(u) du :s

2" + 2" = e.

The funetion h" is an approximation of the Dirae generalized funetion o(t) in

that, for all <p E

c2,

lim ( h,,(t) <p(t) dt = <p(0) = { o(t) <p(t)dt.

,,-1-0 JIR

JIR

The last equality is symbolie and defines the Dirae generalized funetion (see Seetion B24). The first equality is obtained as in the proof of the previous lemma,
this time letting J(u) = <p(u) - <p(0).
Differentiation in the Frequency Domain

We shall see how differentiation in the time domain is expressed in the frequeney
domain.
A1.S. (a) Ifthe integrable signal set) is such that tks(t) E LU~)Jor
alt 1 :s k :s n, then its FT is in Cn , and
THEOREM

(-2imls(t) ~ s(k)(v)
(b) IJ the signal set) E
integrable, then

cn and if it is,

S(k)(t) ~ (2inv)k s (v)

Proof"

Joraltl:S k:s n.
together with its n first derivatives,
Jor altl :s k :s n.

(a) In the right-hand side ofthe expression

s(v) =

e-2invIs(t)dt,

A12 Inversion Forrnula

21

we can differentiate k times under the integral sign (see Theorem 15 and the
hypothesis t ks(t) E Lb(IR)) and obtain
s(k)(v)

(_2i:n:t)k e -2irrvt s(t) dt.

(b) It suffices to prove this for n = 1, and iterate the result. We first observe that
limlaltoo s(a) = O. Indeed, with a > 0, for instance,
s(a)

and therefore, since s'(t)

= s(O) +

s'(t) dt,

Lb(IR), the limit exists and is finite. This limit must be

obecause s(t) is integrable. Now, the Fr of s'(t) is

[ e-2irrvts'(t)dt = lim [+a e-2irrvts'(t)dt.

atoo

La

JIR

Integration by parts yields

a
i : e-2irrvts'(t)dt

= (e-2irrvts(t)):: + i:a(2i:n:V)e-2irrvtS(t)dt.

It then suffices to let a tend to

00

to obtain the announced result.

EXERCISE AI.IS.

Let s(t) be a stable signal with a Fourier transform with compact

support. Show that s(t) E Coo, that all its derivatives are integrable, and that the
kth derivative has the FT (2i:n:v)k s (v).

EXERCISE AI.16.

Give a differential equation satisjied by the Gaussian pulse, and

use it to deduce its Fourier transform. Could you do the same to prove (6)?

The Beat Equation

We now pay our tribute to the founder and give the solution of the heat equation,
which was announced in the introduction. Recall that the heat equation relative to
an infinite rod is the partial differential equation

aB

at =

a2B
K a2x'

(17)

where B(x, t) is the temperature at time t and at location x of the rod with heat
conductance K, and with the given initial temperature distribution
B(x, 0) = f(x).

We assume that

(18)

is integrable.

Let ~ 1-* 8(~, t) be the Fr of x 1-* B(x, t). (We take different notations because
the variable with respect to which the Fr is taken is not the time variable t but the
space variable x.) In the Fourier domain, Eq. (17) becomes
d 8(c t)
--:-,"-'= -K(4:n:2~2)e(~, t),
dt

22

Al.

Fourier Transfonns of Stable Signals

with the initial condition

e(~,

0) = F(H

where F(O is the Fr of fex). The solution is

e(~, t)

= F(~)e-4n2K~2f.

Since x 1-+ (4JrKt)-1/2 e (4Kf)-1/2 x 2 has the Fr ~

multiplication formula gives
1

1-+

e-4n2K~\ the convolution-

8(x, t) = (4JrKt)-Z J~ fex - y)e-i'ii dy,

or
8(x, t)

..Jrr

fex - 2-JKiy)e-yl dy .

As we mentioned earlier, Fourier considered the finite rod heat equation, which
receives a similar solution, in terms of Fourier series rather than Fourier integrals
(see Chapter A2). The efficiency of the Fourier method in solving differential or
partial differential equations of mathematical physics has been, after the pioneering
work of Fourier, amply demonstrated 12 .

12See, for instance, the classic text of 1. N. Sneddon, Fourier Transfonns, McGraw-Hill,
1951; Dover edition, 1995.

A2
Fourier Series of Locally Stable
Periodic Signals

A21

Fourier Series in L loc

Fourier Coefficients

A periodic signal is neither stable nor of finite energy unless it is almost everywhere
null, and therefore, the theory of the preceding Chapter is not applicable. The
relevant notion is that of Fourier series. (Note that Fourier series were introduced
before Fourier transforms, in contrast with the order of appearance chosen in this
text.) The elementary theory of Fourier series of this section is parallel to the
elementary theory of Fourier transforrns of the previous section. The connection
between Fourier transforrns and Fourier series is made by the Poisson sum formula,
of which we present a weak (yet useful) version in this chapter.
A complex signal s(t) is called periodic with period T >
for all t E ~,
s(t

+ T) =

(or T -periodic) if,

s(t).

AT -periodic signal s(t) is locally stable, or locally integrable, if s(t)

that is,

Is(t)1 dt <

Lb([O, Tl),

00.

A T -periodic signal s(t) is locally square-integrable if s(t)

Is(t)1 2 dt <

P. Brmaud, Mathematical Principles of Signal Processing

Springer Science+Business Media New York 2002

00.

E L~([O,

Tl), that is,

24

A2. Fourier Series of Locally Stable Periodic Signals

One also says in this case that s(t) hasfinite power, since
lim
A-+oo

..!.. {A
A

10

Is(t)12

=..!..
T

(T Is(t)12 dt <

10

00.

As the Lebesgue measure of [0, T] is finite,L~([O, Tl) c Lt([O, Tl). (See Theorem 19 of the appendix.) In particular, a finite-power periodic signal is also locally
stable.
We are now ready for the basic definition.
DEFINITION A2.I. The Fourier transform {sn}, n
periodic signal s(t) is defined by theformula

Sn

= -I

Z, of the locally stable T-

n dt,
s(t)e- 2'lJr TI

(19)

and Sn is ca lIed the nth Fourier coefficient of the signal s(t).

A2.I. Compute the Fourier coefficients of the T -periodic function s(t)

such that on [0, T), s(t) = t.

EXERCISE

EXERCISE

A2.2. Let s(t) be a locally stable T -periodic signal. Defining

ST(t) = s(t)I[O,TJ(t),

show that the nth Fourier coefficient Sn ofs(t) and the FT i:;(v) OfST(t) are linked
by
(20)

A2.3. Compute the Fourier coefficients of the T -periodic signal s(t)

such that on [-T /2, +T /2), s(t) = 1[-a~,+a~l(t), where a E (0, 1).

EXERCISE

A2.4. Let s(t) be a T -periodic locally stable signal with nth Fourier
coefficient Sn. Show that limlnltoo Sn = O.

EXERCISE

One often represents the sequence {Sn}nEZ of the Fourier coefficients of a Tperiodic signal by "spectrallines" separated by 1/ T from each other along the
frequency axis. The spectralline at frequency n / T has the complex amplitude Sn
(see Fig. A2.1). This is sometimes interpreted by saying that the FT of s(t) is
s(v)

= I)n(V nEZ

f)'

where (t) is the Dirac generalized function (see Section B2-4).

EXERCISE A2.5. Let s(t) be a T -periodic locally stable signal with nth Fourier
coefficient Sn. What is the nth Fourier coefficient of s(t - a), where a E llV What
can you say about the period and the Fourier coefficients of the signal s(t / a),
where a > O?

A21 Fourier Series in Lloc

25

Figure A2.l. From the Fourier transform to the Fourier coefficients

Convolution-MuItiplication Rule
A2.1. Let x(t) be a T -periodie locaily stable signal, and let h(t) be a
stable signal. The signal

THEOREM

y(t)

(21)

h(t - s)x(s)ds

is almost everywhere weil defined, T -periodie, and locaily stable. Its nth Fourier
coefficient is
Yn
A

= hA(n)
T x n,

(22)

where h(v) is the FT of h(t) (see Fig A2.2).

Proof

We have
L1h(t - s)llx(s)1 ds =

where
hT(u)

=L

la T IhT(t - s)llx(s)1 ds,

h(u

+ nT).

nEZ

Now

x(v)

I -q,I I I I I I

-~

-~

123

fI(v)

"

"

Figure A2.2. Filtering aperiodie signal

26

A2. Fourier Series of Locally Stable Periodic Signals

and hence by the usual argument (see the proof of Theorem A 1.1),

Ih(t - s)llx(s)1 ds <

for almost aIl t

00,

lR. Thus, y(t) is almost everywhere weIl defined by (21). Also,

y(t

+ T) =
=

L
L

x(t

+T -

s)h(s)ds

x(t - s)h(s)ds,

which shows that y(t) is periodic with period T. The same argument as in the proof
of Theorem AU shows that y(t) is locally stable. FinaIly,

l
=- l 1

Yn = -1

1
T

n dt
y(t)e- Z'l1!'j't

n dt ds
hT(t
- s)x(s)e- Z'l1!'j't

A22

Inversion Formula

The Poisson Kernel

In the proof of the Fourier series inversion formula, the Poisson kernel will play
a role similar to that of the Gaussian pulse in the proof of the Fourier transform
inversion formula of the previous seetion.
The Poisson kernel is the family of functions Pr : lR
Pr(t)

f-+

<C, 0 < r < 1, defined by

rlnleZin ~t.

nEZ

For fixed r, Pr is T -periodie, and elementary computations reveal that

Pr(t)

= LrneZin~t + Lrne-zin~t_l
n~O

n~O

(23)

A22 Inversion Formula

27

and therefore,
Pr(t) :::: O.

(24)

Also,
T 2
-1 I+ / Pr(t) dt
T -T/2

(25)

1.

In view of the above expression of the Poisson kernel, we have the bound
1 [
Pr(t)dt <
T [-t,+t]\[-e,+s]
-

(1 - r 2 )
2
1 1 _ e 2i1ry 1

'

and therefore, for all e > 0,

. -1 [
hm
Pr(t)dt
rtl T [-t,+t]\[-e,+e]

= O.

(26)

Properties (24)-(25) make of the Poisson kernel a regularizing kernel, and in

particular,
!im
rtl

-1
1

+t

-t

cp(t)Pr(t) dt

= cp(O),

for all bounded, continuous cp : ffi. -+ <C (same proof as in Lemma ALl).
The following result is similar to the Fourier inversion formula for stable signals
(Theorem A1.4).
THEOREM A2.2. Let set) be aT -periodie localty stable complex signal with Fourier
coefficients {sn}, n E Z. lf

(27)
then, for almost alt t E ffi.,
set)

= LSne+2i1rYI.

(28)

nEZ

lfwe add to the above hypotheses the assumption that set) is a continuousfunction,
then the inversion formula (28) holds for all t.
Proof

The proof is similar to that of Theorem A1.4. We have

LSnrlnle2i1ryl

11+[

= _
T

nEZ

-t

s(u)Pr(t - u)du,

(29)

and
!im
rtl

10r I10r
T

s(u)Pr(t - u) du - S(t)1 dt
T

= 0,

that is: The right-hand side of (29) tends to set) in Lb([O, T]) when r t 1. Since
LnEZ ISn I < 00, the function of t in the left-hand side of (29) tends toward the

28

A2. Fourier Series of Locally Stable Periodie Signals

function LnEZ sne+2irr(n/T)t, pointwise and in L~([O, T]). The result then follows
from Theorem 25.
The statement in the case where set) is continuous is proved exactly as the
corresponding statement in Theorem A1.4.

As in the case of stable signals, we deduce from the inversion formula the
uniqueness theorem.
A2.1. Two locally stable periodic signals with the same period T
that have the same Fourier coefficients are equal almost everywhere.

COROLLARY

EXERCISE

A2.6. Compute

using the expression ofthe Fourier coefficients ofthe 2-periodic signal set) such
that
fort

[-1,+1].

A2.7. Let x(t) be a T -periodic locally stable signal with nth Fourier
coefficient x n such that

EXERCISE

L InlPlxnl <

00.

nEZ

Show that x(t) is p times differentiable and that if the pth derivative is locally
integrable, its nth Fourier coefficient is (2i7T!.f)P Xn.
The Weak Poisson Formula
The Poisson sum formula takes many forms. The strong version is
(30)

This aesthetic formula has a number of applications in signal processing (see Part
B).

The next result establishes the connection between the Fourier transform and
Fourier series, and is central to sampling theory. It is a weak form of the Poisson
sum formula (see the discussion after the statement of the theorem).
'THEOREM A2.3. Let set) be a stable complex signal, and let 0 < T < 00 be
fixed. The series LnEZ set + nT) converges absolutely almost everywhere to a
T -periodic locally integrable function <I>(t), the nth Fourier coefficient of which is
(l/T)s(n/T).

We paraphrase this result as follows: Under the above conditions, the function

<I>(t) :=

L set + nT)
nEZ

(31)

A22 Inversion Forrnula

29

is T -periodie and locally integrable, and its formal Fourier series is

~ I)(!!..) e2inIfI.

Sj(t) =

(32)

nEZ

(We speak of a "formal" Fourier series, because nothing is said about its convergence.) Therefore, whenever we are able to show that the Fourier series represents
the function at t = 0, that is, if <1>(0) = S j(O), then we obtain the Poisson sum
formula (30).
For now, we are saying nothing about the convergence of the Fourier series.
This is why we talk about a weak Poisson's formula. A strong Poisson's formula
corresponds to the case where one can prove the equality everywhere (and in
particular at t = 0) of <I>(t) and of its Fourier series. We shall say more about the
Poisson formula and, in particular, give strong versions of it in Seetion A33. The
version we have here, and that we shaH proceed to prove, is the one we need in the
Shannon-Nyquist sampling theorem (Chapter B2).
Proof:

We first show that <I>(t) is weH defined:

{T L Is(t + nT)1 dt = L

10

nEZ

(T Is(t + nT)1 dt

nEZ

= L
nEZ

10

<n+l l T

Is(t)1 dt

nT

1
R

Is(t)1 dt <

00.

In particular,

+ nT)1

Is(t

<

00

a.e.

nEZ

Therefore, the series LnEZ set +nT) converges absolutely for almost all t. In particular, <I>(t) is weH defined (define it arbitrarily when the series does not converge).
This function is c1early T -periodie. We have

{T

10

1<I>(t)ldt =

{T ILS(t+nT)ldt

10

nEZ

:s {T L

10

Is(t

+ nT)1 dt =

nEZ

(ls(t)1 dt <

1R

Therefore, <I>(t) is stable. Its nth Fourier coefficient is

cn(<I

= -1
T

<I>(t)e- 2"''in l dt

= ~ {T {L set + kT)! e-2inIfI dt

T

10

kEZ

00.

30

A2. Fourier Series of Locally Stable Periodic Signals

=~
T

{T

Ja

!LS(t + kT)e-

2i1C ',f(t+kTl!

dt

kEZ

{
2 n
1 ~ (n )
= T1 JJR
s(t)e- l1C'it dt = T S T .

We have a function as weH as its formal Fourier series. When both are equal
everywhere, we obtain the strong Poisson sum formula. The next exercise gives
conditions for this.1t will be improved by Theorem A3.12.
EXERCISE

(a)

A2.S. Let set) be a stable signal with the FT s(v), and suppose that

LnEZ

set

+ nT) is a continuous function,

(b) LnEZ Is(n/T)I <

Show that, for all t

and

00.

lR.,

Ls(t+nT)= L
nEZ

nEZ

s(f)e

2i1C ',ft.

A3
Pointwise Convergence
of Fourier Series

A31

Dini' s and Jordan' s Theorems

The inversion formula for Fourier series obtained in Chapter A2 requires a rather
strong condition of summability of the Fourier coefficients series. Moreover, this
condition implies that the function itself is almost everywhere equal to a continuous
function. In this seetion, the dass of functions for which the inversion formula holds
is extended.
Recall Kolmogorov's negative result (see the Introduction):
THEOREM A3.1. There exists a locally integrable 2rr -periodic function f : ~
for which the Fourier se ries diverges everywhere.

ce

This result challenges one to obtain conditions that a locally integrable 2rrperiodic function f must satisfy in order for its Fourier series to converge to
f. Recall that the Fourier series associated with a 2rr-periodic locally integrable
function f is the formal Fourier series
(33)
where cn(f) is the nth Fourier coefficient
cn(f)

_1
2rr

j+Jr f(u)e-inu du.

(34)

-Jr

The series (33) is calledformal as long as one does not say something about its
convergence in some sense (pointwise, almost everywhere, in LI, etc). If one has
P. Brmaud, Mathematical Principles of Signal Processing
Springer Science+Business Media New York 2002

32

A3. Pointwise Convergence ofFourier Series

no more than the condition that f is 27T -periodic and locally integrable, the worst
can happen, as Kolmogorov's theorem shows.
The purpose of this section is to find reasonable conditions guaranteeing
convergence as n -+ 00 of the truncated Fourier series
+n

sI (x) = L

ck(f)e ikx .

(35)

k=-n

We have to specify (1) in what sense this convergence takes place and (2) what
the limit iso Ideally, the convergence should be pointwise and to fitself. The next
exercise gives a simple instance where this is true.
EXERCISE

A3.1. Assume that the trigonometric series

Sn(t)

+n

Ck eikt

k=-n

converges uniformly to some function f(t). Show that in this case, for all k E Z,
Ck = ck(f).

Dirichlet's Integral

We will first express the truncated series

this we write

sI (x) = L+n { -1
k=-n

27T

= - 1 j+Jr
27T -Jr

sI in a form suitable for analysis. For

j+Jr

.} .
f(s)e- ,ks ds e'kx

IL I
-Jr

+n eik(x-s)
k=-n

f(s)ds.

Elementary computations give

+n

L eikt =
k=-n

sin((n

1
-2

)t)

(36)

sin(t /2)

(the function in the right-hand side is called the Dirichlet kerne!) and therefore,
f

Sn (x)

1 j+Jr sin((n + i)(x - s))

-2
. (( _ )/2)
fes) ds.
7T -Jr
sm x s

Performing the change of variable x - s = u and taking into account the fact that
fand the Dirichlet kernel are 27T-periodic, we obtain
f

Sn (x)

1 j+Jr sin((n + !)u)

. (/2)
fex
-Jr
sm u

-2
7T

+ u)du.

The right-hand side of (37) is called the Dirichlet integral.

(37)

A31 Dini's and Iordan's Theorems

If we let f(t)

33

=1 in (35), we obtain 1; on substituting this in (37),

j+Jr sin((n + !)u) du-I.
_

27f

(38)

sin(u/2)

-Jr

Therefore, for any real number A,

I
S!(x)-A=27f
or, equivalently,
S!(x) - A

= -

27f

iJr

j+Jr sin((n
+ ! )u)
.
2
(f(x+u)-A)du

sin((n

+ ! )u)
2

sm(u/2)

(39)

sm(u/2)

-Jr

{fex

+ u) + fex

- u) - 2A}du. (40)

Therefore, in order to show that, for a given x E IR, S! (x) tends to A as n -+ 00,
it is neeessary and suffieient to show that the Diriehlet integral in the right-hand
side of (39) eonverges to zero as n -+ 00.
The localization principle states that the eonvergenee of the Fourier series is a
loeal property. More preeisely:
THEOREM A3.2. lf fand gare two locally integrable 27f -periodic complex-valued
functions such that, for a given x E IR and some 8 > 0, it holds that f (t) = g(t)
whenever t E [x - 8, x + 8], then

lim{S!(x) - S!(x)}

ntoo

Proof"

= O.

Using (39) we have

s! (x) -

S!(x)

= -2

j+Jr

7f

= 27f

-Jr

sin((n

+ !)u) Ilul:::8

fex

+ u) -

g(x

. ( /2)

sm u

+ u)

du

j+Jr sin((n + !)u) w(u) du,

-Jr

where
w(u)

= l lul ->8

fex

+ u) -

g(x
. ( u /2)
sm

+ u)

is integrable over [0, 27f]. The last integral therefore tends to zero as n -+
the Riemann-Lebesgue lemma.

00

by

We now state the general pointwise convergence theorem.

THEOREM A3.3. Let f be a locally integrable 27f-periodic complex-valued
function, and let x E IR and A E IR be given. Then

lim S!(x) = A

ntoo

34

A3. Pointwise Convergence of Fourier Series

if,for some

< 8

:s Ti,

.1
8

~(u)
+ !)u) - du =

.
smn

11m
ntoo

~(u)

= f(x

0,

(41)

+ u) + f(x - u) - 2A.

(42)

uj2

where

Proof" Taking ga constant equal to A, we have Sn(g) = A, and therefore we are

looking for a sufficient condition guaranteeing that Sn(f) - Sn(g) tends to as n
tends to 00. By the localization principle, it suffices to show that

lim
ntoo

[8 sinn + !)u)

10

.~(u)

Sln(uj2)

du = 0.

(43)

The two integrals in (41) and (43) differ by

1
8

sinn

+ !)u) v(u)du,

(44)

where

v(u)

= ~(u) {U~2

Sin(~j2)}

is integrable on [0, 8]. Therefore, by the Riemann-Lebesgue lemma, the quantity

(44) tends to zero as n --+ 00.

Dini's Theorem

THEOREM A3.4. Let f be a 2Ti-periodic locally integrable complex-valued

function and let x E IR. Iffor some < 8 :s Ti and some A E IR, the function
t --+

f(x

+ t) + f(x - t) - 2A

is integrable on [0, 8], then

lim
ntoo

S! (x) = A.

Proof" The hypothesis says that the function ~(u)ju, where ~ is defined in (42),
is integrable, and therefore condition (41) ofTheorem A3.3 is satisfied (RiemannLebesguelemma).

We shall give two corollaries ofDini's result.

A3.1. If a 2Ti -periodie locally integrable complex-valued function

f(t) is Lipschitz continuous of order IX > about x E IR, that is,

COROLLARY

If(x
then limntoo

+ h) - f(x)1

S! (x) = f(x).

= O(lhIO')

as h --+ 0,

A31 Dini's and Jordan's Theorems

Proof

35

Indeed, with A = f(x),

f(x

+ t) + f(x

- t) - 2A I < K _1_

ItI 1

-0:'

for some constant K and for all t in a neighborhood of zero, and 1/ltI 1-0: is
integrable in this neighborhood, because I-ex< 1. Dini's theoremA3.4 concIudes
the proof.

COROLLARY A3.2. Let f(t) be a 21T-periodic locally integrable complex-valued

function, and let x E lR be such that

f(x

+ 0) =

+ h)

lim f(x
hW

and

f(x - 0) = lim f(x - h)

hW

exist and are finite, and further assume that the derivatives to the left and to the
right at x exist. Then
. SI() _
I1m
n X -

f(x

+ 0) + f( x

Prao!"

- 0)

ntoo

By definition, one says that the derivative to the right exists if

lim f(x

+ t) -

f(x

+ 0)

ttO

exists and is finite, with a similar definition for the derivative to the left. The
differentiability assumptions imply that
lim f(x

+ t) -

f(x

+ 0) +

f(x - t) - f(x - 0)

ttO

exists and is finite and therefore that

tjJ(t)
f(x + t)

+ f(x

- t) - 2A

is integrable in a neighborhood of zero, where

2A

= f(x + 0) + f(x

- 0).

Dini's theorem A3.4 concIudes the proof.

EXAMPLE

A3.1. Apply the previous theorem to the 21T -periodic function defined

by
f(t)

=t

when 0 < t :::; 21T.

Onefinds
t

= 1T -

"" sin(nt)
~

nEZ

n#O

For t

2-n

when 0 < t < 21T.

= 0, we can directly check that the sum of the Fourier series is

!(f(O+)+f(O-)) = !(0+21T)=1T,

36

A3. Pointwise Convergence of Fourier Series

as announced in the last corollary. For t

= n /2, we obtain the remarkable identity

1

nIl

4=1-3+:5-7+
Jordan's Theorem

Jordan 's convergence theorem features funetions of bounded variation.

A3.1. A real-valued function q; : lR
variation on the interval [a, b] C lR if

DEFINITION

f-+

lR is said to have bounded

n-l

sup
'D

L !q;(Xi+l) -

q;(Xi)! <

(45)

00,

i=O

= {a = Xo

where the supremum is over all subdivisions D

<

Xl

< ... < Xn

= b}.

We quote without proof the fundamental result on the strueture of bounded

variation funetions.
THEOREM A3.5. A real-valued function q; has bounded variation over [a, b] if and
only ifthere exist two nondecreasing real-valuedfunctions q;l, q;2 such that,for all
tE [a, b],

q;(t) = q;l (t) - q;2(t).

(46)

In partieular, for all X E [a, b), q; has a limit to the right q;(x + 0); for all
X E (a, b], it has a limit to the left q;(x - 0); and the diseontinuity points of q;(t)
in [a, b] form a denumerable set, and therefore a set ofLebesgue measure zero.
A3.6. Let f be a 2n-periodic locally integrable real-valuedfunction
of bounded variation in a neighborhood of a given X E lR. Then

THEOREM

lim

st (x) =

f(x

+ 0) +

f(x - 0)

ntoo

(47)

The proof is omitted.

EXERCISE

A3.2. Let f

I-B

E L~(lR).

Show that, for any B > 0,

+ j(v)e2irrvt dv

= 2B

{ f(t

JR

+ s)sine (2Bs) ds,

and use this to study the pointwise convergence of the left-hand side as B tends to
infinity, along the lines of the current chapter.

The funetion
2B sine (2Bt)

is also ealled Dirichlet's kernei.

A31 Dini's and Jordan's Theorems

37

Let!t and h be the 2rr -periodic functions defined on (-rr, +rr]

EXERCISE A3.3.

by

!t(x) = x,
Compute their Fourier coefficients, and use this to compute

(_l)n,

n~l

Integration of Fourier Series

Let f(t) be a real-valued 2rr-periodic locally integrable function. Denoting by

Cn the nth Fourier coefficient of f(t), we have C n = c~ because f(t) is real.
Therefore, the Fourier series of f(t) can be written as
00

!ao

+ L{an cos(nx) + bn sin(nx)},

(48)

n=l

where, for n ::: 1,

2Jr

an = -1
rr

f(t)cos(nt)dt,

bn

= -1
rr

2Jr

f(t)sin(nt)dt.

Of course, the series in (48) is purely formal when no additional constraints are
put on f(t) in order to guarantee its convergence. Now, the function F(t) defined
for tE [0, 2rr) by
F(t) = Iat(f(X) - !ao)dx

(49)

is 2rr-periodic, is continuous (observe that F(O) = F(I) = 0), and has bounded
variation on finite intervals.
Therefore, by Jordan's theorem its Fourier series converges everywhere, and for
all x E lR,
00

F(x)

= !A o + L{A n cos(nx) + Rn sin(nx)},

n=l

where, for n ::: 1,

An = -1
rr

2Jr

F(t) cos(nt) dt

1 [
sin(nx)
=- F(x)-rr
n

= - -1
nrr 0

2Jr

]2Jr
0

-1
nrr

2Jr

(f(t) - !ao) sin(nt) dt

b
f(t)sin(nt)dt = _..!:,
n

38

A3. Pointwise Convergence of Fourier Series

and, with a similar computation,

B
Therefore, for all x

n=

..!.. (2n F(t) sin(nt) dt

n

10

= an
n

lR,

1
~
F(x) = zA
o+ ~

{an-;; sm(nx)
. - -;;
bncos(nx) }.

(50)

The constant A o is identified by setting x = 0 in (50):

1
~bn
zA o = L...- - .

n=l

(51)

Since A o is finite we have shown, in particular, that L~l bn/n converges for any
sequence {b n }n2:1 of the form
bn =

..!.. (2n J(t) sin(nt) dt,

n

10

where, J(t) is areal function integrable over [0, 2n].

Gibbs' Overshoot Phenomenon

We dose this section by mentioning a phenomenon typical of the behavior of a
Fourier series at a discontinuity of the function. Gibbs' overshoot phenomenon
has nothing to do with the failure of the Fourier series to converge at a point of
discontinuity of the corresponding function. It concems the overshoot of the partial
sums at such a point of discontinuity. An example will demonstrate this effect.

I~ ;

Consider the 2n -periodic function defined in the interval ( - n,

+ n] by

ifx > 0,

J(x)

if x < 0,

n
2

x
2

----

if x < O.

The partial sum of its Fourier series is

f
_ ~ sin(nx)
Sn (x) - L...- - - .
k=l
n

sI

By Dini's theorem, the partial sum

(0) converges pointwise to (1/2)(/(0+) +
J(O- = n /2. However, we shall see that for some A > n /2 and sufficiently
large n,

S!(~)

::: A.

(52)

Therefore, there exist a constant c > 0 and a neighborhood No of 0 such that

::: c whenever x E No - {O}. This constitutes Gibbs' overshoot

IsI (x) - sI (0)1

A32 Fejer's Theorem

39

phenomenon, which can be observed whenever the function has a point of discontinuity. The proof of (52) for this special case keeps most of the features of the
general proof, which is left for the reader. In this special case,
f

Sn (x)

Now,

sin((n

2 sin('it)

-_lX
o

+ !)t)

sin((n

+ !)t) dt -

-----;----"-- + -21 cos(nt))

2sin(!t)

sin(nt)
t

ll

+ -

dt

(sin(nt)Cos(!t)

- - dt

x
-.

2 sin(!t)

sm(nt)

dt

cos(!t)
1

2 sin('it)

1)

-t

dt

cos(nt)dt.

The last two integrals converge uniformly to zero (by the uniform version of the
Riemann-Lebesgue lemma). Also,

~ sin(nt)

A32

- - dt

sin(t)
- dt ::::: 1.18 -:rr:rr
> -

tot

Fejer's Theorem

sI

The Fourier (t) series of a 2:rr-periodic locally integrable function I converges

to I(t) for a given t only under certain conditions (see the previous section).
However, Cesaro convergence of the series requires much milder conditions. For
a 2:rr -periodic locally stable function I, Fejer's sum
(53)
behaves more nicely than the Fourier series itself. In particular, for continuous
functions, it converges pointwise to the function itself. Therefore, Fejer's theorem
is a kind of inversion formula, in that it shows that for a large dass of periodic
functions (see the precise statement in Theorem A3.11 ahead), the function can be
recovered from its Fourier coefficients.

40

A3. Pointwise Convergence of Fourier Series

Fejer's Kernel
Take the imaginary part of the identity
n-l
' " ei(k+l/2)u

e iu / 2

1_

inu
e .

1- e lU

k=O

to obtain

sI

Starting from Dirichlet's integral expression for (t) [cf, Eq. (37)], we obtain, in
view of the identity just proven, Fejer's integral representation of a! (x),
a!(x)

{+Jr

LJr

Kn(u)f(x-u)du=

(+Jr

LJr

Kn(x-u)f(x)du,

(54)

where
I

Kn(t) =

sin2(~nt)

(55)

2 1

2n:rr sin ('it)

is, by definition, Fejer's kernel. It has the following properties:

and [letting f(t)

= 1 in (54)],

i:

(56)

Jr Kn(u) du = 1.

(57)

Also (the proofs are left as an exercise),

lim Kn(t) = 1,

(58)

(59)

ntoo

and, for all e S :rr,

lim

ntoo

+C

Kn(u) du = 1.

-6

The last four properties make ofFejer's kernel a regularization kerneion [-:rr, +:rr]
(by definition of a regularization kernel).

Cesaro Convergence for Fourier Series of Continuous Functions

We first treat the case of continuous functions, because the result can be obtained from the basic principles of analysis, in particular, without recourse to
the Riemann-Lebesgue lemma.
THEOREM

A3.7. Let f(t) be a 2:rr-periodic continuousfunction. Then

lim

sup

ntoo XE[-Jr,+Jr]

la! (x) - f(x)1

= O.

(60)

A32 Fejer's Theorem

Proof'

i:

41

From (54) and (56), we have

la! (x) - l(x)1 ::s

KnCu) I/(x - u) - l(x)1 du

1+
+8

-8

=A+B.

(61)

[-n,+n]\[-8,+8]

For a given 8 > 0, ehoose 8 sueh that I/(x - u) - l(x)1 ::s 8/2 when lul ::s 8.
Note that I is uniformly eontinuous and uniformly bounded (being aperiodie and
eontinuous funetion), and therefore 8 ean be ehosen independently of x. We have
A::s

81+8 Kn(u)du ::s 2'8

-8

and, ealling M the uniform bound of I,

B ::s 2M [

Kn(u) du.

J[-n,+n]\[ -8,+8]

By (57) and (59), B ::s 8/2 for n sufficiently large. Therefore, for n suffieiently
large, A + B ::s 8.

Fejer's theorem for eontinuous periodie funetions is the key to important approximation theorems. The first one is for free. We eall a trigonometrie polynomial
any finite trigonometrie sum of the form

L
+n

p(x) =

Ck eikx .

-n

Let I (t) be a 2n -periodic continuous function. Select an arbitrary

> O. Then there exists a trigonometric polynomial p(x) such that

THEOREM A3.8.
8

sup

I/(x) - p(x)1 ::s

8.

tE[-n,+n]

Proof'

iill.

Use Theorem A3.7 and observe that a! (x) is a trigonometrie polynom

From this, we obtain the Weierstrass approximation theorem.

Let I : [a, b] 1-+ C be a continuousfunction. Select an arbitrary
> O. There exists a polynomial P(x) such that

THEOREM A3.9.
8

sup I/(x) - P(x)1 ::s

8.

tE[a,b]

Jf, moreover,

is real-valued, then P can be chosen with real coefficients.

Proof' First, suppose that a = 0, b = 1. One ean then extend I : [0, 1]] 1-+ C
to a funetion still denoted by I, I : [-n, +n]] 1-+ C, that is eontinuous and
sueh that IHn) = I( -n) = O. By Theorem A3.8, there exists a trigonometrie

42

A3. Pointwise Convergence of Fourier Series

polynomial p(x) such that

sup If(x) - p(x)l:s

sup

tE[D, I]

tEl

-rr,+rr]

If(x) - p(x)1

e
:s -.

Now replace each term e ikx in p(x) by a sufficiently large portion of its Taylor
series expansion, to obtain a polynomial P(x) such that
e
sup IP(x) - p(x)1 :s -.
tE[D,I]
2
Then
sup If(x) - P(x)1
tE[D, I]

:s -.
2

To treat the general case f : [a, b] f-+ C, apply the result just proven to cp
[0, 1] f-+ ce defined by cp(t) = f(a + (b - a)t) to obtain the approximating
polynomial rr(x), and take P(x) = rrx - a)j(b - a.
Finally, to prove the last statement of the theorem, observe that
If(x) - Re P(x)1

:s

If(x) - P(x)l

Fejer's Theorem

We shall first obtain for the Fejer's sum the result analogous to Theorem A3.3.
First, from (54), we obtain
a!(x)

irr sin. 222(!nu) {f(x+u)-f(x-u)}du;

= -1

sm Czu)

2nrr D

(62)

therefore, for any number A,

1

a!(x)-A= 2nrr
THEOREM

irr sin. 222( !nu) {f(x+u)+f(x-u)-2A}du.

1

sm (:zu)

A3.10. For any x

IR. and any constant A,

lima!(x)

ntoo

if, for some 8 > 0,

. 1
11m
-

ntoo

(63)

(64)

8 .
4J(u)
sm 2(!nu) - 2 D
u

du

= 0,

(65)

where
4J(u)
Proof

= f(x + u) + f(x

(66)

- u) - 2A.

The quantity

In 18

sin 2(!nu) 4J(U) du l <

sin 2 (!u)

- n

18

14J(u)1
sin 2 (!u)

du

A33 The Poisson Formula

tends to 0 as n

We must therefore show that

00.

n
tends to 0 as n

00.

43

sin 2( !nu)

sin 2 (!u)

fjJ(u) du

However, (65) guarantees this because

11 (1

:'S n

. 2( 1

SIll

ZU

) -

1I

12 IfjJ(u)1 du
ZU

tends to 0 as n t 00 (the expression in curly brackets is bounded in [0, 8], and

therefore the integral is finite).

THEOREM A3.11. Let f (t) be a 2JT -periodic locally integrable function and assume

that, for some x E ~, the limits to the right and to the left (respectively, f(x
and f(x - 0), exist. Then

lim

Proof"

u! (x) =

+ 0) + f(x

- 0) .
2
Fix 8 > O. In view of the last result, it suffices to prove (65) with
fjJ(u)

= {f(x + u) -

f(x

+ O)} + {f(x -

f(x

11

rJ(c),

sin2(!nu)
1
;
fjJ(u) du

c
n

< -

1ry

sin 2(!nu)

u2

du

+ -1

1 -IfjJ(u)1-

n ~

u2

du.

The last integral is bounded; and therefore, the last term goes to 0 as n
for the penultimate term, it is bounded by Ac, where

A33

(67)

u) - f(x - O)}.

Since fjJ(u) tends to 0 as n ~ 00, for any given c > 0 there exists rJ
rJ :'S 8, such that IfjJ(u)1 :'S c when 0 < u :'S rJ. Now,

o<

+ 0)

00

sin2(!v)

dv <

00.

00.

As

The Poisson Formula

The following corollary of F6jer's theorem will play the key role for the proof of
the Poisson sum formula (Theorem A3.l2).

44

A3. Pointwise Convergence of Fourier Series

COROLLARY A3.1.

that, for some x

Let f be a 2rc -periodie locally integrable function and suppose

lR.,

(a) thefunction f is continuous at x, and

(b) its Fourier se ries
Then A
Proof'

sI (x) converges to some number A.

= fex).

From (b) we see that

lim

ntoo

0'1 (x) = A.

From F6jer's theorem and (a),

lim

ntoo

0'1 (x) = fex).

We have already given a weak version of the Poisson sum formula in Section
A22. A most interesting situation is when the function cI>(t) defined by (31) is
equal to its Fourier series for all t E lR., that is,

Ls(t
nEZ

+ nT) = ~ LS(f) e2in !ft

for all tE R

(68)

nEZ

The next theorem extends the result in Exercise A2.8.

THEOREM A3.12. Let set) be a stable complex signal, and let 0 < T <
fixed. Assume in addition that

(1) LnEZ set

(2)

LnEZ

00

be

+ n T) converges everywhere to some continuous function,

s( f) e2in !ft converges for all t.

Then the strang Poissonformula (68) holds.

Proof' The result is an immediate consequence of both the weak Poisson summation result (Theorem A2.3) and the corollary of F6jer's theorem in Section
A32.

Here are two important cases for which the strong Poisson sum formula holds.
COROLLARY A3.1. Let set) be a stable complex signal, and let 0 < T < 00 be
fixed. If, in addition, L set + nT) converges everywhere to a continuousfunction
that has bounded variation, then the Poissonformula (68) holds.

Praof' We must verify conditions (1) and (2) ofTheorem A3.12. Condition (1)
is part of the hypothesis. Condition (2) is a consequence of Iordan's theorem
A3.6.

A3.1. If set) is continuous, has bounded support, and has bounded

variation, the Poisson sumformula (68) holds.

EXAMPLE

A33 The Poisson Formula

COROLLARY

A3.2. If a stable continuous signal s(t) satisfies

s(t)
s(v)

0(1

oe

as Itl

+\tl"')

+llvl"')

~ 00,

as lvi

~ 00,

for some a > 1, then the Poisson formula (68) holds for alt
Proof

45

(69)

< T <

The result is an immediate corollary of Theorem A3.12.

00.

Convergence Improvement

The Poisson formula can be used to replace aseries with slow convergence by one
with rapid convergence, or to obtain some remarkable formulas. Here is a typical
example. For a > 0,

s(t) = e-2naltl ~ s(v) =

n(a 2

+ v2 )

Since

Ls(t +n)

= Le-2nalt+nl

nEZ

nEZ

is a continuous function with bounded variation, we have the Poisson formula, that
is,

The left-hand side is equal to

2

1 - e- 2na

-1,

and the right-hand side can be written as

Therefore,
1

nO": 1

Letting a

n 1 + e- 2na

a 2 + n 2 = 2a 1 - e- 2na

1
2a 2 .

0, we have

The general feature of the above example is the following. We have aseries
that is obtained by sampling a very regular function (in fact, C OO ) but also slowly

46

A3. Pointwise Convergence of Fourier Series

h(t-2T)

~
[jU[jU[j[j[j~
1

-3T -2T

-T

2T

3T

-3T -2T

-T O T

2T

3T

Figure A3.1. Radar return signal

decreasing. However, because of its strong regularity, its Fr has a fast decay. The
series obtained by sampling the Fr is therefore quickly converging.
Radar Return Signal

Let s(t) be a signal ofthe form

s(t) =

(I>(t -

nT) f(t).

(70)

nE'L

(We may interpret h(t - nT) as a return signal of the nth pulse of a radar after
reftection on the target, and f (t) as a modulation due to the rotation of the antenna. )
The Fr ofthis signal is (see Fig. A3.1)

s(v) = ~ Lh(!!.-) f(v T

nE'L

(t)

!!.-).
T

(71)

EXERCISE A3.4. Show that if(1) f

is integrable, (2) LnE'L h(t -n T) is integrable
and continuous, and (3) LnE'L h(n/T) < 00, then (71) holds true. Find other
conditions.

References
[Al] Ablowitz, M.J. and Jokas, A.S. (1997). Complex Variables, Cambridge University
Press.
[A2] Bracewell, R.N. (1991). The Fourier Transform and Its Applieations, 2nd rev. ed.,
McGraw-Hil1; New York.
[A3] Gasquet, C. and Witomski, P. (1991). Analyse de Fourier et Applieations, Masson:
Paris.
[A4] Helson, H. (1983). Harmonie Analysis, Addison-Wesley: Reading, MA.
[A5] Katznelson, Y. (1976). An Introduetion to Harmonie Analysis, Dover: New York.
[A6] Kodaira, K. (1984). Introduetion to Complex Analysis, Cambridge University Press.
[A7] Krner, T.W. (1988). Fourier Analysis, Cambridge University Press.

References

47

[A8] Rudin, W. (1966). Real and Complex Analysis, McGraw-Hill: New York.
[A9] Titchmarsh, E.C. (1986). The Theory of Funetions, Oxford University Press.
[AlO] Tolstov, G. (1962). Fourier Series, Prentice-Hall (Dover edition, 1976).
[All] Zygmund, A. (1959). Trigonometrie Series, (2nd ed., Cambridge University Press.

Part

Signal Processing

Introduction

The Fourier transform derives its importance in physics and in electrical engineering from the fact that many devices mapping an input signal x(t) into an output
signal y(t) have the following property: If the input is a complex sinusoid e2invt,
the output is T(v)e2invt, where T(v) is a complex function characterizing the device. For example, when x(t) and y(t) are, respectively, the voltage observed at the
input and the steady-state voltage observed at the output of an Re circuit (see Fig.
BO.I), the input-output mapping takes the form of a linear differential equation:
y(t)

+ RCy(t) = x(t),

and it can be readily checked that

T(v)

I
1+ 2irrvRC

The Re circuit is one of the physical devices that transform a signal into another
signal, that satisfy the superposition principle, and that are time-invariant. More
precisely:
R

1'~1'
C
y(t)

x(t)

l"""

,1,,)

Figure BO.I. The Re circuit

52

Part B Signal Processing

(a) If Yl (t) and Y2(t) are the outputs corresponding to the inputs Xl (t) and X2(t),
then AlYl (t) + A2Y2(t) is the output corresponding to the input AlXl (t) + A2X2(t);
(b) If y(t) is the output corresponding to x(t), then y(t - -r) is the output
corresponding to x(t - -r).
Such physical devices are caIled (homogeneous linear) filters.
A basic example is the convolutional filter, for which the input-output mapping
takes, in the time domain, the form
y(t)

h(t - s)x(s)ds,

where h(t) is caIled the impulse response, because it is the response of the filter
when the Dirac pulse 8(t) is applied at the input. Indeed,
h(t) =

h(t - s)8(s) ds.

If the impulse response is integrable, the output is weIl defined and integrable, as
long as the input is integrable. Then, by the convolution-multiplication rule, the
expression of the input-output mapping in the frequency domain is
y(v) = T(v)x(v),

where T (v) is the frequency response, that is, the Fr of the impulse response:
T(v) =

h(t)e-2i1Cvt dt.

Observe that if the input is x(t) = e-2i1Cvt, the output is weIl defined and equal to

h(s)x(t - s)ds

h(s)e- 2i1CV (t-s)ds

= T(v)e-2i1Cvt,

in accordance with what was said in the beginning of this introduction.

In the particular case of the RC filter, the solution of the differential equation
with arbitrary initial condition at -00 is indeed of the convolution type, with the
impulse response

The RC filter is a convolutional filter, and it contains the typical features of

the more general filters. In the general case, since a filter is a mapping, we shall
have to define its domain of application. Depending on this domain, the inputoutput mapping takes different forms. In the above informal discussion of the
RC circuit, there are a frequency-domain and a time-domain representation and
also a representation in terms of a linear homogeneous differential equation. The
latter is not general. In fact, when it is available, the transmittance is a rational
function ofthe frequency v. The corresponding filters, caIled rational filters, form
an important class, and Chapter BI gives the basic concepts concerning analog
(that is, continuous-time) filters.

Introduction

53

In addition to filtering, there are two fundamental operations of interest

in communications systems: frequency transposition and sampling. Frequency
transposition is a basic technique of analog communications. It has two main applications, the first of which is transmission. Indeed, the Hertzian channels are in
the high-frequency bands-in fact, much higher than the one ofbrute signals such
as the electric signals carrying voice, for instance-and consequently, the latter
have to be frequency-shifted. The second reason is resource utilization and is related to frequency multiplexing, a technique by which signals initially occupying
the same frequency band are shifted to nonoverlapping bands and can then be simultaneously transmitted without mutual interference. From a mathematical point of
view the theory of frequency transposition (or, equivalently, of band-pass signals,
to be defined in Chapter BI) is not difficult. It remains interesting because of the
special phenomena associated with this technique, such as cross-talk in quadrature
multiplexing and dispersion phenomena.
In digital communications systems, an analog signal s(t) is first sampled, and the
result is a sequence of sampies s(n T), n E Z. It is important to identify conditions
under which the sampie sequence faithfully represents the original signal. The
central result of Chapter B2 is the so-called Shannon-Nyquist theorem, which
says that this is true if the signal s(t) is stable and continuous and if the support
of its Fr s( v) is contained in the interval [-1/ T, + I/Tl. The original signal can
then be recovered by the reconstruction formula:
s(t) =

L s (nT) sinc (f - n) .
nEZ

The theory of sampling is an application of the results obtained in Part A, and

in particular of the Poisson sum formula. The above reconstruction formula has
many sourees, 1 and its importance in communications was fully realized by Claude
Shannon and Harald Nyquist.
The Shannon-Nyquist sampling theorem is the bridge between the analog
(physical) world and the discrete-time (computational) world of digital signal processing. The reader will find in the main text abrief discussion of the interest of
digital communications systems. Therefore, a large portion of this Part B is devoted
to discrete-time signals (Chapters B2-B4).
As we have already mentioned, the Poisson sum formula is the key to the sampling theorem. It also plays a very important role in the numerical analysis of
the discrete Fourier transform considered as an approximation of the continuous Fourier transform (see Chapter B3) and also in the intersymbol interference
problem (see Chapter B2).
The study of the interaction between discrete time and continuous time is not
limited to the sampling theorem. For instance, we prove that filtering and sampling
ISee J.R. Higgins, Five short stories about the cardinal series, Bult. Amer. Math. Soc.,
12, 1985,45-89.

54

Part B Signal Processing

cmumute for base-band signals. This is not a difficult result, but it is of course a
fundamental one because in signal processing, one first sampies and then performs
the filtering operation in the sampled domain, since one of the advantages of digital
processing comes precisely from the difficulty of making analog filters.
One advantage of analog processing is that it is instantaneous. To maintain
competitivity, the signal processing algorithms have to be fast. For instance, the
discrete Fourier transform is implemented by the so-called fast Fourier transform,
an algorithm whose principle we briefty explain in this Part. Subband coding also
has a fast algorithm associated with it. It is a data compression technique. The
signal is not directly quantized, but instead, it is first analyzed by a filter bank, and
the output of each filter bank is quantized separately. This allows one to dispatch
the compression resources unequally, with fewer bits allocated to the subbands
that are less informative (see the discussion in Chapter B4). Subband coding is
the last topic of Part Band introduces the sections on multiresolution analysis in
Part D.

BI
Filtering

B 11

Impulse Response and Frequency Response

Convolutional Filter
We introduce a particular and very important dass of filters.
DEFINITION Bl.l. The transformation fram the stable signal x(t) to the stable
signal y(t) defined by the convolution

y(t)

h(t - s)x(s)ds,

(1)

where h(t) is stable, is ca lied a convolutional filter. This filter is called a causal
filter if h(t) = 0 for t < O.

The signal y(t) is the output, whereas the signalx(t) is the inputofthe linear filter
with impulse response h(t). Informally, if x(t) is the Dirac generalized function
8(t) (an impulse at time 0), the output is (see Fig. Bl.1)

h(t - s)8(s) ds = h(t),

whence the terminology.

A causal filter responds only after it has been stimulated. For this reason, it
is sometimes also called a realizable filter (Fig. B 1.1. features a causal impulse
response). For such filters, the input-output relationship (1) becomes (note the
P. Brmaud, Mathematical Principles of Signal Processing
Springer Science+Business Media New York 2002

56

BI. Filtering

8(t)

f\h(t~
>

impulse

>

impulse response

Figure B 1.1. Impulse and impulse response

upper limit of integration)

y(t)
DEFINITION

[t

oo

h(t - s)x(s) ds.

(2)

B1.2. The Fourier transform ofthe (stable) impulse response h(t),

T(v) =

h(t)e-2i1Cvt dt,

(3)

is ca lied the frequency response.

If the input is the complex sinusoid x(t)

= e2i1Cvt, by (1), the output is

y(t) = T(v)e2i1Cvt.

(4)

(Note that the output is weH defined by the convolution formula, even though in
this particular case the input is not integrable.)
EXERCISE Bl.1. Let y(t) be the output of a stable and causal convolutional filter
with impulse response h(t) [see (2)]. Let

z(t)

1
t

h(t - s)x(s)ds,

~ 0,

be the output of the same filter, when the input x(t) is applied only from time t = 0
on. Show that

lim Iz(t) - y(t)1

tt+oo

= O.

A More General Definition

Convolutional filters are only a special dass of filters. A more general definition
is as foHows. Denote by C ~ the set of functions of lR. into C.
DEFINmON Bl.3. Let D(12) be a set of functions from lR. into C with the two
following properties:

(a) It is closed under linear operations;

() it is closed under translation.
12 : D(12)

1-+

C ~ is ca lied a homogeneous linear filter with domain D(12) if:

(i) 12 is linear, and

(ii) 12 is time-invariant.

B 11 Impulse Response and Frequency Response

The meaning ofproperties (a) and () is the following: (a) XI (t), X2(t)
E

===}

x(t - T)

D('c).

AI, A2

+ A2X2(t)

AIXI (t)

E D('c);

and () x(t)

E D('c),

The meaning of properties (i) and (ii) is the following: (i) XI (t), X2(t)
AI, A2

E C,XI(t)~

A2Y2(t); (ii) x(t)

EXERCISE

YI(t),X2(t)

E D('c),

Y2(t)

B1.2. Show that if e 2i :rcvt

E D('c),

e 2mvt

D('c),

lR

===}

D('c),

AIXI(tHA2X2(t) ~ AI(t)YI(tH

===}

lR, x(t) ~ y(t)

57

x(t - T) ~ y(t - T).

===}

then

T(v)e 2mvt

(5)

for some complex number T(v).

The function T(v) is called thefrequency response ofthe filter. Every frequency
response is of the form

T(v)
where G(v)
EXAMPLE

= G(v)ei(v),

(6)

= IT(v)1 is the amplitude gain and (v) = Arg T(v) is the phase.

Bl.l. Let
D('c) = {x(t) :

or
D('c) = {x(t)

+ e(t) :

Ix(t)1 dt < oo},

Ix(t)1 dt <

00

and e(t)

E},

where E is the set of complex finite linear combinations of complex exponentials.

For any signal in D('c), the right-hand side of (1) is welt defined, and we can
therefore define the filter ,C with domain D('c) by the input-output relationship
(1). Thefrequency response, as defined by (5), is then the FT ofh(t).
Let h(t) E L~(lR). We shalt see in Part C that the FT h(v) = T(v)
of h(t) can be defined and that it is in L~(lR). We take D('c) = LUlR) and define
,C by the input-output relationship
EXAMPLE B1.2.

y(t)

T(v)x(v)e 2i :rcvt dv,

(7)

where xCv) is the FT ofthe input x(t) E L~(lR). The right-hand side of(7) has a
meaning since T(v) and xCv) being in L~(lR) implies that T(v)x(v) is in Lt(lR)
(see Theorem 20 of the appendix).
EXAMPLE B 1.3. If T ( v) is an arbitrary function, not necessarily in L~ (lR), one can
always define a filter ,C by the input-output relation (7), provided one chooses for
domain D('c) the set of signals x(t) such that the right-hand side has a meaning.

58

B 1. Filtering

-B

+B

Low-pass (B)
~2B

----7

~2B

-VQ

----7

+VQ

Band-pass (vQ, B)

Figure B 1.2. Low-pass and band-pass frequency responses

Low-Pass, Band-Pass, and Hilbert Filters
The low-pass and band-pass filters (see Fig. B1.2) that we now define belong to
the category of Example B 1.2.
One calls low-pass (B) a filter with frequency response
T(v) = 1[-B,+B](v),

(8)

where B is the cut-offfrequency. One calls band-pass (B, va), where 0< B < va,
a filter with frequency response
T(v)

= 1[-vo-B,-vo+Bj(v) + l[vo-B,vo+Bj(v),

(9)

where Va is the center frequency, and 2B is the bandwidth.

Hilbert's filter (see Fig. B 1.3) belongs to the category of Example B 1.3. It is the
filter with frequency response
where T(O) = O.

T(v) = i sgn (v),

(10)

One possible domain for Hilbert's filter is the set of stable (resp., finite-energy)
signals whose FT has compact support. The amplitude gain of Hilbert's filter is 1
(except for v = 0, where the gain is zero), and its phase is
(v)

Jr /2

if v > 0,

if v

-Jr /2

if v < O.

+i

= 0,

r - ,- - - - - -

,0
- - - - - - - ' , -i

Hilbert filter

Figure B1.3. Hilbert frequency response

(11)

B 11 Impulse Response and Frequency Response

59

There is no function admitting the frequency response (10). There is, in fact,
a generalized function (in the sense of the theory of distributions) with Fr equal
to T(v). However, in signal theory, the Hilbert filter is used only in the theory of
band-pass signals (see Section BI2). For such signals the Hilbert filter coincides
with a bona fide convolutional filter:
EXERCISE Bl.3. Show that the output y(t) ofthe Hilbertfilter, corresponding to
a stable signal x(t) having an FT x(v) that is null outside the frequency band
[- B, + B], can be expressed as

y(t)

=-

R.

x(t - s)

2 sin 2 (n Bs)
ns

ds.

Differentiation and Integration as Filters

Let D('c) be the set of signals

where

Ix(v)1 dv

x(t)

<

x(v)e2iJrvt dv,

and

00

Ivllx(v)1

(12)

<

00.

Such signals are continuous and differentiable with derivative

~ x(t) =

[(2inv)x(v)e2iJrvt dv.
(13)
JR.
dt
(Apply the theorem of differentiation under the integral sign; 15 ofthe appendix).

The mapping x(t) ~ dx(t)/dt is a linear filter, called the differentiating filter, or
differentiator, with frequency response
T(v)

= 2inv.

(14)

Let D('c) be the set of signals of the form (12), where

~ Ix(v)1 dv

JIR.

<

00

and

[lx(v)1 dv

JIR lvi

<

00.

The signal
y(t)

= [

x.(v) e2iJrvt dv
2mv
is in the domain of the preceding filter (the differentiator), and therefore,

JIR

~ y(t) = [ x(v)e2iJrvt dv = x(t).

dt
JIR
The transformation x(t) ~ y(t) is a homogeneous linear filter, which is called the
integrating filter, or integrator, with frequency response

T(v) = - . - .
2mv

(15)

60

BI. Filtering

y(t)

x(t)

y(t)

x(t)

.c2 *.c1

series

.c 2 +.c l

parallel

~----,-----:~ y (t)

Figure BlA. Series, parallel, and feedback configurations

Series, Parallel, and Feedback Configurations

We now describe the basic operations on filters (see Fig. B1.4). Let C] and C2
be two convolutional filters with (stable) impulses responses h](t) and h 2 (t) and
frequency responses T](v) and T2(V), respectively.
The series filter C = C 2 * C] is, by definition, the convolutional filter with
impulse response h(t) = (h] * h 2)(t) and frequency response T(v) = T](v)T2(V).
It operates as folIows: The input x(t) is first filtered by Cl, and the output of C] is
then filtered by C 2 , to produce the final output y(t).
The parallel filter C = C] + C 2 is, by definition, the convolutional filter with
impulse response h(t) = h](t) + h 2(t) and frequency response T(v) = T](v) +
T2(V). It operates as folIows: The input x(t) is filtered by Cl, and "in parallel," it
is filtered by C 2 , and the two outputs are added to produce the final output y(t).
The feedback filter C = CI/(1 - C] * C 2 ) is, by definition, the convolutional
filter with impulse response frequency response
T(v) =

T](v)

1 - T](v)T2 (v)

This filter will be a convolutional filter if and only if this frequency response is the
PT of a stable impulse response. 1fthis is not the case, one may define the feedback
filter by the input-output relation
A

y(v)

T](v)

1 _ T](v)T2 (v)x(v)

with, for instance, a definition along the lines of Example B 1.2.

B 11 Impulse Response and Frequency Response

61

The filter 'cl is the forward loop filter, whereas 'cl is the feedback loop filter.
The forward loop processes the total input, which consists ofthe input x(t) plus
the feedback input, that is, the output y(t) processed by the feedback loop filter.
EXERCISE

B1.4. Consider the function

1
T(v) = 1 + 4n2v2'

Give the impulse response of the convolutional filter with the above jrequency
response T (v). Interpret the filter as a feedback filter.
Filtering of Decomposable Signals

We introduce the notion of a decomposable signal, because it allows one to rewrite

the results conceming Fourier transforms and Fourier series in a unified manner, without recourse to symbolic expressions in terms of the Dirac generalized
functions or, more generally, to the theory of distributions.
DEFINITION

Bl.4. The signal set) is called decomposable ifit can be put into the

form
set)
where

e2i :rr:vt p,(dv),

(16)

p, is a complex measure on IR. whose total variation 1p,1 isfinite.

We recall that a complex measure of finite total variation is, by definition, a

mapping /L : B(IR.) ~ <C (B(IR.) is the Borel sigma-field on IR.; see the appendix)
ofthe form

where /LI and /L2 are signed measures of finite total variation. A signed measure
of finite total variation is a mapping /L : B(IR.) ~ IR. of the form
/L(C)

= /L1(C n A) -

/L2(C n ),

for some A E B(IR.) and all C E B(IR.), where /LI and /L2 are measures on (IR., B(IR.))
of finite total mass.
EXAMPLE Bl.4. Let set) be a periodic signal with period T that is stable over its
period and whose Fourier coefficients satisfy the condition

L ISnl <

00.

nEZ

Denote the Dirac measure at the point a by 8a (dv) and set

p,(dv)

= LSn8q,(dv).
nEZ

62

BI. Filtering

This measure is signed, has total variation

LnEZ ISn I <

00,

and since

LSne2inyt = ( e2invt fl(dv),

nEZ
lIR
we seefrom the inversionformula that (16) holds; therefore, set) is decomposable.

The measure fl appearing in (16) will be called the spectral decomposition of

the signal set).

BI.I. Let x(t) be a decomposable signal with spectral decomposition

THEOREM

/Lx:

x(t) =

e2invt flxCdv),

(17)

and let he,) be the impulse response, assumed stable, of a convolutionalfilter:F.

The integral on the right-hand side of
y(t)

h(t - s)x(s)ds

is weil defined, and the spectral decomposition ofy(t) is

fly(dv)
Proof

LL

Ih(t - s)lle2invsllflxl (dv)ds

and hence

(L

Ih(t - s)llx(s)1 ds <

On the other hand,

y(t)

=
=

L
L

L
(L

Ih(t)1 dt) Iflxl(lR) <

00,

00.

e2invs flxCdV )) ds

(h(t - s)

e 2inv(t-s)

(18)

T(v)flx(dv).

e- 2inv(t-s)h(t - s)

dS) flxCdv),

andhence
(19)

EXAMPLE

BI.5. In light of(18), one can interpret Eq. (22) ofTheorem A2.I:

A(n)A
T x n,

Yn = h
A

where h(t) is a stable impulse response and x(t) is a locally stable periodic signal
withperiod T: IfLnEZ l.inl < 00, then LnEZ IYnl < 00, since h(v) is bounded.

B 11 Impulse Response and Frequency Response

63

The two signals x(t) and y(t) are therefore decomposable, and (19) can be written
lLy(dv)

LYneq,(dv)

LXnh(f) eq,(dv)

nEZ

nEZ

= h(v)ILAdv) =

T(v)iLAdv).

Rational Filters as Differential Equations

The RC and LRC circuits are well-known filters, and they belong to the class of
analog rational filters, which we proceed to define fonnally.
Let
q

P(z)

= ao + Latzt,

Q(z)

= bo + Lb1z l

1=1

(20)

1=1

be two polynornials in the complex variable z. The coefficients ap and bq are

nonzero, so that the degree of P is p, and the degree of Q is q. Moreover, we
assurne that P(z) does not have purely imaginary roots:

For all v

P(z) has no roots in iR

(21)

Q(2inv)
T (v) = -P-(2-in-v-) .

(22)

IR, define

We define a linear time-invariant filter CC, D(I: as follows. First, we define the
domain

D(I:) = {x(t);x(t), xCv), and vqx(v)

L~(IR)}.

(23)

We first observe that any function x(t) in the domain is differentiable up to order
q and that its jth derivative is
x(j)(t) =

(2inv)j x(v)e2irrvt dv.

We now define the application itself:

I: : x(t)

y(t)

(24)

T(v)x(v)e2irrvt dv.

(25)

One has to verify that the integral in (25) is weIl defined. Indeed,
bounded because P(z) has no imaginary root. In particular,
for some K <

IT(v)llx(v)1 .:::: KIQ(2inv)llx(v)1

Therefore, IT(v)x(v)1 is integrable for all x(t)

weIl defined for all such x(t).

I/I P (2i n v) I is

00.

D(I:), and the integral in (25) is

In fact, T(v)x(v)v k is integrable for all k, 0 .:::: k .:::: p. To check this, observe
that Ivl k /IP(2inv)1 is bounded for all k .:::: p because P(2inv) is bounded away

64

BI. Filtering

from zero (P(z) has no imaginary root) and Ivl k /IP(2iJTv)1 behaves as Ivl k 00. Therefore, the output y(t) is differentiable up to order p, and for j :::s p,
y<j)(t)

(2iJTv)jT(v)x(v)e2invt dv.

at

(26)

From (24), (26), and (22), it follows that the input x(t) and the output y(t) are
linked by the differential equation

+L
p

aoy(t)

= box(t) + L
q

aly(I)(t)

1=1

b1x(l)(t),

1=1

that is, symbolically,

p(:t ) y(t)

= Q(:t )

(27)

x(t).

This is the time-domain relation corresponding to the frequency-domain relation

P(2iJTv).Y(v) = Q(2iJTv)x(v).

Differential Equations as Rational Filters

We now consider the inverse problem: Let x(t) E D(L:)-in particular, x(t) is
differentiable up to order q -and let y(t) be a solution ofthe differential equation
(27). Is it possible to express this solution as
( Q(2iJTv) x(v)e2invt dv ?

Ai P(2iJTv)

The answer is "no, in general" and "yes, asymptotically" if we impose the following
condition:
P(z) is strict1y stable,

(28)

that is, the real parts of all the roots of P(z) are strict1y negative. Then, y(t), t ~ to,
the solution of (27) with arbitrary initial conditions y(to) = Yo, Y6j)(to) = Y6 j )
(1 :::s j :::s p - 1), satisfies

( l

lim y(t) -

ttoo

IR

Q(2iJTv)
.
x(v)e 2lJrvt dv
P(2m v)
A

= O.

(29)

Proof" The general solution of (27) is the sum of a particular solution of (27) and
of the general solution of the differential equation without a right-hand side,
p(:t ) y(t)

= O.

(30)

Therefore, since
( Q(2iJTv) x(v)e2invt
P(2iJTv)

JIR

is a particular solution of (27), we have to show that limttoo z(t) = 0 for the general solution of (30). This follows from the theory of linear differential equations

B 11 Impulse Response and Frequency Response

65

because the characteristic polynomial P(z) of (30) has all its roots in the open left
half complex plane (see [B5]).

If P(z) is not strictly stable, there are initial conditions such that

y(t) does not tend to 0 as t -+

ImIR

Q(2inv) ~
.
.
x(v)e217fvt dv
P(2mv)

00.

EXAMPLE Bl.6. Consider the LRC circuit (see Fig. BI.5). Its input and output
are related through the differential equation

LCy(T) + RCj(t) + y(t)

= x(t),

where jet) and y(t) are thefirst and second derivatives ofy(t). The roots ofthe
characteristic polynomial
P(z)

= 1 + RCz + LCz 2

are given by the formula

-R JR2 - 4L/C
2L

z = ----'--,----'-and their real parts are always strictly negative. Therefore, the system is strictly
stable, and the permanent regime when the input is x(t) E D(C) is
y(t)
We note that Q(z)

2'

= JlR 1 + RC(2inv) + LC(2inv)2 x(v)e !7fvt dv.

== 1 in this example, and therefore,
D(C)

= {x(t) : x(t), xCv) E L~(lR)}.

Rational Filters as Convolutional Filters

Going back to the general case described by Eqs. (20)-(25), we pose the problem:
Is the filter of convolutional type? The answer is yes if and only if q < p. Indeed,
consider the factorization of P(z):

n
r

P(z) = ap

(z -

Zk)m k ,

k=l

1'~1'

x(t)

c 1-

y(t)

l111111111111I11111

Figure B 1.5. The LRC circuit

66

BI. Filtering

where Zk is a root of order mk. We have the decomposition

and therefore,

1
0

-00

.
tj- 1
- - - eZkte-217rvt

(j - I)!
tj-l

00

_ _ _ ezkte-217rvt

(j - I)!

dt

=-

dt

=+

..

(2inv - Zk)1

..

(2inv - Zk)1

(Remember that the case Re (Zk) = 0 has been exc1uded.) Defining

h(t) =

1
'""'
k tj - e Zkt
~
~('-1)'
k;Re(zkl>O j=l }
.

for t < 0,

(31)

we have
h(t)

Fr

-+

mk

L L (2inv
k=l j=l

k~ Zk)j .

The input-output mapping is therefore

y(t)

q-p

k=O

lR

= LCXkX(k)(t) + J.IR h(t -

In particular,
y(t)

s)x(s)ds.

h(t - s)x(s) ds

(32)

(33)

if and only if q < p. If, moreover, P(z) is strict1y stable (no roots in the c10sed
right half-plane), then, as the expression (31) of the impulse response shows, the
impulse response is causal, and the filter is then called realizable.
EXERCISE

BI.5. Give the impulse response of the LRC filter in the case R 2 <

4LjC.
We observe that the input-output relationship (32) is meaningful for all x(t) E
D(,C'), where D(,C') consists of the stable complex signals that are differentiable

B 11 Impulse Response and Frequency Response

67

up to order max(O, q - p). Therefore, one can consider that the filter (C', D(C',
where L' is described by (32), is an extension ofthe original filter (C, D(C. We
may consider that (32) is an extension of the differential equation (27). For some
functions of the extended domain, the input-output relationship is not a differential
equation.
EXERCISE Bl.6.

Give the extendedfiltercorresponding to the differential equation

~y
=
2

y" -

x" -

~x
3 .

Butterworth Filters

We consider the problem of implementing an approximation of the ideal lowpass (B) filter (with cut-off frequency B) by means of a stable and realizable
filter with real impulse response h(t). Let T(v) be the frequency response of the
approximating filter. The following family of filters, called Butterworth filters, has
been proposed:

1+

(ir'

(34)

(As n -+ 00, the filter looks more and more like an ideallow-pass filter.) One
seeks T (v) of the form
T(v)

K
P(2inv)'

where P(z) has all its roots strictly to the left of the imaginary axis in order to
guarantee stability and causality.
The roots of the polynomiall + (v / B)2n are
O::;l::;2n+1.

We reorder these roots in such a way that VI, vi, ... , vn , v~, are the 2n roots, where
VI, .. , Vn have strictly positive imaginary parts. We shall allocate VI, . , Vn to
T(v), thus proposing
T(v)

(35)

This is the frequency response of a real filter (i.e., T*(v) = T( -v because any
root among VI, ... , Vn is purely imaginary or it can be associated with another root
symmetrie with respect to the imaginary axis (see Fig. B 1.6).
In the case n = 2, we find

68

BI. Filtering

1/1

,,

1/0

,,

,
/

\
........... \
...

,,
/

/
/

,,

.........

.... I \ ....

1/*
1

...

1/*

I/i

n=2

n=3

Figure B1.6. Roots of 1 + (vi B)2n for n

= 2 and n = 3

and in the case n = 3,

T(v)

-Vi V2 V3
(v - vd(v - V2)(V - V3)

B1.7. Show that the Butterworth filter

implemented by an LRC circuit.

EXERCISE

Bl2

0/ order

2 can be

Band-Pass Signals

In this section, we give the basic facts conceming frequency transposition and study
the phenomena associated with it, such as cross-talk in quadrature multiplexing,
and channel dispersion.
Complex Envelope

The first relevant notion is that of a base-band signal.

B1.1. A band-pass (vo, B) signal, where B < VO, is a stable signal
whose FT is null if Iv I f/. [- B + vo, Vo + B]. A base-band (B) signal is a stable
signal s(t) whose FT is null outside the interval [- B, +B].

DEFINITION

It will be assumed, moreover, that set) is real and hence that its Fr is Hermitian
even:
sC-v)

= s(v)*.

We are going to show that a real band-pass signal set) has the representation
set)

= met) cos(2nvot) -

n(t) sin(2nvot),

(36)

wherem(t) andn(t) aretwo signals thatare real, andbase-band (B). The base-band
signals met) and n(t) are the quadrature components ofthe band-pass signal set).

Bl2 Band-Pass Signals

/\/\

/\1'\

8(1/)

/\1'\

~S:(I/)

d\

69

~u(1/ )

Figure BI. 7. Complex envelope in the Fourier domain

One way ofproving (36) is to form the analytic signal of s(t)

sa(t)

= 2 10

00

s(v)e2iJrvt dv,

(37)

and then its complex envelope u(t) (see Fig. B1.7)

u(t)
EXERCISE

sa(v

+ vo)e2iJrvt dv.

(38)

BI.S. Show that the FT ofthe signal Re {u(t)e2iJrvot} is s(v), and thus
s(t)

= Re {u(t)e2iJrvot}.

(39)

Let m(t) and n(t) be the real and imaginary parts of u(t):
u(t)

= m(t) + in(t).

(40)

The quadrature decomposition (36) follows from (39) and (40).

EXERCISE

BI.9. Show that

~()
u(v)
mv=
A

n(v)

+ u(-v)*
2

'

u(v) - u(-v)*
2i

(41a)
(41b)

and that

+ vo) + s(v - VO)}l[-B,+Bl(v),

n(v) = - i{s(v + vo) - s(v - vo)}l[-B,+B](v).

m(v) = {s(v

(42a)
(42b)

Frequency Transposition and Quadrature Multiplexing

Frequency transposition is the operation that transforms a real signal m(t), baseband (B), into the band-pass (vo, B) signal
s(t) = m(t) cos 27Tvot.

70

BI. Filtering

The frequency Vo is called the carrier frequency.

The original signal met) is recovered by synchronous detection: One first multiplies the received signal set) (assuming a channel without noise, distortion, or
attenuation) by the carrier cos 2rrvot:
2s(t) cos(2rrvot) = 2m(t) COS2(2rrvot)

= met) + met) cos(4rrvot),

and the signal m(t)cos4rrvot, which is band-pass (2vo, B), is eliminated by the
low-pass (B), which leaves met) intact.
Since a real signal such as met) has a Hermitian symmetrie Fr, the frequency
transposition technique uses a bandwidth 2B, and therefore, there is a waste of
bandwidth: One should be able to transmit two real signals in the base-band (B) on a
bandwidth of 2B. There are several ways of doing this. One ofthem is quadrature
multiplexing (quadrature amplitude modulation, or QAM). In this technique, in
order to transmit two real base-band (B) signals met) and n(t), one sends the
signal set) = met) cos(2rrvot) - n(t) sin(2rrvot).
B1.10. Let set) be a base-band (B) signaloffinite energy. What is the
support ofthe FTofthe signal s(t)2?

EXERCISE

Showthat, in ordertorecoverm(t)(resp., n(t), onecanmultiply

set) by 2 cos(2rrvot) (resp., 2 sin(2rrvot) and thenpass the resulting signal through
a low-pass (B) (see Fig. Bl.8).
EXERCISEB1.11.

Band-Pass Filtering

When the band-pass signal (36) is passed through a filter with frequency response
T(v), we may, without loss of generality, consider that T(v) = 0 if lvi fj [vo B, Vo + B], since filtering is expressed in the frequency domain by multiplication
of the Frs. Hence it will be assumed that the impulse response h(t) of the filter is
also a band-pass (vo, B) function.
The output signal y(t) has as Fr
y(v) = T(v)s(v),
cos(27rl/ot)

m(t)

2 cos(27rl/ot)

m(t)

-+-oo~

n (t) -+-OO---Y
sin(27rl/ot)

(43)

n(t)
2 sin(27rl/ot)

Figure B 1.8. Quadrature multiplexing

B12 Band-Pass Signals

71

and it is therefore also band-pass (va, B).

B1.12. Show that ifwe denote by v(t) and u(t) the complex envelopes
ofy(t) and s(t), respectively, then

EXERCISE

v(v)

= T(v + va)u(v).

(44)

The equality (36) is a base-band representation of the band-pass filtering equality

(43).

We shall describe two effects that are specific of frequency transposition. The
first one is the phenomenon of cross-talk in quadrature multiplexed channels.

Cross-Talk
Suppose we use quadrature multiplexing; we thus send two band-pass messages
m(t) and n(t) in the form
s(t) = m(t)cos(2nvat) - n(t)sin(2nvat).

Ideal reception (without distortion in the channel) is performed by synchronous

detection whereby m(t) and n(t) are recovered. Their Frs are given by (42a) and
(42b), respectively.
Suppose there is distortion in the channel and that, consequently, the received
signal is s'(t). We then obtain, after synchronous detection, m'(t) and n'(t) with
respective Frs
m'(v)

= {s'(v + va) + s'(v -

n'(v)

=-

i{s'(v

+ va) -

Va)}l[-B,+Bl(v),

s'(v - Va)}l[-B,+Bl(v).

Let us assume that the distortion s(t) -+ s'(t) is a linear filtering with frequency
response T(v). Let us note that T(v) is Hermitian symmetric, as it is the Fr of a
real impulse response h(t). We have s'(v) = s(v)T(v), and therefore,
m'(v) = {T(v

+ va)s(v + va) + T(v -

va)s(v - va)}l[-B,+Bl(v),

= - i{T(v + va)s(v + va) - T(v - va)s(v - Va)}l[-B,+Bl(v).

It appears that m' (v) in general cannot be expressed as a function of m( v) alone.
n'(v)

It depends on both m(v) and n(v), and therefore, in general, there is interference
between the two paths. However, under the condition that T(v) be Hermitian
symmetric about Va in the band of width 2B centered on Va, that is,
T(v

+ va) = T*(va -

v)

forall v

[-B, +B],

(45)

or, again, in view of the Hermitian symmetry of T (v) about 0,

T(v

+ va) = T(v -

va)

forall v

[-B, +B],

(46)

then
m'(v) = G(v)m(v),

n'(v)

= G(v)n(v),

(47)

72

B 1. Filtering

T(v)
~+13_

~:::: !-~=~
v
-va-B -Va -va+B

va-B

Va va+B

Figure B1.9. The frequency response in Exercise B1.13

where
G(v) = T(v

+ vo).

(48)

In this case there is only a linear distortion, represented by independent filtering of

m(t) and n(t). After identification ofthe channel (that is, identification of T(v)),
we can therefore recover the signals in the two paths.
EXERCISEBl.13. Suppose thatin the band [vo- B, vo+ B], T(v) has thefollowing
form (see Fig. Bl.9):

T(v

+ Vo) =

+ v,

T(v - Vo)

=A-

v.

Show that m'(t) is a linear combination ofm(t) and (dfdt)n(t).

We shall now study another phenomenon associated with frequency transposition, that of group delay.
Dispersive Channels

A dispersive channel is a homogeneous linear filter with frequency response

T(v) =

(49)

Kei(v),

where K is a complex constant that will be taken equal to unity. This channel transforms the complex sinusoid e2irrvt into the delayed complex sinusoid e i (2rrvt+ (v)),
where (v) is the phase ofthe filter at the frequency v.
Let s(t) be a real signal, band-pass (vo, B), of the form s(t) = m(t) cos 2:rrvot.
Let y(t) be the signal obtained by passing s(t) through the dispersive channel. The
corresponding base-band equivalent filter has the frequency representation
v(v)

= T(v + vo)m(v),

where v(v) is the Fr of the complex envelope v(t) of y(t) [see (43)].
Suppose that in the band [vo - B, Vo + B], the dispersion has a first-order
expansion
(v

+ vo) ~ (vo) + v -a/

av

,
V=Vo

[-B, +B];

Bl2 Band-Pass Signals

73

then (approximately)

where
(vo)

(50)

T ---P 2:rrvo

and
(51)

Therefore, we have

Now,
y(t)

= Re {v(t)e2invot}.

Hence we have
y(t)

The constants

Tp

and

Tg

= m(t -

Tg ) cos 2:rrvo(t -

Tp ).

are the phase delay and group delay, respectively.

(52)

B2
Sampling

B21

Reconstruction and Aliasing

In a digital communication system, an analog signal {S(t)}tE~ must be transformed

into a sequence of binary symbols, 0 and 1. This binary sequence is generated
by first sampling the analog signal, that is, extracting a sequence of sampies
{s(n)}nEZ, and then quantizing which means converting each sampie into a block
ofO and 1.
The first question that arises is: To what extent does the sampie sequence represent the original signal? This cannot be true without further assumptions since
obviously an infinite number of signals fit a given sequence of sampies.
The second question is: How do we efficiendy reconstruct the signal from its
sampies?
The Shannon-Nyquist Theorem
We begin with a general result that will then be applied to the study of
undersampling and both oversampling.
B2.1. Let s(t) be a stable and continuous complex signal with Fourier
transform s( v) E LJ::OR.), and assume in addition that, jJr some 0 < B < 00,

THEOREM

Lls(~)1

nEZ

2B

<00.

P. Brmaud, Mathematical Principles of Signal Processing

Springer Science+Business Media New York 2002

(53)

76

B2. Sampling

Then
LS(v
ja

_1 L S( ~) e-2iJrvfs, a.e.
2B na 2B

+ j2B) =

(54)

Let h(t) be a complex signal oftheform

T(v)e2iJrvt dv,

h(t) =
where T(v)

E LU~).

(55)

The signal
set) = _1
2B

LS(~) h(t - ~)
2B

2B

+ j2B)!

T(v)e2iJrvt dv.

nEZ

(56)

then admits the representation

set)

= [

J~

!LS(V

(57)

JEZ

Proof" By Theorem A2.3, the 2B-periodic function <P(v) =

is locally integrable, and its nth Fourier coefficient is

LjEZ

s(v

+ j2B)

1 [
2 n
2B J~ s(v)e- "'2li V dv,

that is, since the Fourier inversion formula for set) holds (s(v) is integrable) and
it holds everywhere (s(t) is continuous), the nth Fourier coefficient of <p(v) is in
fact equal to

The formal Fourier series of <P( v) is therefore

2~ ~s(2~)e-2iJrfsv.
nE",

In view of condition (53), the Fourierinversion formula holds a.e. (Theorem A2.2),
that is, <p(v) is almost everywhere equal to its Fourier series. This proves (54).
Since the frequency response T(v) E Lb(~), the impulse response h(t) given
by (55) is bounded and uniformly continuous, and therefore set) is bounded and
continuous (the right-hand side of (56) is a normally convergent series-by (53)of bounded and continuous functions). Also, upon substituting (55) in (56), we
obtain
set)

= _1 L
2B

=[

J~

nEZ

S ( ~) [ T(v)e2iJrv(t-fsl dv
2B J~

~)

! L _1 s(
e-2iJrvfs! T(v)e2iJrvt dv.
nEZ 2B
2B

B21 Reconstruction and Aliasing

77

(The interehange of integration and summation is justified by Fubini's theorem

beeause

(L Is( ~ )1) (rk

)IIT(V)I dv =
krna Is( ~
2B

na

Therefore,
set)

dV)

< 00.)

g(v)e2inVI dv,

where
g(v)

IT(v)1

2B

2~ {~S(2~) e-2inviB

T(v).

The result (57) then follows from (54).

We now state the Shannon-Nyquist sampling theorem.

THEOREM B2.2. Let s(t) be a stable and continuous signal whose FT s( v) vanishes
outside [- B, + B], and assume condition (53) is satisfied. We can then recover
s(t)from its sampies s(nj2B), n E Z, by theformula

set)

= LS(~) sine(2Bt 2B

nEZ

n), a.e.

(58)

Proof' This is a direet eonsequenee of the previous theorem, with T ( v) the

frequeney response ofthe low-pass (B). Indeed,
{ LS(V

+ j2B)! T(v) =

s(v)I[-B,+Bl(v) = s(v),

JEZ

and therefore, by (57),

set) =

s(v)e2invI dv

= set).

The seeond equality is an almost everywhere equality; it holds everywhere when

set) is a eontinuous signal (see Corollary Al.2).

If we interpret s(nj2B)h(t - nj2B) as the response of the low-pass (B) when

a Dirae impulse ofheight s(nj2B) is applied at time nj2B, the right-hand side of
Eq. (58) is the response of the low-pass (B) to the Dirae eomb (see Figs. B2.1 and
B2.2)

s.(t) = _1
1
2B

"s(~)
8(t - ~).
2B
2B

L;,
nEa-

(59)

78

B2. Sampling

!21 s (t)

_~~~f~f~f~f'f'f'f",
Figure B2.l. The Dirae eomb of (59)

1/2B
s (t)

s(t)

--7j)(}--+---l

Figure B2.2. Sampling and reeonstruetion

Sampie and Hold

In praetiee, the Dirae eomb is replaeed by a train of "true" funetions. Instead of

the above train of impulses one of the teehniques of reeonstruetion (ealled sampie
and hold) uses the train of rectangles
1,T(t ) --

lL:(n)
n)
2B gT (t - -2B'

2B

nEZ

where gT(t) is a rectangle of base r and unit area,

gT(t)

= -r

1[0

'

Tl(t),

an approximation of the Dirae impulse as r beeomes large. This signal is then

filtered by a low-pass (B), to produee the signal ST(t). We show that the result is a
smoothed version of the original signal:
ST(t)

I1

= r

s(t - u) du.

(Observe that we eannot use Theorem B2.1 as such; why?). Condition (53) implies
that Si,T(t) is integrable and has an Fr given by

= ;(v)

(2~ ~S (2~) e-

2irrfB ) .

The signal ST(t) is obtained by low-pass (B) filtering of the stable signal Si,T(t).
Sinee the impulse response of a low-pass is not integrable, we eannot use the
eurrent version of the eonvolution-multiplieation rule as it iso However, we shall
proeeed formally beeause the result is justified by a more appropriate version of

B21 Reconstruction and Aliasing

79

the convolution-multiplication rule (Theorem C3.4). We therefore have

s,(v) = g,(v)

(1"

2B L;s (n)
2B e- 2in 2Bn l[-B,+Bl(v) ) = g,(v)s(v).
A

nE",

The result then follows by the inversion formula and the convolution-multiplication
formula (the current version, this time).
Aliasing

What happens in the Shannon-Nyquist sampling theorem if one supposes that the
signal is base-band (B), although it is not the case in reality?
Suppose that a stable signal s(t) is sampled at frequency 2B and that the
resulting impulse train is applied to the low-pass (B) with impulse response
h(t) = 2Bsinc (2Bt), to obtain, after division by 2B, the signal
s(t)

= LS(~) sinc(2Bt 2B

nEZ

n).

What is the Fr of this signal? The answer is given by the following theorem,
which is a direct consequence of Theorem B2.l.
THEOREM B2.3. Let s(t) be a stable and continuous signal such that condition
(53) is satisfied. The signal

s(t)

= LS(~) sinc(2Bt 2B

nEZ

admits the representation

s(t)

n)

(60)

J(v)e2inVf dv,

where
J(V)

= !LS(V + j2B)!1[-B,+Bl(V),

(61)

kEZ
~

If s(t) is integrable, then s(v) is its Fr, by the Fourier inversion theorem. This
Fr is obtained by superposing, in the frequency band [- B, +B], the translates
by multiples of 2B of the initial spectrum s( v). This superposition constitutes the
phenomenon of spectrum folding, and the distortion that it creates is called aliasing
(see Fig. B2.3).
EXERCISE

B2.1. Show that if the signal

s(t)

= ( Sin(2Jl' Bt)2
Jl't

is sampled at rate 1/2B and if the resulting train of impulses is filtered by a

low-pass (B) and divided by 2B, the result is the signal

sin(2Jl' Bt)
Jl't

80

B2. Sampling

-w

~
+B +W
-B

s(v + 4B) s(v + 2B)

s(v)

s(v)

s(v - 2B) s(v - 4B)

2B

3B

4B

<--~,
~(v)
+B +W

-W -B

Figure B2.3. Aliasing

EXERCISE

B2.2. Let Vo and B be such that

0< 2B < vo,

and let s(t) be a stable and continuous base-band (B) signal such that
LkEZ Is(kjvo)1 < 00. Consider the train ofimpulses
Si(t)

= ~ L s (~) 8 (t - ~) .
Vo

nEZ

Vo

Vo

Passing this train through a low-pass (vo + B), one obtains a signal a(t). Passing
this train through a low-pass (B), one obtains a signal b(t).
Show that
a(t) - b(t)

= 2 s(t) cos(2Jl'vot).

(We have therefore effected the frequency transposition of the original signal.)
The following exercise gives aversion of the sampling theorem for band-pass
signals.

B2.3. Let Vo = 2K B for some integer K 2: 1, and let m(t) be a stable

base-band (B) signal. Consider the jrequency-transposed version of this signal,
that is, s(t) = m(t) cos(2Jl'vot). Suppose that

EXERCISE

Show that if the impulse train

_1

2B

L s (~)
8 (t - ~)
2B
2B
nEZ

isfiltered by a low-pass (B), we then recover the original signal m(t).

B21 Reconstruction and Aliasing

81

Oversampling

We have seen the effeets of inadapted sampling, that is, sampling at a too slow
rate (undersampling) that results in aliasing, or speetrum folding. We now show
that oversampling ean be exploited to obtain faster rates of eonvergenee in the
reeonstruetion formula.
Assurne that the situation of the Shannon-Nyquist theorem prevails; in partieular, we have the reeonstruetion formula (58). The quantity sine (2Bt - n) therein
is of the order of 1/ n in absolute value and of altemating sign. Therefore, the speed
of eonvergenee of the series on the right-hand side is, roughly, eomparable to that
of

L (_1)n

nEZ

(..!!:.-).
2B

In order to aeeelerate eonvergenee, one ean use oversampling in the following way.
Assurne that supp(s(v)) is eontained in the frequeney interval [- W,
some 0< W < 00. In formula (57) ofTheorem B2.1, ehoose
B

+ W]

for

(62)

(1 +a)W

for some a > 0 and take any integrable function T(v) such that
T(v)

=1

ifv

[-W, +W].

(63)

The resulting signal is then a perfect replica of s(t) since

?=S(V + j2B)) T(v)

= s(v).

JEZ

Therefore, ifwe sampIe at arate 2B largerthan the Nyquistrate 2W, and then filter
the resulting train of impulses with a filter of impulse response h(t), we obtain,
after division by 2B, the signal
_1 '"'
2B L::,
nEtL.

s(..!!:.-)
h(t _ ..!!:.-)
2B
2B '

which is a replica of the original signal s(t) provided the frequency response of
the filter verifies condition (63).
EXERCISE

B2.4. Suppose that

v+B
T(v) = B _ W 1[-B,-w]

+ 1[-w,+w] +

-v+B
B _ W 1[+B,+w].

Give the corresponding impulse response, and study the rate 0/ convergence 0/ the
series on the right-hand side 0/(60).

The series in the reconstruetion formula can decay faster by choosing a smoother
frequency response T(v), since increasing the smoothness of a function increases
the decay of its Fourier transform.

82

B2. Sampling

B22

Another Approach to Sampling

This section presents another approach, more direct and with a broader scope,
to sampling. It acknowledges the fact that a signal is a combination of complex
sinusoids and therefore starts by obtaining the sampling theorem for this type of
elementary signals.
Sampling a Single Sinusoid
Consider the signal
set)

= e2iTrAt ,

where A E R This signal is neither stable nor of finite energy, and therefore it
does not fit into the framework ofthe L'- and L 2 -versions of Shannon's sampling
theorem. However, the Shannon-Nyquist formula remains essentially true.
THEOREM

B2.4. For all t

E ~

and all A E (-1/2T,

= L e2iTrnT

e2iTrAt

nEZ

. (rr
sm
- (t - nT) )
rr T
.
(t - nT)
T

For all B < 1/2T, the convergence is uniform in t

Proo!,

We first prove that for all A E

e2iTrAt

+ 1/2T),

and all t

E [ - B,

. (rr
-

+B].

(-1/2T, + 1/2T),

L e2iTrnTt sm rr-T (A - nT)

nEZ

(64)

)
,

(65)

(A - nT)

where the series converges uniformly for all t E [-B, +B] for any B < 1/2T.
The result then follows by exchanging the roles of t and A.
Let g(t) be the 1/2T-periodic function equal to e2irrt on (-1/2T, +1/2T].
The series in (65) is the Fourier series of g(t). We must therefore show uniform
pointwise convergence of this Fourier series to the original function.
Without loss of generality, we do this for the Fourier series of the 2rr-periodic
function equal to e iat on (-rr, +rr], where the convergence is uniform on any
interval [-c, +c] C (-rr, +rr). By (39) ofSectionA31, it suffices to show that
lim
ntoo

+Tr

leia(t-s) - eiat 1

-Tr

sin((n + .! )s)
2

sin(s /2)

ds

=0

uniformlyon [-c, +c]. Equivalently,

lim

ntoo

+Tr 1 sin(as /2)1

.
sin((n + !)s) ds = O.
-Tr
sm(s /2)

This is true, for instance, by the extended Riemann-Lebesgue lemma A1.3.

B22 Another Approach to Sampling

83

In partieular, if set) is a trigonometrie signal,

M

=L

set)

Yke2ilrVkf,

(66)

k=!

where Yk

C,

Vk E

lR, and if T satisfies

(67)

2T > SUp{IVkl : 1 ~ k ~ M},

we have the Shannon-Nyquist reconstruction formula

sin

+N

set) = lim Ls(nT)

Ntoo -N

(~(t

7/(t -

- nT)
nT)

(68)

B2.1. With a single sinusoid check that you realty need the strict
inequality in (67).

EXERCISE

Sampling a Decomposable Signal

The following extension ofthe sampling theorem for sinusoids is now straightforward:
THEOREM

o<

B <

B2.5. Let J.L be a nonnegative, finite measure on [- B,

and define the signal

+ B],

where

00,

set) = [

e2ilrVf J.L(dv).

(69)

[-B.+B]

Then for any T < 1/2B and for alt t

set)

=L

lR,
sin

s(nT)

nEZ

Proof"

(!f

(t -

7T:

T (t -

nT)

(70)

nT)

Sinee ,Lis finite and the eonvergenee in (64) is uniform in A E [-B, +B],
set)

=[

{L
[-B,+B]

=L
nEZ

nEZ

{[
[-B,+B]

e2ilrvnT

sm
- (t - nT
. (7T:
7T: T
- (t - nT)
T

e2ilrvnT J.L(dv) }

)}

J.L(dv)

. (7T:
sm
- (t - nT))
7T: T
.
- (t - nT)
T

This c10ses for the moment our study of the Shannon-Nyquist sampling theory.
It will be eompleted in Seetion B32 by the theorem of equivalenee of analog and
digital filtering, and in Seetion C22 by the L 2 -version of the sampling theorem.

H2. Sampling

84

B23

Intersymbol Interference

As a further illustration of the weak Poisson sum formula, we consider the problem
of intersymbol interference in digital communication. It does not belong to the
Shannon-Nyquist sampling theory, however, it does concern sampling.
Pulse Amplitude Modulation and the Nyquist Condition
In a certain type of digital communication system one transmits "discrete" information consisting of a sequence {an}nEZ, of real or complex numbers, in the form
of an analog signal

s(t)

= I>n g(t -

nT),

(71)

nEZ

where g(t) is areal or complex function (the "pulse"). Such a "coding" of the
information sequence is referred to as pulse amplitude modulation. Here, T > 0
determines the rate of transmission of information and also the rate at which the
information is extracted at the receiver.
B2.2. Assume that g(t) is a stable signal with FT g(v) and that an is
stable, with transferfunction A(z). Show that s(t) is stable, andgive its FTin terms
of g(v) and A(z).
EXERCISE

At time kT the receiver extracts the sampie

s(kT) =

L ang(kT -

nT),

nEZ

that is,

akg(O) +

L ak-j g(jT).
jEZ

NO

If one only wants to obtain ak from the sampie s(kT), the term

L ak_jg(jT)
jEZ

NO

is parasitic. This term disappears for every sequence ak if and only if

g(jT)

=0

for all j

i= O.

(72)

It turns out that this is equivalent to

(73)

The weak version of the Poisson sum formula of Section A22 is actually all
that is needed to prove the result. Indeed,
THEOREM B2.6. Let g(t) be a continuous and integrable function, and assume
that its FT g( v) is in LUIR). The following two conditions are equivalent:

B23 Intersymbol Interference

85

= Oforall j E Z, j f. 0;
(b) LnE:d{v + !f) = const. almost everywhere.
(a) g(jT)

Proof By the weak version of the Poisson sum formula of Section A22,
Tg( - nT) is the nth Fourier coefficient ofL g(v +n/T) (Note that the continuity
condition on g(t) is used here.) Therefore, if (b) is true, then (a) is necessarily
true. Conversely, if (a) is true, then the sequence {Tg( - nT)}nEZ is the sequence
of Fourier coefficients of two functions, the constant function equal to T g(O),
and LnEZ g( v + n / T), and therefore the two functions must be equal almost
everywhere.

Condition (73) is the Nyquist condition for the absence of intersymbol

interference. 2
The pulses g(t) used in communications always, for reasons both technological and operational (bandwidth resources), have a restricted frequency band
[- W, + W]. The Nyquist condition (73) can be satisfied only if
2W ~

T'

(74)

Therefore, if transmission without interference between symbols is required, the

minimal bandwidth is

2W=2B~ ~

T
In this case, there is no other choice for the corresponding pulse than
I

g(v) = 2B 1[-B,+Bj(v),

that is,
sin(27r Bt)
(75)
.
27r Bt
One dis advantage of such a pulse is linked to questions of numerical stability.
Indeed, let us assurne that the sampling of s(t) is not carried out at the time kT but
at the time kT + ~, where ~ > O. We obtain
sin(27r B~)
sin(27r B~)"
s(kT +~) = ak 27rB~
+ f;;;oa k- j 27rB(~ _ jT)'
g(t) =

We see that the error

Is(kT

+~)

- akl

(76)

does not stay bounded for all bounded sequences {ad, because
I . I
L
NO I~ - JT

=00.

(77)

2S ee H. Nyquist, (1928), Certain Topics ofTelegraph Transmission Theory, Trans. Amer.

Inst. Elec. Eng., 47, 617-644.

86

B2. Sampling

A better pulse from this point of view is the "raised eosine"

g(t)

= sine (2Bt)

eos(27f Bt)
1 _ 16B 2 t 2

'

(78)

whose Fr is
g(v) = eos 2(7fV)
4B 1[-2B,+2Bj(v).
A

(79)

In fact,
sekT

+ D.) =

sine (4BD.)
ak 1 _ 16B 2D. 2

'"
sin(47fBD.)
f#oa n- j 47f B(D. - jT)(1 - 16B2(D. - jT)2)'

and the error (76) is seen to remain bounded whatever the bounded sequenee {ad.
Partial Response Signaling

Another disadvantage of the pulse (75) is that one eannot realize signals with an
Fr that has an "infinite slope" (at - B and

+ B).

We shall see that, with clever encoding, we ean attain the Nyquist limit (74)
(which says that in order to transmit a "symbol" an every T seeonds without
intersymbol interferenee, a bandwidth of at least 2W = 2B ~ 1fT is needed),
without resorting to an unrealizable pulse (with a very large slope).
For example, in the duobinary encoding teehnique, instead of transmitting (7),
one transmits
S'(t) = L(an

+ an+l)g(t -

nT),

(80)

nEZ

that is,
S'(t) = Lang'(t - nT),

(81)

nEZ

where
g'(t) = g(t)

+ g(t + T).

(82)

With the pulse (75) ofminimal bandwidth 2B, starting from (80) we obtain
s'(kT)

= ak + ak-l =

Cb

and from the sequence {Ck} and the initial datum ao we recover the sequenee {ak}'
The interest of this teehnique is that we do not seek to implement Si (t) in the form
(80) using the unrealizable pulse g(t), but rather in the form (81) with a realizable
pulse g'(t). Indeed,
8'(V)

= (1 + e-2iJrvT)g(v)
= 2T eos(7fvT)e-2iJrvT 1[-B,+Bj(v).

B23 Intersymbol Interference

87

This pulse has minimal bandwidth 2B, and, furthermore, it is easier to realize, not
having an infinite slope.
The above is a particular case of the technique of partial response signaling. 3
The general principle is the following: We pretend to use the unrealizable pulse
g(t) given by (75), but in (71) we replace the symbol an by an encoding Cn , say, a
linear encoding
(83)

which gives
S'(t)

= I>ng(t - nT).
nEZ

In order to realize S'(t) it is rewritten in the form

S'(t)

= .~:::>ng'(t - nT),
nEZ

where
g'(t)

= g(t) + y,g(t + T) + ... + Ykg(t + kT)

(84)

is a base-band (B) pulse, in general easily realizable, with FT

8'(v)

= T(v)g(v),

(85)

where
k

T(v) = 1 + LYje-2inVjT = P(e-2invT)

(86)

j=!

and
k

P(z)

= 1 + LYjz j .

(87)

j=!

By sampling at the time

= kT we obtain
s'(kT) = p(z)ak = Ck.

We shall see in Seetion B32 that the sequence fad is deduced from the sequence
{ Ck} by inverse filtering
ak =

1
P(z) Ck

(88)

(we assume that 1/ P(z) is stable and therefore that the corresponding filter is
causal; these notions are discussed in detail in Section B32).
3See A. Lender (1981), Correlative (Partial Response) Teclmiques and Applications
to Radio Systems, in Feher, K. (ed.), Digital Communications: Microwave Applications
(Prentice-Hall: Englewood Cliffs, Ni), Ch. 7..

88

B2. Sampling

B24 The Dirae Formalism

Do We Need Distributions Theory Here?
In the applied literature, the Dirae formalism of generalized funetions is used
profusely. It eonsists of a small set of symbolie roles that are justified by the
classical Fourier theory of the previous ehapters. In signal proeessing, the Dirae
formalism eulminates in the formula giving the Fr of a Dirae eomb. We shall see
that the Poisson sum formula is, for all praetieal purposes, all that is needed to deal
with such a mathematieal objeet in a rigorous way.
We shall see in the next part that the Fourier theory has in the Hilbert spaee
framework a high degree of formal beauty. There was yet another important step to
be made in this direetion. The physicists had introdueed a very useful tool, the Dirae
generalized funetion, associated with a formal ealeulus that was rather pleasant to
use, but that laeked mathematieal foundations. These were established by Laurent
Schwartz, with the elegant theory of distributions (or generalized funetions) and
the equally elegant Fourier theory of tempered distributions. 4
Most engineers are familiar with the so-ealled Dirae funetion o(t), whieh is
"defined" by the property

qJ(t)o(t) dt

= qJ(O),

for all funetions qJ(t). They are aware that there exists no such funetion in the
usual sense with such property, and they take the above formula as a symbolie
way of dealing with a limit situation. In the "prelimit," o(t) is replaeed by a proper
funetion, depending on a parameter, say, n. There are many ehoiees for this proper
funetion on(t), the simplest one being
on(t)

= nree~(t).

Then for sufficiently regular funetion qJ(t) (say, eontinuous),

lim ( qJ(t)on(t) dt

ntoo

JIR

= qJ(O).

Thus, in this point of view, the Dirae funetion is the limit of proper funetions
beeoming more and more eoneentrated around the origin of times while their
integral remains equal to 1. Another eandidate with these properties is the Gaussian
pulse that we have already eneountered in the proof of the inverse Fourier formula:
ha(t)

,2

= - - e--,;;z,
a"fEi

where this time the positive parameter a tends to zero. Observe that the Fr ofboth
on(t) and ha(t) (whieh we have previously eomputed) eonverge pointwise, as the
4Theorie des Distributions, Vols. 1 and 2,1950-1, Hermann, Paris.

B24 The Dirac Formalism

89

eorresponding parameters tend to the appropriate limits, to 1. This is eonsistent

with the formal eomputation of the Fr of the Dirae funetion
8(v)

8(t)e-2irrvI dt

= e-2irrvO = 1.

Another generalized funetion that is omnipresent in the signal proeessing literature

is the Dirae eomb (indeed a eosmetie too1!), also ealled the Dirae pulse train. It is
the T -periodie generalized funetion
Il T (t) =

nT).

8(t -

neZ

If we formally eompute its nth Fourier eoeffieient, we obtain

-1

n dt
8(t)e- 2''" TI

= 1.

Now if we write the eorresponding formal Fourier series

-1

Le

2irr!!.v
T

neZ

'

we observe that its eonvergenee is rather problematic. We ean, however, pursue

the heuristies, and eonsider that the latter sum is the limit as N ~ 00 of the
1/ T -periodie funetion

1 +N

Le

2irr!!.v

-N

whieh is the Diriehlet kernel

'

1 sin(2rr(N + T)
T
sin(rrT)
GraphieaIly, up to a multiplieative faetor 1/ T, such a funetion looks in the vicinity
of 0 like a Dirae funetion: As N ~ 00, it becomes more and more coneentrated
around 0, and its integral in a neighborhood of 0 tends to 1. Therefore, at the limit
we have, invoking the 1/ T -periodicity, the Fourier transform of the Dirac comb

~T(V) = ~ L 8 (v - !!..) .
T

neZ

This overdose of heuristics may weIl be fatal for the more critical mind. However,
in most basic courses in signal analysis, it is administered with the best intentions,
with the exeuse that it saves the student from a painful exposition to distributions
theory. This apology of mathematical euthanasy is founded on wrong premiees.
The first question that one should ask is: Do we need the Dirae comb in signal
analysis? Looking back at the previous chapters, we ean immediately answer NO.1t
is not needed to derive the Shannon-Nyquist theorem, because the Poisson formula
is all that is needed there. Is the Poisson formula harder than distributions theory?
Again, the answer is NO, without surprise, because the distributions theory version

90

B2. Sampling

of the Poisson sum formula is only a small ehapter of distributions theory. (I shall
add that the heuristie derivation of the Poisson sum formula-see the eomment
following the statement of Theorem A2.3 of Chapter l-is mueh more eonvineing
than the usual heuristie derivation of the Fr of the Dirae eomb.)

In fact, the reader may skip this ehapter and proeeed to Chapters 3 and 4 without
damage. On the other hand, the Fourier transform of the Dirae eomb is part of a
well-established tradition in signal analysis that is bound to be etemal due to its
aesthetie appeal. I have therefore devoted the next seetion to the expression of the
classical results of Fourier analysis in the Dirae formalism. It is, however, a purely
symbolie analysis.
The Dirac Generalized Function
The principal formal objeet of the Dirae formalism is the Dirae generalized funetion
8(t), and the first formal rule is the symbolie formula

qy(t)8(t - a)dt

EXAMPLE

= qy(a).

(Dl)

B2.1. By the first symbolie rule,

e-2iJrvt8(t - a)dt

= e-2iJrva,

that is,
Fr
8(t ...,.. a) -+
e- 2IJrva.

In partieular, the Fouriertransform ofthe Dirae generalizedjunetion is the eonstant

junetion equal to l.
B2.2. Let x(t) be a T -periodie signal, and let {xn}, n E Z, be the
sequenee of its Fourier eoeffieients. In Seetion A2I we defi-ned the FT x(t)
symbolieally, by

EXAMPLE

Using the symbolieformula

x(t)

e2iJrvt xCv) dv

and the symbolie rule (Dl), we then have

x(t)

=L
nEZ

J[{

e2iJrvtxn8(V -

-f)

= LXne2iJrft,
nEZ

and we recover the inversionformula of Seetion A2I.

dv

B24 The Dirae Fonnalism

91

EXAMPLE B2.3. Let x(t) be as in the previous example. /fit is the input of afilter
with (stable) impulse response h(t) and withfrequency response T(v) = h(v),
symbolic calculations give for the output

y(t) =

h(v)x(v)e2iJrvt dv

LXn JJ[{[ h(v)e2iJrvt8(v - -f) dv,

nEZ

that is,

The sequence

LYn} of Fourier coefficients ofy(t) is thus

A(n)A
T xn,

Yn = h
A

a result that we already know.

The FT of the Dirac Comb

Consider the Dirae eomb

I:lT(t) =

L 8(t - nT).
nEZ

The seeond symbolie fomula, that we now introduee, gives the FT of this
generalized funetion:
(D2)
EXAMPLE

B2.4.

The Poisson sum formula. The formal Plancherel-Parseval

equality

cp(t)I:lT(t) dt =

qJ'(v)Lr;:.(v) dv

gives, upon substituting into it

I:lT(v)

= ~L

nEZ

(v - -f)'

the Poisson sum formula

Multiplication Rule

The third symbolie formula of the Dirae formalism eoneerns the multiplieation of
a Dirae generalized funetion by a funetion in the usual sense:
s(t)8(t - a)

== s(a)8(t - a).

(D3)

92

B2. Sampling

This rule is consistent with the first rule, in that

EXAMPLE

= s(a)cp(a) =

s(t)8(t - a)cp(t)dt

s(a)8(t - a)cp(t)dt.

B2.5. Sampling and Spectrum Folding. The train oJ sampled pulses

Si(t)

= Ls(nT)8(t -

nT)

netz:

may, in view oJ(D3), beJormally written

Si(t) = s(t)Llr(t).
Its symbolic FT is thereJore

Si(V) = s(v) * K;(v)

that is,

Sie v)

"~(
= -1 ~
s vT netz:

Ifwe input TSi(t) into a low-pass [-I/T,

output the signal set) with FT
s(v)

= LS(V netz:

n) .

+ l/T]filter, we thereJore obtain at the

-f)

l[-t,+tl(v).

This is the equation describing spectrum Jolding (see Theorem B2.3).

EXAMPLE

B2.6. The FT oJ a Radar Return Signal. Let us consider the signal

set)

= ( L h(t -

nT)) J(t)

netz:

= (h(t) * Llr(t))J(t) = v(t)J(t).

Its FT is (by the convolution-multiplicationJormula)
s(v)

V(fL)J(V - fL)dfL

Now, on using the rule (D3):

v(v)

= h(v)~r(v)
=~
T

L h(v)8(v netz:

!!.-)
T

B24 The Dirac Formalism

93

On the other hand,

Thuswe have

s(v) = ~ Lh(~) i(v - ~),

nEZ

that is, Eq. (71) 0/ Section B2.3.

The examples above show how the Dirae symbolie ealeulus formally aeeounts
for ea1culations of Fourier transforms. This symbolie ea1culus retrieves formulas
already proven in the framework of the classical Fourier theory in LI, formulas
that have been proved under eertain eonditions of regularity, and of integrability
or summability. The symbolie ea1culus does not say under what eonditions the
final symbolie formulas have a meaning, nor in what sense they must be interpreted (equalities almost everywhere? in LI?). For this reason, the Dirae symbolie
ea1culus must be used with preeaution. From a mnemonic point of view, it ean
be useful, as it allows one to obtain some formulas very quiekly, and "generally"
these formulas are eorreet under eonditions that are "almost always" satisfied in
praetiee.
However, let us emphasize onee more the fact that these formulas have been
obtained rigorously within the framework of Fourier transforms in LI.

B3
Digital Signal Processing

B31

The DFf and the FFT Algorithm

TheDFT
Suppose we need to compute numerically the Fr of a stable signal s(t). In practice
only a finite vector of sampies is available,
s = (so, ... , SN-t>,

where Sn = s(nb.). The Fourier sum of this vector evaluated at pulsations

2k1t / N is the discrete Fourier transform (DFr).
DEFINITION B3.1.
The DFT
(So, ... , SN-d, where

0/ s

(so, . .. ,SN-i) is

Wk

the vector S =

N-i

Sk =

sn e - i (21rkn/N).

n=O

The DFr is an approximation of the Fr, the quality of which depends on the
parameters N and .1.. The first question to ask is: How to choose these parameters
to attain a given precision? As we shall see, the answer is given by the Poisson
sum formula. For the time being, we shall give the basic properties of the DFr
without reference to a sampled signal.
Let a = (ao, ... , aN-i) be a finite sequence of complex numbers. For the Nth
root of unity, we adopt the following notation:

P. Brmaud, Mathematical Principles of Signal Processing

Springer Science+Business Media New York 2002

96

B3. Digital Signal Processing

The finite sequence A

= (A o, ... , AN-I) defined by

N-I

(89)

Am = Lanw';rn
n=O
is the DFT of a = (ao, ... , aN-d.
THEOREM

B3.1. We have the inversion formula

an

1 N-I
N LAmw'Nmn .
m=O

(90)

Proof'

N-I

N-I

~
m(k-n)
= L...t ak L...t w N
.
k=O m=O
~

But if k =1= n,
N-I

~
m(k-n)
L...t w N
=
m=O

since

WN(k-n) _ 1
N

w Nk-n

=0

wZr = 1 when r =1= 0; on the other hand, for k = n,

N-I

N-I

L w~(k-n) = L I = N.
m=O
m=O
If we consider the periodic extensions of the finite sequences a = (ao, ... , aN - d
and A = (Ao, ... , AN-d, defined by
an+kN

= an,

(91)

Eqs. (89) and (90) remain valid since w<;:+kN)n

= w';rn.

The sequences an and Am being N -periodie, the domains of the sums (89) and
(90) can be shifted arbitrarily. In particular, with N = 2M + 1,
+M

Am = L an w';rn ,
n=-M

an =

L A mWN-mn .
2M + 1 m=-M
+M

(92)

In the sequel, we use the above periodie extensions. The relation between the
sequences {an} and {Am} will be symbolized by
(93)

We observe that
(94)

B31 The DFf and the FFf Algorithm

THEOREM

B3.2.

multiplication rule

If an

97

Am and b n ~ B m, we have the convolutionN

(95)

ProoJ-

The proof of (95) consists of a simple verification. In fact,

The change of variable n - k

=r

gives

N-I
N-I-k
"~ b n-kwN
mn = w mk"
mr
N ~ b rWn
n=O
r=-k

But because of the N -periodicity of the sequences {b r } and

can be taken from 0 to N - 1. Therefore,

~ (~akbn_k)w~n = ~ak (w~k %:brw:r)

{w:zn, the last sum

= AmBm.

Equation (95) and the inversion formula (90) give

(96)
Making n = 0 in (96) and taking (94) into account, we obtain the PlancherelParseval equation for the OFf
N-I
1 N-I
LakbZ = N L AmB~.
k=O
m=O

With an

(97)

== bn we obtain the energy conservation formula

N-I

1 N-I

L
lall = N L IA
k=O
m=O

m 2

(98)

The Fast Fourier Transform Aigorithm

The calculation of the OFf of the sequence {an} by formula

requires N - 1 multiplications for m ~ 1 (none for m = 0). If we consider that

the cost of an addition is negligible compared with that of a multiplication, the
calculation of A = (Ao, ... , AN-I) thus requires (N - 1)2 computational units,

98

B3. Digital Signal Proeessing

where one unit corresponds to one multiplication. The fast Fourier algorithm, 5 , also
called the fast Fourier transform (FFT), considerably reduces the computational
complexity. It is based on the following remark.
Let an ~ Am be a DFT pair (note that we are considering a DFT of order 2N
2N

with 2N terms an and 2N terms Am). Define

(0 ::S n ::s N - 1),

and

(the latter DFTs are of order N). A direct calculation shows that

(0 ::s m ::s 2N - 1).

. that Bm+N
Ob servmg
Eq. (99) in two parts:

B m, Cm+N

Cm, and

m+N
W 2N

(99)

-wm
1
2N , we can sp lt

(O::s m ::s N - 1)

(100)

and

(0 ::s m ::s N - 1).

(101)

In order to calculate B m and C m for 0 < m ::s N - 1, we need 2(N - 1)2

computational units. When (100) is used we need N -1 additional multiplications.
The multiplications in (101) are for free since they were done in (100). In total,
the method requires
2(N - 1)2

+N

- 1 = (N - 1)(2N - 3)

units instead of (2N - 1)2 for the direct method. If we have to calculate a DFT of
order N such that

N = 2s ,

(102)

the FFT will take F(N) ::S N 10g2 N computational units. The result is obtained
by induction. Indeed, F(2) = 1, and the considerations above show that
F(2N)

= 2F(N) + N

- 1 ::S 2F(N) + N.

But if F(N) ::S !N 10g2 N, then 2F(N) + N ::S N(log2 N

+ 1) = !2N log2 2N.

The gain in computational complexity with respect to the direct method is thus
of the order of
1 10g2 N
--2 N
5Cooley, J.w., Lewis, P.A.w., and Welch, P.D., The Fast Fourier Transform Algorithm, eonsiderations in the ealeulation of sine, eosine, and Laplaee transforms, 1. Sound
Vibrations, 1970, 12(3),315-337.

B31 The DFT and the FFT Algorithm

99

The above discussion just gives the basic idea of the FFT. For a detailed account
of the algorithmic aspects of the discrete-time Fourier transform, see, for instance,
[B8]. We now turn to the numerical issues behind the DFf.
Numerical Analysis of the DFT
The Poisson sum formula is useful in numerical analysis when approximating a
Fourier integral by a Darboux sum, and this is of course related to the finite Fourier
transform.
Let us recall the Poisson sum formula, assuming that the conditions of validity
are satisfied:
(103)
The expression (103) elucidates the relation between the Fr s(v) of the signal
s(t) and the DFr ofits sampled and truncated version (s( - M ll), ... , s( +M ll,
+M

s(nll)e-2irrn2.J+,.

n=-M

In fact, letting v

= kj[(2M + l)ll] in (103),

Ls(nll)e-i2rr2~~' = ~ LS(~ +
II

nEZ

THEOREM

nEZ

II

(2M

+ l)ll

(104)

).

B3.3. Let s(t) be a signal with support contained in [-M ll, +M ll].

We then have

n~M
Proof:

s(nll)e-2irr 2!:t:,

1 ' " ~(n

II

fas

II

(2M

Just apply formula (104).

k)

+ l)ll

(105)

If the terms corresponding to the indices n =1= 0 in the right-hand side of (105)
were null, only the central term

~(

II s (2M

+ l)ll

would remain. The DFf of (s( -M ll), ... , s( +M ll would then be a sampled
version of the PT, that is,

( II1~s(-Mvd,, II1~)
s(+MvI) ,
where VI = Ij[(2M + l)ll]. But one cannot have a signal s(t) with bounded
support which has FT s(v) also with bounded support. There will thus always be
an error, equal to

100

B3. Digital Signal Processing

This error is the aliasing error. It can be controlled by choosing t:.. small enough for
= [-1/2t:.., 1/2t:..]. But then
M must be adjusted so that s(t) remains zero outside [-M t:.., +M t:..]. Increasing
M increases the computational complexity.

s(v) to be negligible outside the interval [-B, +B]

We shall retain the approximate relation linking the effective bandwidth 2B =

1/ t:.., the effective temporal extension T = 2M t:.., and the complexity N = 2M + 1:
2BT :::::: N.

(106)

Band T are chosen such that s(t) is negligible outside [-T /2, +T /2] and s(v) is
negligible outside [- B, + B]. Precision requires large T and large B, in order to
capture a large amount of the time-frequency content of the signal. This results in
large complexity (measured by N) of the DFT. This in turn requires sophisticated
algorithms such as the FFT in order to reduce the computationalload.

B32

The Z-Transform

Discrete-Time Fourier Transform

A discrete-time signal is, in signal processing, a sampled signal. This section gives
the basic tools of digital signal processing: the Fourier transform (reducing to a
Fourier sum) and the z-transform.
B3.1. A stable discrete-time signal is a sequence {Xn}nEZ of complex
numbers such that

DEFINITION

(107)

Its Fourier sum is the function

i(w)

= LXke-ikW.

(108)

kEZ

We observe that it is a 2JT-periodic function. Also, with the same arguments as

for the FT of a stable signal (see Section All), we observe that it is continuous
and bounded by LnEZ Ixnl
An inversion formula is available:
THEOREM

B3.1. Xn is the nth Fourier coefficient of i(w):

Xn

Proof"

= -1 1+11" i(w)e .

lnW

2JT

dw.

(109)

-11"

MultipIy (108) by einwand integrate from -JT to +JT.

B3.1. Let Xn = s(n/2B), where s(t) is a continuous base-band (B)

signal with FT s(v). Show that the Fourier sum associated with {x n } is

EXERCISE

i(w)

= 2Bs~(W 2B)
2JT .

(110)

B32 The Z-Transform

EXERCISE

101

B3.2. Give the impulse response of the filter with frequency response
exp(cos(w))ei sin(w).

DEFINmoN B3.2. The operation that associates to astahle discrete-time signal

Xn the discrete-time signal

(111)

where h n is a stahle signal, is called convolutional filtering. The signal Yn is the

output of the convolutional filter :F with impulse response h n, and Xn is the input.
When the input signal is the unit impulse at 0,
On = {

ifn = 0,

otherwise,

(112)

the output is Yn = h n, whence the terminology.

When X n and h n are stable, the right-hand side of (111) has a meaning. In fact,

and, in particu1ar,
L

IXkllhn-kl <

00

for al1 n

Z.

kE71

This also shows that Yn is stable.

DEFINmON

B3.3. A causal, or physically realizable, filter is one such that

for all n < O.

(113)

The filter is called causal because if the input Xn is zero for n .::: no the output
Yn is zero for n .::: no. The input-output relation (111) takes, for a causal filter, the
form
n

Xkhn-k.

(114)

= Lhne-inW

(115)

Yn = L
k=-oo

DEFINITION

B3.4. The Fourier sum

h(w)

nE71

is the frequency response of the convolutional filter with stahle impulse response
hn.

102

B3. Digital Signal Processing

Ifwe write i(w) and y(w), respectively, for the Fourier sums ofthe input X n and
the output Yn, the input-output relation (6) reads
y(w) = h(w)i(w).

(116)

Indeed,
y(w)

LYne-inW
nEZ

"
" h n-ke -inw
~~Xk
nEZ kEZ

_" 1

h - -i(n-k)w
Xk e -ikw "
~ n-k e

kEZ

nEZ

EXAMPLE B3.1

(The pure delay).

The input-output relation X n -+

Yn

defined by

= Xn-k

Yn

is a homogeneous filtering with impulse response

hn

1o

1 ifn
ifn

= k,
=1=

k,

and jrequency response

h(w)

= e- ikw .

EXERCISE B3.3 (The smoothing filter).

output relation

1
Yn

= 2N + 1

This is the filter defined by the input+N

k=-N

Xn-k

Show that its jrequency response is

hN(w)
where h(O)

1 sin{(N + !)w}
= - - - ------'=----2N + 1

sin{w/2}

= 1. What is limNtoo hN(w)?

Equivalence of Analog and Digital Filtering

It is important to understand how the operation of analog filtering followed by
sampling can be performed if one chooses first to sample and then to operate in
the sampled (digital) domain. The precise statement and the precise answer are
contained in the theorem below.

B32 The Z-Transform

103

Let s(t) be a stable continuous signal, base-band (B), sampled at the Nyquist
frequency 2B. We obtain the sampled signal
S

. n s(v)dv
( - n) = j+B e 2lJrV2Jj
2B
-B

= _1 j+Jt eiJtW2BS(!!.. w) dw
2Jr

-Jt

Jr

(the inversion formula can be applied because s( v) is integrable, having a bounded

support; on the other hand, the equality of s(t) and flR s( v )e2iJtvt dt holds for all t,
since both quantities are continuous). It is further assumed that

~ Is (2~) I <

00,

(117)

and therefore, the Fourier sum of s(nj2B) is 2BsB jJr )W).

THEOREM B3.2. Let x(t) and h(t) be stable continuous signals, base-band (B),
both satisfying the condition 0/ type (117). Then

(118)

This is the theorem of the equivalence of analog and digital filtering.

Prao!"

The discrete-time signal

2~ t; h (2~ )x(n 2~ k )

Yn =

is stable, and its Fourier sum is

Hence we have
Yn

j+Jt 2Bh~ (B
-Jt
Jr

= -1

2Jr

W)

. dw
x (B
- w ) e lnW
Jr

= j+B h(v)x(v)e2iJtv it; dv.

-B

On the other hand, for the analog signal

y(t)

we have the inversion formula

y(t)

and therefore Yn

y(nj2B).

h(t - s)x(s)ds,

+B

-B

(119)

e2iJtvtx(v)h(v)dv,

104

B3. Digital Signal Processing

Transfer Functions

To every discrete-time signal Xn is associated itsformal z-transform, which is the

formal series
(120)
The formal z-transform of the impulse response h n of a convolutional filter is the
formal transfer function of the filter considered:
(121)
The input-output relation (111) reads as a function of the z-transforms of Xn, Yn,
andh n
Y(z)

H(z)X(z).

(122)

Note, however, that the z-transform of a signal only takes a meaning as a function
of z E C if one gives the domain of convergence of the series defining it.
We use the unit delay operation z defined symbolically by
iXn = Xn-k

With this notation the relation (6) is written

Yn

Lhk(ixn)
nEZ

that is, symbolically,

Yn = H(z)x n .

(123)

In some cases (see the examples below) a function H(z) holomorphic in a ring
{rl < Izl < r2} containing the unit circle {Izl = I} is given. This function defines
a convolutional filter whose impulse response h n is given by the Laurent expansion
(see [B6], Theorem 1.22, p. 53)
H(z) = L

hnz n

(rl < Izl < r2).

(124)

nEZ

In particular, the Laurent expansion at z = 1 is absolutely convergent, and thus

the impulse response h n is stable. The frequency response of the filter is
(125)

Recall that the Laurent expansion is explicitly given by the Cauchy formula
hn = - 1
-H(z) dz,
2irr c zn+l

(126)

B32 The Z-Transform

105

where C is a c10sed path without multiple points that lies within the interior of the
ring of convergence, for example the unit circ1e, taken in the anti-c1ockwise sense.
The method of residues can be used to compute the right-hand side of formula
(126). This equality also takes the form
(127)
The integral in (126) can also, have been computed by the method of residues:
If C is a simple c10sed contour on which f is analytic, except for a finite number
of isolated singular points Z I, ... , ZN, then

1. f(z)dz = 2irr

tab
k=1

where ak is the residue of f at Z = Zk (see [B 1], Chapter 4, pp. 207 and following).
This is the Cauchy residue theorem. In the case where f has a pole of order m at
Z = Zk, the residue at this point is given by formula

ak
EXERCISE

= (m

1
dm- I
_ I)! dz m- I [f(z)(z - Zk)m-I]lz=Zk

B3.4. Compute

1.

3z + 1 d

~ z(z - 1)3 Z.

Series, Parallel, and Feedback Configurations

We now describe the basic operations on digital filters (see Fig. B3.1).
Let'ci and'c2 be two convolutional filters with (stable) impulse responses hi and
h~ and transfer functions H I(z) and H2(Z), respectively.
The series filter ,C = 'c2 * 'cl is, by definition, the convolutional filter with
impulse response h n = (h I * h 2)n and transfer function H(z) = H I(Z)H2(Z). It
operates as follows: The input X n is first filtered by ,cl. and the output of'ci is then
filtered by 'c2, to produce the final output Yn.
The parallel filter ,C = 'cl + 'c2 is, by definition, the convolutional filter with
impulse response hi + h~ and frequency response H(z) = HI(z) + H2(Z). It
operates as follows: The input X n is filtered by 'cl, and "in parallel," it is filtered
by 'c2, and the two outputs are added to produce the final output Yn.
The feedback filter ,C = ,CJ/(l - 'cl * 'c2) is, by definition, the convolutional
filter with impulse response frequency response

H(z)

HI(Z)
1 - H I (Z)H2(Z)

This filter will be a convolutional filter if and only if this frequency response is the
FT of a stable impulse response.

106

B3. Digital Signal Processing

Xn

Yn

Xn

Figure B3.1. Series, parallel, and feedback configurations

.cl

.cl

The filter
is the forward loop filter, whereas
is the feedback loop filter.
The forward loop processes the total input, which consists of the input X n plus the
fed-back input, that is, the output Yn processed by the feedback loop filter.
EXERCISE

B3.1. Consider the filter with impulse response

hn =

(~r

l{n:::O).

Give a feedback representation of this filter.

Rational Transfer Functions
Let
q

P(z)

= 1 + L:>jzj,

Q(z)

j=l

= 1 + Lbel

(128)

e=l

be two polynomials with complex coefficients of the complex variable z. We shall

assurne that P(z) has no roots on the unit circle {Izl = I}.
Let rl = max{lzl : P(z) = 0 and Izl < I};
with modulus strictly smaller than 1.
Let r2 = inf{lzl : P(z) = 0 and Izl > I};
with modulus strict1y larger than I.

r2

rl

= 0 if there is no root of P(z)

= +00 ifthere is no root of P(z)

The function
H( ) = Q(z)
z
P(z)

(129)

B32 The Z-Transform

107

is holomorphic in the ring Cr j,r2 = {rl < Izl < r2} (in the open disk {Izl < r2}
if rl = 0) which contains the unit circle since r2 > 1. We thus have a Laurent
expansion in Crj ,r2
(130)

H(z) = I>nzn,
nEZ

which defines a filter with stable impulse response h n and frequency response

Q(e- iW )
P(e-' W )
(see [BI], Section 3.3, or [B6], Theorem 1.22, p. 153).
H(e iW )

EXAMPLE

B3.1. An integer r 2: 1 and y

H(z) =

C are given. We set

1

(z -

yy

First Case: lyl > 1. The ring 0/ convergence is defined by r2 = Iyl and rl = 0
(thus, in/act we have a disk 0/ convergence {Izl < lylJ that contains the unit
circle). The Laurent expansion is in this case a power-series expansion in the
neighborhood 0/ zero

(Izl < Iyl)

n;:::O
To find the impulse response h n we must expand H(z) as apower series. Rut
(_1)r-l(r - I)!(z - y)-r is the (r - I)st derivative 0/

_1_ = _ 2. (1 + ~ + ~ + ... + ~ + ... ),

z- y
y
Y
y2
and there/ore,for Izl < y,
(-Ir-l(r - I)!(z _ y)-r-l

L n(n y n=r-l

=- -

yn

00

1) ... (n - r

= _ 2. ~ (j + ~ yko

}!

zn-r+l

+ 2) - - ,
yn

I)! zj. _1_ .

yJyr-1

Finally,

(z -

yY

(- IY
~ (j + r - I)!
1 ~
'1'
(
1)
yr r- . j=O
}.yJ

and, identifying this expression with

hn
with hn

= 0 if n <

= (-Ir

O.

Z,

L}:o h j zj, we obtain

(n + r -1)! (2.)r+n,
n!(r - I)!
Y

n 2: 0,

Izl

< y,

108

B3. Digital Signal Processing

Second Case: Ir I < 1. The Laurent expansion is then a power-series expansion

in the neighborhood 0100:
1
--- =
h_nz- n
Changing

into

(z - r)'

l/ s,

nO":O

(-s1- r )-r = (-1)r-rrsr ( s - -r1)-r

Isl< - .
Irl

We can use the previous calculations to obtain

and we obtain the anticausal filter

h_ n

(n - I)!
(r - 1)!(n - r)!

-n-r

,n

r,

where h n = 0 ifn > -r.

Linear Recurrence Equations
If Yn is the output of the filter with transfer function (129) corresponding to the
stable input signal X n, we have y(w) = H(e-iw)x(w), that is,
P(e-iw)ji(w)

Q(e-iw)x(w).

Now P(e-iw)y(w) is the Fourier surn of the signal Yn + L:~=l ajYn-j, and
Q(e-iw)x(w) is the Fourier surn of X n + L:i=l bexn-e. Therefore,
Yn

j=l

e=l

+ LajYn-j = X n + Lbexn-e,

(131)

or, syrnbolically,
P(Z)Yn

= Q(z)xn.

The general solution of the recurrence equation (131) is the surn of an arbitrary
solution and of the general solution of the equation without right-hand side
p

Yn

+ LajYn-j
j=l

= O.

This latter equation has for a general solution a weighted surn of terms of the form
r(n)p-n,
where p is aroot of P(z) and r(n) is a polynornial of degree equal to the multiplicity
of this root minus one. If we are given X n , n E Z, and the initial conditions
Yo, Y-l, ... , Y-p+l, the solution of (131) is cornpletely deterrnined.

B33 All-Pass and Spectral Factorization

109

In order that the general solution never blows up (it is said to blow up if
limlnltoo /Yn/ = (0) whatever the stable input X n, n E Z, and for any initial conditions Y-p+l, ... , Y-l, Yo, it is necessary and sufficient that all the roots of P(z)
have modulus strict1y greater than unity.
A particular solution of (131) is

Yn

=L

k::::O

hkxn-k .

The output Yn is stable when the input X n is stable since the impulse response h n
is itself stable, and therefore Yn does not blow up.
Therefore, we see that in order for the general solution of (131) with stable input
to be stable, it is necessary and sufficient that the polynomial P(z) has all its
roots with modulus strict1y greater than 1.
Xn

B3.1. The rational filter Q(z)/ P(z) is said to be stable and causal
P(z) has all its roots outside the closed unit disk {/z/ ::: I}.

DEFINITION

if

Causality arises from the property that if P(z) has roots with modulus strictly
greater than unity Q(z)/ P(z) = H(z) is analytic inside {/z/ < rz} where rz > 1.
The LaUfent expansion of H(z) is then an expansion as an entire series H(z) =
Lk::::O hkz k, and this means that the filter is causal (hk = 0 when k < 0).
DEFINITION

invertible

B3.2.

The stable rational filter Q(z)/ P(z) is said to be causally

I}.

if Q(z) has all its roots outside the closed unit disk {/z/ :::

In fact, writing the analytic expansion of P(z)/ Q(z) in the neighborhood of zero
as Lk::::O WkZ k, we have

that is,
X

B33

n=

L WkYn-k .

(132)

k::::O

All-Pass and Spectral Factorization

All-Pass Filters
A particular case of a rational filter is the all-pass filter.
THEOREM B3.3. Let Zi (l ::: i ::: L) be complex numbers with modulus strictly
greater than 1. Then the transfer function
L

*_ 1

H(z)=n~
i=l

Z - Zi

(133)

110

B3. Digital Signal Processing

satisfies

IH(z)1

Proof"

<I

if Izl < 1,

= 1

iflzl = 1,

> 1

iflzl > 1.

(134)

Let
ZZi* - 1
Hi()
Z=--Z - Zi

be an arbitrary factor of H(z). If Izl

Fejer's identity
(z - )(z

1, we observe that IHi(z)1

-~) = -~zlz *

I 2

'

1, using

(135)

which is true for Izl = 1, E <C, =I- O. On the other hand, Hi(Z) is holomorphic
on Izl < Iz;J and IHi(O)1 = Iz;J-l < 1. Therefore, we must have IHi(z)1 <
1 on {Izl < Iz;J}' otherwise the maximum modulus theorem for holomorphic
functions would be contradicted. (Recall the maximum modulus theorem: If I is
analytic in a bounded region D and III is continuous in the closure of D, then III
takes its maximum on the boundary of D; see [BI], Theorem 2.66, p. 97, or [B6],
Theorem 1.21, p. 51.) Observing that

IHiCI* ) 1= IHiI(z) I
we see that the resultjust obtained implies that IHi(z)1 > 1 if Izl > 1.

A filter with frequency response H(e- iw ) is a pure phase filter, or all-pass filter,
by definition.1t is called all-pass because its gain is unity: IH(e-iW)1 = 1.
Consider a signal X n such that
O-::;.n-::;.N,

otherwise.
It can be represented by its polynomial z-transform
N

A(z) =

L anZn .
n=O

Let Zl,

Z2, ... , ZN

be its roots. In particular,

A(z)

= aN

n(z N

Zj).

j=l

The effect offiltering X n with an all-pass filter (zrz - I)/(z - Zl) is to replace the
factor z - Zl in A(z) by zrz - 1, but it does not change the energy of the signal.

B33 All-Pass and Spectral Factorization

111

Indeed, the z-transforrn of the resulting signal,

B(z)

* = A(z) zlz

Z - Zl

is such that

and therefore,

Thus,
N

n=O

la n l2 =

L Ib l

(136)

n 2

n=O

At a time 0 ::::: k ::::: N the two signals (ao, ... , aN) and (b o, ... , b N) have already
dissipated the energies
k

Ea(k)

= L lajl2

and

Eb(k)

j=O

=L

Ibj

j=O

l2 .

There is an interesting relation between these partial energies. Writing

A(z) = (z - zl)F(z),

B(z)

= (zrz -

I)F(z),

where
F(z)

= 10 + fIz + ... + IN-I ZN-I ,

we have

= In-I - zdn, bn = ZUn-1 - In

= IN = 0 by convention. Taking the
an

where I-I
subtracting yields

(0::::: n ::::: N),

square of the modulus and

and therefore,
(137)
This shows that if Izii < 1, then (ao, ... , an) is always late with respect to
(b o, ... , bN ) in dissipating its energy.

112

B3. Digital Signal Processing

Fejer's Lemma
EXERCISE B3.2. Let X n be a stable signal with z-transJorm X(z). Define its
autocorrelationfunction Cn by

cn =

Xn+k X;

kEZ

Show that {cn}nEZ E .e~(.Z) and that its Fourier sum is

c(w) = li(w)1 2 = R(e- iUJ ),

where
R(z) = X(z)X(z)*.

The 2:rr-periodic function R(e- iUJ ) in the above exercise has the following
properties:

i:

(138)
n

R(e- iUJ ) <

(139)

00.

Moreover, if X(z) is a rational fraction,

R(e- iUJ ) is a rational fraction in e- iUJ

(140)

The next result is Fejer's lemma, which is also called the spectralJactorization
theorem.
'THEOREM B3.1. Let R(z) be a rational Jraction in z with complex coefficients
such that (138) and (139) are satisfied. Then there exist two polynomials in Z with
complex coefficients, P(z) and Q(z), and a constant c 2: 0, such that P(O)
Q(O) = 1 and

R(e- iUJ ) = c 1 Q(e- iUJ )

P(e- 1UJ )

(141)

Moreover, one can choose P(z) to be without roots inside the closed unit disk, and
Q(z) to be without roots inside the open unit disko
Proof"

R(z) can be factored as

R(z)

= az mo T1 (z -

Zk)m k ,

kEK

where a E C, the Zk are nonnull distinct complex numbers, and the mk

Izl = 1, R(z) is real, and therefore,
R(z)

= R(z)* = a*(z*)mo T1 (z* -

zDm k = a*(z-I)m o

kEK

Therefore, when Izl = 1, there exist b

R(z) = bz ro

T1 (Z-I -

kEK

C and ro

Z such that

T1 ( z - --;Zk1 )m
kEK

ZDmk.

Z. If

B33 All-Pass and Spectral Factorization

113

Therefore, if Izi = 1,

Two rational fractions that coincide when Izi = 1 coincide for all Z E <C. In
particular, a = b, and whenever we have in R(z) the factor (z - zd with IZk I i= 1,
then we also have the factor (z - 1.). We therefore have
Zk

wherelZtl = lforallf E L,andlzjl i= lforallj E J.Weshowthatrt = 2se E N

for all f E L. For this, we write Ze = e- i"'" and observe that in the neighborhood of
We, R(e- i "') is equivalent to a constant times (w - weP and therefore can remain
nonnegative if and only if re = 2st . Since R(e- i"') is locally integrable, then
necessarily Se E N. Therefore,
R(z)

= bzro n(z -

ZjYj

(z _

jE]

~)Sj n(z _ Ze)2s,.

Zj

eEL

Using Fejer's identity (135), we therefore find that R(z) can be put under the form
R(z)

= ciIG(z)1 2 ,

where
G(z)

= n(z jE]

Zj)Sj

n(z -

ze)".

tEL

The function R(e- iw ) can remain real and nonnegative if and only if c ~ 0 and
d = O. Finally, we can always suppose that IZj I < 1 for all j E J (a root Zj is
paired with another root l/zj).

EXERCISE

B3.3. Find a constant c and polynomials P(z) and Q(z) as in Theorem

B3.1, such that

5 - 2cos(w)
1 Q(e- i "')
----)=c
.
3 - cos(w)
P(e-'''')

The proof ofTheorem B3.1 can be specialized to obtain that for any polynomial
p(z) such that p(e- i"') ~ 0 for all w E lR, there exists a polynomial A(z) with
A(O) = 1 and no root inside the closed unit disk, and a constant c ~ 0, such that
p(e- i"')

= cIA(e- i"')1 2

Looking at the proof of B3.1, we see if there exist another polynomial B(z) with
B(O) = 1 and a constant c' ~ 0, such that p(e- iw ) = eil B(e- i "')1 2 , then c = c' and
B(z) = H(z)A(z)

for some all-pass filter H(z).

B4
Subband Coding

B41

Band Splitting with Perfect Reconstruction

Smooth Filter Banks

Let x(t) be a stable base-band (B) real signal that we seek to analyze in the
following sense. For fixed N = 2k we wish to obtain for all I ::::: i ::::: 2k the signals
Xi(t) with Fourier transforms

where Bi is the frequency band

Bi

i- I

= [T

B '2 kB .

From a theoretical point of view the problem is stated with its solution: For each
i, do no more than filter x(t) with a pass-band filter offrequency response IB/v)!
From the practical point of view of digital processing, in the sampIe domain, an
ideal band-pass filter has an infinite impulse response-actually one with rather
slow decay-and this makes the above pure band-pass filters of poor value from a
numerical point of view.
A solution consists of replacing the pure band-pass filters by approximations
with "good" impulse responses, and if possible finite impulse responses (FIR).
However, FIR filters with short impulse response have in general a poor frequency
resolution, and therefore the analysis will not be satisfactory without a careful
choice of the approximate band-pass filters. One also requires perfect synthesis,
P. Brmaud, Mathematical Principles of Signal Processing
Springer Science+Business Media New York 2002

116

B4. Subband Coding

that is,
2k

x(t) = LXi(t),
i=l

where Xi (t) is obtained from x(t) by approximate band-pass filtering on the band Bi.
This means that leakage between contiguous bands must be mutually compensated.
The above is a summary of the numerical problem associated with subband
decomposition of a signal by a filter bank. The second problem is algorithmic:
How to perform efficiently analysis and synthesis? The standard example of an
efficient algorithm is the FFf, which involves successive splitting, and subband
decomposition is another avatar of this idea: The basic block of the algorithm consists of splitting a given band in two, that is, of solving the subband decomposition
problem for N = 2.
Subband coding is one way of performing data compression. Instead of sampling the original signal and then quantifying the resulting sampies with a view
of digitizing them, one performs the sampling and quantifying operations on each
of the outputs Xi(t). If a subband Bi is deemed unimportant it will be allocated
fewer compression resources, that is, only coarsely quantified. The appraisal of
the importance of each subband is generally based on psychological experiments.
The subjective difference between subbands is very marked in two-dimensional
signal processing, where it has been observed that low-frequency components are
the most important from a subjective point of view.
The Basic Algorithm

Since all signals and filters considered in the present chapter are real, we need only
consider positive frequencies, those in the frequency band [0, B]. Ideal splitting of
the frequency band [0, B] uses two ideal band-pass filters, one for the band [0, B 12]
and the other for the band [BI2, B]. We call To(v) and Tl(V) their frequency
responses. Then, as the Shannon-Nyquist theorem suggests, we sampie each output
at rate B, and reconstruction is perfo,Pled by t~o ideal band-pass filters, [0, B 12]
and [B 12, B], respectively. We call To(V and Tl (v) their fr~quency responses (of
course, ifweuse ideal pass-band filters, To(v) = To(v), and Tl (v) = Tl (v); wekeep
different notations because in the nonideal case, the analysis and reconstruction
filters need not be the same).
Consider Fig. B4.1. In the ideal case (ideal pass-band filters), the signals in the
upper branch at levels (){ (Xl (t and Y (Yl (t are identical and equal to the original
signal x(t) filtered by the band-pass [0, B 12]. This follows from the theory of
sampling of Chapter B2, and the details of the operations in the lower branch are
shown in Figure B4.2. Similarly, in the lower branch of Fig. B4.1, the signals at
levels ()( (X2(t and Y (Y2(t are identical and equal to the original signal x(t)
filtered by the band-pass [B 12, B].
As we explained ~efore, the ideal band-pass filters will be replaced by approximations To( v) and To( v) that have most of their energy inside the band [0, B 12],

B41 Band Splitting with Perfect Reconstruction

117

Figure B4.1. Block diagram of subband coding

J3

-B/2

B/2

-13

-B/2

B/2

original signal x(t), base-band (B)

after the band-pass (B/2, B] (To(lI

13 //\f'......//\f'......//\f'.....
~~
~~
~
0

+B

B/2

,
~

+2B

after sampling at rate B

after the reconstruction filter (B/2, B] (Ta(lI

Figure B4.2. Subband coding in the frequency domain

and Tl (v) and Tl (v) that have most of theirs inside [B /2, B]. We insist once more
on the fact that we do not require that To(v) = To(v) nor that Tl(v) = Tl(V),
because we need some freedom in the choice of To(v) and Tl(v) to guarantee
perfect reconstruction. Analysis of the original signal yields the decomposition
(Xl (t), xz(t, whereas synthesis reconstructs y(t) = Xl (t) + xz(t). Synthesis is
called perfect when y(t) = x(t).
The signal at level a is
Xl(t)

L>(~)
ho(t - ~),
2B
2B

_1
2B JEa.
. ~

where ho(t), h l (t), ho(t), h l (t) are the r~spective imp~lse responses corresponding to the frequency responses To(v), To(v), Tl(V), Tl(V). Sirnilarly the signal
Yl (t) at level y in Fig. B4.1 is
Yl(t)

= ~Xl(~)ho(t-

;).

118

B4. Subband Coding

Sampling at the rate 2B gives the sampie sequence

YI(n2B)

LX(~)
ho(~B
2B

= Lk

_1
2B j

- ~).
B

~)
iio(!!.2B
2B

If we set 2B = 1 (this condition can be forced upon the system by a change of

time scale), we find

YI(n)

=L

(142)

LX(j)h o(2k - j)iio(n - 2k),

with a similar expression for the output Y2(t) ofthe lower branch ofFig. B4.1.
Down- and Up-sampling
We shall now express the resuIts in terms of the operations of down-sampling and
up-sampling, and then go back to (142).
Let {xnlnez be a sequence of complex numbers and let m
sequences {Ynlnez and {znlnez defined by

Yn

= Xnm '

= Xn,
=0

nE Z,

N. Consider the

nEZ

and

{ Znm
Zj
For example, with m

if j is not divisible by m.

= 2,
Xo

X2

Xl

Yo

X3

X4

Xs

Y2

YI

and

Xo

Xl

X2

X3

X4

Xs

Zo

Zl

Z2

Z3

Z4

Zs

Z6

Z7

Zg

Z9

ZIO

The sequence {YnlneZ is said to be obtained from the original sequence {xnlnez
by down-sampling by a factor m. The corresponding operation is denoted as m,/...
Up-sampling by a factor m, denoted as mt, is the operation that transforms {xnlnez
into {znlnez.
In this chapter, we are concemed with the case m = 2. For future use, we shall
express the operation of down-sampling by 2 followed by up-sampling by 2 in
terms of z-transforms (see Figure B4.3).
Denote X(z) and R(z) thez-transforms ofthe sequences {x(n)lnez and {r(n)}neZ,
respectively. The sequence {r(n)}neZ is therefore obtained from {x(n)}neZ by

B41 Band Splitting with Perfect Reconstruction

119

@1--_Y--'l(~_)_--1CWI-_~)_r_(n_)

x( n))

XW

RW

Figure B4.3. Down-sarnpling and up-sarnpling

1--------------

r--------------I

" - - - - - - - - ______ 1

ANALYSIS

SYNTHESIS

Figure B4.4. Subband coding in the Z-domain (1 split)

replacing all the entries with an odd index by a zero. Therefore,

R(z) = LX(2n)z2n =
nEZ

~ !LX(n)zn + Lx(n)(-zt) ,
nEZ

nEZ

that is,
R(z)

= !{X(z) + X( -

(143)

z)}.

Going back to (142) and the similar expression for the lower branch of Fig. B4.1,
we see that the whole system is equivalent in the z-domain to Fig. B4.4.
From (143) we see that
y(z)

= ! {X(z)Ho(z) + X( +

!{X(z)H1(z)

z)Ho( - z)} Ho(z)

+ X(Z)Hl( -

Z)}Hl(Z).

Separating the aliasing terms from the rest,

y(z) =

! X(z){Ho(z)Ho(z) + H 1(z)H1(z))

Therefore, aliasing is eliminated if

Ho( - z)Ho(z)

+ H 1(- Z)Hl (z) = 0,

(145)

and perfect reconstruction is obtained provided that

~

Ho(z)Ho(z)

- (z) = 2.
+ H 1(Z)Hl

(146)

120

B4. Subband Coding

B4 2

FIR subband filters

Quadrature Mirrors Filters

In or~r t'2., find a solution of (145) and (146), one ean first fix Ho and then find
H I , Ho, HI in terms of Ho in order to satisfy the no-aliasing eondition (145). Then
one ean determine Ho so that the perfeet reeonstruetion eonditions (146) ean be
satisfied.
Given Ho(z), one possible solution of (145) is 6
HI(z) = Ho(- z),
{

~o(z)

H I (z)

= Ho(z),
= - Ho( -

(147)
z).

Assume that the filter Ho is symmetrie, that is, it has a symmetrie impulse response
= ho(n), n E Z). Then

(h o( - n)

HI(z) = Ho(-z) = L(-lrho(n)zn

nEZ

Ir ho( -

= L (-

n)zn

(symmetry of Ho)

nEZ

= L ( - l tho(n) (~)n
Z

nEZ

Therefore, if Ho is symmetrie,

that is, in terms of pulsations,

HI(e- iw )

= Ho(e-i(Jr-w).

This means that the pulsation speetrum of HI is symmetrie with respeet to that of
Ho with respeet to the frequeney n /2. This is why in this ease Ho and H I are said
to be quadrature mirror filters (QMFs).
Going back to (147)-and without assurning that Ho is symmetrie-the perfeet
reeonstruetion eondition (146) beeomes, in terms of Ho:
Ho(d - Ho( - Z)2 = 2.

(148)

One drawback of the solution (145) is the nonexistenee of a finite impulse response
filter Ho satisfying it. However, we ean relax eondition (146) to
Ho(z)Ho(z)

+ HI(z)HI(z)
= 2z k

(149)

6Esteban, D., and Galand, C. (1977), Applications of quadrature mirror filters to splitband voice-coding schemes, Proc. IEEE Inf. Conf ASSP, Hartford, Connecticut, 191-195.

B42 FIR Subband Filters

121

for some K ::: 1, which means that we accept a delay of K time units to recover
the input, and in this case FIR filters do exist.

B4.1. Taking the no-aliasing condition (147) into account, the relaxed
condition (149) with K = 1 gives

EXAMPLE

Ho(z)2 - Ho( -

= 2z.

(150)

Afamous solution is the Haar filter

Ho(z)

= -J2 (1 + z).

(151)

The relaxed condition (149) allows a "linear phase" corresponding to a delay

K. For K ::: 2, we shall just mention that (149) does not have an exact solution
with a FIR filter.

If in Fig. B4.I, the input signal x(t) is assumed to be band-pass [jB, (j +

I)B], for some j ::: 1, the resulting output of the analyzer is the same as if the
input had been frequency-shifted by j B, to obtain a base-band (B) signal. In fact,
immediately after the sampler at rate B, at level y, we have the same signal, for
both inputs (the pass-band signal x(t), or its base-band version).
Therefore, the analyzer of Fig. B4.1 performs band splitting on the base-band
(B) version of any band-pass [j B, (j + I)B] signal. Consequently, the analyzerof
Fig. B4.4 behaves in the same way, with the additional feature ofbeing independent
of B!
This remark shows that the full program of subband coding can be achieved by
a cascade ofthe analysis (resp., synthesis) structures ofFig. B4.4 (by anticipation,
let us mention that this is similar to the structure of Mallat's algorithm in multiresolution analysis). Fig. B4.5 shows the analysis synthesis of the band [0, B] into
four subbands [0, BI4], [BI4, B12], [BI2, 3BI4], [3BI4, B].
---------------------

1---------------------1
1

I ____________________ J

~---------------------~

ANALYSIS

SYNTHESIS

Figure B4.5. Subband coding in the Z-domain (2 splits)

122

B4. Subband Coding

stage 1

stage 2

stage J
stage 1

stage 2

stage J

Figure B4.6. Octave band filtering

In constant Q-filtering one decides not to split the high-frequency component.

Thus at each stage of analysis only the "coarse" component (corresponding to low
frequency) is further analyzed (see Fig. B4.6). Such a structure is also called a
logarithmicjilter or an octave bandjilter.
EXERCISE B4.2. (a): Verify that filtering by H(z) followed by up-sampling by 2
is equivalent to up-sampling by 2 followed by filtering by H(Z2). Show that the
synthesis part in Fig. B4.6 with J = 3 is equivalent to Fig. B4.7. (b): With the
Haar filter we have

Ho(z)

1
= -J2
(1 + z),

B42 FIR Subband Filters

123

Figure B4.7. Equivalent octave band filtering

Give the impulse response of each of the fOUf filters in Fig. B4.7.

Another Solution
Another class of solutions7 for the no-aliasing condition (145) is

= Z-l Ho( - Z-l),

Ho(z) = Ho(C 1),
~l (z)

(152)

H 1(z) = zHo( - z).

The perfect reconstruction condition (146) then becomes

Ho(z)Ho(Z-l)

+ H o(- z)Ho(- Z-l) =

2.

(153)

Since Ho is a real filter,

HO(Z-l)

= H(z)*

for z

= e-ia>,

and (153) takes the form

Ho(e-ia 12

+ IHo(- e-i a1 2 =

2.

(154)

We shall now exhibit a general solution of (152). We perform a change of

notation that will be convenient in the chapter on multiresolution analysis:
Ho(z) = H(z) =

L hnzn,
neZ

ja>
mo (w) = ,J2Ho(e- ),
ml (w)

1
ja>
= ,J2Hl(e).

7Smith, M.J.T., and Barnwell, m T.P. (1986), Exact reconstruction techniques for treestructured subband coders, IEEE Transactions ASSP, 34, 434-441.

124

B4. Subband Coding

In view of (152), we have

ml(w)

= ei"'mo(w + JT)*,

and the perfect reconstruction condition (154) becomes

Imo(w)12

+ Imo(w + JT)1 2 = 1.

(155)

The solution (152) is in terms of Ho(z), and therefore it suffices to obtain mo(w)
satisfying (155). We seek a finite impulse response filter Ho(z), in which case
mo(w) is a polynomial in e- i "'. We shall in fact look for a solution in the form
1 + ei"')N
mo(w) = ( - 2 L(w),

where N :::: 1, and L(w) is a polynomial in e- i",. Letting

Mo(w) = Imo(w)12,

we have
Mo(w)

+
1+

2N

i", 1

IL(w)1 2

= (cos2 (~))

IL(w)1 2.

But IL(w)1 2 is a real-valued polynomial in e- i "', and therefore it is a polynomial

in cos(w). Since cos(w) = 1 - 2 sin 2(wj2),
Mo(w)

= (COS2(~))N P(sin2(~)),

for some polynomial P. Condition (155) must be satisfied for all w, and therefore
it is equivalent to
(156)
for all y E [0, 1]. Since two polynomials identical on [0, 1] are identical
everywhere, the latter equality is for all y E IR.
The polynomials yN and (1 - y)N have no common roots, and therefore, by
Bezout's theorem, there exist two unique polynomials a and b, of degree::: N -1,
such that
(1 - y)N a(y)

This is true for all y

+ yNb(y) = 1.

IR, and in particular, replacing y by 1 - y,

(1 - y)Nb(1 - y)

+ yN a(1 -

y)

= 1.

By the uniqueness of a and b, it follows that

b(y) = a(1 - y).

Therefore, (157) is
(1 - y)N a(y)

+ yN a(1 -

y)

= 1.

(157)

B42 FIR Subband Filters

125

Therefore, P(y) = a(y) is a solution of (156). We have thuse proven that (156)
admits at least one solution, and by the uniqueness in Bezout's theorem, this
solution is the only one of degree :s N - 1. We have

N-l

a(y) = (1 - y)-N[l - yN a(1 - y)] =

(f+k-l) l + G(yN).

k=O

Since a is a polynomial of degree

at N - 1, and therefore,

:s N -

1, it is equal to its Taylor series truncated

= L (f+k-l) l
N-l

a(y)

k=O

This solution is the unique one with degree :s N - 1. Observe that it is nonnegative
for aH y E [0, 1], and therefore a solution to the initial problem. Call it PN and let
P be the general solution. We have

(1 - y)N (P(y) - PN(y

+ yN (P(1

- y) - P N(1 - y

= 0.

This implies that P - PN is divisible by yN, that is, P(y) - PN(y)

and
(1 - y)N yN Q(y) + yN (1 - y)N Q(l - y

yN Q(y),

= 0,

which implies Q(y) + Q(1- y = 0. That is, Q is symmetrie with respect to 1/2,
and therefore ofthe form Q(y) = R(1/2 - y) for an odd polynomial R.
In summary, the general solution of (156) is
P(y)

= N-l
L (f+k-l) l + yN R
k=O

(1 )
--

(158)

where R(y) is any odd polynomial such that P(y) so defined remains nonnegative
for all y E [0, 1].
Having obtained Mo(w), it remains to extract its square root mo(w). But this can
be done by spectral factorization, using Fejer's lemma.
We shall elose this chapter on the basic principles of subband coding. Note, however, that other solutions were proposed, most notably "biorthogonal solutions,"8
which are more versatile and yield finite impulse response subband filters with
better properties (of symmetry, for instance). We refer to the monograph [B12],
where the reader will find a full and detailed treatment of this topic, as weH as
additional references.

8Vetterli, M. Filter banks allowing peifect reconstruction, Signal Processing, 10 (3),

1986,219-244.

126

B4. Subband Coding

References
[BI]
[B2]
[B3]
[B4]
[B5]
[B6]
[B7]
[B8]
[B9]
[BIO]
[B 11]
[BI2]

Ablowitz, M.J. andJokas, A.S. (1997). Complex Variables, Cambridge University

Press.
Daubechies, I. (1992). Ten Lectures on Wavelets, CBSM-NSF Regional Conf.
Series in Applied Mathematics, SIAM: Philadelphia, PA.
Gasquet, C. and Witomski, P. (1991). Analyse de FourieretApplications, Masson:
Paris.
Haykin, S. (1989). An Introduction to Analog and Digital Communications, Wiley:
New York.
Hirsch, M.W. and Smale, S. (1974). Differential Equations, Dynamical Systems,
and Linear Algebra, Academic Press: San Diego.
Kodaira, K. (1984). Introduction to Complex Analysis, Cambridge University
Press.
Lighthill, MJ. (1980). An Introduction to Fourier Analysis and Generalized
Functions, Cambridge University Press.
Nussbaumer, H.J. (1981). Fast Fourier Trans/orm and Convolution Algorithms,
Springer-Verlag: New York.
Orfanidis, S. (1985). Optimal Signal Processing, McMillan: New York.
Papoulis, A. (1984). Signal Analysis, McGraw-Hill: New York.
Rudin, W. (1966) Real and Complex Analysis, McGraw-Hill: New York.
Vetterli, M. and Kovacevic, J. (1995). Wavelets and Sub-Band Coding, PrenticeHall: Englewood Cliffs, NJ.

Part C

Fourier Analysis in L2

Introduction

The modem era of Fourier theory started when the tools of functional analysisin particular, Lebesgue's integral and Hilbert spaces-became available. Fourier
theory then seemed to have reached the promised land, which is called L 2, the
space of square-integrable complex functions, indeed a Hilbert space.

F. Riesz and E. Fischer were the first to study Fourier series in the L 2 framework. 1
Many ideas of the modem theory of Hilbert spaces were already contained in the
work of these two mathematicians, and they had a c1ear view of the geometrie
aspect ofthe L 2-spaces. They were inspired by aseries of articles by David Hilbert
written after 1904 on the theme of integral equations and in which he gives the
properties of 4(Z). Note, however, that the notion of abstract Hilbert spaces made
its appearance much later than one usually believes, in the years 1927-1930, with
the work of John von Neumann, who was motivated by quantum mechanics. 2

In short, a Hilbert space is a vector space H on the field <C (or lR.), with a
Hermitian (or scalar) product, denoted (., .) or (., .) H, and a special topological
property that we shall now briefly introduce. The Hermitian product induces a
norm, the norm IIxll, or IIxIlH, ofthe vector x E H being

IIxll = (x,x}'1.
I

IF. Riesz, Sur les systemes orthonormaux de fonctions, CRAS Paris, 144, 1907,615619; and E. Fischer, Sur la convergence en moyenne, CRAS Paris, 144, 1907, 1022-1024;
Applications d'un theoreme sur la convergence en moyenne, CRAS Paris, 144, 1907, 11481151.
2His theory was published in the reference text Mathematische Grundlagen der Quantum
Mechanik in 1932.

130

Part C Fourier Analysis in L 2

This allows us to define a limit in H: We say that lim n-4oo x n = x iflimn-4oo IIxn xII = O. Having this notion of a limit, we have the notion of a Cauchy sequence:
A sequence {xnlnEN in H is called a Cauchy sequence if
lim IIxm - xnll

m,n~oo

= O.

To be called a Hilbert space, H must-besides being a vector space on C with a

Hermitian product-be complete with respect to the induced norm. This means
that any Cauchy sequence {xnlnEN in H converges, that is, there exists an x E H
such that

= O.

lim IIxn - xII

n-4OO

Note that for any positive integer k, Ck , considered as a vector space on C with the
usual Hermitian product, is indeed a Hilbert space. B ut there are more sophisticated
Hilbert spaces. For instance, L~(lR), the space of functions f : lR --+ C that are
square-integrable:

If(t)1 2 dt <

00.

In L~(lR), one does not distinguish two functions that are almost everywhere equal.
The Hermitian product is
(f, g)

Saying that limntoo fn

=f

= (f, gh

2 (ITt)

= (

lITt

f(t)g(t)* dt.

in L~(lR) means that

lim ( Ifn(t) - f(t)1 2 dt

ntoo lITt

= O.

Another example of a Hilbert space is the space of functions f : lR --+ C that are
2rr-periodic, and in L~([ -rr, +rr]), that is, square-integrable on [-rr, +rr):

+Jr

-Jr

If(t)1 2 dt <

with the Hermitian product

(f, g) = (f, g)

L~([-Jr,+Jr]) =

00,

L:

Jr

f(t)g(t)* dt.

In L~([-rr, +rr]), one also does not distinguish two functions that are almost
everywhere equal.
A third example is .e~(Z), the set of complex sequences a = {xnlnEz such that

L Ix l

n 2

nEZ

<

00,

Introduction

with the Hermitian product

(a, b) = (a, b) e~('z.) =

131

L
anb~.
neZ

The Hilbert space LUlR.) is a paradise ofFourier transforms, since every function
thereof admits a Fourier transform, and moreover the mapping that associates to a
function its Fourier transform is a bijection from L~(lR.) to itself, and the inversion
formula for Fourier transforms, which gives the latter in terms of the former is
f(t) =

j(v)e+ 2i :n:vt dv.

This is not apreeise statement. In particular, the integrals appearing in the definition
of the transform and in the inversion formula are in some extended sense, and the
equality in the inversion formula is "almost everywhere." To be exact,
f(t)

lim jb
a,btoo -a

j(v)e+ 2i :n:vt dv,

where the limit is in the sense of L~(lR.). A similar interpretation is needed for the
integral defining the Fourier transform.
The beautiful formula of the L 2- theory is the Plancherel-Parseval's formula

j(v)g(v)* dv

f(t)g(t)* dt,

in other words,

(f, g) L~(IR)
where f, g

= (j, g) L~(IR)'

E L~(lR.).

The above results are stated for the Fourier integral transform, but similar results
hold for the Fourier series of periodic functions: Let f be a 2n -periodic function
square-integrable on [0, 2n]; then it admits the representation
f(t)

=L

cn(f)e int .

neZ

This is the inversion formula for Fourier series. Similarly to the Fourier transform
in L~ (lR.), this equality is only almost everywhere, and the sum has to be interpreted
in an extended sense:

where the limit is in the sense of L~([ -n, +n]). This result is in fact a particular
case of the Hilbert basis theorem, which gives the orthonormal expansion

= L(x, en)en
neZ

132

Part C Fourier Analysis in L 2

of any vector x in a Hilbert space H, when {en}nE!\! is a complete orthonormal

system. "Orthonormal" means that
(ek, en)

= lk=n,

and "complete" means that the closure of the vector space consisting of the finite
linear combination of elements of {en }nE!\! is H. (See Chapter C2 for precise definitions.) In this case, the above orthonormal expansion is valid (the series in the
right-hand side converging with respect to the distance induced by the Hermitian
product of H), and moreover, we have Plancherel-Parseval's identity

IIxll 2 =

L 11 (x, e }1I
n

2.

nEZ

The Fourier series development is a particular case of the above very general resuIt,
where H == L~([-x, +x]), and
en(t)

==

~elnt.

",2x

The Plancherel-Parseval formula for Fourier series reads

-

2x

j+Jr f(t)g(t)* dt = L cn(f)cn(g)*,

-Jr

nEZ

where f and gare 2x-periodic functions in L~([ -x, +x]. In terms ofHermitian
products,

Cl
Hilbert Spaces

CII

Basic Definitions

Hilbert space theory is the fundamental tool in Fourier analysis of finite-energy

signals. It is a huge chapter of functional analysis, but we shall only give the
definitions and prove the basic facts used in this book, in particular, the projection
theorem and the theorem of extension of isometries.

Pre-Hilbert Spaces
DEFINITIONCl.l. Let E beavectorspaceoverC(resp., lR)andlet(x, y) -+ (x, y)
be a mappingjrom Ex E to C (resp., IR) such that,forall x, y E E and all a E C
(resp., IR),

(a) (x + z, y) = (x, y) + (z, y),

(b) (ax, y) = a(x, y),
(c) (x, y) = (y, x)* (resp., (x, y) = (y, x)),
(d) (x, x) ::: 0, and (x, x)

= 0 if and only ifx = O.

It is then said that Eis a complex (resp., real) pre-Hilbert space with the Hermitian product (resp., scalar product) (., .). The complex (resp., real) number
(x, y) is the Hermitian (resp., scalar) product ojx and y.
In the above definition and in the sequel, 0 represents the zero of IR or C, or the
neutral element of addition in E. The context will remove ambiguity.
From now on, we shall consider complex pre-Hilbert (and later Hilbert) spaces.
The other choice for the scalar field, IR, leads to formally analogous results.
P. Brmaud, Mathematical Principles of Signal Processing
Springer Science+Business Media New York 2002

134

Cl. Hilbert Spaces

For any x

E, denote

IIxll 2 = (x, x).

(1)

IIx+YII 2 = IIxIl 2 +IIYIl2+2Re{(x,y)}

(2)

Elementary computations yield

for any x, y

E. The parallelogram identity

IIxll 2 + IIYll2 = !(lIx + Yll2 + IIx _ Y1l2)

(3)

is obtained by expanding the right-hand side of (3) using (2).

EXERCISE CI.I. Prove the polarization identity
(x, y)

= 4{lIx +

Yll2 - IIx - Yll2 + i IIx + iYll2 - i IIx - iYIl2}.

(4)

Consequently, two Hermitian products (., h and (., h on E such that 11 . 111 =
11 . 112 are identical.
THEOREM CI.I. For all x, y

E, we have the Schwarz inequality

I(x, y)1

:s IIxll

(5)

x lIyll,

with equality holding if and only if there exist a,

C such that ax + y

= O.

Proof" We do the proof for the real case and leave the complex case to the reader.
We may assume that (x, y) =1= 0; otherwise, the result is trivial. For all E lR,

IIxll 2 + 2(x, y)2 + 2(x, y)211Y1l2

= IIx +

(x, Y)YIl2 ~ O.

This second-degree polynomial in E lR therefore cannot have two distinct real

roots, and this implies a nonpositive discriminant:
I(x, y)1 4

IIxIl 2 1(x, y)1 211y1l2

:s 0,

and thus the inequality (5) holds. Equality in (5) corresponds to a null discriminant,
and this in turn implies a double root of the polynomial. For such a root, IIx +
(x, Y)YIl2 = 0, that is, by Property (d) in Definition CU,
x

DEFINITION CI.2. Two elements x, y

Let Xl,

... , X n E

+ (x, y)y = O.
E

E are said to be orthogonal if (x, y)

= O.

E be pairwise orthogonal. We have Pythagoras' theorem:

(6)

which follows from (2).

THEOREM CI.2. The mapping x -+ IIx 11 is a norm on E, that is, for all x, y
all a E C,
(a) IIxll

0, and IIxll

= 0 if and only ifx = 0,

E,

Cll Basic Definitions

(b)
(c)

135

lIaxll = lalllxII,
IIx + ylI ::: IIxll + lIyll

Proof

The proof of (a) and (b) is immediate. For (c) write

IIx + Yll2 = IIxll 2+ IIYll2 + (x, y) + (y, x)

and

It therefore suffices to prove

(x, y)

+ (y, x)

:::

2l1xllllyll,

and this follows from the Schwarz inequality.

The norm

11 . 11

induces a distance d(, .) on E by

d(x, y) =

IIx - ylI.

(7)

Recall that a mapping d : E x E -+ lR+ is called a distance on E if, for all

x, y, Z E E,
(a') d(x, y) ~ 0, and d(x, y) = 0 if and only ifx
(bi) d(x, y) = d(y, x),
(c') d(x, y) ~ d(x, z) + d(z, y).

= y,

The above properties are immediate consequences of (a), (b ), and (c ) of Theorem

C1.2.
Hilbert Spaces

A metric space is a set E endowed with a distance d. One then says: the metric
space (E, d), or, for short and when the context is sufficiently explicit as to the
choice of the distance, the metric space E. A pre-Hilbert space E is therefore a
metric space for the distance d induced by the Hermitian product.
C1.3. A Hilbert space is a pre-Hilbert space that is complete with
respect to the distance d defined above.

DEFINffiON

Recall that a metric space (E, d) is called complete if any Cauchy sequence in E
converges; that is, if {Xn}n:o:l is a sequence in E such that limm.ntoo d(x n , x m ) = 0,
then there exists x E E such that limntoo d(xn , x) = O.
When considered as a Hilbert space relative to the norm 11 . 11, E will be denoted
H.1f necessary, the notation for the Hermitian product and the norm will explicitly
refer to the space H: We then write (., ')H and 11 . IIH.
C1.1. Let (X, X, f.L) be a measure space. It is proven in the appendix
(Theorem 26) that LUf.L) is a Hilbert space relative to the Hermitian product

EXAMPLE

(j, g)L 2 (/-L)

c

= Jx[ f(x)g(x)* f.L(dx).

136

Cl. Hilbert Spaces

Cl.2. A particular case of Example 3.1 is that in which X = Z, X =

P(Z), and J-t is the counting measure on Z. The corresponding Hilbert space L~(J-t)
is then denoted lUZ). Therefore,

EXAMPLE

4(Z) =

{{Xn}nEZ : X n E

Cforall nE Z and

L Ix l

n 2

< oo}

nEZ

is a Hilbert space with Hermitian product

(x,

Y}e~(z) = LXnY;.
nEZ

C1.2. Show that if h(t) and x(t) are both in L~(lR), then Y = h * x is
weil defined. Find h E L~(lR) such that IIh 11 = 1 and maximizing y(T)for a given
time T. What is the corresponding maximum?
EXERCISE

Cl 2

Continuity Properties

Closed Subspaces
A subset G is said to be c10sed in H if every convergent sequence of G has a limit
in G.

Cl.3. Let GeH be a vector subspace of the Hilbert space H. Endow

G with the Hermitian product that is the restriction to Gof the Hermitian product
on H. Then G is a Hilbert space if and only if G is closed in H.
In this case, Gis called a Hilbert subspace of H.
THEOREM

Proof Assume that Gis c1osed. Let {Xn}nEN be a Cauchy sequence in G. It is a

fortiori a Cauchy sequence in H, and therefore it converges in H to some x, and
this x must be in G, because it is a limit of elements of G and G is c1osed.
Assume that G is a Hilbert space with the Hermitian product induced by the Hermitian product of H. In particular, every convergent sequence {x n }nEN of elements
of G converges to some element of G. Therefore, G is c1osed.

EXERCISE Cl.3.

Let (X, X, J-t) be a measure space. For somefixed constant K, let

G = L~(J-t) n {f; sup If(t)1 :::: K, J-t - a.e.}. Is G a Hilbert subspace of L~(J-t)?
Answer the same question, with G = L~(J-t) n {f; sup If(t)1 < 00, J-t - a.e.}.
WhataboutLUJ-t)n{f;suplf(t)l:::: K(f) < 00, J-t-a.e.}?
DEFINITION C1.4. Let {Xt }tET be an arbitrary collection of elements of H. The
smallest Hilbert subspace of H containing all the vectors Xt, t E T, is called the
Hilbert subspace generated by {Xt }tET, or the Hilbert span of {Xt }tET.
EXERCISE Cl.4. Call G be the Hilbert subspace generated by {XrJtET. Let ,c, the
vector space formed by all finite linear combinations of elements of {Xt }tET. Show
that G = l (the closure of ,c).

Cl2 Continuity Properties

137

The following notation is eonvenient:

12 = span {XI,
G

t E

= span {XI, t

Tl.

E T}.

Paraphrasing the above result, we see that X E span {XI> t E T} if and only if X is
the limit in H of a sequenee of finite linear eombinations of elements of {XI, t E T}.
Continuity of the Hermitian product
CI.4. Let H be a Hilbert space over C with the Hermitian product
( " .). The mappingfrom H x H into C defined by (x, y) t-+ (x, y) is bicontinuous.

THEOREM

Proof:

We have

I(x + h l , Y + h 2 ) - (x, y)1 = I(x, h 2 ) + (h l , y) + (h l , h 2 )1

By Sehwarz's inequality, l(x,h 2 )1 :::
l(h l ,h 2 )1::: Ilh 1 1l1lh 2 11. Therefore,
lim

II h l II.IIh2 11-1-0

In partieular, the norm X

IIxllllh 2 11, l(hl,y)1 ::: lIyllllhlll, and

I(x + h l , Y + h 2 ) -

t-+

(X,

y)1 = O.

IIx 11 is a eontinuous funetion from H to lR+

EXERCISE C1.5. Let (X, X, fL) be a measure space, where fL is a finite measure.
Let {fn}n ::: 1 be a sequence of LUfL) converging to f. Apply Theorem Cl.4 to
prove that limntoo fL(fn) = fL(f). Give a counterexample ofthis property when the
hypothesis that fL is finite is dropped. (Hint: f = 1[0, I] , fn = (1-1/ n) 1[0, 1] +- .. .)
Show that when fL is finite,

= L~(fL) n {f; fL(f) = O}

is a Hilbert subspace of L~(fL).

Note that when fL is not finite, G need not be a Hilbert subspaee of L~(fL).
Wavelet multiresolution analysis will provide a speetacular eounterexample.
Isometry Extension Theorem
DEFINITION CI.5. Let Hand K be two Hilbert spaces with Hermitian products
denoted ( " .) Hand ( " .) K, respectively, and let q; : H t-+ K be a linear mapping
such that,for all x, y E H,

(q;(x), q;(Y)K = (x, y)H.

(8)

Then q; is called a linear isometry from H into K. If, moreover, q; isfrom H onto
K, then Hand Kare said to be isomorphie.

Note that a linear isometry is neeessarily injeetive, sinee q;(x) = q;(y) implies
q;(x - y) = 0, and therefore,
0=

1Iq;(x - y)IIK = IIx - yllH,

and this implies x = y. In particular, if the linear isometry is onto, it is neeessarily

bijeetive.

138

Cl. Hilbert Spaces

Recall that a subset A E E, where (E, d) is a metric space, is said to be dense

in E if, for all x E E, there exists a sequence {xn}ne:! in A converging to x. The
following result will often be used. It is ca1led the isometry extension theorem of
Hilbert spaces or, for short, the isometry extension theorem.

CI.5. Let Hand K be two Hilbert spaces with Hermitian products

( ., .) Hand ( ., .) K, respectively. Let V be a vector subspace of H that is dense
in H, and let cp : V f-+ K be a linear isometry from V to K (cp is linear and (8)
holds for all x, y E V). Then there exists a unique linear isometry (l : H f-+ K
such that the restriction of (l to V is cp.
THEOREM

Proof We sha11 first define (l(x) for x EH. Since V is dense in H, there exists
a sequence {xn}ne:! in V converging to x. Since cp is isometric,

In particular, {cp(xn) }ne:! is a Cauchy sequence in K, and it therefore converges to

some element of K, which we denote (l(x).
The definition of (l(x) is independent ofthe sequence {xn}ne:! converging to x.
Indeed, for another such sequence {Yn}ne:!'

The mapping (l : H f-+ K so constructed is c1early an extension of cp (for x E V,

one can take as the approximating sequence of x the sequence {xn}ne:! such that
Xn

== x).

The mapping (l is linear. Indeed, let x, Y E H, a, E C, and let {xn}ne:!

and {Yn}ne:! be two sequences in V converging to x and y, respectively. Then
{axn + Yn}ne:! converges to ax + y. Therefore,
lim cp(axn + Yn)

ntoo

= (l(ax + y).

However,

= acp(xn) + CP(Yn) ---+ a(l(x) + (l(y)

Therefore, (l(ax + y) = a(l(x) + (l(y)
cp(axn + Yn)

The mapping (l is isometric since, in view of the bicontinuity of the Hermitian

product and of the isometricity of cp,

where {xn}ne:! and {Yn}ne:! are two sequences in V converging to x and y,

respectively.

Cl3 Projection Theorem

Cl 3

139

Projection Theorem

Let G be a Hilbert subspace of the Hilbert space H. The orthogonal complement

of G in H, denoted G1., is defined by
G1. = {z EH: (Z, x) = 0 for all x E G}.

(9)

Clearly, G1. is a vector space over Co Moreover, it is c10sed in H since if {Zn}n~l

is a sequence of elements of G 1. converging to zEH, then, by continuity of the
Hermitian product,
0= lim(Zn,x) = (Z,x)

forallx

ntoo

H.

Therefore, G1. is a Hilbert subspace of H.

Observe that a decomposition x = Y + Z where Y E G and Z E G1. is necessarily
unique. Indeed, let x = Y' + z' be another such decomposition. Then, letting
a = Y - y', b = Z - z', we have that 0 = a + b where a E G and b E G1..
Therefore, in particular, 0 = (a, a) + (a, b) = (a, a), which implies that a = O.
Similarly, b = O.

Cl.6 (Projection theorem). Let x

G such that x - Y E G1.. Moreover,

THEOREM

H. There exists a unique element

lIy -xII = inf lIu -xII

(10)

UEG

Proof"
that

Let d(x, G) = infzEG d(x, z) and let

d(x,

{Yn}n~l

be a sequence in G such

Gi :s d(x, Yni
1
:s d(x, G)2 + -.

(*)

The parallelogram identity gives, for all m, n ::: I,

IIYn - Ym 11 2 = 2(lIx - Yn 11 2

Since ~(Yn

+ Ym)

+ IIx -

Ym 11 2 )

411x - ~(Ym

+ Yn)1I 2

G,

therefore,
IIYn - Ymll 2

:s 2 (~ + ~).

The sequence {Yn}n~l is thus a Cauchy sequence in G, and it consequently converges to some Y E G since G is c1osed. Passing to the limit in (*) gives
(10).

Uniqueness of Y satisfying (10): Let y'

IIx - Y'II

= IIx -

G be another such element. Then

Yll

= d(x, G),

140

CL Hilbert Spaces

and from the parallelogram identity

lIy - y'1I 2 = 211Y - xll 2 + 211y' - xll 2 - 411x _

= 4d(x, G)2 Since !(y

+ y')

411x _ !(y

!(y + Y')11 2

+ y')1I 2.

E G,

Therefore,
lIy - y'1I 2 ~ 0,

which implies that lIy - Y'1I 2

= 0 and therefore, y = y'.

It now remains to show that x - y is orthogonal to G, that is,

forallZEG.

(x-y,z)=O

This is trivially true if z = 0, and we shall therefore assume

y + AZ E G for all A E IR,

IIx -

(y

+ Az)1I 2

:::

=j::.

O. Because

d(x, Gi,

that is,

Since

wehave
- 2A Re {(x - y, z)}
which implies Re {(x - y, z)}
(pure imaginary) leads to

+ A211z 11 2 ::: 0

.for allA

IR,

O. The same type of calculation with A E ilR

Im{(x - y, z)} =

o.

Therefore,
(x - y, z) = O.

That y is the unique element of G such that y - x E G.L follows from the
observation made just before the statement of Theorem C 1.6.

The element y in Theorem C 1.6 is called the orthogonal projection of x on G

(see Fig. CU) and is denoted PG(x).

Projection Principle
The projection theorem states, in particular, that for any x E G there is a unique
decomposition

x = Y +z,

G,

Z E

G.L,

(11)

C13 Projection Theorem

141

xeH

Figure C 1.1. Orthogonal projection

and that y = Pa(x), the (unique) element of G closest to x. Therefore, the

orthogonal projection y = Pa (x) is characterized by the following two properties:
(1) y e G;
(2) (y - x, z) = 0 for all z e G.

This characterization is called the projection principle and is useful in

determining projections.
Projection Operator

The next result features two useful properties of the orthogonal projection operator
Pa
'THEOREM

CI.7. Let G be a Hilbert subspace ofthe Hilbert space H.

(a) The mapping x ~ Pa(x) is linear and continuous; furthermore,

IlPa(x)1I :::: IIxll

forall xe H.

() If Fis a Hilbert subspace of H such that F S;; G, then PF

particular, Pa
Pa (Pa is then called idempotent).

Proof:

Pa

= PF. In

(a) Let Xl, X2 eH. They admit the decomposition

Xj

where Wj e Gl. (i

= Pa(Xj) + Wj

(i

= 1,2),

= 1,2). Therefore,
Xl

+ X2 =
=

where W e Gl.. Now, Xl

+ Pa(X2) + WI + W2
Pa(xt} + Pa(X2) + W,

Pa(XI)

+ X2 admits a unique decomposition of the type

= y+w,
where W e Gl., y e G: namely, y = Pa(XI + X2). Therefore,
Pa(XI + X2) = Pa(XI) + Pa(X2).
Xl

+X2

One similarly proves that

Pa(ax)

= aPa(x)

for all a e G, X e H.

142

Cl. Hi1bert Spaces

Thus PG is linear.
From Pythagoras' theorem applied to x = PG(x) + w,
IIPG(x)1I

+ IIwII 2 = IIx1I 2 ,

and therefore,

Hence, PG is continuous.
() The unique decompositions of x on G and G.L and of PG(x) on F and F.L
are
x

= PG(X) + w,

From these two equalities we obtain

(*)

But (z E G.L) =} (z E F.L) since F ~ G, and therefore v = Z + W E F.L. On the

other hand, PF(PG(X E F. Therefore, (*) is the unique decomposition of x on
Fand F.L; in particular, PF(X) = PF(PG(X.

The next result says that the projection operator PG is "continuous" with respect
to G.
THEOREM CI.S. (i) Let {Gn }n:::l be a nondecreasing sequence ofHilbert subspaces
ofH. ThentheclosureGofUn:::l Gn isaHilbertsubspaceofH and,for all X E H,

lim PGn(x) = PG(x).

ntoo

(ii) Let {G n } be a nonincreasing sequence of Hilbert subspaces of H. Then

nn:::l G n = G is a Hilbert subspace of Hand, for all x E H,

lim PGn(x) = PG(x).

ntoo

Proof: (i) The set Un>l G n is evidently a vector subspace of H (in general,
however, it is not closedflts closure, G, is a Hilbert subspace (Theorem C1.3). To
any Y E Gone can associate a sequence {Yn}n:::h where Yn E G n, and
lim IIY-Ynll=O.

n->oo

Take Y = Pa(x). By the parallelogram identity,

IIPGn(x) - PG(x)1I 2

= lI(x -

PG(x - (x - PG n(x1I 2

= 211x -

PGn (x)1I 2 + 211x - PG(x)1I 2

- 411x - !(PGn(x)

+ PG(xII

C13 Projection Theorem

But since PGn(x)

143

+ PG(x) is a vectorin G,
IIx - !(PGn(x)

+ PG(x))1I 2 ~

IIx - PG(x)1I 2,

and therefore,
IIPGn(x) - PG(x)1I 2 S 211x - PGn (x)1I 2 - 211x - PG(x)1I 2

S 211x - Ynll 2 - 211x - PG(x)1I 2.

By the continuity of the norm,
lim IIx - Ynll 2 = IIx - PG(x)1I 2,
ntoo
and, finally,
lim IIPGn(x) - PG(x)1I 2 = O.
ntoo
(ii) Devise a direct proof in the spirit of (i) or use the fact

Gl..

= clos

(U G;) .

n~1

EXERCISE

Cl.6. Prove the lollowing two assertions:

Let {G n } be a nonincreasing sequence 01 Hilbert subspaces 01 H. Then

nn~IGn = 0 ifand only iflimntoo PGn(x) = Olorall xE H .
Let {Gn}n~1 be a nondecreasing sequence 01 Hilbert subspaces 01 H. Then
clOSUn~1 G n = H ifand only iflimNtoo PGn(x) = xlorall XE H.
NOTATION.

IIG I and G 2 are orthogonal Hilbert subspaces olthe Hilbert space

H,

GI ffi G2 := {z = XI

+ X2

: XI E G, X2 E G2}

is called the orthogonal sum 01 GI and G2.

Riesz's Representation Theorem

C1.6. Let H be a Hilbert space over C and let I : H t-+ C be a
linear mapping; I is then called a (complex) linear form on H. It is said to be
continuous if there exists A ~ 0 such that
DEFlNITION

I/(xd - I(X2)1

AlixI - x211

lor all XI, X2

H.

(12)

The infimum olthe constants A satisfying (12) is called the norm 01 I.

EXAMPLE

C1.3. Let Y

Hand define I: H t-+ C by

I(x)

= (x, y).

(13)

144

Cl. Hilbert Spaces

It is a linear form and, by Schwarz's inequality,

If(XI) - f(X2) I = If(XI -

x2)1

I(XI - X2, y)1 :'S lIyllllxl - x211

Therefore, fis continuous. Its norm is lIyll. To prove this it remains to show that
if K is such that I(XI - X2, y)1 :'S K IIXI - x211foralt Xl, x2 E H, then lIylI :'S K.
It suffi,ces to take Xl = X2 = Y above, which gives IIYll2 :'S K lIylI.
=

We now state and prove Riesz's representation theorem.

THEOREM C1.9.

Let f : H 1-+ C be a continuous linear form on the Hilbert space

H. Then there exists a unique y E H such that (13) is true for alt X E H.

Proof"

Uniqueness.

Let y, y'

H be such that

f(x) = (x, y) = (x, y')

for an X EH.

In particular,
(x, y - y')

The choice x

=y-

y' leads to

=0

for all x

H.

lIy - y'II 2= 0, that is, y - y',

Existence: Consider the kernel of f, N = {u EH: f (u) = O}. It is a Hilbert

subspace of H. We may suppose that f is not identically zero (otherwise, if f == 0,
take y = 0 in (13. In particular, N is strictly included in H. This implies that N1.
does not reduces to the singleton {O} and, therefore, there exists zEN 1., z =f- O.
Define y by
y = f(z)* IIzzll 2

For an x

N, (x, y) = 0; therefore, in particular, (x, y) = f(x). Also,

(z, y)

= (z,

f(z)*

11 zZIl 2 ) = f(z).

Therefore, the mappings x --+ f(x) and x --+ (x, y) coincide on the Hilbert
subspace generated by N and z. But this subspace is Hitself. Indeed, for an
xE

H,

x = (x -

where u

f(x)
f(z)

N and w is colinear to z.

z) + f(x)
z= u + w,
f(z)

C2
Complete Orthonormal Systems

C21

Orthonormal Expansions

The result ofthis section is the pillar ofthe L 2-theory ofFourier series and wavelet
expansions. It concems the possibility of decomposing a vector of a Hilbert space
along an orthonormal base.
The Gram-Schmidt Orthonormalization Procedure
The central notion is that of an orthonormal system:

C2.I. The sequence {en}n~O in a Hilbert space H is called an

orthonormal system of H if it satisfies the following two conditions:

DEFINITION

(cx) (en,ek) = Oforall n =j:.k;and

() lien 11 = Iforalln:::: O.

An orthonormal system {en}n~O isfree in the sense that an arbitrary finite subset
of it is linearly independent. For example, taking (eI, ... , ek), the relation
k

Lcxiei =0
i=1

implies that

CXiei )

=0

1:::

e ::: k.

C2.I. Let {fn}n~O be a sequence of vectors of a Hilbert space H.

Construct {e n }n~O by the Gram-Schmidt orthonormalization procedure:

EXERCISE

P. Brmaud, Mathematical Principles of Signal Processing

Springer Science+Business Media New York 2002

146

C2. Complete Orthonormal Systems

Set p(O) = 0 and eo = 10111 10 11 (assuming 10 i= 0 without loss 01 generality);

Witheo, ... , en and p(n) defined, let p(n + 1) be thefirst index p > p(n) such
that I p is independent 01 eo, ... , en, and define, with p = p(n + 1),

Ip-

L(fp, ej}ej

.,,-11-------;;-11
I
j=!

en +!
Show that

{en}n~O

p - "t(fp, ej}ej

is an orthonormal system.

Hilbert Basis
The following theorem gives the preliminary results that we shall need for the
proof of the Hilbert basis theorem.
THEOREM C2.1. Let {en}n2:0 be an orthonormal system 01 Hand let G be the
Hilbert subspace 01 H generated by {en}n~!. Then:

(a) For an arbitrary sequence {an }n~O 01complex numbers, the se ries Ln>o anen
is convergent in H if and only if {an}n~! E e~, in which case
(14)

(b) For alt x

H, Bessel's inequality holds:

L I(x, e }l2 .::; IIx1l

n

(15)

n~O

(c) For alt x

H, the series L (x, en}en converges, and

n~O

L(x,en}en = PG(x),

(16)

n~O

where PG is the projection on G.

(d) For alt x, y E H, the series Ln>! (x, en}(y, en ) is absolutely convergent,
and
L(x, en}(y, en}*

= (PG(X), PG(y)}

(17)

n~O

Prool:

(a) From Pythagoras' theorem we have

and, therefore, {LJ=oajej}n~o is a Cauchy sequence in H if and only if

{LJ=o la j 12}n~0 is a Cauchy sequence in IR. In other words, Ln~o anen con-

C21 Orthononnal Expansions

147

verges if and only if Ln>O lan 12 < 00. In this case equality (14) follows from the
continuity of the norm, by letting n tend to 00 in the last display.
(b) Accordingto (a) ofTheorem Cl.7, IIxll :::: 11 PGn (x)lI, where G n is theHilbert
subspace spanned by {el, ... , en }. But
n

PGn(x)

= ~)x, ei}ei,
i=O

and by Pythagoras' theorem,

IlpGn (x)11

L I(x, ei}1

2.

i=O

Therefore,
n

IIxll 2 ::::

L I(x, ei}1

2,

i=O

from which Bessel's inequality follows on letting n -+

00.

(c) From (15) and result (a), it follows that the series Ln>o (x, en}en converges.
For any m :::: 0 and for all N :::: m,

and, therefore, by continuity of the Hermitian product,

( X - L(x, en}en, em)
n;::O

=0

for all m :::: O.

This implies thatx - Ln>o(X, en}en is orthogonal to G. Also, Ln>o(X, en}en

Therefore, by the projectlon principle,
PG(x)

G.

= L(x, en}en
n;::O

(d) By Schwarz's inequality in .e~, for all N :::: 0,

(~llx, e")(y, e"),r " (~llx, e")I') (~IIM)I')

~ IIx1l 2 11Y1l2.
Therefore, the series L:'o (x, en ) (y, en )* is absolutely convergent. Also, by an
elementary computation,

148

C2. Compiete Orthononnal Systems

Leuing N -+
product).
DEFINITION

00,

we obtain (17) (using (16) and the continuity of the Hermitian

C2.2. The sequence

{Wn}n~O

oJvectors oJ His said to be total in H

ifit gene rates H.

In other words, the finite linear combination of the elements of {w n }n~O forms
a dense subset of H.

C2.2. Prove that a sequence {wn}n~O oJthe Hilbert space H is total in

H if and only if there is no element oJ H orthogonal to alt the Wn, n ::: 0, except
0, that is, if and only if
EXERCISE

((Z, w n ) = 0 Jor alt n ::: 0)

==}

(z = 0).

(18)

We are now ready for the fundamental result: the Hilbert basis theorem.
THEOREM C2.2. Let {en}n~O be an orthonormal system oJ H. The Jollowing
properties are equivalent:

(a) {en}n~O is total in H;

(b) Jor alt x E H, the Plancherel-Parseval identity holds true:

IIxII 2 =

L I(x, e }1
n

2;

(19)

n~O

(c)Jor alt x

H,
(20)

x = L(x, en}en.
n~O

Proof

(a)=}(c)

Accordingto(c)ofTheoremC2.1,
L(x, en}en = PG(x),
n~O

where G is the Hilbert subspace generated by {en}n>O. Since {en}n>O is total, it

follows by (18) that Gl. = {O}, and therefore PG(x) :: x.
(c)=::}(b) This follows from (a) ofTheorem C2.I.
(b)=::}(a) From (14) and (16),
L

I(x, en}1 2 = IIPG(x)1I 2 ,

n~O

and (19) therefore implies

From Pythagoras ' theorem,

IIxll 2 =

IIPG(x)+x - PG(x)1I 2

= IIPG(x)1I 2 + IIx = IIxll 2 + IIx -

PG(x)1I 2

PG(x)1I 2 ;

C21 Orthononnal Expansions

149

therefore,

IIx -

PG (x)1I 2 = 0,

which implies
x

PG(x).

Since this is true for an x E H, we must have G

H.

==

H, that is, {en}n;::O is total in

A sequence {en }n;::O satisfying one (and then an) of the conditions of Theorem
C2.2 is caned a (denumerable) Hilbert basis of H.

VI

C2.3. Let 1/f be a function in L~ (IR) with the FT

= 2~ 1/, where
I = [-2rr, -rr] U [+rr, +2rr]. Show that {1/fj,n}jEZ,nEZ is a Hilbert basis of L~(IR),
where 1/fj,n(x) = 2 j / 2 1/f(2 j / 2 x - n).
EXERCISE

EXERCISE

C2.4. Let {gj}j;::o be a Hilbert basis of L 2 O, 1]). Show that {gA -

n)ICn,n+1l0}j;::O,nEZ is a Hilbert basis of L 2 (1R). (Here, L 2(I) Denotes the Hilbert

space (equivalence classes) of measurable complex-valued functions defined on

I, with the Hermitian product (f, g) =
f(t)g(t)*dt.)

1/

Biorthonormal Expansions
DEFINITION

C2.3. Two sequences {en}n;::O and {dn}n;::O of aHilbert space H form

a biorthonormal system

if

(0:) (e n , d k ) = Oforall n
() (e n , dn ) = Ifor all n

=f. k,
~

O.

This system is ca lIed complete if, in addition, each ofthe sequences {en}n;::O and
{dn}n;::oforms a total subset of H.

Then we have the biorthonormal expansions

x = L(x, en}dn,
n;::O

x = L(x, dn}en
n;::O

whenever these series converge. Indeed, with the first series, for example, calling
its sum y, we have for any integer m ~ 0,

= L(x, en}(dn , em }
n;::O

(x, e m ).

Therefore,
(x - y, em )

=0

for an m

Since {en}n;::O is total in H, this implies x - y = O.

O.

150

C2. Complete Orthononnal Systems

Separable Hilbert Spaces

An interesting theoretical question is: For what type of Hilbert spaces is there a
denumerable Hilbert basis? Here is a first (theoretical) answer.
DEFINITION C2.4. A Hilbert space H is called a separable Hilbert space
contains a sequence {fn}n:::O that is dense in H.

if it

THEOREM C2.3. A separable Hilbert space admits at least one denumerable

Hilbert basis.

Proof Let {fn }n:::O be a sequence defined in Definition C2.4. Construct from
it the orthonormal sequence {en}n:::O by the Gram-Schmidt orthonormalization
procedure. It is a Hilbert basis because (a) of Theorem C2.2 is satisfied. Indeed,
forany zEH,

(en , Z)

= 0 for all n ~ 0) ==>

(fp, z)

= 0 for alln ~ 0).

In particular, (y, z) = 0 for any finite linear combination of {fp}p:::o, Because

{fp}p:::o is dense, (y, z) = 0 for all y E H. In particular, (z, z) = 0, that is,

z=Q

C22

Two Important Hilbert Bases

The Fourier Basis

The following theorem is the fundamental result of the theory of Fourier series of
finite-power periodie signals.
THEOREM

C2.4. The sequence

{en ( )} def {

1 2i Jr
../Te

!l. . }
T

'7J
fLj,

is a Hilbert basis of L~([O, T]).

Proof One first observes that {enO, n E Z} is an orthonormal system in
L~([O, T]). It remains to show that the linear space it generates is dense in
L~([O, T]) (Theorem C2.2).
For this, let f(t) E L~([O, T]) and let fN(t) be its projection on the Hilbert subspace generated by {enO, -N ::: n ::: N}. The coefficient of en in this projection
is cn(f) = (f, en)L~([O.Tl)' we have

+N

{T

L Icn(f)1 + Jo
n=-N
2

If(t) - fN(t)1 2 dt

(T

= Jo

If(t)1 2 dt.

(21)

C22 Two Important Hilbert Bases

151

(This is Pythagoras' theorem for projections: 11 PG(X) 11 2 + IIx - PG(X) 11 2 = IIx 11 2 .)

In particular, LnEZ Icn (f)1 2 < 00. It remains to show ((b) of Theorem C2.2) that
lim

Ntoo

[T

10

We assume in a first step that

qJ(X)

If(t) - fN(t)1 2dt

= 0.

f is continuous. For such a function, the formula

Jex

+ t)Jet)* dt,

where
J(t)

=L

f(t

+ nT)l(o.nCt + nT),

nEZ

defines a T -periodic and continuous function qJ. Its nth Fourier coefficient is
Cn(qJ)

- + t)f(t)*
- dt ) e- 2.IITTXn dx,
= T1 10[T( fex
1

{T _

= T 10
=

f(t)*

{{T _

10

fex

+ t)e-

2.

- 2
T1 10{T f(t)* {ft+T f(s)et

IITTX

n
IITTS

dx dt

ds e

n
IITTt

dt

Since LnEZ Icn (f)1 2 < 00 and qJ(X) is continuous, it follows from the Fourier
inversion theorem for locally integrable periodic functions that, for all x E IR,
qJ(x)

=L

Icn(f)1 2e 2i Jl"j-x.

nEZ

In particular, for x = 0,

and therefore, in view of (21),

lim

Ntoo

(T

10

If(t) - fN(t) 12 dt

= 0.

It remains to pass from the continuous functions to the square-integrable functions. Since the space C([O, Tl) of continuous functions from [0, T] into C is dense
in L~([O, Tl) (Theorem 27), with any 8 > 0, one can associate qJ E C([O, Tl) such
that IIf - qJlI ::::: 813. By Bessel's inequality, IIfN - qJNII 2 = 1I(f - qJ)N1I 2 :::::

152
11

f -

C2. Complete Orthonormal Systems

cP 11 2 , and therefore,

IIf - fNIl S IIf - cplI + IIcp - CPNII + IIfN - CPNII

S IIcp - CPNII
S

+ 211f - cpll
8

IIcp - CPNII + 2 3,

For N sufficiently large, 11 cP - cP Nil S 8/3. Therefore, for N suffieiently large,

IIf-fNIIS8.

The Cardinal Sine Basis

The LI-version of the Shannon-Nyquist theorem of Seetion B2l eontains a

eondition bearing on the samples themselves, namely,
(22)

The simplest way of removing this unaesthetie condition is given by the L 2 _

version of the Shannon-Nyquist theorem.
THEOREM

C2.5. Let set) be a base-band (B) signaloffinite energy. Then

lim ( Iset) -

Ntoo

llR

bn sine (2Bt - n

n=-N

where
bn

L+N

+B

j2 dt = 0,

(23)

s(v) e2irrvfB dv,

-B

Proof' Let L~(lR; B) be the Hilbert subspaee of LUlR) consisting of the finiteenergy eomplex signals with a Fourier transform having its support eontained in
[ - B, +B]. The sequenee
(24)

where h(t) == 2B sine (2Bt), is an orthonormal basis of L~(lR; B). Indeed, the
functions of this system are in L~(lR; B), and they form an orthonormal system
sinee, by the Planeherel-Parseval formula,

+B

-B

e2irrvkz-;

dv

= 2B

x ln=k.

C22 Two Important Hilbert Bases

153

It remains to prove the totality of the orthonormal system (24) (see Theorem C2.2).
We must show that if g(t) E L~(IR.; B) and

2~) dt = 0

g(t)h(t -

for all n

Z,

(25)

then g(t) == 0 as a function of L~(IR.; B) (or, equivalently, that g(t) = 0 almost

everywhere).
Condition (25) is equivalent (by the Plancherel-Parseval identity) to
[+B g(v)e 2irrv ii;- dv

LB

=0

for all n

Z.

(26)

But we have proven in the previous section that the system {e2irrvn/2B }nEZ is total in
LUIR.; B); therefore, (26) implies g(v) = 0 almost everywhere, and consequently,
g(t) = 0 almost everywhere.
Expanding s(t)

E L~(IR.;

B) in the Hilbert basis (24) yields

+N

s(t) = lim LCn

Ntoo -N

n
h(t - - ) ,
v2B
2B
Mn

(27)

where the limit and the equality in (27) are taken in the L 2-sense (as in (23, and
Cn

[s(t)

Jrw.

~h(t -

..!!...-)dt.
2B

v2B

By the Plancherel-Parseval identity,

Cn

+B

-B

s(v) - - e 2l1tv2ij dv.

./fii

An Apparent Paradox

Note that since s( v) is in L 2 and of compact support, it is also in LI, and therefore
the Fourier inversion formula is true and the reconstruction formula takes the
farniliar form
s(t) = Ls(..!!...-) sinc(2Bt - n).
nEZ
2B

(28)

This is essentially true, but not quite.

Indeed, imagine that someone tells you the following: Look, I have an proof that
aL 2-signal s(t), base-band (B) is almost everywhere zero! Here is my cute proof.
Of course, the reconstruction equality is in the sense of equality of L 2-functions,
and in particular, it holds only for almost all t. Now, let me change the original
signal to obtain a new signal s'(t) differing froms(t) only atthe times n/2B, where
I set s'(n/2B) = O. I now apply the reconstruction formula and obtain that s'(t) is
almost everywhere zero. But s' (t) and s(t) are almost everywhere equal. Therefore,
s(t) is almost everywhere zero! Quod erat demonstrandum.

154

C2. Complete Orthonormal Systems

The flaw in the above "proof' is that the Fourier inversion formula holds only
almost everywhere, and maybe not at the sampling times. Therefore, formula (28)
is true only if the Fourier inversion formula can be applied at all the times of the
form nj2B. This is the case if s(t) is continuous, because the inversion formula
then holds everywhere.
We see that the continuity hypothesis always pops up. We cannot expect a much
better version of the sampling theorem in the LI or L 2 framework. Indeed, since
s(v) is integrable, the right-hand side of
s(t) =

s(v)e2invtdv

is continuous, and s(t) is therefore almost everywhere continuous.

We have a sampling theorem for sinusoids and for decomposable signals (Theorem B2.5), and those signals are neither in LI nor in L 2 . Note, however, that they
are continuous.

C3
Fourier Transforms of Finite Energy
Signals

C31

Fourier Transform in L 2

A stable signal as simple as the rectangular pulse has a Fourier transform that is not
integrable, and therefore one cannot use the Fourier inversion theorem for stable
signals as it iso However, there is aversion of this inversion formula that applies
to all finite-energy functions (for instance, the rectangular pulse). The analysis
becomes slightly more involved, and we will have to use the framework ofHilbert
spaces. This is largely compensated by the formal beauty of the results, due to the
fact that a square-integrable function and its Fr play symmetrical roles.

The Isometrie Extension

We start with a technical result. We use 1(.) to denote the function I : lR 1-+ C;
in particular, I(a + .) is the function la : lR 1-+ C defined by la(t) = I(a + t).
THEOREM

C3.1. Let set)

LUlR). The mapping Irom lR into L~(lR) defined by

t -+ set

+ .)

is uniformly continuous.
Proof:

We have to prove that the quantity

L'S(t

+ h + u) -

set

+ u)1 2 du

= L1S(h

+ u) -

s(u)1 2 du

tends to 0 when h -+ o. When sO is continuous and compactly supported, the

result follows by dominated convergence. The general case where sO E LUlR)
P. Brmaud, Mathematical Principles of Signal Processing
Springer Science+Business Media New York 2002

156

C3. Fourier Transforms of Finite-Energy Signals

is obtained by approximating sO in L~(I~) by continuous compact1y supported

functions (see the proof ofTheorem A1.4).

From Schwarz's inequality, we deduce that

t -+ (s(t

+ .), s( . ) L~(R)

is uniformly continuous on ~ and bounded by the energy of the signal.

The above function is
(29)

t -+ LS(t +x)s*(x)dx

and called the autocorrelation function of the finite-energy signal s(t). Note that
it is the convolution s(t) * s(t), where s(t) = s( -t)*.
THEOREM C3.2. lfthe complex signal s(t) lies in L~(~) n L~(~), then its FT s(v)
belongs to L~(~) and

L Is(t)1 2 dt

=L

(30)

Is(v)1 2 dv.

Praof: The signal s(t) admits s(v)* as FT, and thus by the convolutionmultiplication rule,
(31)

Consider the Gaussian density function

h,y(t)

,2

= --e-2,;2.
a,J2ii

Applying the result in (14) of Chapter Al, with (s

observing that ha(t) is an even function, we obtain
L Is(v)1 2 ha(v)dv

=L

(s

* s)(t) instead of s(t), and

* s)(x)ha(x)dx.

(32)

Since ha(v) = e- 2:rr 2a2x 2 t I when a .,j.. 0, the left-hand side of (32) tends to
Is(v)1 2 dv, by dominated convergence.

IR

On the other hand, since the autocorrelation function (s * s)(t) is continuous and
bounded, the quantity
L (s

tends when a

.,j..

* s)(x)ha(x)dx =

L (s

* s)(ay)hj(y)dy

0 toward
L (s *s)(O)hj(y)dy

by dominated convergence.

= (s *s)(O) = L

Is(t)1 2 dt,

C31 Fourier Transform in L 2

157

From the last theorem, we have that the mapping cp: s(t) -+ s(v) from LboR.) n
L~(lR) into L~(lR) thus defined is isometrie and linear. Sinee LI n L 2 is dense

in L 2, this linear isometry ean be uniquely extended into a linear isometry from
LUlR) into itself (Theorem C1.5). We will eontinue both to denote by s(v) the
image of s(t) under this isometry and to eall it the FT of s(t).
EXERCISE

C3.1. Show thatfor s(t) E L~(lR),

lim ( 1s(v)

Ttoo

J~

-1

+T

s(t)e-2irrVldt 1 dv.

(33)

-T

The above isometry is expressed by the Plancherel-Parseval identity:

THEOREM

C3.3. If SI (t) and S2(t) are finite-energy, complex signals, then

SI(t)S2(t)* dt =

EXERCISE

(34)

SI(V)S2(V)* dv.

C3.2. Show that

dv =
Im ( Sin(JTv))2
JTV

1.

THEOREM

C3.4. If h(t) E Lb(lR) and x(t) E L~(lR), then

y(t)

(35)

h(t - s)x(s) ds

is almost everywhere weil defined and in LUlR). Furthermore, its FT is

y(v) = h(v)x(v).

(36)

Proof Letus first show that f~ h(t-s)x(s) ds is welldefined. Forthis weobserve

..
that on the one hand
L1h(t - s)llx(s)1 ds :::: L1h(t - s)l(l

Ix(s)1 2)ds

= L1h(t)1 dt + L1h(t -

s)llx(s)1 2 ds,

and on the other, for almost all t,

L1h(t - s)ll(x(s)1 2 ds <

00,

sinee Ih(t)1 and Ix(t)1 2 are in Lb(lR). Therefore, for almost all t,

L1h(t - sllx(s)1 ds <

00,

and y(t) is almost everywhere well defined. We now show that y(t)

LUlR).

158

C3. Fourier Transforrns of Finite-Energy Signals

Using Fubini's theorem and Schwarz's inequality, we have

LIL

h(t - S)X(S) dS

=
=

LIL
LL{L

2
I

dt

h(u)x(t - U)dUr dt

x(t - u)x(t - v)* dt} h(u)h(v)* du dv

~ (L Ix(s)1 dS) (L Ih(u)1 du Y<

2

00.

For future reference, we rewrite this as

IIh

* XIlL~(lR) ~

(37)

IIhIlLt(lR)lIxIlL~(lR)'

The signal (35) is thus in L~(IR) when h(t) E LUIR) and x(t)
furthermore, x(t) is in Lb(IR), then y(t) is in Lb(IR). Therefore,
x(t) E Lb(lR)

n L~(IR) -+

E LUIR).

y(t) E Lb(lR) n L~(IR).

If,

(38)

In this case, we obtain (36) by the convolution-multiplication formula in L I .

We now suppose thatx(t) is in L~(IR) (butnot necessarily in Lb(IR. The !signal
XA(t) = x(t)l[-A,+Aj(t)

isinLb(lR)nL~(IR)andlimxA(t)

in L~(IR). Introducing
YA(t)

= x(t)inL~(IR).Inparticular,limxA(v) = x(v)

h(t - s)xA(s)ds,

= h(V)XA(V), Also, lim YA(t) = y(t) in L~(IR) [use (37)], and thus
limYA(v) = y(v) in L~(IR). Now, since limxA(v) = x(v) in L~(IR) and h(v) is

bounded, lim h(V)XA(V) = h(v)x(v) in L~(IR). Therefore, we have (36).

we have YA(V)

EXERCISE

C3.3. Use the Plancherel-Parseval identity to prove that

{dt

llR (t 2 + a 2 )(t 2 + b2 ) EXERCISE

7t:

ab(a

+ b)'

C3.4. Show that

H

= {w(t) E L~(IR);tw(t) E L~(IR), vw(v) E L~(IR)}

is a Hilbert space when endowed with the norm

IIwllH = (lIwll~2

+ IItwll~2 + IIvwll~2P .

Show that the subset 0/ H consisting 0/ the C oo -functions with compact support
is dense in H. Hint: Select any q; in a Coo-function with compact support, with

C32 Inversion Formula in L 2

159

integral equal to 1, and equal to 1 in a neighborhood ofO, andfor any wEH,

consider the function w(t)cp(t / n) ncp(nt).

C32

Inversion Fonnula in L 2

So far, we know that the mapping cp : L~(~) 1-+ L~(~) defined in Seetion C31
is linear, isometrie, and into. We shall now show that it is onto, and therefore
bijeetive.
THEOREM

C3.5. LetS(v) be the FT of s(t)

E L~(~).

Then

cp : s( -v) --+ s(t),

that is,
s(t)

= lim

Atoo

+A

-A

s(v)e 2i Jl'vt dv,

(39)

(40)

where the limit is in LU~), and the equality is almost everywhere.

We shall prepare the way for the proof with the following result.
LEMMA

C3.1. Let u(t) and v(t) be two jinite-energy signals. Then

u(x)v(x)d.x

u(x)v(x)d.x.

(41)

Proof' If (41) is true for u(t), v(t) E L~(~) n L~(~), then it also holds for
u(t), v(t) E L~(~). Indeed, denoting XA(t) = x(t)I[-A,+Aj(t), we have

u A(x )(v;)(x) d.x

(l7;)(x)v A(x) d.x,

that is, (UA, VA) = (UA, VA)' Now UA, VA, UA, and VA tend in LU~) to u, V, u,
and V, respeetively, as A t 00, and therefore, by the eontinuity of the Hermitian
produet, (u, v) = (u, v).
The proof of (41) for stable signals is aeeomplished by Fubini's theorem:

u(x)v(x)d.x

L {L
= L {L
=L
=

u(x)

v(y)e- 2i Jl'xy d Y } d.x

v(y)

u(x)e- 2i JI'XY d Y } dy

v(y)u(y)dy.

160

C3. Fourier Transforms ofFinite-Energy Signals

Proof of(39); Let g(t) be a real signal in L~(lR), and define f(t)
g-(t) = g(-t). We have j(v) = g(v)*. Therefore, by (41):

g(x)j(x)dx =

= (g~)(t), where

g(x)f(x)dx

(x)(x)* dx.

Therefore,
IIg -

But IIgll 2

f112

j) + IIfII 2

= IIgll 2 -

2Re (g,

= IIgll 2 -

211g11 2 + IIfII 2 .

= IIgll 2 and IIfll 2 = IIg1l 2 . Therefore, IIg - f112 = 0, thatis,

g(t) = j(t).

(42)

In other words, every real (and, therefore, every complex) signal g(t) E L~(lR)
is the Fourier transform of some function of LUIR). Hence, the mapping q; is
onto.

C3.5. Show that if a stable signal is base-band (that is,

compact support), then it also has afinite energy.

EXERCISE

if its FT has

We dose this seetion by showing how the LI Fourier inversion theorem was
lirnited in scope, since it does not take much for a stable signal not to have an
integrable FT.
C3.6. Show that if a stable signal is discontinuous at a point t
FT is not integrable.

EXERCISE

= a, its

C4
Fourier Series of Finite Power Periodic
Signals

C41

Fourier Series in Lfoc

Let us consider the Hilbert space e~ of complex sequences a

that LnEZ lan l2 < 00, with the Hermitian product
(a, b}e~

= {an}, n

Z, such

= L>nb~

f:

and the Hilbert space L~([O, T], dt/T) of complex signals x = {x(t)}, t
such that
Ix(t)1 2 dt < 00, with the Hermitian product
(x, Y}L~([O,T],~) =
THEOREM

C4.1. Formula
Sn

= -1
T

(43)

nEZ

dt
Jorx(t)y(t)* T'
T

n dt
s(t)e- 2"17r'i/

lR.,

(44)

(45)

defines a linear isometry sO --+ {Sn} lram L~([O, T], dt / T) onto e~, the inverse

01 which is given by

s(t)

= L::Sne2i1l"f/ ,

(46)

nEZ
where the se ries on the right-hand side converges in L~([O, T], dt / T), and the
equality is almost everywhere. This isometry is summarized by the PlancherelP. Brmaud, Mathematical Principles of Signal Processing
Springer Science+Business Media New York 2002

162

C4. Fourier Series of Finite-Power Periodic Signals

Parseval identity:

LXnY~ = ~

nEZ

(47)

(Tx(t)y(t)* dt.
10

The result follows from general results on orthonormal bases of Hilbert

spaces, since the sequence

Prao!"

{e n (-)}

~ {Jre 2i1t T'}'

nE Z,

is a complete orthonormal sequence of L~([O, Tl. dt / T) (Theorem C2.4).

The L 2 inversion theorem tells us that if s (t) is a T -periodic complex signal

with finite power, then
+N

L~([O, Tl)

Sn e2i1t (n/T)t ------+ set).

-N

Ntoo

In general, for an arbitrary sequence of functions of L~([O, Tl), convergence

in L~([O, Tl) does not imply convergence almost everywhere. However, for
sequences of partial Fourier series, we have the surprising Carleson's theorem:
THEOREM C4.2. The Fourier se ries 01 a T -periodie signal s(t) with finite power
converges almost everywhere to set).

This result shows that the situation for finite-power periodic signals is pleasant,
in contrast with the situation prevailing locally stable periodic signals (remember
Kolmogorov's result, Theorem A3.1). The proof ofCarleson's result is omitted; it
is rather technical. It also shows that the L 2 framework is very adapted to Fourier
series, since everything works "as expected."
Discrete-Time Fourier Transform of Finite-Energy Signals
Let lb be the space of sequences In, n
discrete-time signals).
THEOREM

C4.3. lb

Z, such that LnEZ Ilnl <

00

(stable

l~, that is, a discrete-time stahle signal has finite energy.

Prao!" Let A = {n: IX n I ~ I}. Since LnEZ IX n I < 00, then necessarily
card(A) < 00. On the other hand, if Ixnl ::::: 1, Ix n l2 ::::: Ixnl, whence

The situation for discrete-time signals is therefore in contrast with that of

continuous-time signals, for which there exist stable signals with infinite energy
that and finite-energy signals that are not stable.

C42 Orthonormal Systems of Shifted Functions

163

Let L~(2n) be the Hilbert space of functions j: [-n, +n] ~ C such that
f~: Ij(w)1 2 dw < 00 provided with the Hennitian product
(],

C4.4.

THEOREM

g)L~(2n) =

l+

_1
2n

-n

j(w)g(w)* dw.

There exists a linear isomorphism between LU2n) and .e~

dejined by
fn =

l+

-n

j(w)e inw dw ,
2n

j(w)

=L

fn e- inw .

(48)

nEZ

In particular, we have the Plancherel-Parseval identity

L fng: = -2n1 l+

f(w)g(w)*
dw.

(49)

-n

nEZ

This is arestatement of Theorem C4.3.

Proof:

C4 2

Orthonormal Systems of Shifted Functions

We give a necessary and sufficient condition in the frequency domain for a system
of shifted functions to be orthonormal.
THEOREM C4.5. Let g(t) be afunction of L~(lR.) andjix 0 < T <
and sufficient condition for the family offunctions

00.

{g(. - nT)}nEZ

A necessary
(50)

to fonn an orthononnal system of L~(lR.) is

Ig(v

nEZ

-f )1

= T

almosteverywhere.

(51)

Proof: The Fourier transform g( v) of g(t) E L~(lR.) is in L~(lR.) and, in particular,

Ig(v)1 2 is integrable. By Theorem A2.3, LnEZ Ig(v + (n/T1 2 is (I/T)-periodic
and locally integrable, and T In~. g(t) g(t - nT)* dt is its Fourier coefficient (this
follows from the Plancherel-Parseval formula:
T

g(t)g(t - nT)* dt

=T

Ig(v)12e-2invnT dv

The definition of orthonormality of system (50) is

g(t) g(t - nT)* dt

= I n=o.

The proof then follows the argument in the proof of Theorem B2.6.

164

C4. Fourier Series of Finite-Power Periodic Signals

Riesz's Basis
The following notion will play an important role in multiresolution analysis.
DEFINITION

C4.1. A system offunctions of LU~)

{w( - nT)}nEz

(52)

is said to form a Riesz basis of some Hilbert subspace Vo of LU~)

(a) it spans Vo, and
(b)forall sequence, {ckhEZ ofe~(Z),

AL 1ck1 2 ::s
k

where 0 < A ::s B <

00

(IL

Jffi.

kT)1

CkW(t -

2
dt

::s B L

kEZ

if

Icd,

(53)

kEZ

are independent ofthe Ck.

Thefunction LkEZ Ck w(t-kT) has theFouriertransform LkEZ cke-2inkTvw(v),

and, therefore, by the Plancherel-Parseval identity, the term between the bounds
in (53) is equal to

(IL

Jffi.

Cke-2itrkTVW(V)12 dv

kEZ

Also,

Now, any function c(v) E L~([O, I/T]) has the form LkEZcke-2itrkTv, where
LkEZ ICkl 2 < 00, and (53)is therefore equivalent to
AT

1fT

Ic(v)1 2 dv::s

1 IC(V)121~
1fT

::s BT Jo

fT

Iw(v

Ic(v)1 2dv

f )1 1
2

dv

C42 Orthononnal Systems of Shifted Functions

for any c(v)

E L~([O,

165

IjT]). It then foIlows that

ATS~lw(v+ ~)12 SBT

a.e.

(54)

C4.6. Let (w( - n T) }neZ be a Riesz basis of some Hilbert subspace

Vo C L~(R). Define the function g E L~(R) by its Fourier transform

THEOREM

(55)

Then (g(. - nT)}nez is a Hilbert basis ofVo.

In view of (54), the function gis weIl defined and in L~(R). Since (51)
is obviously satisfied, it foIlows that the system {g(. - nT)}neZ is orthonormal.

Proof"

We must now show that the Hilbert space Vo spanned by (g(. - nT)}neZ is in
fact identical to Vo.1t suffices to show that the generators of Vo belong to \10, and
vice versa. Define

In view of condition (54), a( v) is (1 j T)-periodic and offinite power, and it therefore

admits a Fourier representation

a(v) = L ane-2i7CvnT,
neZ

for some sequence {an}

E l~(Z).

Since

w(v) = a(v)g(v),

it foIlows that
w(t) = L ang(t - nT).
neZ

Therefore, the generators of Vo are in Vo. The converse is true by the same argument
since

g(v) = a(v)-Iw(v),
where a(vr l is also, in view of condition (54), a (ljT)-periodic of finite
power.

C4.1. Let h(t) be the function of FT (ljJ2B)I[-B.+Bj(v) for some

B > O. Show that there exists no orthonormal basis of L~(R; B) of the form

EXERCISE

{k g (t -

2~) }nd where

g(t) = LCkh (t keZ

~),
2B

166

C4. Fourier Series of Finite-Power Periodic Signals

where {cn} nEZ is in CU7-), unless only one of the Cn is nonzero. Show that if the
Fourier sum c(w) ofthe sequence {cn}nEZ is such that A < !c(w)!2 < B for some

o<

:s

B <

L~(lR; B).

00 and for all w,

then

{k g (t -

2~)}

nEZ

is a Riesz basis of

References
[Cl]
[C2]
[C3]
[C4]
[C5]

Daubechies, I. (1992). Ten Lectures on Wavelets, CBSM-NSF Regional Conf.

Series in Applied Mathematics, SIAM: Philadelphia, PA.
Gasquet, C. and Witomski, P. (1991). Analyse de Fourier et Applications, Masson:
Paris.
Halmos, P.R. (1951). Introduction to Hilbert space, Che1sea, New York.
Rudin, W. (1966). Real and Complex Analysis, McGraw-Hill, New York.
Young, N.Y. (1988). An Introduction to Hilbert Spaces, Cambridge University
Press.

Part

Wavelet Analysis

Introduction

Although Fourier theory had reached in the L 2 framework a formal mathematical

beauty, it was not entirely satisfactory for important applications in signal processing. Indeed, in many situations, the information contained in a signal is localized
in time and/or in frequency. The typical example is a piece of music, which is
perceived as a succession of notes welliocalized in both time and frequency. The
usual Fourier transform is not adapted to the analysis of music because for a given
frequency (a note) it is related to the total energy of all occurrences of this note in
the entire piece.
This led Dennis Gabor 1 to propose a windowed Fourier transform, whose idea
is very natural. If f(t) is the signal to be analyzed, the local information at time
t = bis contained in the time-Iocalized signal
f(t)w*(t - b),

where w(t) is the time window function, a function negligible outside a relatively
small interval around zero.
Given a window w(t), the local information at time b is obtained by computing
the Fourier transform of the last display:
Wf(v, b):=

f(t)w*(t - b)e-2invt dt

(j, Wv,b),

where
Wv,b(t)

= w(t -

b)e2invt.

ITheory ofCommunications, J. Inst. Elec. Engrg., Vol. 93, pp. 429-457,1946.

170

Part D Wavelet Analysis

We see that the time information is collected in a time interval around time b of
width of the order of that of the time window. Now, from the Plancherel-Parseval
identity,

Therefore, we see that the frequency information is collected in a frequency interval

around v of the width of the time window's Fr.
These observations point to a fundamental limitation of the windowed Fourier
transform, in relation with the uncertainty principle, which states that time resolution is possible only at the expense of frequency resolution, and vice versa.
Indeed, the wider the function, the narrower its Fourier transform, and vice versa.
Of course, this is an imprecise statement, but it is already substantiated by the
Doppler theorem

(Note that as a varies, the energy of lai! !(at) remains the same.) The so-called
Heisenberg uncertainty principle makes the above limitation more explicit and
states that

where a f is the mean-square width of a function ! E L~(lR) (see the definition

in the first lines of Section D 11). The conclusion is that as long as we resort
to windowed Fourier transforms, time resolution and frequency resolution are
antagonistic: If one is increased, the other is necessarily decreased, keeping the area
awafij ofthe time-frequency box above the lower bound of 1/41T. However, the real
inconvenience of the windowed Fr is the fixed shape of the time-frequency box. In
many occasions, it is interesting to have a time-frequency box that adapts itself to
the time-frequency point analyzed. For instance, a discontinuity (abrupt change)
of a signal takes place in a short time and involves high frequencies. Therefore, at
high frequencies, the time dimension of the time-frequency box should be small.
Also, since it takes time to determine the frequency of a low-frequency sinusoid,
at low frequencies the time dimension of the time-frequency box should be large.
Motivated by these imperatives, lean Morlet2 proposed the wavelet transform in
order to take into account the need for an adaptive time-frequency box.

2 Sampling theory and wave propagation, in NATO ASI series, Vol. 1, Survey in Acoustic
Signal! Image Processing and Recognition, C.H. ehen, ed., Springer-Verlag, Berlin, 233-

261,1983.

Introduction

171

In the wavelet transfonn, the role the family of functions Wv,b(t) plays in the
windowed Fourier transfonn is played by a family

(t - b)

1fta,b(t) = lai -1/2 1ft -a-

a, b

IR, a

=I- 0,

where 1ft(t) is called the mother wavelet. The wavelet transfonn (WT) of the
function f E L~(IR) is the function
C j(a, b) = (j, 1fta,b) =

f(t)1ft;,b(t) dt,

By the Plancherel-Parseval identity,

C j(a, b)

= (j, :(fra,b) =

a, b

IR, a

=I- 0.

!(V):(fr;,b(V) dv,

where

:(fra,b(V) = laI 1/ 2 e- 2i 1l'Vb:(fr(av).

From the above expression, it appears that the wavelet transfonn C j (a, b) analyzes
the function f(t) in the time-frequency box

[b

+ am -

au, b + am

8- m 8-]
+ au] x [-m - -,
- + -

m
m/

(where m is the center of 1ft, u is its width; and 8- are the corresponding objects
relatively to :(fr; see the details in the main text). Assuming
> 0, we see that
the frequency window is centered at v =
a and has width 28- / a; therefore, the
ratio center/width

Q=

28-

is independent of the frequency variable a. The area of the time-frequency box

is constant, but its shape varies with the frequency v =
a analyzed. For high
frequencies it has a large time dimension, and for small frequencies it has a small
time dimension, which is the desired effect.

m/

The interest of the wavelet transfonn for signal analysts is that they can "read"
it to extract infonnation about the time-frequency structure that is otherwise
blurred in the brute signal by concurrent phenomena and subsidiary effects. For instance, they can detect the appearance time of a phenomenon linked to a particular
frequency (e.g., the time at which a particular atom starts to be excited).
A fonnula, called the identity resolution, allows us to reconstruct, under mild
conditions that we shall make precise in due time, the function from its wavelet
transfonn:

f(t)

= ~ { ( C j(a, b)1fta,b(t) dll ~b ,

K JlRJlR
a

where K is some constant depending on 1{1. In this sense, wavelet analysis is

continuous, in that the original function of L 2 is reconstructed as a continuous linear

172

Part D Wavelet Analysis

combination ofthe continuous wavelet basis 1J!a,b(t) = lal-l/21J!e~b) , a, bE lR.,

a =1= O. One would rather store the original function not as a function of two
arguments, but as the doubly indexed sequence of coefficients of a decomposition
along an orthonormal base of L 2 , {1J!j,nb,nEZ, called the wavelet basis,

L
LU, 1J!j,k)1J!j,k.
JEZ kEZ

where
j

1J!j,n(t) = 2 2 1J!(2J t - n),

and where 1J!(t) is the mother wavelet. The multiresolution analysis of Stephane
Mallat3 is one particular way of obtaining such orthonormal bases and is the main
topic of Part D. Similarly to the continuous wavelet transform, the coefficients of
the multiresolution decomposition can be used to analyze a signal. But multiresolution analysis is also a tool for data compression. Indeed, with a good design of
the mother wavelet, many wavelet coefficients are small and can be neglected. The
coefficients that are not neglected are quantized more or less coarsely, depending,
for instance, on the frequency index. This is of course reminiscent of subband
coding, and this resemblance is not at all a coincidence. Mallat's algorithm of
analysis-synthesis is of the same form as the subband analysis-synthesis algorithm. Multiresolution analysis can be considered as a systematic way of doing
subband analysis. One of its advantages is to place the latter in a framework where
the mathematical issues ensuring the efficiency of the algorithms are more easily
dealt with.
Perhaps one of the most striking advantages of multiresolution analysis over
classical Fourier analysis is in the way it handles discontinuities. Consider, for
instance, the signal on top of Figs. DO.la and DO.lb,4 which has a spike. In
both figures the middle signal is a Fourier series approximation of the top signal, whereas the bottom signal is a wavelet approximation of the top signal. In Fig.
DO.I a, only the first 60 coefficients of the expansions (Fourier or wavelet) are used
to produce the approximated signals, and it appears that there is not a dramatic
difference between the two approximations. This is not the case, however, when,
as in Fig. DO.I b, one uses the 60 largest coefficients. The advantage of the wavelet
approximation is then obvious.

3Multiresolution approximation and wavelets, Trans. Amer. Math. Soc., 315, 1989,6988.
4Reproduced with the kind permission of Martin Vetterli.

Introduction

173

0.5

of---------'
o

100

200

300

400

500

600

700

800

900

1000

100

200

300

400

500

600

700

800

900

1000

200

300

400

500

600

700

800

900

1000

0.5

o
o

0.5
01------"--"--"'\

100

(a)

0.5
01------'

100

200

300

400

500

600

700

800

900

1000

300

400

500

600

700

800

900

1000

300

400

500

600

700

800

900

1000

0.5

o
o

0.5
01------'

100

200

(b)

Figure DO.I. Wavelet VS. Fourier

Dl
The Windowed Fourier Transform

D 11

The Uncertainty Princip1e

Fourier analysis, weH as wavelet analysis have an intrinsic limitation, which is

contained in the uncertainty principle. In order to state this result, we need a
definition of the "width" of a function. Here is the one that suits our purpose.
Root Mean-Square Widths

Let w : lR 1-+ C be a nontrivial function in L 2 Define the centers of w and W,

respectively, by
mw

= -1

mfjj

= _1_

Ew IR

Ew

t Iw(t)1 2 dt,

JIR

v Iw(v)1 2 dv,

where E w is the energy of w(t):

Ew =

Iw(t)1 2 dt =

Then define

and

P. Brmaud, Mathematical Principles of Signal Processing

Springer Science+Business Media New York 2002

Iw(v)1 2 dv.

176

D 1. The Windowed Fourier Transforrn

w,

The numbers o"w and 0"1ii are the root mean-square (RMS) widths of w and
respectively. Note that m w and mlii are not always defined. When they are weIl
defined, o"w and 0"1ii are always defined but may be infinite. Therefore, we shall
always assurne that

and

Ivllw(v)1 2 dv

<

00,

to guarantee at least the existence of the centers of w(t) and of its Fr.
EXERCISE D1.I. Check if the centers m w and mlii are well defined, and then
compute o"w and 0"1ii and the product O"wO"Iii in thefollowing cases:

w(t)
w(t)

= l[-T,+Tl
= e-a1tl ,

a > 0,

w(t) = e- at2 ,

a > O.

Suppose that the centerm w ofthefunction w

Show that the quantity

EXERCISEDI.2.

L 2 iswell defined.

It - toI 2 Iw(t)1 2 dt

is minimized by to = m w .
Heisenberg's Inequality
THEOREM

DI.I.

Under the conditions stated above, we have Heisenberg's

inequality
(1)

Proof: We assurne that the window and its Fr are centered at 0, without loss of
generality (see Exercise D1.3). Denoting the L 2 -norm of a function f by IIfII, we
have to show that
~

IItwll x IIvwll :::: 4n IIwll '

We first assurne that w is a COO-function with bounded support. In particular,
td(v)

and therefore,

= (2inv) w(v),

(2)

D 11 The Uncertainty Principle

177

Thus, it remains to show that

1

x IIw'lI ::: "2 IIw1l 2

IItwll

(3)

By Schwarz's inequality,

x IIw'lI :::

IItwll

I(tw,

w')1 ::: IRe{(tw, w')}I.

Now,
2Re{(tw, w')}

= (tw, w') + (w', tw) = l

=

t(ww'*

+ w'w*)dt

tlw(t)121~~ -lIW(t) 12 dt = 0 -

IIw1l 2

This gives (3) in the case where w E Coo . We now show that (3) is true in the
general case. To see this, we first observe that it suffices to prove (3) in the case
where w belongs to the Hilbert space
H = {w

L~(lR); tw(t) E L~(lR), vw(v) E L~(lR)},

1

with the norm 11 w 11 H = (11 W11 2 + IItw 11 2 + 11 vwll2) 2: (if w(t) is not in this space,
Heisenberg's inequality is trivially satisfied). Then we use the fact that the subset of
H consisting ofthe Coo -functions with compact support is dense in H (see Exercise

C3.4). The result then follows from the continuity ofthe Hermitian product.
Equality in (1) takes place if and only if w(t) is proportional to a Gaussian signal
e- ct2 , where c > O. We do the proof only in the case where it is further assumed
that WES and is real. Observe that all the steps in the first part of the proof remain
valid since for such a function, tlw(t)121~~ = O. Equality is attained in (1) if and
only if the functions tw(t) and w'(t) are proportional, say,
w'(t)

=-

ctw(t),

and this gives

w(t) = Ae-ct2 ,

where c > 0 necessarily, because w(t)

EXERCISE

L2.

D1.3. Show that, for arbitrary to E lR, Vo E lR,

lI(t - to)wll

(Hint: Consider the function g(t)

~
1
lI(v - vo)wll ::: 4n

IIwll .

(4)

= e-2invot w(t + to).)

The above resulttells us that in Heisenberg's inequality (1), the numbers a w and
can be taken to be, respectively, the root mean-square width of w around any
time to and the root mean-square width of w around any frequency vo. In particular,
to can be the center of w(t), and Vo can be the center of w(v). This version is the
most stringent, since the RMS widths around the centers are the smallest RMS
widths (see Exercise D1.2).
aliJ

178

D 1. The Windowed Fourier Transforrn

D 1 2

The WFf and Gabor' s Inversion Formula

Windows

Let f(t) be the signal to be analyzed. The local information at (around) time t = b
is contained in the time-Iocalized signal
f(t)w*(t - b),

(5)

where w(t) is the time window function, the support of which is included in a
relatively small interval about zero. Typical examples are the rectangular window
w(t) = l[-a,+aj(t)

and the Gaussian window

wherea > O.
Given a window w(t), the local information at time b is obtained by computing
the Fourier transform of (5):
Wf(v, b):=

f(t)w*(t - b)e-2irrvt dt.

(6)

The choice of w* instead of w in (5) is purely for notational cornfort. For

example,
Wf(V, b)

= (j, Wv,b),

(7)

the Hermitian product in L~(lR) of f(t) with

Wv,b(t) = w(t - b)e2irrvt,

(8)

where we assume, of course, that fand w are complex-valued functions that have
finite energy.
L 2. One caUs the function Wf : lR x lR -+ C
defined by (6) the windowed Fourier transform (or WFT) of f associated with the
window w.
DEFINITION

DI.I. Let w, f

When the rectangular window is chosen, W f is called the short-time Fourier

transform of f. If the window is Gaussian, the function Wf is called the Gabor
transform of f.
Inversion Formula
THEOREM

DI.2. Under the foUowing assumptions,

(a) w E Lb(lR)

(b)

n LUlR),

Iw(t)1 2 dt

= 1,

D 1 2 The WFf and Gabor' s Inversion Formu1a

179

(c) Iwl is an evenfunction,

we have, lor all I

E L~(lR),

the energy conservation formula

LLIWf (v,b)1 2 dvdb = LI/(t) 12 dt,

(9)

and the reconstruction formula, (inversion formula),

lim ( I/(t)

Atoo

JR.

-1 JR.(

= O.

Wf(V, b)Wv,b(t) dv dbl2 dt

Ivl:",A

(10)

Note that assumption (c) is automatically satisfied if w(t) is areal function.

Assumption (b) is just a convention: The window can always be normalized to
have an energy equal to 1, and the wavelet transform is then just multiplied by a
constant.

Proof

The proof is technical and can be skipped in a first reading. We define

IA(t) =

1 JR.(

Wf(v, b)Wv,b(t) dv db.

Ivl:",A

Using the Plancherel-Parseval identity, (7) becomes

Wf(v, b)

= (I, Wv,b),

where
Wv,b(f.1,)

= e-2irr (/l--v)b W(Jl_ v)

(11)

is the Fourier transform of (8). Therefore,

Wf(v, b) = e-2irrvb L I(Jl)w(Jl- v)*e 2irr /l- b dJl.

(12)

The function Jl -+ I(Jl)w(Jl - v)* is in LI n L 2. (It is in LI as the product of

two L 2-functions; it is in L 2 because E L 2 and Wis bounded, being the Fourier
transform of an L I-function.) By the Plancherel-Parseval identity,

L IWf(v, b)1 2 db = L

=L

IL

I(Jl)w(Jl- v)*e 2irr /l- b dJl

db

1!(Jl)w(Jl - v)*1 2 dJl,

and, therefore,
L L IWf(v, b)1 2 dbdv = L L II(Jl)1 2Iw(Jl- v)1 2 dJldv

=L

{1!(Jl)1 2 L IW(Jl - v)1 2 dV} dJl

=L

{11(Jl)1 2 L Iw(v)1 2 dV} dJl.

180

D 1. The Windowed Fourier Transform

Equality (9) foHows because

Iw(t)1 2 dt = 1,

Iw(v)1 2 dv =

Ij(JL)1 2 dJL

IJ(t)1 2 dt (Plancherel-Parsevalidentity).

We show that the function JA is weH defined, that is, (v, b) -+ Wj(v, b)Wv,b(t) is
integrable over [- A, + A] x R In view of (12),
IA(t) := j+A { IWj(v, b)llwv,b(t)1 dv db

-A JIR

By Schwarz's inequality and the Plancherel-Parseval identity, and using assumption (b),

1.r-I{j(.)W(. - V)*}(b)IIW(t - b)1 db

:s (
=

2 )1/2 (

l.r-I{j(.)w(. - v)*}1 db

(L

Iw(t - b)1 2 db

)1/2

l.r-I{j(.)w(. - V)*}1 2 db )1/2

Ij(JL)w(JL -

v)*1 2 dJL

(lj12 * Iw 12 ) (v)

:= h(v).

This function h(v) is in LI, being the convolution product oftwo LI-functions. In
particular,
IA(t):s

+A
-A Ih(v)1 dv < 00,

and JA is therefore weH defined. Using a previous calculation, we have

JA(t)

where
g(v):=

+A
-A g(v) dv,

.r-I{j(.)w(. - v)*}(b)w(t - b)e2inv (t-b) db.

Dl2 The WFT and Gabor's Inversion Forrnula

181

By the Plancherel-Parseval identity,

g(v) =

=
Therefore,
fACt) =

!(f.1.)w(f.1. - v)*w(f.1. - v)e 2in /Lt df.1.

!(f.1.)lw(f.1. - v)1 2 e2in /Lt df.1..

i: (L
A

!(f.1.)lw(f.1. - v)1 2 e2in /Lt df.1. ) dv.

In order to change the order of integration in the above integral, we first verify that
(v, f.1.) -+ 1!(f.1.)llw(f.1. - v)1 2 is integrable over [- A, + Al x lR. But I!I E L 2 and
Iwl 2 E L 1 ; therefore, I!I * Iwl 2 E L 2 , and the integral of an L 2-function over a
finite interval is finite. But this integral is just

i: (L
A

1!(f.1.)llw(v - f.1.)1 2d f.1.) dv

i: (L
A

1!(f.1.)llw(f.1. - V)1 2d f.1.) dv

since Iwl is even. We can now apply Fubini's theorem to obtain

fACt)

L(i:
L (i:
=L
A

!(f.1.)lw(f.1. - v)1 2e2in /Lt dV) df.1.

!(f.1.)e 2in /Lt

Iw(f.1. - v)1 2 dV) df.1.

!(f.1.)C{JA(f.1.)e 2in /Lt df.1.,

where 0:::: C{JA(f.1.) :::: 1, in view of assumption (b). In particular, !C{JA

fA

= F-

E L 2,

and

(jC{JA).

We now show that limA too fA = f in L 2. For this, we write, using the PlancherelParseval identity,
IIf -

fAlli2 = IIF- 1!

- F- 1!C{JAlli2

= IIF- 1{!(1 -

C{JAmi2

Wehave

/L-A

-00

Iw(Y)1 2 dy

1+00
/L+A

Iw(Y)1 2 dy,

182

D 1. The Windowed Fourier Transform

and, therefore, if I/LI ::; A12,

o ::; 1 ::;

((JA(/L)

-A 12

-00

Iw(Y)1 2 dy

= y(A) -+

1+00 Iw(Y)1

AP

dy

(A -+ (0).

Also,

-+ 0

sinee

(A -+ (0)

L 2 . Therefore, finally,

lim

Atoo

111 - IAlli2 = O.

From (7) and the Planeherel-Parseval identity, we obtain the two expressions
for the windowed Fourier transform:
(13)

where WV.b is defined by (11). We assume (without diminishing the generality of

the diseussion to follow) that w and W are funetions eentered at zero, that is,

tlw(t)1 2 dt

=0

and

vlw(v)1 2 dv

= O.

The RMS width a w is an indicator of the loealization of Wv.b about b, whereas

ofthe loealization of WV.b about v. The reetangle [b - a w , b +
+ aJij] is the loeal time-frequeney box about (b, v) analyzed by
the windowed Fourier transform at (b, v).
It is of interest to have a sharp resolution, that is, to make the area 4awaJij of
the time-frequeney box as small as possible. However, windows have the basic
limitation eontained in the uneertainty prineiple, whieh says that
aJij is an indieator
a w ] x [v - aJij, v

(14)

awaJij:::: 4n '

with equality if and only if w(t)

shows that.

==

Ae-ct2 ,

where c

4n2a~. The last result

D1.3. The Gabor window is optimal, in the sense that it minimizes the
uncertainty aJijaw.

THEOREM

Dl2 The WFf and Gabor's Inversion Formu1a

183

The windowed Fr is a continuous transform, in that the local time-frequency

content of a signal is contained in a function of two continuous arguments. It would
be interesting to have a discrete version, that is, a decomposition of the signal along
a Hilbert basis. More specifically, one asks the question: Is there a window w(t)
such that the family {lj!m,n}mE71,nE71, where

lj!m,n(t)

= e2irrmtw(t -

n),

(15)

is an orthonormal basis of L 2 (ffi.)?

01.1. Show that the answer to the previous question is positive for the
rectangular window w(t) = 1[0,1](t).

EXERCISE

Although such "atomic" windowed Fr bases do exist, they turn out to be very
bad from the view point of time-frequency resolution, as the following result,
called the Balian-Low theorem,5, shows.
THEOREM 01.4.

If{ lj!m,n}mE71,nE71, where lj!m,n is defined by (15)(with gEL 2 (ffi.),

is an orthonormal basis of L 2 (ffi.), then at least one of the following equalities is
true:

t 2Iw(t)1 2dt =

EXERCISE

00

or

v 2Iw(v)1 2dv =

00.

01.2. Show that the system {lj!m,n}mE71,nE71, where

lj!m,n(t)

= e2irrmt e-a (t-n)2,

()( > 0, is not an orthonormal basis of L 2 (ffi.).

SR. Balian, Dn principe d'incertitude fort en theorie du signal et en mecanique quantique,

CR Acad. Sei. Paris, 292, Sero II, 1981, 1357-1361; F. Low, Comp1ete sets of wave packets, A Passion for Physics-Essays in Honor of Geoffrey Chew, 17-22, World Scientific:
Singapore, 1985.

D2
The Wavelet Transform

D21

Time-Frequency Resolution of Wavelet

Transforms

Definition of the Wavelet Transform

We mentioned in the introduction to Part D the shortcomings of the windowed

Fourier transform. This chapter gives another approach to the time-frequency
issue of Fourier analysis. The role played in the windowed Fourier transform by
the family of functions
Wv,b(t) = w(t - b)e+ 2i :rrvt,

b, v E IR,

is played in the wavelet transform (WT) by a family

1/Ia,b(t) -_

lai -1/2 1/1

(t - b)
-a-

a, b

IR, a =I 0,

(16)

where 1/I(t) is called the mother wavelet. The function 1/Ia,b is obtained from the
mother wavelet 1/1 by successively applying a change of time scale (accompanied
by a change of amplitude scale in order to keep the energy constant) and a time
shift (see Fig. D2.1).
DEFINITION D2.1. The wavelet transform ofthe function f E

C j : (IR - {On x IR

1-*

C defined by

Cj(a, b)

= (j, 1/Ia,b) =

P. Brmaud, Mathematical Principles of Signal Processing

Springer Science+Business Media New York 2002

LUIR) is the function

f(t)1/I:,b(t)dt.

(17)

186

D2. The Wavelet Transform

1lV2
1
1
1
1

1
1
1
1
1
3.51 1
1 1

,-------, 1

1
-1

1y'2
1

0 11

1+1
1
1

-1 L - J

17.5
1

-y'2

1 1

-V2 L.....

'ljJ(t)

'ljJ2,7.5(t)

'ljJ! ,3.5 (t)

Figure D2.1. Dilations and translations

The Adaptive Time-frequency Box
By the Plancherel-Parseval identity,

C j(a, b) =

(j, Vla,b)

(18)

j(v) VI;,b(V) dv,

where
(19)

Let m", and a", be, respectively, the center and RMS width of the mother wavelet
1/1 , respectively defined by
m",

= -1

E",

1lR
I

11/I(t)1 2 dl,

aJ = _1_
r(I E", llR
and similarly define m;;; and a;;;, where
and width of 1/1a,b are, respectively,

m",)211/1(1)1 2 dt,

VI is the Fourier transform of 1/1. The center

b+am""

aa""

whereas the center and width of Vla,b are

1
a
We shall simplify notations by writing

-m;;;,

1
a

-a;;;.

We see that C j(a, b) is the result of the analysis of the function

frequency box (see Fig. D2.2)

[b+am-aa,b+am+aa]x

[ma -

~a

m
+ ~J.
a
a

in the time-

D22 The Wavelet Inversion Forrnula

187

Figure D2.2. Time-frequency tiling in wavelet analysis

Let us assurne that fii > O. The frequency window is then centered at v = fii / a
and has width 2a / a; therefore,
Q-

center frequency
fii
-bandwidth
- 2a

is independent of the frequency variable a. This is called constant-Q jiltering.

Calling v = fii / a the center frequency, we see that the area of the box is constant
and equal to 4a but that its shape changes with the frequency v = 1/a analyzed.
For high frequencies it has a large time dimension, and for small frequencies it has
a small time dimension (see Fig. D2.2). The interest of such features is discussed
in the introduction to this chapter.

a,

We shall see in the next subsection that in order to guarantee perfect reconstruction of the signal from its wavelet transform, the center of :(f must be zero. Also,
the center of the wavelet itself can be taken equal to zero without loss of generality.
The Fourier transform of a wavelet has bumps at positive and negative frequencies
(see Example D2.3, the Mexican hat). The centers of the bumps then play the role
of the center of the wavelet in the first part of the above discussion (where fii was
assumed to be nonzero ).

D22

Wavelet Inversion Formula

Under mild conditions, there exists a wavelet inversion formula similar to the WFT
inversion formula.
THEOREM

D2.1. Let 1jJ : ~

f-+

~ be a mother wavelet such that 1jJ E LI

11jJ(t)1 2 dt

1,

n L 2,
(20)

188

D2. The Wavelet Transfonn

and

1~(v)12 dv =

K <

lvi

JR.

00.

(21)

Define 1/1a,b E L 2 n LI by (16). To the function J E L 2 is associated its wavelet

transJonn C j : IR\{O} X IR -+ C, defined by (17). Then

{ IC j(a, b)1 2 da ~b = ( IJ(t)1 2 dt

a
JR.

K JR.\/OjJR.

(22)

and

(C j(a, b)1/Ia,b(t) da ~b ,

J(t) = K1 {

(23)

JR.\/OjJR.

in the sense that Je -+ J in L 2 as e -+ 0+, where

Je(t) = -1 ~~
K

dadb
C j(a, b)1/Ia,b(t) 2a

R.x/lal:::eJ

Proof' First observe that 1/Ia,b has the same energy as

From (18) and (19),
C j(a, b)

= lal 1j2

1/1, equal to unity.

f !(v)~*(av)e-2i7rvb

dv

The function inside the curly brackets is in L I because it is the product of two
L 2-functions, and it is in L 2 as it is the product of an L 2-function with a bounded
function (~ is bounded because 1/1 E LI). By the Plancherel-Parseval identity,

L'C j(a, b)1 2 db

= laILIF;I{!(v)~*(av)}12 (b) db
= laILI!(v)121~(av)12 dv,

and therefore,

11

R. R.

dbda
IC j(a, b)1 2 2- =
a

11

R. R.

~
2
da
IJ(v)1 2 11/I(av)1
dvla I
A

= LI!(V)1 2 K

dv

=K

LIJ(t)1 2 dt.

D22 The Wavelet Inversion Formula

189

This proves (22). To prove (23), first compute

C j(a, b)1/!a,b(t) db

l(a) =

lal l / 2

L
L

F;I{!(v)V;*(av)}(b)1/!a,b(t)db

= lal l / 2

!(v)V;*(av)F;I{1/!a,b(t)}(v)dv,

where we have used the Planchere1-Parseval identity. Now,

= lal l / 2 V;(av)e 2i Jl'vt,

F;I{1/!a,b(t)}(V)

and therefore,

and, for c > 0,

J(t) =

l(a)

( { !(v)IV;(av)1 2 e2i Jl'vt dV) da .

K J11al?}

d~
a

lai

K J11al?} JR

With a view to applying Fubini's theorem we must check that the function
~

(a, v) -+ IJ(v)II1/!(av)1 ~

is integrable in the domain lR. x {la I ~ c}. We have

1

11

R lal?

IJ(v)II1/!(av)1 -

lai

JR

1+ + {

11

1+

J1VI?1

Ix I

= 11

1V;(x)1 2

J1xl?IVI

+1

-I

1!(v)1 dv <

dx) dv

+h

1!(v)1 ( (

-I

:s K

1V;(x)1 2

(
J1xl?lvl

-I

But

lai

J1al?

= { 1!(v)1 (

da dv

1V;(av)1 2 da) dv

= { 1!(v)1 ( (
JR

00

lxi

dX) dv

(25)

190

D2. The Wavelet Transfonn

because!

L2e~) and, in particular, !

lz =

l!ev)1

Ivl:::l

(1

L 2 e[ - 1, + 1]). Also,
2

11/Iex)1 dX) dv
Ixl:::slvl Ix I

But

l11/let)1 2 dt = 1,
and, therefore, using Schwarz' inequality,

lz

< -1

(1

<

l!ev)1 dv
lvi

- e Ivl:::l

- c

IJev)1 2 dv )1/2
A

Ivl:::l

(1

-dV)I/2
Ivl:::l v 2

We can therefore change the order of integration in (25):

~

KJset)

= { !ev)e 2i ;rvt ( ( 11/Ieav)1 da) dv

JlR
Jlal:::s
lai

!ev)e 2i ;rvt gsev)dv

= F-1{!gs}(t).

Since we want to prove that Js -+ J in L 2 , we must evaluate the L 2 -norm of

J- Js:

K 2 11J

- Jsll 2 = K 2 I1F- 1{j

Js}1I 2 = lI!eK - gs)1I 2

1 1

= { IK - gsev)1 2 1!ev)1 2 dv =
+
= A + B.
JlR
Ivl:o:eIvl>e1/ 2

On

{lvi

:s c- I / 2 },
gsev) = {
Jlal:::s

l1freav)1 2 da
lai

= {
Jlxl:::SIVI

l1frex)1 2 dx
lxi

1/ 2

D22 The Wave1et Inversion Formu1a

191

where

Ke

:s K

and

lim K e

e-+O+

= K.

Therefore,

Also,

since

L 2 . We have therefore proved that

IIf - fell

--+ 0

as

--+ 0+.

The proof is lengthy because we have only required

D2.2.
for almost all t.
THEOREM

Proof"

If f

L 2 n LI and

We start from (24):

[Ca) -da

Ja

a2

11

Ja Ja

LI, the inversionformula (23) is true

~
~
2 2
dvda
f(v)Il{!(av)1
e "rvt -

a2

= [ j(v)e2irrvt ( [
lJa

= K

f to be in L 2

lJa

1~(av)12 da) dv
lai

g(v)e2irrvt dv.

This quantity is almost everywhere equal to K f (t) by the Fourier inversion formula
in LI.

Recall that if f is continuous the equality in the Fourier inversion formula holds
for all t and, therefore, (23) is then true for all t E R
Oscillation Condition

Since ~ is continuous (l{!

is, to say,

LI), the assumption (21) implies that ~(o) = 0, that

l{!(t) dt

= o.

(26)

192

D2. The Wavelet Transfonn

In most situations it suffices to verify (26), and then (21) follows. For example, if
1/I(t) and t1/l(t) are integrable, then Vi is Cl; therefore, if Vi(o) = 0, the quantity
IVi(v)1 2 /lvl is integrable in a neighborhood of zero and therefore on ~, since at
infinity there is no problem, due to the hypothesis 1/1 E L 2 (which implies that

Vi E L2 ).

EXAMPLE

02.1 (Modet's pseudo-wavelet).

1/I(t)

= ye-

t2 / 2

Morlet used the mother wavelet

cos(5t),

where y is a normalization factor that makes the energy equal to unity. The
theoretical problem here is that

Vi(O) =

1/I(t) dt > O.

However, the numerical results obtained with this wavelet were satisfactory
because the value of Vi(O) is in fact very smalI.
EXAMPLE

02.2 (Haar wavelet).

The Haar wavelet

ifO ~ t < ~,
if ~ ~ t < 1,

otherwise,
satisfies the conditionsfor the reconstructionformula (23) to be valid. Here
~

1/I(v)

= le. -;rrv

1 - cos(7l' v)
7l'V

EXAMPLE 02.3. In practice, a mother wavelet 1/1 should be weil localized in

addition to Vi, and it should also be oscillating (so as to guarantee at least that
1/I(t) dt = 0). Derivatives ofthe Gaussian pulse are goodfor this purpose. For
example, the second-order derivative, called the Mexican hat (see Fig. D2.3),

IR

1/I(t) "" (1 - t 2 )e- t2 / 2

with the Fourier transform
~

1/I(V) "" v eis interesting because both

2 2 2
rr v

1/1 and Vi are rapidly decreasing C OO -functions.

We shall now give a pictorial example. Fig. D2.4 shows a simple signal and
Fig. D2.5 shows its wavelet transform. The mother wavelet used is not given,
since it is irrelevant to this qualitative illustration. In the latter figure, the time
axis is horizontal, and the time axis vertical, the bottom part corresponding to
high frequencies. We observe the good time localization and the fact that sharp
discontinuities are represented in the bottom part.

D22 The Wavelet Inversion Formula

193

~ilr-----~----II
(a)

-10

-8

-6

-4

-2

JJ(/:::

10

(b)

-10

-8

-6

-4

-2

1
10

(c)

j:~
-10

-8

-6

-4

-2

Figure D2.3. The Mexican hat

10

194

D2. The Wavelet Transform

0.8

0.6

0.4

0,2

01 - - - - - - - '

~.2

__

500

L _ _ _ _ _- - - '

__
__
___
__
__
__
__
1000
1500
2500
4000
2000
3000
3500
~

Figure D2.4. Spike

+ sinusoid

Figure D2.5. The wavelet transform of the signal in Fig. D2.4

~~

D3
Wavelet Orthonormal Expansions

D31

Mother Wavelet

The wavelet analysis of Chapter D2 is continuous, in that the original function of

L 2 is reconstructed as an integral, not as a sumo One would rather store the original
function not as a function of two arguments, but as the doubly indexed sequence of
coefficients of a decomposition along an orthonormal base of L 2 Multiresolution
analysis is one particular way of obtaining such orthonormal bases.
In the remainder, we adopt a slightly different definition of the Fourier transform.
The FT j(w) ofthe signal f(t) is now defined by
j(w)

f(t)e- iwt dt.

The inversion formula, when it holds true, takes the form

f(t) = _1 [j(w)e iwt dw, a.e.,

2n

JR

and the Plancherel-Parseval identity, when it holds true, reads

[ f(t)g(t)* dt = _1 [j(w)g(w)* dw.

JR

2n

JR

Also, the necessary and sufficient condition for {cp( . - n) }nEZ to be an orthonormal
sequence of L~(IR) (Theorem C4.5) now reads

L 1<7J(w + 2kn)1

= 1, a.e.

kEZ

P. Brmaud, Mathematical Principles of Signal Processing

Springer Science+Business Media New York 2002

196

D3. Wavelet Orthonormal Expansions

One reason for abandoning the definition in terms of the frequency v is that
the topic of MRA involves a mixture of analog signals and of digital filtering,
and digital signal processing is traditionally-as in the present text---dealt with in
terms of the pulsation w.
Scaling Function and MRA
DEFINITION D3.1. A multiresolution analysis (MRA) of L 2 = L~(IR) consists of a
function cp E L 2 together with afamily {Vj}jez of Hilbert subspaces of L 2 such
that

(a) {cp(. - n)}neZ is an orthonormal basis ofVo,

(b)for all j
(c) fE Vo

Z, Vj

{::=}

Vj+1 (the Vj 's are said to be nested; see Fig. 4.7),

f(2 j .) E Vj ,

(d) njez Vj = 0 and c10S(Ujez Vj ) = L 2 .

The function cp is called the scaling function of the MRA. The index j represents
a resolution level: The projection Pj f of a function f E L 2 on Vj is interpreted
as the observation of this function at the resolution level j.
Usually, the projection on Vo is the function itself, in which case the projections
at all levels j 2: 0 are identical. The projection at level 0 is, in applications, the
one offered by the recording device.
Observe that, since the mapping f -+ ,J2 f(2 ) is an isometry from Vo onto VI
and since (cp(. - n) }neZ is an orthonormal basis of Vo, the set (,J2 cp(2 . - n) }neZ
is an orthonormal basis of VI. More generally, {CPj,n}nez is an orthonormal basis
of Vj , where
(27)

With respect to (d), recall that (Exercise C1.6)

nV
jeZ

= 0

{::=}

lim P_j/ = 0

J .... +OO

for all fE L 2

Figure D3.1. Nested subspaces

(28)

D31 Mother Wavelet

197

](1/)

-7f

j(.) E Va

1/

+7f

}(I/)

~.
o
+27f

1/

Vi

j(.)

j(.)

E V- 1

](1/)

1/
Shannon multiresolution
Figure D3.2. Nesting in the Shannon MRA

and
closU Vj = L 2
jE71
EXERCISE

{:::::::}

.lim Pj / = /

J-->+OO

D3.1 (Shannon scaling function).

Vj

= {f E

L 2 : supp (1)

for all /

L2

(29)

For alt j E Z, define

c [- 2 j 1T, + 2 j 1T]}.

Define the function rp by its Fourier trans/orm

$(w)

ll[-rr,+rrj(w)

(see Fig. D3.2). Verify that {Vj }jE71 is a multiresolution analysis 0/ L 2 associated
with the scaling function rp.
EXERCISE

Vj

D3.2 (Haar scaling function).

= {f E

L2

: /

For alt j E Z, define

is a.e. constant on the intervals (kTj, (k

+ l)Tj]).

Define
rp(t)

ll(o,l](t)

(see Fig. D3.3). Verify that {Vj }j E71 is a multiresolution analysis 0/ L 2 associated
with the scaling function rp.

We shall see later that some regularity of the scaling function is desirable.

198

D3. Wavelet Orthonormal Expansions

,...........J--I
L...I

Vo

j(.) E V-I

16

j(.)

Haar multiresolution
Figure D3.3. Nesting in the Haar MRA
D3.2. The function ({J is said to belong to Sr for some rEN if ({J is r
times continuously differentiable with rapidly decreasing derivatives, in the sense
that

DEFINITION

I({J(k)(t)1

:s

C kp

= 0, 1, ... , r,

and all p

N.

This is a multiresolution analysis for wh ich the scaling function ({J

r-smooth.

sr is called

(1

+ Itl)P

for k

(30)

The Haar and Shannon scaling functions are not in Sr (for any rEN).
Conditions (a), (b), (e), and (d) of Definition 4.3 are not independent. In fact,
the first part of (d) is always true under conditions (a), (b), (e), whereas the latter
conditions are almost sufficient for the second part of (d). The result below makes
this statement precise.
D3.1. Suppose that eonditions (a), (b), and (e) of the definition of an
MRA are satisfied. Then njEZ Vj = 0. Moreover, ifep is eontinuous at the origin,
then
THEOREM

lep(O) I =1= 0

{::=>

closU Vj

= L~(l~).

(31)

jEZ

In this ease, neeessarily, I?CO) I = 1.

Proo!" The first statement will be proven in the more general Theorem D4.1.
We now prove the second statement. (The proof is technical and can be skipped
in a first reading.) Denote by Ta the translation operator defined by Taf(x) =

(x - a). We shall first show that the Hilbert space W

invariant under translations.

clos ( Uj EZ Vj ) is

D31 Mother Wave1et

We begin with dyadic translations (a = m2-c, where e, mEZ). Let

Therefore, for any given c: > 0, there exist jo E Z and h E Vjo such that 11 f
c:. For all j ::: jo, h is also in Vj . In particular,
h

= L c{q;(2 j

199

f E W.
- h 112 :s

-k),

kE'lL

and the function

Tm2 -(h

L c{q;(2

j .

-2 j - Cm - k)

kE'lL

is therefore in Vj if j :::

e (because 2 j - Cm is then in Z). Therefore, for all j

and Tm 2-( h E Vj This means that Tm 2-( f is c:-c1ose to Vj for all j :::
and the arbitrariness of c:, we deduce that Tm 2-( f E W.
Let now a
c E (a - 8, a

:::

e,

e. From this

lR be arbitrary. Given c: > 0, there exists 8 such that, for all

+ 8),

(use Theorem C3.1 stating that the map a 1--+ Taf is uniformly continuous). In
particular, we can find a dyadic number c for which the above inequality is satisfied.
Since Tc! E W and c: is arbitrary, we deduce that Ta f E W.
We now proceed to the proof of (31). We assurne that ~is continuous at 0 and
that I~(O)I -=1= O. Therefore, ~(w) -=1= 0 on (-c, +c), for some c > O. Consider any
function g orthogonal to W, that is, orthogonal to all f E W. Since W is invariant
under translations, for all x E lR and for all f E W,

L
L

0=

f(x

+ t)g(t)* dt.

By the Plancherel-Parseval identity,

0=

eiwx j(w)g(w)* dw,

for all x ER The function jg* E L~(lR), and therefore, by the Fourier inversion
theorem in L I , j g* = 0 almost everywhere.
In particular, with f(t) = 2 j q;(2 j t) (indeed, such f E Vj C W), we obtain
~(Tjw)g(w)*

= 0,

a.e.

Since ~(2-jw) -=1= 0 if w E (-2 j c, +2 j c), we have that g(w) = 0 if w E

(-2 j c, +2 j c). This being true for all j E Z, we have that g, and therefore g,
is almost everywhere null. We have thus proven that the only function in L~(lR)
orthogonal to W is the null function. Therefore, W is exactly L~(lR).

200

D3. Wavelet Orthonormal Expansions

Assurne now that W

In particular,

= L~(IR). Let 1 be the function with the Fr j = 1[-1.+1].

By (29),
lim 111 - Pj/1I2

jt+ oo

= 0,

and therefore, by continuity of the norm,

.

.hm

Jt+ oo

1
rr

IIPj/1I 2 = 1I/\1z =-,

that is,

Itim 11

+00

LU, qJj.k)qJj,kiI~ = ~.rr

ke'Z

We have by the Plancherel-Paseval identity for orthonormal bases (Theorem C2.2),

11

LU, qJj,k)qJj,kll~ = LI (l(t)qJj,-k(t)* dtl2

ke'Z

ke'Z JIR

The last sum equals, by another Plancherel-Paseval identity,

For large enough j, [-2- j, +2- j] c [-rr, +rr], and therefore the last displayed
expression is 2 j times the sum of the squared absolute values of the Fourier
coefficients of 1[-2-i,+2-ijr. Therefore, by the appropriate Plancherel-Parseval
identity,

L 1_1 j+2-i e-ikwq;(w)* dwl2 = _1 j+2-i \q;(w) \2 dw = ~.

ke'Z

2rr

-2-j

2rr

-2-j

rr

Therefore,

By continuity of q; at 0, this limit is also ~ \qJ(0)\2. Therefore, \qJ(0) \ = 1.

D31 Mother Wavelet

201

Wavelet Expansion

We shall suppose in the sequel that the scaling functions cp satisfy 1qJ'(0) I > 0, and
then take (without further loss of generality)
qJ'(0) = 1.

(32)

DEF1NITION D3.3.

A wavelet orthonormal basis 01 L 2

basis olthelorm {1/Ij,n}j,nez, where

= L~(lR) is an orthonormal
(33)

The function
expansion

1/1 is then called the mother wavelet of the wavelet basis. The

1 = LL(f, 1/Ij,k)1/Ij,k
jeZ

(34)

keZ

is called the wavelet expansion of I.

A wavelet orthonormal basis can be obtained from an MRA in the following
way. Let Wj be the orthogonal complement of Vj in Vj+l:
(35)

Vj+l = Vj EB Wj.
From property (d) of the definition of MRA,
L2

= EBWj .

(36)

jeZ

Also, from (e),

1E

Wo ~

1(2 j .) E

Wj.

Therefore, ifwecan exhibit an orthonormal basis of Wo ofthe form {1/1( . -n)}nez,

then {1/1 j.n }neZ is an orthonormal basis of Wj. Therefore, by (36) {1/1 j,n b,nez is an
orthonormal basis of L 2
Recall that Pj is the projection on Vj . We have
Pj+I!

= Pj/ + L (f, 1/Ij.k)1/Ij,k

keZ

for all

1E

L2.

(37)

Pj 1 is the result of observing 1 at the resolution level j: As j increases, the

resolution increases (note that Vj C Vj+l); the difference

Pj+I! - Pj/

= L(f, 1/Ij,k)1/Ij.k
keZ

is the additional detail required to pass from the resolution level j to the higher
resolution level j + 1.
A first issue is: How to compute the mother wavelet 1/1 from the scaling function
cp? The next question is: How to obtain a scaling function cp? Finally, one would
like to obtain a mother wavelet with "good" numerical properties, that is fast
convergence of the wavelet expansion (34).

202

D3. Wavelet Orthonormal Expansions

D32

Mother Wavelet in the Fourier Domain

We address the first issue, that of explicitly finding a mother wavelet given a scaling
function.

D3.1. We seek to obtain the mother wavelet corresponding to the Haar

scaling function. Recall that VI is the Hilbert space of L 2 -functions that are constant almost everywhere on the intervals (nI2, (n + 1)/2]. The mother wavelet 1/1
must be ofthis type since Wo C VI. Thefunction ofnorm 1, with support (0,1],
EXAMPLE

1/I(t)

!],

+1

ift

(0,

-1

ift

(!' 1],

does it. To see this, it suffices to verify that any f E VI with support (0, 1] is a
linear combination of cp and 1/1 and that cp and 1/1 are orthogonal. Orthogonality
is obvious. Any fE VI such that supp(f) E (0, 1] is oftheform
f(t)

!],

ift

(0,

ift

(!' 1].

and we therefore have the decomposition (see Fig. D3.4)

a+

a-

f = - 2 - cp + -2-1/1
The function 1/1 is called the Haar mother wavelet.

4" ----,

l---

f(t)

L --

4"

01

1
2

~x
4

'Cl

'ljJ(t)

+1

lx
2

-1

21
1
1
1
1

--~

Figure D3.4. Haar decomposition

D32 Mother Wavelet in the Fourier Domain

203

We now give a simple example of wavelet analysis to illustrate the notions

of projection and detail. Fig. D3.5a gives (from top to bottom) a signal and its
successive projections on the nested subspaces at decreasing resolution levels,
whereas Fig. D3.5b gives (from top to bottom) the successive projections on the
detail subspaces at decreasing resolution levels. In particular, the second function
in Fig. D3.5b is the difference between the first and second functions in Fig. D3.5a.
The general case will now be treated. We shall obtain a necessary condition for
the scaling function ({J to be a scaling function. First, since {({J(' - n)}nEZ is an
orthonormal system, we have, by Theorem C4.5,

L I$(w + 2kn)1

= 1,

a.e.

(38)

kEZ

The scaling function ({J E Vo and therefore, ({J E VI. Requirements (a) and (e) in
the definition of an MRA imply that {({JI,n}nEZ is a Hilbert basis of VI. and we
therefore have the expansion ({J = LnEZ hn({JI,n, that is,
({J

= v'2

L h n({J(2 . -n),

(39)

nEZ

where
(40)

In the Fourier domain (39) reads

~
((J(w)

1M ~hne'""'
inW~(W)
'i({J "2

-v2

nEZ

'

that is,
(41)
where mo(w) is the 2n-periodic function defined by
mo(w)

1M ~hne
'""'

-v2

-inw .

(42)

nEZ

It is called the low-pass filter MRA, because mo(O) = 1 (recall the running
assumption that $(0) = 1; see (32)). Substituting identity (41) in (38) gives

204

D3. Wavelet Orthonormal Expansions

300~

200

100
O~

__

____- L____

50

100

::~'

____L -_ _

150

200

'~'

_ _ _ _- L____~____L-~~____~

250

100

150

400

450

500

AM' '~
,~-, LJJ

- L_ _ _ _~_ _ _ _~_ _ _ _L -_ _ _ _L -_ _~~

- J_ _ _ _~_ _ _ _- L_ _ _ _

50

350

'''A~'

100~~-,
__
-L____
O~

300

200

250

300

350

400

450

500

300~

200

100
O~

__- J_ _ _ _
50

_ _ _ _- L_ _ _ _~_ _ _ _- L_ _ _ _~_ _ _ _~_ _ _ _~_ _ _ _~_ _~~

100

150

200

250

300

350

400

450

500

300~
200
100
O~

__

____- L____

50

100

150

____L -__

200

____- L____

250

300

____L -_ _

350

400

____

450

500

300~

200

100
O~

__- J_ _ _ _
50

_ _ _ _- L_ _ _ _- L_ _ _ _- L_ _ _ _~_ _ _ _~_ _ _ _L -_ _ _ _~_ _~~

100

150

200

250

300

350

400

450

500

300

350

400

450

500

(a)

50

100

150

200

250

,oo~
-10:
50

100

150

200

250

300

350

400

450

500

,oo~
-10:
50

100

150

200

250

300

350

400

450

500

100~
-10:
50

100

150

200

250

300

350

(b)

Figure D3.5. Haar wavelet analysis

400

450

500

D32 Mother Wavelet in the Fourier Domain

205

Therefore,

or, equivalently,
(43)
The filter with frequeney response eiwmo(w + n)* is ealled the high-pass filter
of the MRA. Eqn. (43) shows that the high-pass and the low-pass filters altogether
extraet the whole energy eontained in the band [-n, +n].
We now eharaeterize the spaees V-I and Vo. This will be a preliminary to the
eharaeterization of W_I, the orthogonal eomplement of V-I in Vo. Onee this is
done, we shall obtain the eharaeterization of Wo and then the mother wavelet
itself.
LEMMA

D3.1. f E V-I

if and only if it has an FT of the form

f(w)

= m(2w)mo(w)qJ'(w),

for some 2n -periodic function m

Proof" Indeed, any f

{CP-I,n}nEZ, that is,

E l~.

+n]).

V-I ean be deeomposed along the orthonormal basis

f(t)
where {Cn}nEZ

E L~ ([ -n,

(44)

1
I>kCP( -t - k),
",2 kEZ
2

(45)

Taking the FT, we obtain

f(w)

= hI>ke-i2kWqJ'(2w).
kEZ

This is (44) (using (41) and defining m(w)

= v'2 LkEZ Cke-ikw).

Conversely, eonsider a funetion defined by (44), where m is a 2n-periodie

funetion in L~([-n, +n]). We show that fis in L~(lR.). First, observe that it is
of the form h(w)qJ'(w), where h is a 2n-periodic funetion in L~([ -n, +n]) (sinee
m E L~([ -n, +n]) and sinee mo is bounded in view ofEq. (43. Now
[ Ih(w)qJ'(w)1 2 dw

JR

=L

j+Jr Ih(w)1 21qJ'(w + 2k1T)1

j +Jr 2

kEZ

-Jr

-Jr

Ih(w)1 dw <

+00.

dw

206

D3. Wavelet Orthononnal Expansions

This proves that E L~(lR). Since

E L~(lR), it is the Fr of a function I E
LUlR). Tracing back the computations in the first part of the proof, we obtain that
(45) holds true, with {cn}nEZ E e~, which implies that I E V-j.

LEMMA

D3.2.

Vo

if and only if it has an FT 01 the lonn

f(w) = d(w)$(w),

lor some 2rr -periodie function d

Proof Indeed, let
{fPo,n }nEZ, that is,

(46)

L~([ -rr, +rr]).

Vo. It can be decomposed along the orthonormal basis

= L dkfP(t -

I(t)

k),

(47)

kEZ

where {dn}nEZ

e~. Taking the Fr, we obtain

f(w) =

L dkeikwqJ(w) = d(w)$(w),
kEZ

where d E L~([-rr, +rr]). Arguing as in the proof ofLemma D3.l, we can show

that any function of the form (46) is the Fr of a function I E Vo.

Consider the mapping U : Vo !--+ L~([ -rr, +rr]) defined by UI = d (where d

is defined by (46)). Clearly, this mapping is linear, and
IIUfllh([-rr,+rr])

= IIdlli~([_rr,+rr]) = 2rr

By the polarization identity, for all

(j, g) L~(R)

I, g E

L Id l

k 2

kEZ

= 2rrll/ll~.

(48)

Vo,

= 2rr (UI, U g) L~([-rr,+rr])'

(49)

We are now ready to state and prove the Fourier characterization of Wo, the
Hilbert space of details at level O.
THEOREM

D3.2. The function

f(w)

Wo

if and only if

= ei~mo(~ + rr)* V(w)$(~) ,

(50)

lor some 2rr -periodie function v in L~( -rr, +rr).

Proof

Observe that it is equivalent to prove that the function lEW_I if and

only if
(51)

for some 2rr-periodic function v in LU -rr, +rr).

Let lEW-I, that is, I E Vo and 1..1 V-I. Being in Vo, I has a representation
oftype (46). By (49) and the characterization (44) of V-I, the orthogonality of I
and V-I is equivalent to

0=

(2rr

10

d(w)m(2w)*mo(w)* dw,

D32 Mother Wavelet in the Fourier Domain

for all2Jr-periodie funetion m

0=

E L~([ -Jr,

207

+Jr]). This ean also be written

Ln: m(2w)* [d(w)mo(w)* + d(w + Jr)mo(w + Jr)*] dw.

The funetion in the square braekets is therefore orthogonal to all g

and therefore,

E L~([O,

d(w)mo(w)* + d(w + Jr)mo(w + Jr)* = 0

+Jr]),
(52)

almost everywhere in [0, +Jr] (and therefore almost everywhere in [-Jr, +Jr)).
Define
mo(w) = (mo(w), mo(w + Jr.

In view of the identity (43), this is a unitary veetor in C 2 eonsidered as a 2dimensional veetor spaee (with sealar field C). The veetor
m'o(w) = (mo(w

+ Jr)*, -mo(w)*)

is unitary and orthogonal to mo(w). Defining

do(w) = (d(w), d(w + Jr,

we have, by (52), do.lmo(w). Therefore,

do

= (w)m'o(w),

where
(w) =

(do, m'o(w)

= d(w)mo(w + Jr) - d(w + Jr)mo(w).

In partieular,
(w + Jr)

= -(w + 2Jr),

a.e.

or, equivalently,
(w) = -(w + Jr), a.e.,
whieh implies in partieular that is 2Jr-periodie. It is also in L~([ -Jr, +Jr)).
Indeed,
j(w) = d(w)qJ(w) = (w)mo(w + Jr)*qJ(w),

and therefore,

1n: I(w)1 2 dw = 111: I(w)12(lmo(w)12 + Imo(w + Jr)1 2) dw

208

D3. Wavelet Orthononnal Expansions

where the last equality follows from (48). Defining

gives the representation (51).

Conversely, suppose that
j(w) = eiwmrf,w

+ n)*v(2w)qy(w),

for some 2n-periodic function v in L~([ -n, +n]). That is,

j(w) = d(w)ifI..w),

where
d(w)

= eiwmriw + n)*.

Since Imo(w)1 ::: 1, this implies that d(w) E L~([-n, +n]). Therefore, I E Vo
(Lemma D3.2). Also, from the expression of d(w), do(w) = eiwv(w)m'o(w), and
therefore
do..lmo(w),

that is, d(w)mo(w)* + d(w + n)mo(w + n)* = O. By Lemma D3.1 and Eq. (48),

this implies that 1.1 V-I. But also I E Vo. Therefore, I E Wo.
We are now ready for the main result of this subsection, the Fourier characterization of the mother wavelet in terms of the scaling function and of the high-pass
filter.
THEOREM

D3.3. The junction 'tjJ is a mother wavelet if and only if

:V;(w)

= eiW/2mo(~ + n V(w)qy(~) ,

lor some 2n-periodicjunction v such that Iv(w)1

(53)

= 1 almost everywhere.

Proof" Since:V; is of the form (50) with lvi = 1, it is in L~(IR) (by the now
standard argument) and, therefore, it is the Fr of a function 'tjJ E L~(IR), which is
in Wo by Lemma D3.2. By Lemma D3.2 again, any function g E Wo has an Fr of
the form
g(w)

= eiW/2mo(~ + n s(w)qy(~) ,

for some 2n-periodic function s in L~([ -n, +n]. In particular, since

g(w)

Since sv*

E L~([ -n,

= s(w)v(w)*:V;(w).

+n],
s(w)v(w)*

= LCke-inw,
nEZ

v-I

= v*,

D32 Mother Wavelet in the Fourier Domain

for some sequence (cn}nEZ

E l~,

and therefore,

g(t)

= I:>k1fr(t -

209

n).

nEZ

Therefore, the translates of

orthonormal because

1fr

generate Wo. The system

L 11fr(w + 2br)1

= 1,

{1fr(. -

n) }nEZ is

a.e.,

kEZ

as can be checked by the usual routine.

Conversely, let 1fr be an orthonormal wavelet. Being in Wo, it is of the form (50).
By the usual calculations, we find that

L 11fr(w + 2krr)1

= Iv(w)1 2 ,

kEZ

and therefore, by the orthonormality condition, Iv(w)1 2

= 1.

In summary, the mother wavelet is of the form

(54)
where ml(w) = eiWmo(w+rr)*v(2w) is a high-pass filter (Iml(rr)1
that the scaling function cp and the low-pass filter are related by

q;(w) = mo (~) q;(~) .

= 1). Werecall
(55)

We also have the identity

(56)
These three relations fuHy describe the MRA: (55) is called the dilation equation
and teHs us that Vj - 1 is obtained by low-pass filtering Vj ; (54) teHs us that Wj - 1
is obtained by high-pass filtering Vj; and (56) guarantees that there is no loss of
energy.
The choice v

==

1 leads to
1fr(w)

= eiW/2mo(~ + rr)* q;(~),

or, equivalently, up to the sign,

1fr(t)

=.J2

L(-It-1h"'-n_lcp(2t - n),

(57)

where h n is defined by (40).

EXAMPLE D3.1 (The Haar Wavelet). We shall obtain the Haar wavelet by the
general method just described. Recall that the scaling function is cp(t) = I[o,1](t)

210

D3. Wavelet Orthonormal Expansions

(b)

(a)

r-

0.5

0.5

-0.5

-0.5

-1

-1

-3

-2

-1

-3

-2

-1

(d)

(c)

1.5

1.5 ,-----------~--~--~--____,

0.5

0.5

OL-------~--~--~--~

0.1

0.2

0.3

0.4

0.5

0.1

0.2

0.3

0.4

0.5

Figure D3.6. Haar scaling function and the corresponding wavelet (left: scaling
function; right: wavelet; top: time domain; bottom: frequency domain)
and, therefore,
hn

= v'2

q;(x )q;(2x - n)* dx

forn = 0, I,
otherwise,
and, using (57),
1jI(t) = q;(2t) - q;(2t - I).

Thus, we recover the Haar wavelet (Fig. D3.6).

EXAMPLE

D3.2 (Shannon wavelet).

Here

fi(w) = I[-n,+nj(w),

and, therefore,
q;(t) =

sin(nt)
nt

D33 Mallat's A1gorithm

211

Wefirst choose mo such that (41) holds, i.e.,

$(2w) = mo(w)$(w).

Therelore, necessarily,
mo(w)

= $(2w)

on

[-n, +n],

that is,

By periodicity,

=L

mo(w)

$(2w + 2kn).

kEZ

Ourchoice 011/1 is as in (53), with v(w) = _ie- iw :

V!(2w)

= - e-iwmo(w + n)*$(w)
= - e- iw

(L

$(2w + 2kn

kEZ

=-

e- iW ($(2w

= _e- iw

+ 1) $(w)

+ n) + $(2w -

(1 [_lI"._lj](w) + 1[+lj,+lI"](w).

This gives the Shannon wavelet (Fig. D3.7)

1/I(t) = cos(ln(t _
2

D33

!
2

sin( !n(t 2

!
2

~n(t _ ~)

Mallat's Algorithm

Mallat's algorithm6 is a fast algorithm for obtaining from the projection at a given
level the wavelet's coefficients at coarser levels of resolution.
Let 1 be a function in L~(IR).lts projection on Vj , the resolution space at level
j, is

Pjl

= LCj,n'Pj,n,
nEZ

where
(58)
6Mal1at, S. A theory of multireso1ution signal decomposition: The wave1et representation, IEEE Transactions on Pattern Analysis and Machine Intelligence, 11, 1989,
674-693.

212

D3. Wavelet Orthonormal Expansions

(a)

(b)

0.5

0.5

-0.5

-0.5

-1
-15

-10

-5

10

-1
-15

15

(c)

1.5

-10

-5

10

15

(d)

1.5

0.5

0.5

0.1

0.2

0.3

0.4

0.5

0.1

0.2

0.3

0.4

0.5

Figure D3. 7 Shannon sealing funetion and the eorresponding wavelet (left: sealing
funetion; right: wavelet; top: time domain; bottom: frequeney domain)

Its projeetion on Wj , the spaee of details at level j, is

Dj/

= Ldj,nVrj,n,
nEZ

where
(59)

and wehave
(60)

Denote by Cj and d j the sequenees {Cj,n}nEZ and {dj,n}nEZ, respeetively. The purpose of Mallat's algorithms is to decompose the funetion f, that is, to pass from
CM to dM-I, dM-I, ... , do, Co, and to reconstruct that is to pass, from co, do,d l ,
" " d M to CM.
The sequenee d M - lo d M - lo ... , do, Co is the wavelet encoding ofthe wavelet data
CM' We shall explain the interest of this eneoding onee we have derived Mallat's
algorithm.

D33 Mallat's Algorithm

213

Since the function cp(t/2) is in V-I, and V-I C Vo, and since {cp(. - n)}nEZ is a
Hilbert basis of Vo, we have the decomposition

2CP(2 t ) = L

ancp(t

+ n),

nEZ

where

11m cp(-t)cp(t+n)dt.
1

a n =2

lR

Therefore,

=2

L::!.
2

-cp( -(2' t - 2n))

= 2-9- LakCP(2jt -

2n

kEZ

+ k),

that is,
CPj-l.n

= V2L a kCPj,2n-k.

(61)

kEZ

Sirnilarly, since the function 1/1'(t/2) is in W_I, and W_I C Vo, and since {cp(. n)}nEZ is a Hilbert basis of Vo, we have the decomposition

where
n

= ~ [ 1/1'(~t)cp(t+n)dt.
21lR

Therefore, it follows by computations sirnilar to those above that

1/1'j-l,n

= V2LkCPj,2n-k.
kEZ

Denoting the low-pass filter by

mo(w)

=L

ane inw

nEZ

and the high-pass filter by

ml(w)

= Lneinw,
nEZ

we have, from (55) that

qy(2w) = mo(w)cp(w),

(62)

214

D3. Wavelet Orthonormal Expansions

and, from (54) that

In Theorem D3.3, we now make the particular choice of the mother wavelet
corresponding to v(w) = 1:
that is,
L

= L ( _l)n+la;_neinw.

n einw

nEZ

nEZ

Therefore,
(63)

Substituting (61) in (58), we obtain

Cj-l.n

(64)

hLaZcj,Zn-k.
kEZ

Similarly, substituting (62) in (59), we obtain

d j - 1n

(65)

hLZCj,Zn-k.
kEZ

These are the basic recursions of the decomposition algorithm (see Fig. D3.8).
The recursions for the reconstruction algorithm (see Fig. D3.9) are obtained from
(60), (61), and (62). This gives
Cj,n

EXERCISE

=h

[azk-nCj-l,k
L
kEZ

(66)

D3.1. Show that for the Haar wavelet,

Cj-l,k

+ Zk-ndj-l,k]'

(64)

(64)

+ Cj,Zk
h

Cj,Zk-l

(64)

(64)

~CM-I ~CM-2 ~

(~

dM-I

(6~

dM-2

Cl

(64)

~Co

(6~

Figure D3.8. Mallat's decomposition algorithm

~CM-I~CM

Figure D3.9. Mallat's reconstruction algorithm

D33 Mallat's Algorithm

215

and
dj -

I k

Cj,2k-1 -

"fi

Cj.2k

We shall now evaluate the algorithmic complexity of the decomposition

algorithm. (Similar results hold for the reconstruction algorithm.)
For this we suppose that the low-pass and high-pass filters, mo and ml, respectively, have finite impulse responses, that is, the sequences {an }nEZ and {n }nEZ have
a finite-number (say, K) of nonzero terms. Suppose that the infinite-dimensional
vector CM has in practice a finite number N of nonzero terms (say, after truncation).
Then there are approximately N /2 terms in CM-I, and therefore, in view of (64),
the passage from CM to CM-I costs approximately K N /2 multiplications; so does
the passage from CM to d M -I. For the decomposition algorithm, we therefore have
approximately
N
(K2

N)

+ K 4' + ... + K 2 M + K 2M = K N

multiplications. The complexity of Mallat's algorithm is therefore linear in data

size.
Note that Mallat's algorithm encodes N numbers into N numbers. Thus the
compression gain seems to be null. However, only a few terms in the sequence of
details dj,e, e E Z, j = M - 1, ... ,0, are nonnegligible, provided the MRA is
sufficiently smooth. The smoothness issue is discussed in Chapter D5.

D4
Construction of an MRA

D41

MRA from an Orthonormal System

The Fourier structure of an MRA is now elucidated, and we know how to obtain a
wavelet basis when an MRA is given. This chapter gives two methods for obtaining
anMRA.
In the previous chapter, we started from a nested family of resolution spaces
{Vj bEZ and we discovered a scaling function q; in rather simple examples. Now,

obtaining the scaling function from a given nested family of resolutions spaces can
be a difficult task in general. However, if we are interested in a wavelet basis rather
than in a given family of resolution spaces, we might as weH start from a given
function q; E L 2 with the property that {q;(. - n)}nEZ is an orthonormal system,
and define the resolution spaces in an ad hoc manner guaranteeing that q; is indeed
the corresponding scaling function.
If q; is to be the scaling function, there is but one choice for the resolution spaces,
namely,
Vj

= span {q;j,n

:n

Z}.

An inspired choice of q; will make the Vj 's nested as required, and this has to
be verified because there is no reason why it should be so when one starts from an
arbitrary orthonormal system {q;(. - n)}nEZ, A necessary and sufficient condition
for this is that
q;(t)

= L cnq;(2t nEZ

P. Brmaud, Mathematical Principles of Signal Processing

Springer Science+Business Media New York 2002

n),

(67)

218

D4. Construction of an MRA

for some sequence

{cn}nEZ E

e~(Z) or, equivalently, that the dilation equation

(~) fi(~)

fi(w) = mo

(68)

holds for some 2JT -periodic function mo in L~( -JT, +JT). We must also verify that
conditions (d) in the definition of, an MRA are satisfied. By Theorem D3.1, it
suffices that fibe continuous at the origin and that Ifi(O) I = 1.

Meyer's Wavelet
Define cp by

fi(w)

2JT

lflwl S

3'

. 2JT
3 -

l f - < Iwl <

where v is a smooth function

4JT
3 '

(69)

otherwise,
(C k or C OO ) such that

v(x)

{~

ifx SO,
ifx:::l

(70)

and
v(x)

+ v(l

- x)

1.

(71)

U sing (71) it is easy to verify that

L lfi(w + 2kJT)1

1,

kEZ

and, therefore, {cp(. - n)}nEZ is an orthonormal system. We must now verify that
the Vj are nested, and for this it suffices to verify that Vo c Vj or, equivalently,
that cp E Vj. But this is true if and only if there exists a 2JT -periodic function mo
of finite power such that

fi(w) = mo

(~) fi(~) .

It turns out that

mo(w)

= Lfi(2w+4kJT)
kEZ

accomplishes what is required. In fact,

~(w)
=~
cp(w)cp
"2 '

D41 MRA from an Orthononnal System

since the supports of $(w + 2klr) and of $(w /2) do not overlap if k

$(~) = 1

if w

=1=

219

O. But since

supp(~,

we have

$(w)$(~) = $(w),
as desired. We obtain a mother wavelet by formula (53) of with v(w)
gives

1. This

which gives
e iw / 2 sin(
:;j;(w)

~ v (2~ Iwl -

1))

~ v (2~ Iwl -

1))

e iw / 2 cos(

0
EXERCISE D4.1.

.f 2JT
3

1 -

4JT

if3

::slwl::s
::s Iwl ::s

4JT

3'
8JT

(72)

3'

otherwise.

Let P be a probability measure on lR with support in [-8, +8] C

[-1' +1]' and define q;(t) by its Fourier trans/orm

Check that q;(t) is indeed in LUlR) and that the system {q;(. -n ]}nEZ is orthonormal.
Check that the dilation equation (55) holds with

mo(~) =

4JT

q;(w)

iflwl::S 3'

otherwise.

Show that q;(t) so defined is the scaling function 0/ some multiresolution analysis
and that a mother wavelet is given by its Fourier trans/orm

Examine the case where P is the Dirac measure at O.

220

D4. Construction of an MRA

D42

MRA from a Riesz Basis

Now we do not impose orthonormality. To be specific, we have an L 2 -function W

such that
W(t)

= I>nw(2t -

(73)

n),

nEZ

where {cn}nEZ

eUZ), and we define the resolution spaces by

Vj

= span {Wj,n

:n

(74)

Z}.

Of course, condition (73) guarantees that these spaces are nested. In order to obtain
a Hilbert basis of Vo, we use Theorem C4.6 which says that under the "frame"
condition

o< a

L Iw(w + 2krr)1

~<

(75)

00,

kEZ

the system {cp(. - n)}nEZ is a Hilbert basis of Vo, where

~

cp(w)

w(w)

= "~ Iw(w + 2k:rr)1 2 .

(76)

kEZ

Here we shall also have to verify that

For this we can use the following result.

THEOREM

D4.1. Let W

E L~(lR.)

o< a

satisfy

L Iw(w + 2krr)1

~ <

(77)

00,

kEZ

and define

Vj

= span {Wj,k : k E Z}.

(78)

Suppose that the Vj are nested. Then

nVj

(79)

= 0.

JEZ

Proof The inequalities (77) are equivalent to the existence of A > 0, B <
such that

0< AllfII 2 ~

L l(f, wo,k)1

~ BllfII 2 ,

kEZ

for all

Vo, and therefore equivalent to

0< AllfII 2 ~

kEZ

l(f, wj,k)1 2 ~ BllfII 2 <

00,

00

(80)

D42 MRA from a Riesz Basis

221

for all 1 E Vj , 1 # 0, and allj E Z.

With any 1 E njEZ Vj and s > 0, one can associate a compactly supported and
continuous function 1 E L~(~) such that 11 1 - 111 :::: s, and therefore on denoting
the orthogonal projection on Vj by Pj , we have

111 Therefore, for all j

Pjlll

= IIPj(f - j)11 :::: 111 - 111 :::: s.

Z,

By (80),

Now, with c > 0 such that

suppIC[-c,+c]

M=supll(x)l,

and

XEIR

we have
1(1, wjk)1 2 = ITj

Ixl<c

II(x)w(Tjx - k)1

dxl2

Iw(Tjx -k)1 2 dx,

:::: TjM22cl
Ixl<c

where the last inequality is Schwarz's inequality. Therefore,

k +2- jC
. Iw(x)1 2 dx.
1(1, wjk)1 2 :::: 2cM 2
k-2- 1 c

Assuming j to be large enough for 2- j

respect to k E Z,

::::

1/2 to hold, and then summing with

where
A(c,j)

= I)k-Tjc,
kEZ

k+Tjc].

222

D4. Construction of an MRA

By the dominated convergence theorem this term tends to 0 as j -+ 00. In particular, there exists a j such that 11 Pj 111 ::: 8, and therefore 11 f II ::: 28. Since 8 is

arbitrary, this implies 11 111 = 0, which proves (79).

Let us now see how this program works in a classic example:
Franklin's Wavelet

Take 7 for w the piecewise linear spline

w(x)

={

I - Ix I if 0 ::: Ix I ::: 1,

(81)

otherwise,

and observe that (73) is verified. More explicitly,

w(x) = !w(2x

1) + w(2x)

( sinz'-~(' I )

)2

!w(2x - 1).

(82)

The Fourier transform of w is

w(w)

Wehave

L Iw(w + 2krr)1

= ~

~ cos(w)

kE'L

= ~ ( 1 + 2 cos2 (~) )

(One way to prove this is to compute the Fourier coefficients of the left-hand side

w(t)w(t

+ n)* dt,

and this immediately gives the result. Note the generality of the method and its
interest when w(t) is compactly supported.) The mother wavelet is then obtained
from (76). This gives

cp(w)

= w(w)

.f3
1 + 2cos 2

(W))1/2 .
"2

(83)

If we can compute, at least numerically, the Fourier coefficient Cn in

7pranklin, P. A set of continuous orthogonal functions, Math. Ann., 100, 1928,522-529.

D43 Spline Wavelets

223

then we obtain q;(t) as

q;(t)

= L::>nW(t -

n).

nEZ

The corresponding low-pass filter mo(w) is

mo(w)

(jJ(2w)

= -:::::-- =
q;(w)

cos 2

W))1/2
1 + 2cos (
(
(2) 1+2cos (w)2 ,
2 -

and this leads to an expression for the mother wavelet's Fourier transform. Again
the (numerical) evaluation of the Fourier coefficients of the function factoring (jJ( w )
yields an evaluation of 'ifr(t) in terms of the translates of q;(2x).

D43

Spline Wavelets

Franklin's wavelet is a particular case ofthe Battle-Lemarie spline wavelets, which

are now described. We first introduce a family of functions, the basis splines, or
B-splines, that play an important role in numerical analysis, in the theory of spline
approximation. The B-spline functions B n (t), n :=:: 0, are defined recursively by
Bo(t)

= 1[O,1](t)

and, for n :=:: 1,

B n+1(t) = (Bo

For n

[I

* Bn)(t) =

Bn(x) dx.

I-I

= 3, we have
ifO~t~1,

if 1

~ t ~

2,

if t < 0,
the rest of the function being obtained by symmetry around 2.
In the general case, Bn(t) is (for n :=:: 1) in cn- I , its support is the interval
[0, n + 1], and

Bn(x)dx

Wehave
Bo(w)

= e-i~sinc

1.

(2:)'

and, therefore, in the Fourier domain, the recurrence defining the B-splines
becomes by the convolution-multiplication rule
Bn+I(W)

= e-i~sinc

(2:) Bn(w).

224

D4. Construction of an MRA

This gives, for n 2: 0,

= (.W
e-'Tsinc

Bn(w)
A

We observe that Bn(O)

(w))n+!
2Jr

(84)

= 1 and that in the neighborhood ofO

Bn(w)

=0

(IWI~+! ) .

(85)

We shall now seek a scaling function for the B-spline of order n. From the
observation
e-iIsinc

(2:) = e-i~ cos (4:) x e-i~sinc (4:)'

it follows that

where
mo(w) =

(e- iI cos (~)) n+! =

(1 +

-iw )n+!
;)

The impulse response of the low-pass filter of the MRA is therefore

.fi n+!
hk = 2n+! (k)'

0~k~ n

+ 1,

and the scaling equation is

1 n+!

Bn(t)

= --;; L(~+!)Bn(2t 2

k).

k=O

In view of the estimate (85), the series

IBn(w + 2kJr)1 2

(86)

kEZ

is absolutely convergent. Using (84) and the estimate

sin(w)

-->-,
w

we have, for Iwl

Jr
- 2'

O<w< -

Jr,

IBn (~) 2 = /Sin~I) /2n+2 2: (~yn+2

1

Therefore, there exist positive finite constants A and B such that

A ~

IBn(w + 2kJr)1 2 ~ B,

kEZ

and {Bn (. - k)} kEZ constitutes a Riesz basis of the Hilbert subspace that it generates.
In order to compute the scaling function of the MRA, we need the following lemma.

D43 Spline Wavelets

LEMMA

225

D4.1. There exists a polynomial Pn of degree n such that

L IBn(w + 2br)1

= Pn(cos(w.

(87)

kEZ

Moreover, the coefficients of this polynomial are rational and can be computed
recursively.
Proof

Denote the left-hand side of (87) by Fn(w). Inserting (86) in(84) gives
Fn(w)

= (sin (~) )2n+2 Gn(w),

where
Gn(w)

=L

kEZ

h- + 1Tk) 2n+2'
(J)

One verifies easily that, for n 2: 1,

Gn(w)

"
= n(2n2+ 1) Gn(w),

and, therefore,
Fn(w)

(. (W))2n+2 ( Fn-l(W) )"

2
n(2n + 1) sm "2
(sin (~))2n

We introduce the new variable y = cos(w), and define the function Pn by Fn(w) =
Pn(y). Since Fo(w) = 1, we have Po(y) = 1. The recursion in the last display
becomes
Pn(y)

= n(2n + 1) (1 -

y)

n+ld (Pn-1(y)
dw (1 _ y)n .

Using the differentiation rules

d
dw

d
dy

- = (- sin(wand
d2

d2

-dw 2 = (-y)+ (1 - l ) - ,
dy
dy 2
we obtain, after simplification and rearrangement,
Pn(y)

2
(n(n
n(2n + 1)

+ 1 + nY)Pn-l(Y) + (1 + (1

y)(2n

- y)2(1

+ (2n -

l)y)Pn- 1

+ y)P~'--l)'

Therefore, if Pn - 1 is a polynornial of degree n - 1, then Pn is a polynornial of

degree n. The conclusion follows since Po is indeed a constant.

226

D4. Construction of an MRA

The general method of the previous section gives for scaling function (fi = (fin

Therefore,
cp(t)

L CkBn(t -

k),

kEZ

where the Ck'S are given by the power-series expansion

r = LCkl.

(Pn(z+r' )
Observe that Ck =

Ck.

Also, since the function in the left-hand side is analytic,

ICkl
for some

Ipl

kEZ

~ plk l ,

< 1. In particular, the scaling function cp(t) has exponential decay.

We now proceed to compute the mother wavelet. We have to compute the impulse
response of the low-pass filter mo(w). We have
1

_ cp(2w) _ Bn(2w) ( Pn(cos(w )

mo(w) - - - - - A - cp(w)
Bn(w)
Pn(cos(2w

2:

that is,
.

l+e- WW )

mo(w) = (

n+!

(Pn(COS(W)2
Pn(cos(2w

We compute the Fourier expansion

1

( Pn(cos(w )
Pn(cos(2w

where
qk

= q-k = -

rr (

2:

"
-ikw
= '~qke
,

kEZ

Pn(cos(w )
Pn(cos(2w

2:

cos(kw)dw.

Therefore,

The mother wavelet is then

1/r(t)

= v'2L(-ll-IL k - 1CP(2t kEZ

k).

D43 Spline Wavelets

Putting all this together, we finally obtain

1/!(t)

= L bk B n (2t -

k),

kEZ

where

br = hL(-ll-ILk-1qr-k.
kEZ

227

D5
Smooth Multiresolution Analysis

D51

Autoreproducing Property of the Resolution

Spaces

The axiomatic framework of multiresolution analysis is Fourier analysis in L 2 ,

and the convergence of the wavelet expansion is therefore in the L 2- norm. The
smoothness properties of the scaling function and of the mother wavelet are, however, of great interest to obtain fast L 2 -convergence of the wavelet expansion, or
to obtain pointwise convergence of this expansion.
We first recall a definition.
D5.1. Let r ::: 0 be an integer. The junction q; : lR -+ C is said to be
in Sr iffor all n E N, all 0 .::: k .::: r, there exist finite nonnegative constants Ck,n
such that, for all x E :IR,
DEFINITION

q;

)1

(k)(
X

<
-

(1

Ck,n

+ Ix Dn

(88)

D5.2. Consider an MRA with scaling junction q;. Let r be an integer.

The MRA is called r-smooth if q; E Sr.

DEFINITION

We now consider an r-smooth multiresolution analysis. We associate with it the

function q : lR X lR -+ C, called the kernel of the MRA and defined by
q(x, t) =

L q;*(x -

n)q;(t - n).

neZ

P. Brmaud, Mathematical Principles of Signal Processing

Springer Science+Business Media New York 2002

(89)

230

D5. Smooth Multiresolution Analysis

U sing the inequality

(1

wehave
Iq(x, t)1

:s L

+ laD(l + IbD ::: 1 + Ib -

al,

IqJ(x - n)llqJ(t - n)1

nEZ

""

CO,k+2
nDk+2 (1

:s nEZ
~ (l + Ix -

<C 2
- O,k+2 nEIU'" (1
It foUows that, for aU k
that, for all x, t E IR,

+ Ix -

CO,k+2
nDk+2

+ It -

nD 2 (l

+ It -

nl)2 (1

+ Ix -

tDk '

N, there exists a finite nonnegative constant Ck such

(90)

In particular, for each t E IR, the function qt : IR --+ C defined by qt(x) = q(x, t)
is in L 2 , and the development of any function f E Vo along the orthonormal basis
{qJn}nEZ = {qJ(' - n)}nEZ

takes the form

f(t) =

q(x, t)f(x)dx.

(91)

DEFINITION D5.3.

Let E be some set, and let H be a Hilbert space offunctions f :

E --+ CwiththeHermitianproduct(, }.IfthereexistsafunctionK: ExE --+ C
such thatfor each x E E, thefunction K(x, .) E H, and fex) = (K(x, .), j}, H
is called an autoreproducing Hilbert space with reproducing kernel K.

D5.1. Let E be some set, and let H be a Hilbert space of functions

f : E --+ C with the Hermitian product (., .). Suppose that for each x E E,
the mapping f --+ fex) from H to C is continuous. Show that H is then an
autoreproducing Hilbert space.
EXERCISE

Equation (91) therefore teUs us that Vo is an autoreproducing Hilbert space with

reproducing kernel q(x, t). Similarly, for all mEZ, Vm is an autoreproducing
Hilbert space with reproducing kernel qm (x, t), where
(92)

We know (Theorem D3.1) that 1$(0)1 = 1, and we can assume without loss of
generality that $(0) = 1. Therefore, in view of property (38),
$(2klr)

= l[k=o}.

(93)

D52 Pointwise Convergence Theorem

231

It follows from this and the weak Poisson formula (Theorem A2.3) that

L qJ(x - n) = 1.

Finally, from (89) and (94),

qm(x, t)dx

for m

(94)

(95)

1,

= 1, and therefore for all mEZ.

EXERCISED5.2. Give the kernel q(x, t) ofthe Haar MRA (qJ(t)

the Shannon MRA (qJ(t) = sin(rrt)jrrt).

= I[o,l](t) and of

In general, the kernel of an MRA does not have a c10sed form, and the examples
in the previous exercise are exceptions.

D52
Let f

Pointwise Convergence Theorem

E L~(lR),

and denote by fm its projection on Vm. We have

fm(t)

qm(x, t)f(x)dx,

where qm is the autoreproducing kernel of Vm, defined by (92). This kernel representation allows us to obtain pointwise convergence results, in the manner
of Dirichlet's pointwise convergence analysis of Fourier series. We need some
preliminary results on the kernel.

DEFINITION D5.4. Let {m}mEZ be a sequence of functions m : lR x lR ~ C.

It is called a quasi-positive delta sequence if it satisfies the three following three
conditions:
(a) There exists afinite nonnegative constant K such that

Im(x, t)ldx < K, for all t E lR, all mEZ.

(96)

(b) There exists afinite nonnegative constant c such that

lim

mtoo

t+c

t-c

m(x, t)dx

1,

(97)

= 0.

(98)

uniformly with respect to t in compact sets.

(c) For all y > 0,

tim

sup 18m (x, t)1

mtoo Ix-tlo::y

EXERCISE DS.3. Show that Fejer's kernel sequence

t)

= 2rr (m + I tsm2 e(
) I[-n,+nj(x '2 x - t
sin 2 (m+l(x -

8m (x, t)

t)

232

D5. Smooth Multiresolution Analysis

is a quasi-positive delta sequence. Show that Dirichlet's kerne! sequence

8m(x, t) =

sin (m + !)(x - t))

. ('
) 1[-n,+nj(x - t)
2:n: sm 2:(x - t)

is not a quasi-positive delta sequence.

THEOREM D5.1.

lfthe MRA is r-smooth, the sequence {qm}m E Z defined by (92)

is a quasi-positive delta sequence.

Proof

We first prove property (a) of Definition D504:

Iqm(x, t)ldx

L
=L
=

2mlq(2mx, 2m t)1 dx
Iq(x, 2m t)1 dx

<C2

J[{ (1 + lxi?

dx=K<oo

'

where we have used inequality (90).

We now prove property (b) ofDefinition D504: Let c > 0 be finite. We have

t+c

t-c

qm(X, t)dx

12m (t+c)

q(x,2mt)dx

~Q-~

where we have used (92) and (95). But

2mt-2mc
1
_2mc
dx
=
C2j
1/,1< C2j
-00
(1 + Ix - 2m tl)2
-00
(1

+ Ixl)2 dx '

and this quantity tends to zero as m tends to infinity. A similar conclusion holds
for h Therefore, property (b) ofDefinition D504 is satisfied.
For property (c) ofDefinition D504, it suffices to observe that

2mlq(2m x 2mt)1 < C 2m_ _ _ _ __=_

,
2
(1+2 mlx-tI)2

in view of (90).
We can now state the main result.
THEOREM D5.2.

lf I E L~(JR;)nL~(JR;) is continuous on (a, b), then the projection

Im = PVm I converges pointwise to I, uniformlyon compact subintervals [a, ]

(a, b), as m -+

00.

D52 Pointwise Convergence Theorem

233

Proof" This is an immediate consequence of Theorem D5.1 and of the

regularization lemma below.

LEMMA D5.1. Let {8m }mEZ be a quasi-positive delta sequence. Let f E Lb(lR) be
continuous on (a, b), and define for all mEZ thefunction fm by

fm{t)

8m(x, t)f(x)dx.

Then

tim fm{t)

m--+oo

= f(t)

uniformlyon any compact subinterval [a, ] E (a, b).

Proof"

For y > 0, write

fm{t)

= ( 8m(x, t)f(x)dx = ft+ y + f+oo + jt- y

J'R.

= f(t)

+f

t-y

HY

Hy

-00

8m(x, t) dx

t-y

t+ y 8 (x, t)(f(t) - f(x dx

m
t-y

+ (f+oo + jtt+y

y)

-00

= A+B +(C).

Let [a, ]

E (a, b), t E [a, ]. Let c be as in (b) of D5.4. Choose y such that

y < c, + y < b, a - y > a. For any 0 < 8 < 1, further restrict y so
that If(x) - f(t)1 < 8 whenever t E [a, ] and Ix - tl < y (in which case both t
and x are in a compact subinterval contained in (a, b), and we can then invoke the

o<

uniform continuity of f in this closed interval). We then have

IBI

~ 8 ft+ y 18m(x, t)1 dx ~ 8

ICI

t-y

J'R.

18m(x, t)1 dx

~ 8K

and
sup 18m(x, t)1 { If(x)1 dx

J'R.

Ix-tl2:Y

~8

J'R.

If(x)1 dx

for large enough m. Also, for large enough m,

1

1-

f~:y 8m(x, t)dxl ~ 8

(use property (b) of the definition of quasi -delta sequences, and the fact that
lim ft+c 8m(x, t)dx
mtoo t+y
uniformly with t
8.

=0

~, and the same for the limit of J/~:). Therefore, If(t) - AI ~

234

D5. Smooth Multiresolution Analysis

Putting all this together, we have for large enough m

I/(t) - Im(t)1

::s I/(t) -

AI

+ IBI + ICI ::s Me + e III + Ke,

where M

= SUPtE[a.1I/(t)l.

D53

Regularity Properties ofWave1et Bases

In the wavelet expansion

1= L(f, 1{Ij.n}1{Ij,n =
j,n

L d j,n1{lj,n,
j,n

where

it is highly desirable from a numerical point ofview that the coefficients dj,n decay
rapidlyas IJ I, Iml -+ 00, thus ensuring fast convergence ofthe wavelet expansion.
This is not the case, however, even for smooth functions (say, I E C oo n L 2 ) if
no further conditions are imposed on the mother wavelet 1{1. To understand this
and see what type of conditions 1{1 should satisfy, let us examine the asymptotic
behavior of
dj,o

as

J -+

00.

= 2 j /2

Let 2 j = N and set

d(N)

= JN

l(x)1{I(2 j x)* dx

l(x)1{I(Nx)* dx

= JNa(N).
A Taylor expansion of I (assumed to be C OO ) with Lagrange residue gives

where
RK(X)

(x

t)K

K!

lK+l)(t) dt.

We assurne that the scaling function has a Fourier transform at 0 equal to 1, which
implies that the mother wavelet has a null Fourier trans form at 0 or, equivalently,
that it integrates to O. Therefore,
a(N)

1'(0) JLl
N2 1!

1"(0) JL2
N3 2!

+ ... +

I(K)(O) JLK
N K +1 K!

+r

(K)
N

D53 Regularity Properties of Wavelet Bases

235

where the J,Lk'S are the wavelet moments:

J,Lk

x k1/l(x)*dx,

and the rest is readily bounded above by

rN(K) .:::

NK+2 '

for some finite nonnegative c. In particular, a wavelet with moments that are zero
up to order K implies
deN) .:::

-JN N~+2'

We shall see how the smoothness of 1/1 relates to moment conditions.

D5.3. Let 1/1

THEOREM

Sr and assume that {1/Ijkli,kEZ is a Hilbert basis of

LUIR), where

Then

(99)

Let N be a dyadic integer (that is, N = 2- j oko) such that 1/I(N) #- 0 (the
existence of N follows by the density of dyadic integers in IR and by the fact that
1/1 is continuous and not identically zero).
Let j > 1 be sufficiently large for 2j N to be an integer. By orthogonality

o=

2j

=
Passing to the limit j

~ 00

1/1 (x )1/t(2 j x

- 2 j N) dx

+ N)1/I(y)dy.

1/t(Tjy

(*)

gives, we have by dominated convergence

1/I(N)

1/I(y)dy

= O.
Suppose that (99) is true for k = 1, ... , n -

= O.

Therefore, (99) is proved for k

1, where n .::: r. We have the Taylor

expansion
1/t(x)

=L
n

k=O

1/I(k)(N)

(x

N)k

k!

+ rn(x)

(x

N)n

n!

where rn(x) is uniformly bounded. Choose N such that 1/I(n)(N)

in (*), we obtain
0=

LI~

1/I(k)(N)

#- O. Substituting

(2-:~l + rn(Tj y + N) (2-~~)n } 1/I(y) dy

236

D5. Smooth Multiresolution Analysis

= 1jI(n\N)

{ Tjn yn 1jI(y)dy

J1I?

n!

+(

J1I?

r n (2- j y

+ N) 2- jn y n 1jI(y)dy.
n!

By dominated convergence, the last integral goes to zero as j -+

1jI(n)(N) l

yn1jl(y)dy

00,

and therefore,

= 0 .

Here is an apparent paradox relative to the moment conditions, and especially

to the condition
l1jl(X)dx

= 0,

which is always satisfied and implies that the projection Im

(100)

Im(x)dx = 0,

a surprising fact at first glance, since the function

not such that

= Pvml satisfies

that is analyzed is in general

Im(x)dx = O.

(101)

There is actually no contradiction since one cannot pass in the limit m -+ 00 in

(100) to obtain (101): Convergence of Im to I is in L 2 , and this does not imply
that
lim { Im(x)dx

mt oo

J1I?

= (

J1I?

Im(x)dx.

In Mallat's algorithm one first computes the projection Po I that is the approximation of I at the resolution level 0, and then the coarser resolution approximations
Pj I, j :s -1. As we have just seen, the moment conditions on 1jI are useful for
the first part of the algorithm. For the second part fast decay of the coefficients
hn =

<p(x)<p(2x - n)* dx

is needed for rapid numerical convergence. An ideal situation is when only a finite
number of h n are nonzero, which is guaranteed ifthe scaling function has compact
support. Note that if this is the case, then the compactness of the scaling function
carries over to the mother wavelet, and this is why one usually talks of compact
wavelets rather than compact scaling functions.
Let us mention at this point that if we start from a Riesz basis of Vo, as in the
method explained in Section D42, the compactness of w (there defined) does not
imply compactness of the scaling function. In the face of this negative statement
one needs to be reassured about the transmission of exponential decay from w to
<p. As a matter of fact the situation is not too bad, and a result in this direction is,
for example, Proposition 5.4.1 in [D3]. We end this section by showing how the
decay ofthe scaling function is transmitted to the coefficients h n Localization of

References

237

the scaling function can be taken in many related senses. We mentioned previously
one of them, namely ({J E Sr. Another definition of localization could be

L+
(1

Ixl)m l({J(x)1 2 dx <

00,

forallm E N.

(102)

It follows from (102) that for finite constants Cm ,

Ixl~A

(103)

1({J(x)12 dx::: : : '

for all A > 0, mE N. By Schwarz's inequality,

../211
: : . /2 (1

Ihni:::

({J(x)({J(2x -n)dxl

Ixl~A

1({J(2x - n)1 2dx)1/2

+../2 (1

1({J(x)1 2 dx)1/2 ,

Ixl::::A

Ixl~A

and therefore with a proper choice of A, saya = n, in view of the tail majorization
(103), we obtain

Ihn I :::

Dm

(1

+ n)m

for all m E N,

(104)

where the Dm are finite. Thus, the Fourier coefficients of mo are rapidly decaying
and this implies that mo E C oo .
The topic of compact wavelets is an important one, but it is rather technical. The
interested reader is refered to [D3] for the detailed theory.

References
[01]
[02]
[03]
[04]
[05]
[06]
[07]
[08]
[09]

Blatter, C. (1998). Wavelets, a Primer, A. K. Peters: Natick, MA.

Chui, C.K. (1992). An lntroduction to Wavelets, Academic Press: New York.
Oaubechies, I. (1992). Ten Lectures on Wavelets, CBSM-NSF Regional Conf
Series in Applied Mathematics, SIAM: Philadelphia, PA.
Hemandez, E. and Weiss, G. (1996). A First Course on Wavelets, CRC Press: Boca
Raton,FL.
Kahane, J.-P. and Lemarie-Rieusset, P.G. (1998). Stfries de Fourier et Ondelettes,
Cassini: Paris.
Mallat, S. (1998). A Wavelet Tour 0/ Signal Processing, Wiley: New York.
Meyer, Y. (1993). Wavelets Algorithms and Applications, SIAM: Philadelphia, PA.
Vetterli, M. and Kovacevic, J. (1995). Wavelets and Sub-Band Coding, PrenticeHall: Englewood Cliffs, NJ.
Walter, G. (1994). Wavelets and Other Orthogonal Systems with Applications,
CRC Press: Boca Raton, Fl.

Appendix

The Lebesgue Integral

Introduction
Integration is almost as old as mathematics. It is at least as old as Greek
mathematics,8 since Eudoxus and Archimedes used the exhaustion method to
compute the volume ofvarious solids, in particular, the pyrarnid and the cone. 9
The modem theory of integration is intimately linked to Fourier series. Indeed,
Bernhard Riemann (1826-1866) developed his theory of integration as a tool for
studying Fourier series, the theme of his memoir of habilitation to professorship at
the University of Gottingen. Also, Renri Lebesgue (1875-1941), who conceived
his theory of integration in the period from 1902 to 1906, stated in a 1903 artic1e: "[
am going to apply the notion of integral to the study of the trigonometrie expansion
offunetions that are not integrable in the sense of Riemann."

The Riemann integral has a few weak points, the two main ones being that

8Sir Thomas Heath, A History of Greek Mathematics; Vol. I: From Thales to Euclid,
Clarendon Press, Oxford, 1921; Dover edition, 1981.
9Exhaustion is the procedure by which we compute, for instance, the volume of the cone
of height h and circular base of radius R, as the limit of a heap of circular tiIes:
lim

h
2:>' (k)2
-R -.
n
n
n

ntoo k=!

242

Appendix

(1) The dass of nonnegative functions which are Riemann-integrable is not large
enough. Indeed, some functions have an "obvious" integral, and Riemann's integration theory denies it, while Lebesgue's theory recognizes it (see Example 9), and
its stability properties under the limit operation are too weak.
(2) The Riemann integral is defined with respect to the Lebesgue measure (the
"volume" in ffi.n), whereas the Lebesgue integral can be defined with respect to a
general abstract measure, a probability for instance.
The last advantage is an excellent argument to convince a student to invest a
little time in the study of the Lebesgue integral, because the return is considerable.
Indeed, the Lebesgue integral ofthe function f with respect to the measure p, (see
the meaning in the first chapter), modestly denoted by

Ix

fex) p,(dx),

contains a variety of mathematical objects, for instance, the usual Lebesgue integral
on the line,

f(x)dx,

and also the Lebesgue volume integral. An infinite sum

can also be viewed (with profit) as a Lebesgue integral with respect to the counting
measure on Z. The Stieltjes-Lebesgue integral

f(x)dF(x)

with respect to a function F of bounded variation, the expectation of a random

variable Z:

E[Z]
are also in the scope ofLebesgue's integral. For the student who is reluctant to give
up the expertise dearly acquired in the Riemann integral, it suffices to say that any
Riemann-integrable function is also Lebesgue-integrable and that both integrals
then coincide.
Is Lebesgue's theory hard to grasp? Not at all, because most of the results are
very natural, and in that respect, the Lebesgue integral is much easier to manipulate
correctly than the Riemann integral. A tedious (but not difficult) part is the step-bystep construction of the Lebesgue integral. However, if one just gives a summary
of the main steps without going into the details, this is usually not a cause of
frustration for the student interested in applications. The really difficult part is the
proof of existence of certain measures, but students usually do not mind admitting
such results. For instance, there is an existence theorem for the Lebesgue measure

The Lebesgue Integral

243

e (the "length") on JEt It says: There exists a unique measure e on JR: that gives to
the intervals [a, b] the measure b - a. Of course, in order to understand what all the
fuss is about, and what kind of mathematical subtleties hide behind such a harmless
statement, we shall have to be more precise about the meaning of "measure". But
when this is done, one is very much ready to approve the statement although the
proof is not immediate. Of course, in this appendix, the proofs of such "obvious"
results are not given. In fact, the goals of this appendix are to provide a tool and to
give a few tips as to how to use it safely. The reader who has no previous knowledge
ofintegration theory will therefore be very much in the situation of the new recipient
of a driving license who takes the road in spite of her inexperience. Experience is
best acquired on the road, and the main text contains many opportunities for the
student to check her reflexes and to apply the roles that are briefly explained in the
appendix. The student wishing to purehase good insurance is directed to the main
companies, a few of which are listed in the bibliography of this appendix.
Farewell and bon voyage!

Measurable Functions and Measures

In this section, the basic steps in the construction of Lebesgue's integral are
described, and the elementary properties of the integral are stated.
We first recall the notation: N, Z, Q, JR:, C are the sets of, respectively, integers,
relative integers, rationals, real numbers, complex numbers; iR = JR: U {+oo, -oo};
N+ and JR:+ are the sets of positive integers and nonnegative real numbers; iR+ =
JR:+ U {+oo}.
Sigma-Fields

P(X) is the collection of all subsets of an arbitrary set X; card(X) is its cardinal,
that is, the "number" of elements in it.
DEFINITION

1. Afamily X

P(X) of subsets of X is ca lied a sigma-fie1d on X

(a) X E X,
() (A E X) ===} (:4 EX),
(y) (An E X for all n E N) ===}

if

(U~oAn EX).

One then says that (X, X) is a measurable space.

Two extremal examples of sigma-fields on X are the gross sigma-field X =
{0, X} and the trivial sigma-field X = P(X).
The following situation is often encountered in measure theory: One has a collection of elementary sets, easy to describe mathematically, and one needs to define
a sigma-field that contains these elementary sets and that is not too big.

2. The sigma-field generated by a collection of subsets C c P(X)

is, by definition, the smallest sigma-field on X containing all the sets in C. It is
denoted by a(C).
DEFINITION

244

Appendix

Let {Xi}' i E I, be the collection of all sigma-fields on X containing C. This

collection is not empty, because P(X) belongs to it. Furthermore, one readily
checks that ni EI X; (by definition, the collection of subsets of X that belong to
all the Xi, i E I) is a sigma-field. It contains C and, obviously, it is the smallest
sigma-field containing C. This proves the existence of 0" (C).
For the next definition, the reader not familiar with abstract topology may take
X = ]Rn with the Euclidean topology.
DEFINITION 3. Let X be a topologieal spaee and let 0 be the eolleetion of open
sets defining the topology. The sigma-field B(X) = 0"(0) is ealled the Borel sigmafield on X associated with the given topology. A set B E B(X) is ealled a Borel
set ofX.

If X = ]Rn is endowed with the Euclidean topology, the Borel sigma-field B(]Rn)
is denoted Bn. For n = 1, we write B(]R) = B. For I =
I j , where I j is
a general interval of]R (I is then called a general reetangle of ]Rn), the Borel
sigma-field B(I) on I consists of all the Borel sets contained in I.

Hi=-l

1. B(]Rn) is generated by the eolleetion C of all reetangles ofthe type

TI7=1 (-00, ail, where ai E Qfor all i E {I, ... ,n}.

THEOREM

Measurable Functions
One ofthe central notions ofLebesgue's integration theory is that of a measurable
function.

4. Let (X, X) and (E, e) be two measurable spaees. Afunetion f :

E is said to be measurable with respeet to X and e if

DEFINITION

f-1(C)

forall

CE

e.

This situation is denoted in various ways:

f: (X,X)

(E,e),

or

fEeiX,

or

fEX,

where the third notation will be used only when (E, e) = (I, B(I, I being a
general rectangle of]Rn, provided the context is clear enough as to the choice of I.
If f : (X, X) ~ (]Rk, Bk) one says that f is a Borelfunetion from X to ]Rk.
(However, this is not quite standard terminology; in the standard terminology,
(X, X) must be some (]Rn, Bn).)
Let B be the sigma-field on i: generated by the intervals of type ( - 00, a], a E :IR.
A function f : (X, X) ~ (i:, B), where (X, X) is an arbitrary measurable space,
is called an extended Borel funetion, or simply a Borel funetion. As for functions
f : (X, X) ~ (]R, B), they are called real Borelfunetions. In general, in a sentence
such as "f is a Borel function defined on X," the sigma-field X is assumed to be
the obvious one in the given context.
It seems difficult to prove measurability since most sigma-fields are not defined
explicitly (see the definition of Bn, for instance). However, the following result
often simplifies the task.

The Lebesgue Integral

245

THEOREM 2. Let (X, X) and (E, E) be two measurable spaces, where E = a(C)
for some collection C of subsets of E. Then f : (X, X) f-+ (E, E) if and only if

f-I(C)

for all C

C.

One immediate application of this result is

1. Let (X, X) be a measurable space and let n ~ I be an integer. Then
f = (fl, ... , fn) : (X, X) f-+ (IRn, Bn) if and only iffor all a = (al, ... , an) E
IQt, {f :s a} E X. (Here
EXAMPLE

{f:S a}:= {x

IR n : !;(x):s a;joralll:s

i:s n}.)

The proof follows immediately from 2 and the definition of Bn.

EXAMPLE 2. Let X and E be two topological spaces with respective Borel sigmafieids B(X) and B(E). Any continuous function f : X f-+ E is measurable with
respect to B(X) and B(E).

The above result is a direct consequence of Theorem 2 and of the abstract

definition of continuity: f : X f-+ Eis said to be continuous if f-I( 0) is an open
set of X whenever 0 is an open set of E.
Measurability is stable by composition:
THEOREM

3. Let (X, X), (Y, y), and (E, E) be three measurable spaces, and let
f-+ (Y, y), g : (Y, y) f-+ (E, E). Then f := g 0 q; : (X, X) f-+ (E, E).

q; : (X, X)

(This follows immediately from the definition ofmeasurability.)

The next result shows that the set of Borel functions is stable by the "usual"
operations.
THEOREM 4. (i) Let f, g : (X, X)
Borel functions.

f-+

(IR, B). Then f g, f

+ g, (f/ g ) I golD are real

(ii) Let fn : (X, X) f-+ (IR, 8), n E N. Then lim infntoo fn and lim SUPntoo fn
are (possibly extended) Borel functions, and the set

{lim sup fn = lim inf fn} = {3 lim fn}

ntoo
ntoo
ntoo
belongs to X. In particular, if {3limntoo fn}
(possibly extended) Borelfunction.

X, the function limntoo fn is a

Measures
DEFINITION 5. Let (X, X) be a measurable space and let /-L : X f-+ [0,00] be
a setfunction such thatforany denumerablefamily {An}n:::1 ofmutually disjoint
sets in X,

(105)

246

Appendix

The setfunction JL is called a measure on (X, X), and (X, X, JL) is ca lied a measure
space.
Property (105) is the sigma-additivity property.
The following three properties are easy to check:

JL(0) = 0;
(A ~ B and A, B E X)
(An E X for all nE N)

===}
===}

(JL(A):::: JL(B));
(JL(U~oAn):::: L~o JL(An)).

3. Let a E X. The measure Ba defined by Ba(C) = lc(a) is the Dirac

measure ata EX. The setfunction JL : X 1-+ [0,00] defined by

EXAMPLE

L aj lai(C),
00

JL(C) =

j=O

where aj

i:+ for all i

N, is a measure denoted JL

= L~o aj Bai.

4. Let {a n}n:::l be a sequence ofnonnegative numbers. The setfunction

JL : P(Z) 1-+ [0,00] defined by JL(C) = LnEC an is a measure on ('I." P(Z)). If
an == 1 we have the counting measure V on '1." where v( C) = card (C).
EXAMPLE

Next theorem introduces Lebesgue's measure.

THEOREM

5. There exists one and only one measure l on (~, S) such that

la, b]) = b - a.

(106)

This measure is called the Lebesgue measure on R

6. Let JL be a measure on (X, X).

If JL(X) < 00 the measure JL is called a finite measure.
If JL(X) = 1 the measure JL is called a probability measure.
Ifthere exists a sequence {Kn }n:::l of X such that JL(Kn) < oofor all n ::: 1,
and U~l K n = X, the measure JL is called a sigma-finite measure.
A measure JL on (~n , sn) such that JL( C) < 00 for all bounded Borel sets C is
called aRadon measure.
DEf1NITION

EXAMPLE 5.

The Dirac measure Ba is a probability measure. The counting measure

on 'I., is a sigma-finite measure. Any Radon measure on (~n , sn) is sigma-jinite.
The Lebesgue measure is aRadon measure.
V

Cumulative Distribution Function

DEf1NITION 7.

Afunction F : ~ 1-+ ~ is called a cumulative distribution function

(c.dj.) ifthefollowing properties are satisfied:

1. Fis nondecreasing;
2. F is right-continuous;
3. F admits a left-hand limit, denoted F(x-), at all x

The Lebesgue Integral

EXAMPLE

247

6. Let f-L be aRadon measure on (lR, B), and define

FJL(t)

f-LO, tD

ift 2:

0,

-f-Lt,OD

ift <

0.

(107)

This is a c.d.f (use next lemma), and, moreover,

FJL(b) - FJL(a)

= f-La, bD,

FJL(a) - FJL(a-) = f-L({a}).

FJL is called the c.d.f of f-L.

From the last formula, we deduce that any point set {a}, a E lR has null Lebesgue
measure, and therefore, any countable subset of lR (Ql, for instance), has null
Lebesgue measure.
The following lemma features the sequential continuity properties of measures.

1. Let (X, X, f-L) be a measure space. Let {An}n~1 be a non-decreasing

(that is, An ~ A n+ 1for all n 2: 1) sequence of X. Then

LEMMA

f-L

(0
n=1

An)

= lim t
ntoo

f-L(A n).

(l08)

Let {Rn }n~ 1 be a nonincreasing (that is, Rn+1 ~ Rn for all n 2: 1) sequence of X
such that f-L(R no ) < 00 for some no E N+. Then
f-L
Proof:

(n
n=1

Rn)

= lim +f-L(Rn).

(l09)

n,!,oo

We shall prove (108). This equality follows direct1y from sigma-additivity

since
n-I

f-L(A n) = f-L(Ad

+L

f-L(Ai+1 - Ai)

i=1

and
f-L(QAn)

= f-L(A 1) + ~f-L(Ai+I-Ai)'

The proof of (l09) is left to the reader.

The necessity of the condition f-L(Rno ) < 00 for some no is illustrated by the
following counterexample. Let v be the counting measure on Z, and for all n 2: 1
define Rn = {i E Z : lil 2: n}. Then v(R n) = + 00 for all n 2: 1, and
v

(0

Rn)

= v(0) = 0.

We now state a fundamental existence result, which generalizes Theorem 5.

248

Appendix

6. Let F : lR
(lR, ) such that Ffl- = F.

THEOREM

f--+

lR be a c.d.j. There exists a unique measure fL on

This result is easily stated, but it is not trivial.lt is typical of the existence results,
which ans wer the following type of question: Let C be a collection of subsets of X
with C c X, where X is the sigma-field on X generated by C. Given a set function
u : C f--+ [0,00], does there exist a measure fL on (X, X) such that fL(C) = u(C)
for all C E C, and is it unique?
The reason why such results are nontrlvial is that the sigma-field generated by
C is not explicitly constructed. It is therefore not easy to say what fL( C) should be
when one does not really know what a typical C E X should look like!

Negligible Sets
A very important concept in measure and integration theory is that of negligible
sets, with the correlated notion of "almost everywhere."

8. A fL-negligible set is a set contained in a set N E X such that

fL(N) = O.
Let P be some property relative to the elements x E X, where (X, X, fL) is
a measure space. One says that P holds fL-almost everywhere (fL-a.e.) if the set
{x EX: x does not satisfy P} is a fL-negligible set.
DEFINITION

For instance, if I and g are two Borel functions defined on X, the expression

I Sg

fL-a.e.

means that

fL({X : I(x) > g(x)})

= o.

The following result is easy to prove.

THEOREM

7. A countable union 01 fL-negligible sets is a fL-negligible set.

The following result is used several times in the main text.

THEOREM 8. Iltwo continuousfunctions I, g : lR
everywhere equal.

f--+

lR are l-a.e. equal, they are

Praol: Let t E lR be such that I(t) =j:. g(t). For any c > 0, there exists s E
[t - c, t + c] such that I(s) = g(s). (Otherwise, the set {t; I(t) =j:. g(t)} would
contain the whole interval [t - c, t + c] and therefore could not be of null Lebesgue
measure.) Therefore, one can construct a sequence {tnk:::l converging to t and
such that l(tn) = g(tn) for all n ~ 1. Letting n tend to 00 yields l(t) = g(t), a
contradiction.

The Integral
Having defined measures and measurable functions, we are ready to construct the
abstract Lebesgue integral.

The Lebesgue Integral

249

The Simple Case

A Borel function

I: (X, X) 1-+ (R B) ofthe type

k

I(x) = Lai 1A ,(x),

i=l

where k E N+, al,"" ak E lR., Al,"" A k E X, is called an elementary Borel

function (defined on X).
The following result is the key to the construction of the Lebesgue integral:
Let I : (X, X) 1-+ (iR, B) be a nonnegative Borelfunction. There exists a nondecreasing sequence {/n}n2:l olnonnegative elementary Borelfunctions
that converges pointwise to I.

THEOREM 9.

Proof"

Take
In(x)

n2- n -l

kr n 1Ak ,n(x),

L
k=O

where
Ak,n

= {x

EX: kr n < I(x) ~ (k

For any nonnegative elementary Borel function

form

+ l)r n }.

I : (X, X)

1-+

(lR., B) of the

I(x)

= Lai lA/X),
i=l

where ai E lR.+, Ai E X for all i E {l, ... , k}, one defines the integral of I with
respect to J1" denoted

Ix

I dJ1"

or

Ix

I(x) J1,(dx),

or

J1,(f),

by
(110)

The Case of Nonnegative Measurable Functions

If I

: (X, X) 1-+ (iR, B) is nonnegative, the integral is defined by

Jx{ I

dJ1,

= lim t {
ntoo

Jx In dJ1"

(111)

where {fn}n2:l is a nondecreasing sequence ofnonnegative elementary Borel functions In : (X, X) 1-+ (lR., B) such that limntoo t In = I. This definition can be
shown to be consistent, in that the integral so defined is independent of the choice

250

Appendix

of the approximating sequence. Note that the quantity (111) is nonnegative and
can be infinite. It can be shown that if I ::: g, where I, g : (X, X) 1--+ (~, 13) are
nonnegative, then

In particular, if
1+ = max(f, 0)

and

1- = max( - I, 0),

wehave

and therefore,
(112)
Integrable Functions
DEFINITION 9. A measurable function
JL-integrable function if

Ix III

I : (X, X)

dJL <

1--+

(~, 13) is called a

(113)

00.

In this case (see (112)) the right-hand side of

(114)
is meaningful and defines the left-hand side. Moreover, the integral of
respect to JL defined in this way is finite.

with

The integral can be defined for nonintegrable functions in certain circumstances;

for example, it is defined in the nonnegative case even when the function is not
integrable. More generally, if I : (X, X) 1--+ (~, 13) is such that at least one ofthe
integrals
1+ dJL or
1- dJL is finite, one defines

Ix

Ix

(115)
This leads to one of the forms "finite minus finite," "finite minus infinite," and
"infinite minus finite." The case JL(f+) = JL(f-) = + 00 is rigorously excluded
from the definition, because it leads to the indeterminate form "infinite minus
infinite."
Counting Measure and Dirac Measure

The results in the following two examples are easy to prove.

The Lebesgue Integral

251

Any funetion I : Z t-+ lR. is measurable with respeet to P(Z) and .

With the measure IJ. defined in Example 4, and with I ~ Olor instanee, we have

EXAMPLE 7.

=L
00

1J.(f)

anl(n).

n=!

EXAMPLE 8.

Let Ca be the Dirae measure at point a

(lR., ) is ca-integrable, and
ca(f)

X. Then any I : (X, X)

t-+

= I(a).

Elementary Properties of the Integral

First, recall that for all A EX,

Ix

lA dlJ.

= IJ.(A).

(116)

Also, recall the notation JA I dlJ. for Jx lAI dlJ..

THEOREM 10. Let I, g : (X, X)
a, bE R Then

t-+

(i:,13) be IJ.-integrable funetions, and let

(a) al + bg is IJ.-integrable and lJ.(al + bg) = alJ.(f) + blJ.(g),

(b) if I = 0 IJ.-a.e., then 1J.(f) = 0; If 1= g IJ.-a.e., then 1J.(f) = lJ.(g),
(e) if I :s g IJ.-a.e., then 1J.(f) :s lJ.(g),
(d) 11J.(f)1 :s IJ.(I/I),
(e) if I ~ 0 IJ.-a.e. and 1J.(f) = 0, then I = 0 IJ.-a.e.,
(f) if 1J.(1 Af) = Olor all A E X, then I = 0 IJ.-a.e..
(g) if I is IJ.-integrable, then III < 00 IJ.-a.e.
For a complex Borel function I : X t-+ C (Le., I =
t-+ (lR., )) such that IJ.(I/I) < 00, one defines

!I + ih, where fI, h :

(X, X)

(117)

The extension to complex Borel functions of the properties (a), (b), (d), and (f) in
Theorem lO is immediate.
Riemann and Lebesgue
The following result tells us that all the time spent learning about the Riemann
integral has not been in vain.
THEOREM 11. Let I : (lR., ) t-+ (R ) be Riemann-integrable. Then it is
Lebesgue-integrable with respeet to l, and the Lebesgue integral is equal to the
Riemann integral.
EXAMPLE 9. The eonverse is not true: The funetion I defined by I (x) = 1 if x E Ql
and I (x) = 0 if x f/. Ql is a Borel funetion, and it is Lebesgue-integrable with its
integral equal to zero beeause {I =I- O} = Ql, has l-measure zero. However, I is
not Riemann-integrable.

252

Appendix

EXAMPLE

10. The function f : (JR;, B)

fex)

(R 8) defined by

x
1 +x2

does not have a Lebesgue integral, because

+
f (x)

- - 2 1[0

l+x

'

oo)(x)

f-(x)

and

=-

- - 2 1(-00 Oj(X)

l+x

'

have infinite Lebesgue integrals. However, it has a generalized Riemann integral

lim
Atoo

+A

-A

x
--2

l+x

dx

= O.

Limits Under the Integral

The three main results that we need to know in this book are the Lebesgue theorems
(when can we interchange the order of limit and integration ?), the Tonnelli-Fubini
theorems (when can we interchange the order of integration in a multiple integral ?),
and the theorem of completeness of L 2

Lebesgue, Fatou, and Beppo Levi

The following result of Beppo Levi is often called the monotone convergence
theorem.

12. Let fn : (X,.1:') ~ (JR;, 8), n ~ 1, be such that

(i) fn ~ O/L-a.e.,
(ii) fn+l ~ fn /L-a.e.
Then there exists a nonnegativefunction f : (X,.1:') ~ (i, B) such that

THEOREM

lim
ntoo

fn = f

/L-a.e.

and

Jx[ f

d/L = lim
ntoo

t [

Jx fn d/L.

The next result is a useful technical tool called Fatou 's lemma.

13. Let fn : (X, X) ~ (i, B), n ~ 1, be such that fn ~ 0 /L-a.e. for

1. Then

THEOREM

all n

J[x (limntooinf fn) d/L :'S limntooinf

(1
x

fn d/L) .

(118)

The domina ted convergence theorem is also called the Lebesgue theorem:

14. Let fn : (X, .1:') ~ (i, B), n ~ 1, be such that, for some function
f: (X,.1:') ~ (i, B) and some /L-integrablefunction g : (X,.1:') ~ (i, B),

THEOREM

(i) lim fn
ntoo

= f,

/L-a.e.,

The Lebesgue Integral

(ii) Ilnl

:s

Igl JL-a.e.lorall n

253

1.

Then

Ix

= !~~

I dJL

(Ix

In dJL ) .

The results in Theorems 12 and 14 ensure that under certain circumstances limit
and integration may be interchanged (that is, JL(lim In) = lim JL(fn. The classical
counterexample that shows this is not always true is the following:
EXAMPLE

11. For (X, X, JL)

In(x)

= (IR., {3, l), define

=0

1
n

Ixl>-

if

1
n

:s x :s 0,
1
n

One has
lim In(x)
ntoo
that is, limntoo In
lor alt n ~ 1.

= 0 JL-a.e.

=0

if

x =j:. 0,

Therelore, JL(limntoo In)

= O. However,

JL(fn)

=1

Differentiation Under the Integral

A very useful application of the dominated convergence theorem is the theorem

of differentiation underthe integral sign. Let (X, X, JL) be a measure space and let
(a, b) C IR.. Let I : (a, b) x X f-+ IR. and, for all t E (a, b), define It : X f-+ IR.
by It(x) = I(t, x). Assume that for all t E (a, b), It is measurable with respect
to X, and define, if possible, the function I : (a, b) f-+ IR. by the formula
let)

Ix

I(t, x) JL(dx).

(119)

THEOREM 15. Assume thatlor JL-almost alt x thefunction t "rl I(t, x) is continuousatto E (a, b)andthatthereexistsaJL-integrablefunctiong : (X, X) f-+ (i:, )
such that I/(t, x)1 :s Ig(x)1 JL-a.e. lor alt t in a neighborhood V 01 to. Then
I : V f-+ IR. is welt defined and is continuous at to. Furthermore, assume that

(a) t -+ I(t, x) is continuously differentiable on V lor JL-almost alt x,

() For some JL-integrable function h : (X, X) f-+ (i:, )

(t, X)/ < Ih(x)1

/ al
at
-

JL-a.e ..

Then I is differentiable at to and

I I (to)

1-

al (to, x) JL(dx).
x at

(120)

254

Appendix

Proof" Let (tn}n~l be a sequence in V \ {to} such that limntoo tn = to, and define
fn(x) = f(t n, x), f(x) = f(to, x). Then, by dominated convergence,
lim I(tn) = I(to).
ntoo
Also,

l(tn) - I(to)
=
tn - to

---'--'-~

and for some 9

f(tn, x) - f(to, x) dx
JL( ),
x
tn - to

(0, 1), possibly depending upon n,

If(tn,:~ =~(to, x) I ~ Iir (to + 9(tn -

to), X)I

The latter quantity is bounded by Ih(x)l. Therefore, by dominated convergence,

) JL(dx)

lim l(tn) - I(to) = { (lim f(tn, x) - f(to

ntoo
tn - to
Jx ntoo
tn - to

{ af
= Jx
(to, x) JL(dx).

at

The Fubini Theorem

Product Measures
Let (Xl, Xl, JLl) and (X2, X2, JL2) be two measure spaces where JLl and JL2 are
sigma-finite measures.
Define the product set X = Xl X X2 and the product sigma-field X = Xl X X 2,
where by definition the latter is the smallest sigma-field on X containing all sets
ofthe form Al x A 2, where Al E Xl. A 2 E X2.
THEoREM 16. There exists unique measure JL on (Xl

JL(A l x A2)
for all Al

X2, Xl x X2) such that

= JLl(A l )JL2(A2)

(121)

Xl> A 2 E X 2.

The measure JL is the product measure of JLl and JL2, and is denoted JLl JL2.
The above result extends in an obvious manner to a finite number of sigma-finite
measures.
EXAMPLE 12. The typical example of a product measure is the Lebesgue measure
on the space (Rn, Bn): It is the unique measure in on that space that is such that

in( Ai) =

i(Ai) forall Al,.., An

B.

The Lebesgue Integral

255

Tonnelli and Fubini

Going back to the situation with two measure spaces (the case of a finite number
of measure spaces is similar) we have the following result:

17. Let (X I, XI, Ji,1) and (X I, X 2, Ji,2) be two measure spaces in which
Ji,1 and Ji,2 are sigma-finite. Let (X, X, Ji,) = (XI x X2, XI X X 2, Ji,1 Ji,2).
(A) ToneIli. lf I is nonnegative, then, lor Ji,1-almost all XI, the function X2 --+
I(XI, X2) is measurable with respect to X 2, and
THEOREM

XI

--+ (

I(XI, X2) Ji,2(dx2)

X2

is a measurablefunction with respect to XI. Furthermore,

Ix I dJi,

= Ixl [Ix/(XI' X2) Ji,2(dx2)] Ji,1 (dxd.

(123)

(B) Fubini. If I is Ji,-integrable, then, lor Ji,1-almost all XI, the function X2 --+
I(XI, X2) is Ji,2-integrable and XI --+ JX2 I(XI, X2) Ji,2(dx2) is Ji,2-integrable, and
(123) is true.
In this text we shall refer to the global result as the Fubini-Tonelli theorem.
Part (A) says that one can integrate a nonnegative Borel function in any order
of its variables. Part (B) says that the same is true of an arbitrary Borel function if
that function is Ji,-integrable. In general, in order to apply Part (B), one must use
Part (A) with I = 1I1 to ascertain whether or not J 1I1 dJi, < 00.
EXAMPLE 13. Consider thefunction I defined on XI x X2

= (1, 00) x (0, 1) by

thelormula

Wehave

= h(X2)

However;

1
1

h(X2) dx2

1=

00
(-

0,

h(xI)) dxl,

since h ~ f-a.e. on (0, 00). We therelore see that successive integrations yield
different results according to the order in which they are perjormed. As a matter
ollact, I(XI, X2) is not integrable on (0,1) x (1,00).

256

Appendix

Integration by Parts
THEOREM

18. Let

interval (a, b)

111 and 112 be two sigma-finite measures on

(R 8).

For any

lR,

111 a, b ])112a, b])

= {

~a.bl

111 a, t]) 112(dt)

+ ( 112a, t)) 111 (dt).

~a.bl

(124)

Observe that in the first integral we have (a, t] (c1osed on the right), whereas in
the second integral we have (a, t) (open on the right).
Proof' The proof consists of computing the l1-measure of the square (a, b] x
(a, b] in two ways. The first one is obvious and gives the left-hand side of (124).
The second one consists of observing that 11a, b] x (a, b]) = I1(Dd + I1(D2),
where D I = {(x, y);a < y ::::: b, a < X::::: y} and D 2 = (a, b] x (a, b] \ D I . Then
I1(D 1) and I1(D2) are computed using Tonelli's theorem. For instance,
I1(Dd =

and

L(L

I D ,(x, y)111(dx)

I D ,(x, y)111(dX)) 112(dy)

l\a<x:o;yjI11(dx)

= 111a, y]).

Let 11 be aRadon measure on (lR, 8) and let F/i- be its c.dJ. The notation

g(x) F/i-(dx)

stands for IIR g(x) l1(dx). When this integral is used, it is usually called the
Lebesgue-Stieltjes integral of g with respect to F w With this notation, (124)
becomes

F 1(b)F2(b) - F1(a)F1(b)

= (

~.bl

F 1(x)dF2(x)

+(

~a.~

F2(X-) dFl(X),(125)

where F i := F/i- i (i = 1,2). This is the Lebesgue-Stieltjes version of the

integration by parts formula of ca1culus.

The Spaces LP
For a given P :::: 1, L~(I1) is, roughly speaking (see the details below), the collection
of complex-valued Borel functions J defined on X such that
IfIP dl1 < 00.
We shall see that it is a complete normed vector space over C, that is, a Banach
space. Of special interest to Fourier analysis is the case P = 2, since L~(I1) has
additional structure that makes of it a Hilbert space.

Ix

Let (X, X, 11) be a measure space and let J, g be two complex-valued Borel
functions defined on X. The relation R defined by

(fRg)

(f = g l1-a.e.)

The Lebesgue Integral

257

is an equivalence relation, and we shall denote the equivalence dass of I by {f}.

Note that for any p > 0 (using property b of 10),
(fRg)

(Ix

===}

III P dJL

Ix Igl

dJL) .

The operations x, +, *, and multiplication by a scalar a

equivalence dass by
{f}

+ {g} = {f + g},

{f}*

= {f*},

a{f}

= {af},

C are defined on the

{f}{g}

= {fg}.

The first equality means that {f} + {g} is, by definition, the equivalence dass
consisting of the functions 1+ g, where land g are abritrary members of {f}
and {g}, respectively. A similar interpretation holds for the other equalities.
By definition, for a given p ::: 1, L~(JL) is the collection of equivalence dasses
{f} such that
IIIP dJL < 00. Clearly, it is a vector space over C (for the proof
recall that

Ix

CII;

Igl)

s ~ III P + ~ Igl P

since t -+ t P is a convex function when p ::: 1).

In order to avoid cumbersome notation, in this seetion and in general whenever
we consider LP -spaces, we shall write I for {f}. This abuse of notation is harmless
since two members of the same equivalence dass have the same integral if that
integral is defined. Therefore, using loose notation,
(126)

The following is a simple and often used observation.

THEOREM 19.

Let pandq be positive realnumberssuchthat p > q.lfthemeasure

JL on (X, X, JL) is finite, then L~(JL) ~ Lt(JL). In particular, L~(JL) ~ L~(JL).

Proof' From the inequality la Iq S 1 + la IP, true for all a E C, it follows that
JLOIlq) S JL(I) + JLOIIP). Since JL(I) = JL(lR.) < 00, JL(IIlq) < 00 whenever
JL(IIIP) < 00.

Hlder's and Minkowski's Inequalities

THEOREM

20. Let p and q be positive real numbers different Iram 1 such that

-p + -q = 1
(p and q are then said to be conjugate), and let I, g : (X, X) t-+ (i,8) be
nonnegative. Then, we have Hlder's inequality
(127)

In particular,

if I, g

E L~(lR.),

then Ig E L~(lR.).

258

Proof'

Appendix

Let
A

(Ix (fP) dJl)l/P ,

(Ix

(gq) dJl )l/q

We may assurne that 0 < A < 00, 0 < B <

inequality is trivially satisfied.
Define F = J/ A, G = g/ A, so that

Ix FP dJl Ix
=

00,

because otherwise Hlder's

Gq dJl = 1.

The inequality

F(x)G(x)::::: - F(x)P
p

+ -1 G(x)q

(*)

is trivially satisfied if x is such that F(x) = 0 or G(x) = O. If F(x) > 0 and

G(x) > 0, define
sex)

= p In(F(x)),

= q In(G(x)).

tex)

From the convexity of the exponential function and the assumption that 1/ p
l/q = 1,
es(x)/p+t(x)/q :::::

.!.. es(x) + .!.. et(x),

p

and this is precisely the inequality (*). Integrating this inequality yields

1
x

1
p

1
q

+ - = 1,

(FG)dJl::::: -

and this is just (127).

THEOREM 21.

Let P ~ 1 and let J, g : (X, X)

that

Ix

jPdJl < 00,

1-+

(i:, B) be nonnegative and such

Ix gP dJl <

00.

Then, we have Minkowski's inequality

Proof" For p = 1 the inequality is obvious. Therefore, assurne p > 1. From

Hlder's inequality,

Ix J(f +

g)p-l dJl :::::

[Ix jP dJl

r/ [Ix
p

(f + g)(p-l)q

r/

The Lebesgue Integral

259

and

Ix g(f

+ g)p-l dlL

[Ix gP dlL flP [Ix (f + g)(p-l)q flq

Adding together the above two inequalities and observing that (p - l)q = p, we
obtain

One may assume that the right-hand side of (128) is finite and that the lefthand side is positive (otherwise the inequality is trivial). Therefore, !x(f +
g)P dlL E (0,00). We may therefore divide both sides of the last display by
[Jx (f + g)P dlL q. Observing that 1 - 1/q = 1/P yields the desired inequality
(128).
For the last assertion of the theorem, take p = q = 2.

THEOREM 22. Let p ::: 1. The mapping vp

L~(IL)

vp(f) = (Ix'/'p dlL

1-+

[0,00) defined by

)I IP

(129)

defines a norm on L~(IL).

ProoJ-

Clearly, vp(cxf) = Icxlvp(f) for all cx

UX

C, I

E L~(IL).

Also, (vp(f) = 0) <===>

I/IP dlLjlP = 0 ===> (f = 0).
Finally, vp(f + g) S vp(f) + vp(g) for all I, g E L~(IL), by Minkowski's
inequality.
Therefore, vp is a norm.

Riesz-Fischer Theorem
We shall denote vp(f) by IIfll p. Thus L~(IL) is a normed vector space over C,
with the norm 11 . 11 p and the induced distance

dp(f, g)

= 111 -

gllp.

THEOREM 23. Let p ::: 1. The distance d p makes 01 L~ a complete normed space.

In other words, L~(IL) is a Banach space for the norm

ProoJ-

11 .

I p

To show completeness one must prove that for any Cauchy sequence

(fn}n~1 of L~(IL) there exists I E L~(IL) such that limntoo dp(fn, f) = O.

Since {fn}n~1 is a Cauchy sequence (that is, limm,ntoo dp(fn, Im) = 0), one can
select a subsequence (fn,}i~1 such that

dp(fni+l - In) S Ti.

Let
k

gk

=L

i=1

I/ni+l - Inil,

(*)

260

Appendix
00

g=

;=1

I/ni+' - Ini I

By (*) and Minkowski's inequality we have IIgk I p :s 1. Fatou's lemma applied to the sequences {gfk~:1 gives IIgli p :s 1. In particular, any member of
the equivalence c1ass of g is finite /L-almost everywhere, and therefore

L (jni+' (X) 00

In, (X) +

ln/x))

;=1

converges absolutely for /L-almost all x. Call this limit I(x) (set I(x)
this limit does not exist). Since

= 0 when

k-I

In,

+L

;=1

(jni+' - Ini) = Ink'

we see that

I = lim Ink /L-a.e.

ktoo

One must show that I is the limit in Lt(/L) of Unkk~:I' Let e > O. There exists an
integer n = N(e) such that II/n - Im I p :s e whenever m, n 2: N. For all m > N,
by Fatou's lemma we have

Jx[ 1I -

ImIPd/L:S liminfll/ni - ImIPd/L:S e P

I~OO

Therefore, I - Im E Lt(/L), and consequently, I

the last inequality that
lim

m--+oo

111 - Imllp

Lt(/L). It also follows from

= O.

Terminology. For p 2: 1, Lt(/L) is a Banach space (a complete normed vector

space) over Co This phrase will implicitly assume that the norm is defined as in
(129). When /L is the Lebesgue measure on jRn, we write Lt(jRn) instead of Lt(/L)
(with a slight symbolic inconsistency).
In the proof of Theorem 23 we obtained the following result.
THEOREM 24. Let Unk:1 be a convergent sequence in Lt(/L), where p 2: 1, and
let I be the limit.
A subsequence {/ni k::1 can then be chosen such that

lim f,n

;too

'

=I

/L- a.e.

(130)

Note that the statement in (130) is about functions and not about equivalence
c1asses. The functions thereof are any members of the corresponding equivalence
c1ass. In particular, since when a given sequence of functions converges /L-a.e. to
two functions, these two functions are necessarily equal/L-a.e.

References
THEOREM 25.

If{fnln:o:! converges both to f in L~{J.1) and to g f-L-a.e., then f

261

=g

f-L-a.e.

A most interesting special case of LP -space, particularly in view of its relevance

to signal processing, is when p = 2. In this case
THEOREM

26. L~(f-L) is a complete normed space with respect to the norm

IIfII =

[Ix

Ifl 2 df-L

f/2

This norm is derived from a Hermitian product, namely,

(j, g) =

Ix

fg* df-L

in the sense that

IIfII 2

= (j, f).

L~(f-L) is a Hilbert space over C (see Section CII).

Approximation Theorems
We now quote the approximation results used in the main text.

27. Let f E L~(lR.), P ~ 1. There exists a sequence {fnln:o:! of continuous functions fn : IR t-+ C with compact support that converges to f in

THEOREM

L~(IR).

(To have compact support means, for a continuous function, to be null outside
some c10sed bounded interval.)
Let f E L~(IR), P ~ 1. There exists a sequence {fnln:o:! offunctions
fn : IR t-+ C which are finite linear combinations ofindicatorfunctions ofintervals,
that converges to f in L~(IR).

THEOREM 28.

29. Let fE Lt([-n, +nD be a 2n-periodicfunction (that is, f(t) =

f(t + 2n)forall t E IR, and J~: If(t)1 d t < (0). There exists a sequence {fnln:o:!
of functions fn : IR t-+ C with continuous derivatives that converges to f in
Lt([ -n, +n D.

THEOREM

References
[Dl]
[D2]
[D3]
[D4]
[D5]

de Barra, G. (1981). Measure Theory and Integration, EIlis Horwood: Chichester.

Halmos, P.R. (1950). Measure Theory, Van Nostrand: New York.
Royden, H.L. (1988). Real Analysis, 3rd ed., MacMillan: London.
Rudin, W. (1966). Real and ComplexAnalysis, McGraw-Hill: New York.
Taylor, A.E. (1965). General Theory of Functions and Integration, Blaisdell,
Waltham, MA, Dover edition, 1985.

Glossary of Symbols

P(X), the collection of all subsets of set X.

card (X), or

lXI, the cardinal of set x; the number of elements in X.

N, the integers.
N+, the positive integers.
Z, the relative integers.
IR, the reals.
IR+, the positive reals.
C, the complex numbers.

z*, the complex conjugate of z

C.

(a, b], interval of IR open to the left, c10sed to the right; and similar notation for
the other types of intervals.
Re(z), the real part of z E C.

Im(z), the imaginary part of z

J : IR

1-+

C.

C, a function from IR to C; equivalent notation: J, JO, J(t).

J(n), the nth derivative of J; J(O)

= J.

J * g, the convolution product of J and g:

Cf * g)(t) =

J(t - s)g(s)ds =

g(t - s) J(s)ds.

264

Glossary of Symbols

f*n, the nth convolution product of 1 by itself:

1* = I; 1*(n+1) = 1

* f*n.

lA, the indicator function of a set A; lA (t) = 1 if t

h(t)

A, =

otherwise.

= l(t)l[o,Tj(t).

sinc (f) = sin(:rrt)/m, the cardinal sine function.

rectT (t)

l[_~,+~j(t),

the rectangle function.

l, the Lebesgue measure on lR; l([a, b])

=b-

a.

a.e., almost everywhere with respect to the Lebesgue measure.

f-L-a.e., almost everywhere with respect to the measure f-L.
L~(lR), the set (equivalence c1asses) of measurable functions

that

flR

I/(t)IP dt <

f:

1 : lR

L~([a, b]), the set (equivalence c1asses) ofmeasurable functions

such that

~ C such

00.

I/(t)IP dt <

1 : Ca, b]

~ C

00.

L~,loc(lR), the set (equivalence c1asses) of measurable functions

that I(t)l[a,bj(t) E L~(lR) for all Ca, b] c lR.

1 : lR ~

C such

Lfoc' short for L~,loc(lR).

l~(Z), the set of complex sequences

{Xn}nEZ

such that

LnEZ IXn 12

<

00.

Cn , the set of ntimes continuously differentiable functions

1 : lR ~ C.

COO, the set of infinitely differentiable functions 1 : lR

C.

1 : lR ~ C.
C~, the set of continuous functions 1 : lR ~ C with bounded support.
C([O, T]), the set of continuous functions 1 : [0, T] ~ C.
Co, the set of continuous functions

V, the set oftest functions q; : lR

C; C oo and compact support.

V', the set of distributions on lR; the set of linear forms on V.

S, the set of functions

1 : [0, T]

C in C oo and with all its derivatives rapidly

decreasing.
Sr. the set of functions 1 : [0, T]
order r rapidly decreasing.

C in

er and with all its derivatives up to

S', the set of tempered distributions on lR; the set oflinear forms on S.
(x, Y) H, the Hermitian product of x, y EH, H Hilbert space.

IIx 11 H

= (x, x) H )1/2, the norm ofx EH, H Hilbert space.

265

x..ly; x is orthogonal to y; (x, y) H = O.

G.L, the orthogonal complement of G.
PG, the orthogonal projection on G.

= (I/2n) J~rr f(t)e- int dt, the nth Fourier coefficient of f.

Sn(f) = L~: ck(f)e+ ikt , the Fourier series.

cn(f)

S(f)

= LnEZ cn(f)e+ int , the formal Fourier series development.

j(v)

= JR. f(t)e-2irrvt dt, the Fourier transform of f.

H(z)

= LnEZ hnz n, the z-transform of {hn}nEZ,

CPj,n(t)

= 2 j / 2cp(2 j t -

n).

Index

Aliasing, 79, 100

all-pass, 110
almost everywhere, 248
amplitude gain, 57
analytic signal, 69
autocorrelation function, 14, 156
autoreproducing Hilbert space, 230
B-splines, 223
band-pass, 58
base-band, 68
Bessel's inequality, 146
biorthonormal system, 149
Borel function, 244
Borel set, 244
Borel sigma-field, 244
bounded support, 8
bounded variation, 36
Butterworth filter, 67
C.dJ., 246
cardinal sine, 10
Cauchy sequence, 135
causal filter, 55, 101, 109
complete, 135
complex envelope, 69
complex signal, 7

convolution, 14
convolution-multiplication rule, 14,25
convolutional filter, 55, 101
counting measure, 246
cut-off frequency, 58
Decomposable signal, 61
dense, 138
differentiating filter, 59
dilation equation, 209
Dini's theorem, 34
Dirac comb, 91
Dirac measure, 246
Dirichlet integral, 32
Dirichlet kerneI, 32, 36
dispersive channel, 72
distance, 135
distribution function, 246
dominated convergence, 252
down-sampling, 118
Elementary Borel function, 249
Fejer's idenlity, 110
Fejer's kerneI, 40
Fejer's lemma, 112
Fatou's lemma, 252

268

Index

feedback filter, 60, 105

finite-energy signal, 8
finite measure, 246
finite-power signal, 24
FIR,115
formal Fourier series, 31
Fourier coefficient, 24
Fourier inversion formula, 17
Fourier sum, 100
Fourier transform, 9
Franklin's wavelet, 222
frequency response, 56, 101
frequency transposition, 69
Ff,9
Fubini's theorem, 255
Gabor transform, 178
Gaussian pulse, 10
Gibbs's phenomenon, 38
Gram-Schmidt,145
group delay, 73
Hlder's inequality, 257
Haar filter, 121
Haar mother wavelet, 202
Haar wavelet, 192
Heisenberg's inequality, 176
Hermitian product, 133
Hilbert basis theorem, 148
Hilbert space, 135
Hilbert span, 136
Hilbert subspace, 136
Hilbert's filter, 58
Impulse response, 55, 101
indicator function, 8
integrating filter, 59
integration by parts, 256
isometry extension theorem, 138
isomorphic, 137
Jordan's theorem, 36
Kernel of an MRA, 229
Lebesgue measure, 246
Lebesgue-Stieltjes integral, 256
linear form, 143
linear isometry, 137

localization principle, 33
locally integrable, 8, 23
locally square-integrable, 8
locally stable, 23
low-pass, 58
Mallat's algorithm, 211
measurable function, 244
measure, 246
measure space, 246
metric space, 135
Mexican hat, 192
Meyer's wavelet, 218
Minkowski's inequality, 258
monotone convergence, 252
Morlet's pseudo-wavelet, 192
mother wavelet, 185,201
MRA,196
multiresolution analysis, 196
Norm, 134
norm of a linear form, 143
Nyquist condition, 85
Octave band filter, 122
orthogonal, 134
orthogonal complement, 139
orthogonalprojection,140
orthogonalsum, 143
orthonormal system, 145
Parallelogram identity, 134
partial response signaling, 87
periodic signal, 23
phase, 57
phase delay, 73
Plancherel-Parseval identity, 157, 162
Poisson sum formula, 91
polarization identity, 134
pre-Hilbert space, 133
probability measure, 246
product measure, 254
product sigma-field, 254
projection principle, 141
projection theorem, 140
pulse amplitude modulation, 84
Pythagoras' theorem, 134

QMF,120

Index
quadrature components, 68
quadrature mirror filter, 120
quadrature multiplexing, 70
quasi-positive delta sequence, 231
Radon measure, 246
realizable filter, 55, 101
rectangular pulse, 9
regularization lemma, 19
regularizing function, 19
reproducing kernel, 230
residue theorem, 105
resolution level, 196
Riesz basis, 164
Riesz's representation theorem, 144
Riesz-Fischer theorem, 259
root mean-square width, 176
Sampie and hold, 78
scalar product, 133
scaling function, 196
Schwarz inequality, 134
Shannon wavelet, 210
short-time Fourier transform, 178
sigma-additivity, 246

sigma-fie1d, 243
sigma-finite measure, 246
spectral decomposition, 62
spectral factorization, 112
spectrum folding, 79
stable, 7, 100
synchronous detection, 70
Tonelli's theorem, 255
total, 148
transfer function, 104
triangular pulse, 14
Uncertainty princip1e, 175
up-sampling, 118
Wavelet orthonormal basis, 201
Weierstrass theorem, 41
WFT,178
WFT inversion formula, 179
window function, 169, 178
windowed Fourier transform, 178
WT, 185
Z-transform, 104

269