A Tutorial on

Wavelets and their Applications

Martin J. Mohlenkamp

University of Colorado at Boulder

Department of Applied Mathematics
[email protected]

This tutorial is designed for people with little or no

experience with wavelets. We will cover the basic
concepts and language of wavelets and computational
harmonic analysis, with emphasis on the applications
to numerical analysis. The goal is to enable those who
are unfamiliar with the area to interact more
productively with the specialists.

Thanks to Gregory Beylkin, Willy Hereman, and

Lucas Monzón for help with this tutorial.
 Goals

• Enable the wavelet novice to interact

more productively with the specialists, by

– Introducing the basic concepts and

language

– Doing some physics related examples

– Explaining why people like them

Not Goals

• Give the history and assign credit

• Convince you that wavelets are better

than any particular technique for any
particular problem
 Outline

• Multiresolution Analysis

• wavelets (traditional)
• properties
• fast algorithms

• Connections with Fourier Analysis

• Local Cosine and the phase plane

• 1D Schrödinger example
• wavelet packets

• Operators in Wavelet Coordinates

• density matrix example

• Review and Questions

 Multiresolution Analysis

A multiresolution analysis is a decomposition

of L2(R), into a chain of closed subspaces

· · · ⊂ V2 ⊂ V1 ⊂ V0 ⊂ V−1 ⊂ V−2 ⊂ · · · ⊂ L2 (R)

such that

1. Vj = {0} and
T
j∈Z
j∈Z Vj is dense in L2(R)
S

2. f (x) ∈ Vj if and only if f (2x) ∈ Vj−1

3. f (x) ∈ V0 if and only if f (x − k) ∈ V0

for any k ∈ Z.

4. There exists a scaling function ϕ ∈ V0

such that {ϕ(x − k)}k∈Z is an orthonormal
basis of V0.
 Multiresolution Analysis

Let Wj be the orthogonal complement

of Vj in Vj−1:

Vj−1 = Vj ⊕ Wj ,
so that

L2(R) =
M
Wj .
j∈Z

Selecting a coarsest scale Vn and finest scale

V0, we truncate the chain to
Vn ⊂ · · · ⊂ V 2 ⊂ V 1 ⊂ V 0
and obtain
n
M
V0 = Vn Wj .
j=1

From the scaling function ϕ we can define

the wavelet ψ, such that {ψ(x − k)}k∈Z is an
orthonormal basis of W0.
 Multiresolution Analysis

Example: Haar wavelets


 1 for 0 < x < 1
ϕ(x) =
 0 elsewhere.

V0 = span({ϕ(x − k)}k∈Z) are piecewise

constant functions with jumps only at
integers.






1 for 0 < x < 1/2
ψ(x) = −1 for 1/2 ≤ x < 1 ,


 0

elsewhere.
W0 = span({ψ(x − k)}k∈Z) are piecewise
constant functions with jumps only at
half-integers, and average 0 between integers.
 Multiresolution Analysis

V0
B
B
B
B
B
B
B
? BNB
V1 W1
B
B
B
B
B
B
B
? BNB
V2 W2
B
B
B
B
B
B
B
? BNB
V3 W3

V0 = V3 ⊕ W 3 ⊕ W 2 ⊕ W 1
 Where’s the Wavelet?

Since Wj is a dilation of W0, we can define

ψj,k = 2−j/2 ψ(2−j x − k)

and have
Wj = span({ψj,k (x)}k∈Z).

In this example,
 
 
 
 
 
W1 = span 



 
 
 
 

 

W2 = span 




 

W3 = span 



 The Wavelet Zoo

φ ψ
1
1
0.8
0.5
0.6

0.4 0.2 0.4 0.6 0.8 1

Haar 0.2 -0.5

0.2 0.4 0.6 0.8 1 -1

1.5
1.25
1 1
0.75 0.5
0.5
0.25 -1 -0.5 0.5 1 1.5 2
daub4 0.5 1 1.5 2 2.5 3
-0.5

-0.25 -1

1 1
0.8
0.6 0.5

0.4
0.2 -4 -2 2 4 6

daub12 -0.2
2 4 6 8 10 -0.5

-0.4 -1

1.5 1
0.5
1
-2 -1 1 2 3
-0.5
0.5
-1
coif4 -1.5
-2 -1 1 2 3 -2

1
0.5
0.8
0.6 -7.5 -5 -2.5 2.5 5 7.5
0.4
-0.5
coif12 0.2
-1
-5 -2.5 2.5 5 7.5 10
-0.2
 Vanishing Moments

Wavelets are usually designed with vanishing

moments:
Z +∞
ψ(x)xm dx = 0, m = 0, . . . , M − 1,
−∞
which makes them orthogonal to the low
degree polynomials, and so tend to compress
non-oscillatory functions.

For example, we can expand in a Taylor series

x M −1
f (x) = f (0) + f 0(0)x + · · · + f (M −1)(0)
(M − 1)!
(M ) xM
+f (ξ(x))
M!
and conclude
M
 

 (M ) x 
|hf, ψi| ≤ max f (ξ(x)) .
x  M!

Haar has M = 1.
 Quadrature Mirror Filters

Wavelets are designed through properties of

their “quadrature mirror filter” {H, G}.
1
Haar ↔H = √ [1, 1]
2
1
G= √ [1, −1]
2
1 √ √ √ √
daub4 ↔H = √ [1 + 3, 3 + 3, 3 − 3, 1 − 3]
4 2
G = [H(3), −H(2), H(1), −H(0)]}
(The values are usually not in closed form.)

For instance, vanishing moments

Z +∞
ψ(x)xm dx = 0, m = 0, . . . , M − 1,
−∞
are a consequence of

G(i)im = 0,
X
m = 0, . . . , M − 1.
i
 Trade-offs

You can get

• higher M
• more derivatives
• closer to symmetric
• closer to interpolating (coiflets)

if you pay by increasing the filter length,

which causes

• longer (overlapping) supports, and so

worse localization.

• slower transforms

The cost is linear (in M etc.).

 Fast Wavelet Transform

Sample onto the finest resolution and then

apply the “quadrature mirror filter” {H, G}.

V0 • • • • • • • •

A

A
H

AU
A
G

• • • • ◦ ◦ ◦ ◦ W1
V1
A

A
H


 AU
A
G

V2 • • ◦ ◦ W2

A

A
H

AU
A
G

V3 • ◦ W3

The total cost of this cascade is 2 · N · L,

where L is the length of the filter.
 There are many, many ’lets

By loosening the definitions, you can get

• symmetric
• interpolating
• 2D properties (brushlets)
• ...

Rules of thumb:

• Use a special purpose wavelet if and only

if you have a special need.

• Use one vanishing moment per digit

desired (truncation level).

• Do not use more derivatives than your

function typically has.
 Multiwavelets
(Polynomial version on [0, 1])

Fix k ∈ N, and let Vn be the space of

functions that are polynomials of degree less
than k on the intervals (2n j, 2n(j + 1)) for
j = 1, . . . , 2−n − 1, and 0 elsewhere.

V0 is spanned by k scaling functions.

W0 is spanned by k multiwavelets.

By construction, the wavelets have k

vanishing moments, and so the same sparsity
properties as ordinary wavelets.

The wavelets are not even continuous, but

this allows weak formulations of the
derivative, which allows better treatment of
boundaries.
 Connections with Fourier Analysis

Fourier analysis gives an understanding of

frequency, but “non-stationary” signals beg
for space (time) localization.

This need motivates Computational Harmonic

Analysis and its tools, such as wavelets and
local cosine.

The theory and intuition are still based on

Fourier analysis.
 Local Cosine Basis

Partition the line (interval, circle) with

· · · ai < ai+1 · · · , Ii = [ai, ai+1 ] .

Construct a set of bells: {bi(x)}

P 2
with i bi (x) = 1, bi(x)bi−1 (x) even about ai,
and bi(x)bj (x) = 0 if j 6= i ± 1.

Construct the cosines which are even on the

left and odd on the right
 v !
2 (j + 1/2)π(x − ai) 
u
j
 u
c (x) = cos .
 i
t
ai+1 − ai ai+1 − ai 

j
{bi(x)ci (x)} forms an orthonormal basis with
fast transform based on the FFT.
 Phase Plane Intuition

If a function has ‘instantaneous frequency’

ν(x), it should be represented by those Local
Cosine basis elements whose rectangles
intersect ν(x). There are max{∆l, 1} of these.
6

ν(x)

1/l

|I| = l x
 Local Cosine Phase Plane

Consider the eigenfunctions of −∆ − C/x,

whichqhave instantaneous frequency
νn = C/x + λn .

6
ξ

νn

P
-
x

We can get an efficient representation (with

proof) by adapting, but this is not very
flexible, especially in higher dimensions.
 Wavelet Phase Plane

The multiresolution analysis divides the phase

plane differently:

6
ξ

W1

W2

W3
W4
V4 -
x

For wavelets this is intuitive but not rigorous.

 Wavelet Phase Plane

Consider
q again the instantaneous frequency
νn = C/x + λn .

6
ξ

-
x

We get an efficient representation without

adapting, so the location of the discontinuity
is not important.
 Tones

Sustained high frequencies, such as those in

spherical harmonics

are a problem for Wavelets, but not for local

cosine.

To enable wavelets to handle such functions,

“wavelet packets” were developed.
Wavelet Packets and Best Basis Searches

Idea: by filtering the wavelet spaces, we can

partition phase space in different ways:

V0 • • • • • • • •

A

A
H

AU
A
G

• • • • ◦ ◦ ◦ ◦ W1
V1
A
A

A
A
H



AU
A
G H




G
AU
A

V2 • • ◦ ◦

A
A
A
A

A
A
A
A
H

AU
A
G H



G H
AU
A


 AU
A
G H

AU
A
G

V3 • ◦

Any choice of decompositions gives a wavelet

packet representation.

A fast tree search can find the “best basis”.

 Wavelet Packets Phase Plane

A



 AU
A

A
A


AU
 A


 AU
A

If we choose:
A
A



 AU
A

AU
 A

A
A



 AU
A
AU


 A

ξ6

then our phase

plane looks like:

-
x
 Operators

There are many competing, adaptive ways to

represent functions.

It is more interesting to consider operators

and develop operator calculus.

Α1 Β 21 Β 31 Β 41 ^
d1 d1

=
Γ 21 2 ^2
Α2 Β 32 Β 42 d d

Γ1
3
Γ2
3 Α 3 Β 43 d3 ^3
d
Γ 14 Γ 42 Γ 43 Τ 3 s3 ^s 3

Many operators are sparse in Wavelet bases.

 Operators in the Nonstandard Form

The nonstandard form gives a more isotropic,

and often more sparse, representation.

Α1 Β1 d1 d^ 1

Γ1 s1 ^s 1

=
d2 ^
Α2 Β2 d2

Γ2 s2 ^s 2

Α 3 Β3 d3 ^3
d
Γ 3Τ 3 s3 ^s 3
 Operators in Wavelets

Hamiltonian

300
−∆ −
|x|

Density Matrix
with 15
eigenfunctions
 Operators in Wavelets

Some operators can be computed rapidly.

1
straightforward disretization
full wavelet representation
0.9 adapted wavelet projection at 1e−8
adapted wavelet projection at 1e−5

0.8

0.7
Significant coefficient ratio

0.6

0.5

0.4

0.3

0.2

0.1

0
0 5 10 15 20 25 30 35 40
Number of iterations

Here the density matrix is computed via the

sign iteration

T0 = T /||T ||2
Tk+1 = (3Tk − Tk3 )/2, k = 0, 1, . . .
 Philosophical Review

Multiscale assumption: Efficient when high

frequencies (sharp features) happen for a
short amount of time/space.

Cleanly adaptive: Refine or coarsen the

“grid” by adding or deleting basis
functions.

Automatically adaptive: Simply truncate

small coefficients.

Tunable: Decide on the needed precision and

properties first, then choose which
wavelet.