Digital Image Processing

Image Restoration
Concept of Image Restoration

Image restoration is to restore a degraded image back to

the original image while image enhancement is to
manipulate the image so that it is suitable for a specific
application.

Degradation model:
g ( x, y )  f ( x, y )  h( x, y )   ( x, y )

where h(x,y) is a system that causes image distortion and

h(x,y) is noise. (Images from Rafael C. Gonzalez and Richard E.
Wood, Digital Image Processing, 2nd Edition.
Noise Models
Noise cannot be predicted but can be approximately described in
statistical way using the probability density function (PDF)
Gaussian noise:
1  ( z   ) 2 / 2 2
p( z )  e
2
Rayleigh noise
 2 ( z  a )2 / b
( z  a )e for z  a
p( z )   b
 0 for z  a

Erlang (Gamma) noise

 a b z b1
 ( z  a )e az for z  0
p( z )   (b  1)!
 0 for z  0
Noise Models (cont.)

Exponential noise

p( z )  ae  az
Uniform noise
 1
for a  z  b
p( z )   b - a
 0 otherwise

Impulse (salt & pepper) noise

 Pa for z  a

p( z )   Pb for z  b
0 otherwise

PDF: Statistical Way to Describe Noise

PDF tells how much

each z value occurs.

(Images from Rafael C. Gonzalez and Richard E.

Wood, Digital Image Processing, 2nd Edition.
Image Degradation with Additive Noise

g ( x, y )  f ( x, y )   ( x, y )

Degraded images

Original image

Histogram

(Images from Rafael C. Gonzalez and Richard E.

Wood, Digital Image Processing, 2nd Edition.
Image Degradation with Additive Noise (cont.)

g ( x, y )  f ( x, y )   ( x, y )

Degraded images

Original image

Histogram

(Images from Rafael C. Gonzalez and Richard E.

Wood, Digital Image Processing, 2nd Edition.
Periodic Noise

Periodic noise
looks like dots
In the frequency
domain
(Images from Rafael C. Gonzalez and Richard E.
Wood, Digital Image Processing, 2nd Edition.
Estimation of Noise

We cannot use the image

histogram to estimate
noise PDF.

It is better to use the

histogram of one area
of an image that has
constant intensity to
estimate noise PDF.

(Images from Rafael C. Gonzalez and Richard E.

Wood, Digital Image Processing, 2nd Edition.
Periodic Noise Reduction by Freq. Domain Filtering
Degraded image DFT
Periodic noise
can be reduced by
setting frequency
components
corresponding to
noise to zero.

Band reject filter Restored image

(Images from Rafael C. Gonzalez and Richard E.
Wood, Digital Image Processing, 2nd Edition.
Band Reject Filters
Use to eliminate frequency components in some bands

Periodic noise from the

previous slide that is
Filtered out.

(Images from Rafael C. Gonzalez and Richard E.

Wood, Digital Image Processing, 2nd Edition.
Notch Reject Filters
A notch reject filter is used to eliminate some frequency components.

(Images from Rafael C. Gonzalez and Richard E.

Wood, Digital Image Processing, 2nd Edition.
Notch Reject Filter:
Notch filter
Degraded image DFT (freq. Domain)

(Images from Rafael C. Gonzalez and Richard E.

Wood, Digital Image Processing, 2nd Edition.
Noise Restored image
Example: Image Degraded by Periodic Noise
Degraded image

DFT
(no shift)

DFT of noise Noise Restored image

(Images from Rafael C. Gonzalez and Richard E.
Wood, Digital Image Processing, 2nd Edition.
Mean Filters
Degradation model:

g ( x, y )  f ( x, y )  h( x, y )   ( x, y )

To remove this part

Arithmetic mean filter or moving average filter (from Chapter 3)

ˆf ( x, y )  1
 g ( s, t )
mn ( s ,t )S xy
Geometric mean filter mn = size of moving window
1
  mn
fˆ ( x, y )    g ( s, t ) 
 ( s ,t )S 
 xy 
Geometric Mean Filter: Example

Original Image
image corrupted
by AWGN

Image Image
obtained obtained
using a 3x3 using a 3x3
arithmetic geometric
mean filter mean filter

AWGN: Additive White Gaussian Noise (Images from Rafael C. Gonzalez and Richard E.
Wood, Digital Image Processing, 2nd Edition.
Harmonic and Contraharmonic Filters
Harmonic mean filter
mn
fˆ ( x, y )  Works well for salt noise
1

( s ,t )S xy g ( s, t )
but fails for pepper noise

Contraharmonic mean filter mn = size of moving window

 g ( s , t ) Q 1
Positive Q is suitable for
fˆ ( x, y ) 
( s ,t )S xy eliminating pepper noise.
 g ( s, t )
( s ,t )S xy
Q Negative Q is suitable for
eliminating salt noise.

Q = the filter order

For Q = 0, the filter reduces to an arithmetic mean filter.
For Q = -1, the filter reduces to a harmonic mean filter.
Contraharmonic Filters: Example

Image Image
corrupted corrupted
by pepper by salt
noise with noise with
prob. = 0.1 prob. = 0.1

Image Image
obtained obtained
using a 3x3 using a 3x3
contra- contra-
harmonic harmonic
mean filter mean filter
With Q = 1.5 With Q=-1.5

(Images from Rafael C. Gonzalez and Richard E.

Wood, Digital Image Processing, 2nd Edition.
Contraharmonic Filters: Incorrect Use Example

Image Image
corrupted corrupted
by pepper by salt
noise with noise with
prob. = 0.1 prob. = 0.1

Image Image
obtained obtained
using a 3x3 using a 3x3
contra- contra-
harmonic harmonic
mean filter mean filter
With Q=-1.5 With Q=1.5

(Images from Rafael C. Gonzalez and Richard E.

Wood, Digital Image Processing, 2nd Edition.
Order-Statistic Filters: Revisit
Original image
subimage

Statistic parameters
Mean, Median, Mode,
Min, Max, Etc.

Moving
window

Output image
Order-Statistics Filters
Median filter

fˆ ( x, y )  median g ( s, t )
( s ,t )S xy

Max filter

fˆ ( x, y )  max g ( s, t ) Reduce “dark” noise

( s ,t )S xy
(pepper noise)
Min filter

fˆ ( x, y )  min g ( s, t ) Reduce “bright” noise

( s ,t )S xy (salt noise)

Midpoint filter

ˆf ( x, y )  1  max g ( s, t ) min g ( s, t )

2  ( s ,t )S xy ( s ,t )S xy 
Median Filter : How it works
A median filter is good for removing impulse, isolated noise

Salt noise
Pepper noise
Median

Sorted
Degraded image Moving array
window
Salt noise Filter output
Pepper noise
Normally, impulse noise has high magnitude
and is isolated. When we sort pixels in the
moving window, noise pixels are usually
at the ends of the array.
Therefore, it’s rare that the noise pixel will be a median value.
Median Filter : Example
1 2
Image
corrupted
by salt-
and-pepper
noise with
pa=pb= 0.1

3 4

Images obtained using a 3x3 median (Images

filter from Rafael C. Gonzalez and Richard E.
Wood, Digital Image Processing, 2nd Edition.
Max and Min Filters: Example

Image Image
corrupted corrupted
by pepper by salt
noise with noise with
prob. = 0.1 prob. = 0.1

Image Image
obtained obtained
using a 3x3 using a 3x3
max filter min filter

(Images from Rafael C. Gonzalez and Richard E.

Wood, Digital Image Processing, 2nd Edition.
Alpha-trimmed Mean Filter

Formula:

1
fˆ ( x, y ) 
mn  d
 g ( s, t )
( s ,t )S xy
r

where gr(s,t) represent the remaining mn-d pixels after

removing the d/2 highest and d/2 lowest values of g(s,t).

This filter is useful in situations involving multiple types

of noise such as a combination of salt-and-pepper and
Gaussian noise.
Alpha-trimmed Mean Filter: Example
1 2
Image
additionally
Image corrupted
corrupted by additive
by additive salt-and-
uniform pepper
noise noise

Image 2 Image 2
obtained obtained
using a 5x5 using a 5x5
arithmetic geometric
mean filter mean filter

(Images from Rafael C. Gonzalez and Richard E.

Wood, Digital Image Processing, 2nd Edition.
Alpha-trimmed Mean Filter: Example (cont.)
1 2
Image
additionally
Image corrupted
corrupted by additive
by additive salt-and-
uniform pepper
noise noise

Image 2
Image 2 obtained
obtained using a 5x5
using a 5x5 alpha-
median filter trimmed
mean filter
with d = 5
Alpha-trimmed Mean Filter: Example (cont.)

Image Image
obtained obtained
using a 5x5 using a 5x5
arithmetic geometric
mean filter mean filter

Image
Image obtained
obtained using a 5x5
using a 5x5 alpha-
median filter trimmed
mean filter
with d = 5
Adaptive Filter
General concept:
- Filter behavior depends on statistical characteristics of local areas
inside mxn moving window
- More complex but superior performance compared with “fixed”
filters
Statistical characteristics:
Local mean: Noise variance:
1
  2
mL  g ( s, t )
mn ( s ,t )S xy
Local variance:
1
 
2
L  (
mn ( s ,t )S xy
g ( s , t )  m L ) 2
Adaptive, Local Noise Reduction Filter
Purpose: want to preserve edges

Concept:
1. If sh2 is zero,  No noise
the filter should return g(x,y) because g(x,y) = f(x,y)

2. If sL2 is high relative to sh2,  Edges (should be preserved),

the filter should return the value close to g(x,y)

3. If sL2 = sh2,  Areas inside objects

the filter should return the arithmetic mean value mL
Formula:
 2
ˆf ( x, y )  g ( x, y )   g ( x, y )  m 
L2 L
Adaptive Noise Reduction Filter: Example
Image Image
corrupted obtained
by additive using a 7x7
Gaussian arithmetic
noise with mean filter
zero mean
and s2=1000
Image
Image obtained
obtained using a 7x7
using a 7x7 adaptive
geometric noise
mean filter reduction
filter

(Images from Rafael C. Gonzalez and Richard E.

Wood, Digital Image Processing, 2nd Edition.
Adaptive Median Filter
Purpose: want to remove impulse noise while preserving edges
Algorithm: Level A: A1= zmedian – zmin
A2= zmedian – zmax
If A1 > 0 and A2 < 0, goto level B
Else increase window size
If window size <= Smax repeat level A
Else return zxy
Level B: B1= zxy – zmin
B2= zxy – zmax
If B1 > 0 and B2 < 0, return zxy
where zmin = minimum grayElselevel
return zmedian
value in Sxy
zmax = maximum gray level value in Sxy
zmedian = median of gray levels in Sxy
zxy = gray level value at pixel (x,y)
Adaptive Median Filter: How it works
Level A: A1= zmedian – zmin Determine
A2= zmedian – zmax whether zmedian
If A1 > 0 and A2 < 0, goto level B is an impulse or not

Else  Window is not big enough

increase window size
If window size <= Smax repeat level A
Level B: Else return zxy
Determine
 zmedian is not an impulsewhether zxy
is an impulse or not
B1= zxy – zmin
B2= zxy – zmax
If B1 > 0 and B2 < 0,  zxy is not an impulse
return zxy  to preserve original details
Adaptive Median Filter: Example

Image corrupted Image obtained Image obtained

by salt-and-pepper using a 7x7 using an adaptive
noise with median filter median filter with
pa=pb= 0.25 Smax = 7

More small details are preserved

(Images from Rafael C. Gonzalez and Richard E.

Wood, Digital Image Processing, 2nd Edition.
Inverse Filter
From degradation model:
G ( u, v )  F ( u, v ) H ( u, v )  N ( u, v )

after we obtain H(u,v), we can estimate F(u,v) by the inverse filter:

G (u, v ) N ( u, v )
Fˆ (u, v )   F (u, v ) 
H (u, v ) H (u, v )

Noise is enhanced
when H(u,v) is small.
To avoid the side effect of enhancing
noise, we can apply this formulation
to freq. component (u,v) with in a
radius D0 from the center of H(u,v).
In practical, the inverse filter is not
Popularly used. (Images from Rafael C. Gonzalez and Richard E.
Wood, Digital Image Processing, 2nd Edition.
Inverse Filter: Example

Result of applying Result of applying

Original image the full filter the filter with D0=40

Blurred image Result of applying Result of applying

Due to Turbulence the filter with D0=70 the filter with D0=85
0.0025( u 2  v 2 )5 / 6
H ( u, v )  e
Wiener Filter: Minimum Mean Square Error Filter
Objective: optimize mean square error: e 2  E ( f  fˆ ) 2  
Wiener Filter Formula:
 H *
( u , v ) S ( u , v ) 
Fˆ (u, v )   f
2 G ( u , v )
 S f (u, v ) H (u, v )  S (u, v ) 
 H * ( u, v ) 
 2 G ( u , v )
 H (u, v )  S (u, v ) / S f (u, v ) 
 1 H ( u, v )
2

 2 G ( u , v )
 H (u, v ) H (u, v )  S (u, v ) / S f (u, v ) 
where
H(u,v) = Degradation function
Sh(u,v) = Power spectrum of noise
Sf(u,v) = Power spectrum of the undegraded image
(Images from Rafael C. Gonzalez and Richard E.
Wood, Digital Image Processing, 2nd Edition.
Approximation of Wiener Filter
Wiener Filter Formula:
 1 H ( u , v )
2

Fˆ (u, v )   2 G ( u , v )
 H (u, v ) H (u, v )  S (u, v ) / S f (u, v ) 

Difficult to estimate
Approximated Formula:

 1 H ( u , v )
2

Fˆ (u, v )   2 G ( u , v )
 H (u, v ) H (u, v )  K 
Practically, K is chosen manually to obtained the best visual result!
Wiener Filter: Example

Result of the Result of the inverse

Original image full inverse filter filter with D0=70

Blurred image Result of the

Due to Turbulence full Wiener filter
Wiener Filter: Example (cont.)

Result of the inverse

Original image filter with D0=70

Blurred image Result of the

Due to Turbulence Wiener filter
Example: Wiener Filter and Motion Blurring
Image Result of the Result of the
degraded inverse filter Wiener filter
by motion
blur +
AWGN sh2=650

sh2=325
Note: K is
chosen
manually

sh2=130 (Images from Rafael C. Gonzalez and Richard E.

Wood, Digital Image Processing, 2nd Edition.