Digital Image Processing

Image Restoration and

Reconstruction

1
 Image Restoration/Denoising

► Image restoration: recover an image that has been

degraded by using a prior knowledge of the degradation
phenomenon.

► Model the degradation and applying the inverse process in

order to recover the original image.

11/17/2023 2
 A Model of Image Degradation/Restoration
Process

► Degradation
 Degradation function H
 Additive noise  ( x, y )

11/17/2023 3
 A Model of Image Degradation/Restoration
Process

If H is a linear, position-invariant process, then

the degraded image is given in the spatial domain by
g ( x, y )  h ( x , y ) f ( x, y )   ( x, y )

11/17/2023 4
 A Model of Image Degradation/Restoration
Process

The model of the degraded image is given in

the frequency domain by

G (u , v)  H (u , v) F (u , v)  N (u, v)

11/17/2023 5
 Noise Sources

► The principal sources of noise in digital images arise during

image acquisition and/or transmission

 Image acquisition
e.g., light levels, sensor temperature, etc.

 Transmission
e.g., lightning or other atmospheric disturbance in wireless
network

11/17/2023 6
 Noise Models (1)
Why it is required to know about different Noise
Models?
► White noise
 The Fourier spectrum of noise is constant

► With the exception of spatially periodic noise, we assume

 Noise is independent of spatial coordinates
 Noise is uncorrelated with respect to the image itself

11/17/2023 7
 Noise Models (2)

 Gaussian noise
Electronic circuit noise, sensor noise due to poor illumination and/or
high temperature

 Rayleigh noise
Usually presents in Range imaging techniques like radar range images.

11/17/2023 8
 Range Imaging (1)

 Also called High Dynamic Range Imaging (HDRI).

 HDRI is an imaging technique that allows for a greater
dynamic range of exposure than would be obtained through
any normal imaging process.

 It is now popularly used to refer to the process of

tone mapping* together with bracketed** exposures of
normal digital images, giving the end result a high, often
exaggerated dynamic range

* Tone mapping is a technique used in image processing and computer graphics to

map a set of colours to another; often to approximate the appearance of HDRI in media
with a more limited dynamic range
** bracketing is the general technique of taking several shots of the same subject using
different or the same camera settings

http://en.wikipedia.org/wiki/High_dynamic_range_imaging
http://www.webopedia.com/TERM/H/High_Dynamic_Range_Imaging.html
11/17/2023 9
 Range Imaging (2)

 The intention of HDRI is to accurately represent the wide

range of intensity levels found in real scenes ranging from
direct sunlight to shadows

 HDRI, also called HDR (High Dynamic Range) is a feature

commonly found in high-end graphics and imaging software

http://en.wikipedia.org/wiki/High_dynamic_range_imaging
http://www.webopedia.com/TERM/H/High_Dynamic_Range_Imaging.html

11/17/2023 10
 Range Imaging: Examples (1)
http://en.wikipedia.org/wiki/High_dynamic_range_imaging

Tower Bridge in
Sacramento,
California

11/17/2023 11
 Sydney Harbour
Bridge HDRi
Range Imaging: Examples (2) produces greater
detail and fewer
http://en.wikipedia.org/wiki/High_dynamic_range_imaging shadows

11/17/2023 12
11/17/2023 Old Saint Paul’s Wellinton, New Zealand 13
 Noise Models (3)

 Erlang (gamma) noise: Laser imaging

 Exponential noise: Laser imaging

 Uniform noise: Least descriptive; Basis for numerous random

number generators

 Impulse noise: quick transients,

such as faulty switching

11/17/2023 14
 Gaussian Noise (1)

The PDF of Gaussian random variable, z, is given by

1  ( z  z ) 2 /2 2
p( z )  e
2

where, z represents intensity

z is the mean (average) value of z
 is the standard deviation

11/17/2023 15
 Gaussian Noise (2)

The PDF of Gaussian random variable, z, is given by

1  ( z  z ) 2 /2 2
p( z )  e
2

 70% of its values will be in the range

(   ), (    )
 95% of its values will be in the range

(  2 ), (   2 )
11/17/2023 16
 Rayleigh Noise

The PDF of Rayleigh noise is given by

2  ( z  a )2 / b
 ( z  a )e for z  a
p( z )   b
 0 for z  a

The mean and variance of this density are given by

z  a  b / 4
b(4   )
 
2

11/17/2023 17
 Erlang (Gamma) Noise

The PDF of Erlang noise is given by

 a b z b 1  az
 e for z  0
p ( z )   (b  1)!
0 for z  a


The mean and variance of this density are given by

z b/a
 2  b / a2

11/17/2023 18
 Exponential Noise

The PDF of exponential noise is given by

 ae  az for z  0
p( z )  
0 for z  a

The mean and variance of this density are given by

z  1/ a
 2  1/ a 2

11/17/2023 19
 Uniform Noise

The PDF of uniform noise is given by

 1
 for a  z  b
p( z )   b  a
 0 otherwise

The mean and variance of this density are given by

z  ( a  b) / 2
 2  (b  a) 2 /12

11/17/2023 20
 Impulse (Salt-and-Pepper) Noise

The PDF of (bipolar) impulse noise is given by

 Pa for z  a

p ( z )   Pb for z  b
0 otherwise


if b  a, gray-level b will appear as a light dot,

while level a will appear like a dark dot.

If either Pa or Pb is zero, the impulse noise is called

unipolar
11/17/2023 21
11/17/2023 22
 Examples of Noise: Original Image

11/17/2023 23
 Examples of Noise: Noisy Images(1)

11/17/2023 24
 Examples of Noise: Noisy Images(2)

11/17/2023 25
 Periodic Noise

► Periodic noise in an image arises typically from electrical or

electromechanical interference during image acquisition.

► It is a type of spatially dependent noise

► Periodic noise can be reduced significantly via frequency

domain filtering

11/17/2023 26
 An Example of Periodic Noise

11/17/2023 27
 Estimation of Noise Parameters (1)
The shape of the histogram identifies the closest PDF match

11/17/2023 28
 Estimation of Noise Parameters (2)

Consider a subimage denoted by S , and let ps ( zi ), i  0, 1, ..., L -1,

denote the probability estimates of the intensities of the pixels in S .
The mean and variance of the pixels in S:
L 1
z   zi ps ( zi )
i 0
L 1
and  2   ( zi  z ) 2 p s ( z i )
i 0

11/17/2023 29
 Restoration in the Presence of Noise Only
̶ Spatial Filtering

Noise model without degradation

g ( x, y )  f ( x, y )   ( x, y )
and
G (u, v)  F (u, v)  N (u, v)

11/17/2023 30
 Spatial Filtering: Mean Filters (1)

Let S xy represent the set of coordinates in a rectangle

subimage window of size m  n, centered at ( x, y ).

Arithmetic mean filter

f ( x, y )  1

mn ( s ,t )S xy
g ( s, t )

11/17/2023 31
 Spatial Filtering: Mean Filters (2)

Geometric mean filter

1
  mn
f ( x, y )  
 g ( s, t ) 
 ( s ,t )Sxy 

Generally, a geometric mean filter achieves smoothing

comparable to the arithmetic mean filter, but it tends to
loose less image details in the process

11/17/2023 32
 Spatial Filtering: Mean Filters (3)
Harmonic mean filter
f ( x, y )  mn
1

( s ,t )S xy g ( s, t )

It works well for salt noise, but fails for pepper noise.
It does well also with other types of noise like Gaussian
noise.

11/17/2023 33
 Spatial Filtering: Mean Filters (4)

Contraharmonic mean filter

 g ( s, t )Q 1
f ( x, y )  ( s ,t )S xy


( s ,t )S xy
g ( s, t )Q

Q is the order of the filter.

It is well suited for reducing the effects of salt-and-

pepper noise. Q>0 for pepper noise and Q<0 for salt
noise.
11/17/2023 34
 Spatial Filtering: Example (1)

11/17/2023 35
 Spatial Filtering: Example (2)

11/17/2023 36
 Spatial Filtering: Example (3)

11/17/2023 37
 Spatial Filtering: Order-Statistic Filters (1)

Median filter
f ( x, y )  median g ( s, t )
( s ,t )S xy

Max filter
f ( x, y)  max g ( s, t )
( s ,t )S xy

Min filter
f ( x, y)  min g ( s, t )
( s ,t )S xy

11/17/2023 38
 Spatial Filtering: Order-Statistic Filters (2)

Midpoint filter
f ( x, y )  1  max g ( s, t )  min g ( s, t )
2  ( s ,t )S xy ( s ,t )S xy 

11/17/2023 39
 Spatial Filtering: Order-Statistic Filters (3)

Alpha-trimmed mean filter

1
f ( x, y ) 
mn  d
 g (s, t )
( s ,t )S xy
r

We delete the d / 2 lowest and the d / 2 highest intensity values of

g ( s, t ) in the neighborhood S xy . Let g r ( s, t ) represent the remaining
mn - d pixels.

11/17/2023 40
11/17/2023 41
11/17/2023 42
11/17/2023 43
 Spatial Filtering: Adaptive Filters (1)

Adaptive filters

The behavior changes based on statistical characteristics

of the image inside the filter region defined by the mхn
rectangular window.

The performance is superior to that of the filters

discussed earlier
Better for local noise reduction

11/17/2023 44
 Adaptive Filters:
Adaptive, Local Noise Reduction Filters (3)

An adaptive expression for obtaining f ( x, y)

based on the assumptions:
 2
f ( x, y )  g ( x, y )    g ( x, y )  m 
L2 L

where, mL , the local mean of the pixels in S xy and  L2 , the local variance of the pixels in S xy .

11/17/2023 45
11/17/2023 46
 Linear, Position-Invariant Degradations

H is linear
H af1 ( x, y )  bf 2 ( x, y )   aH  f1 ( x, y )  bH  f 2 ( x, y )
f1 and f 2 are any two input images.

An operator having the input-output relationship

g ( x, y )  H  f ( x, y )  is said to be position invariant
if
H  f ( x   , y   )  g ( x   , y   )
for any f ( x, y ) and any  and  .
11/17/2023 47
 Linear, Position-Invariant Degradations
 
f ( x, y )    f ( ,  ) ( x   , y   ) d d 
 
Assume for a moment that  ( x, y)  0
if H is a linear operator,
g ( x, y )  H  f ( x, y ) 

 H   f ( ,  ) ( x   , y   )d d  
  
Superposition (or
    Fredholm)
  integral of the
  H  f ( ,  ) ( x   , y   )  d d  first kind
 
 
  f ( ,  ) H  ( x   , y   )  d d  Impulse
 
response
11/17/2023 48
 Linear, Position-Invariant Degradations

Assume for a moment that  ( x, y)  0

if H is a linear operator and position invariant,
H  ( x   , y   )   h( x   , y   )
g ( x, y )  H  f ( x, y ) 
 
  f ( ,  ) H  ( x   , y   )  d d 
  Convolution
  integral in 2-D
  f ( ,  )h( x   , y   )d d 
 

11/17/2023 49
 Linear, Position-Invariant Degradations

In the presence of additive noise,

if H is a linear operator and position invariant,

 
g ( x, y )    f ( ,  )h( x   , y   )d d    ( x, y )
 

 h ( x, y ) f ( x, y )   ( x, y )

G (u , v)  H (u , v) F (u , v)  N (u , v)

11/17/2023 50
 Estimating the Degradation Function H
► Three principal ways to estimate the degradation function
1. Observation: By close observation and analysis,
especially areas of high contrast/high signal strength using
sharpening filter, etc. If a cleaner subimage Gs(u,v) is
obtained, estimate Hs(u,v) as follows using LPI property

2. Experimentation: Simulate an impulse function by

imaging a brightest dot (to reduce effect of noise) with the
same imaging setting as it was while capturing the noisy
image to be restored. Assuming LPI propert and FT of
impulse as constant A, H(u,v) is estimated as
11/17/2023 51

3. Mathematical Modeling
 Mathematical Modeling (1)

► Environmental conditions cause degradation

A model about atmospheric turbulence

 k ( u 2  v 2 )5/6
H (u , v )  e
k : a constant that depends on
the nature of the turbulence

11/17/2023 52
11/17/2023 53
 Mathematical Modeling (2)

► Derive a mathematical model from basic principles

E.g., An image blurred by uniform linear motion between

the image and the sensor during image acquisition

11/17/2023 54
 Mathematical Modeling (3)

Suppose that an image f ( x, y ) undergoes planar motion,

x0 (t ) and y0 (t ) are the time-varying components of motion
in the x- and y -directions, respectively.
The optical imaging process is perfect. T is the duration
of the exposure. The blurred image g ( x, y )
g ( x, y )   f  x  x0 (t ), y  y0 (t ) dt
T

T  j 2 ux0 ( t )  vy0 ( t ) 
11/17/2023 G(u, v)  F (u, v)  e dt 55
0
 Mathematical Modeling (4)

T  j 2 ux0 ( t )  vy0 ( t ) 
H (u , v)   e dt
0

Suppose that the image undergoes uniform linear motion

in the x-direction only, at a rate given by x0 (t )  at / T .
T
H (u , v)   e  j 2 ux0 (t ) dt
0
T
  e  j 2 uat /T dt
0

T
 sin( ua )e  j ua

 ua
11/17/2023 56
11/17/2023 57
 Inverse Filtering: How to restore if we have H

An estimate of the transform of the original image

 G (u , v)
F (u , v ) 
H (u , v)

 F (u , v) H (u, v)  N (u , v)
F (u , v) 
H (u , v)
N (u, v)
 F (u , v) 
H (u , v)
11/17/2023 58
 Inverse Filtering

 N (u , v)
F (u , v)  F (u , v) 
H (u , v)

1. We can't exactly recover the undegraded image

because N (u, v) is not known.
2. If the degradation function has zero or very
small values, then the ratio N (u, v) / H (u, v) could
easily dominate the estimate F  (u, v).

11/17/2023 59
 Minimum Mean Square Error (Wiener)
Filtering
 N. Wiener (1942)

 Objective
Find an estimate of the uncorrupted image such that the mean
square error between them is minimized


e 2  E ( f  f )2 
Addresses problem of Inverse Filtering. It incorporates both the degradation function H
and the statistical properties of the noise into the restoration process

11/17/2023 60
 Minimum Mean Square Error (Wiener)
Filtering

  1 | H (u , v ) | 2

F (u , v)    G (u , v)
 H (u , v) | H (u , v) |  K 
2

K is a specified constant. Generally, the value of K

is chosen interactively to yield the best visual results.

11/17/2023 61
 Minimum Mean Square Error (Wiener)
Filtering

11/17/2023 62
Left:
degradated
image

Middle:
inverse
filtering

Right:
Wiener
filtering

11/17/2023 63
 Some Measures (1)

Singal-to-Noise Ratio (SNR)

M 1 N 1

 | F (u , v ) | 2

SNR  u 0 v 0
M 1 N 1

 | N (u
u 0 v 0
, v ) | 2

This ratio gives a measure of the level of information

bearing singal power to the level of noise power.

11/17/2023 64
 Some Measures (2)

Mean Square Error (MSE)

M 1 N 1 2
1
MSE=
MN
   f ( x, y)  f ( x, y ) 
x 0 y 0

Root-Mean-Sqaure-Error (RMSE)
M 1 N 1

 f ( x, y ) 2
RMSE  M 1 N 1
u 0 v 0

 | f (
u 0 v 0
x , y )  f ( x, y ) |2

11/17/2023 65