DIGITAL IMAGE PROCESSING

IMAGE RESTORATION
 Basic Idea
• With Image Restoration one tries to repair errors or distortions
in an image, caused during the image creation process.
• In general our starting point is a degradation and noise model:
g(x, y) = H ( f(x, y) ) + (x, y)
Image restoration assumes a degradation model that is known
or can be estimated.
• Image restoration try to recover original image from degraded
with prior knowledge of degradation process.
• Restoration involves modeling of degradation and applying the
inverse process in order to recover the original image.
• Although the restore image is not the original image, its
approximation of actual image.
 Image Enhancement vs. Image
Restoration
• Image enhancement: process image so that the result is more
suitable for a specific application, is largely a subjective
process.
• Image restoration: recover image from distortions to its
original image, is largely an objective process.
• Image enhancement techniques are basically heuristic
procedure designed to manipulate an image in order to take
advantage of the psychophysical aspects of human visual
systems.
• In contrast, the restoration approach usually involves
formulating a criterion of goodness that will yield an optimal
estimate of the desired result.
 Model of Image Degradation/Restoration
Degradation process is modeled as a degradation function that
together with an additive noise, operates on an input image f(x, y) to
produce a degraded image g(x, y).

Spatial Domain: g(x, y)  f (x, y)* h(x, y) (x, y)

Frequency Domain: G(u, v)  F(u, v)H (u, v)  N(u, v)
 Noise Models

• The principal sources of noise in digital images arise

during image acquisition and/or transmission. The
performance of imaging sensors is affected by a variety
of environmental factors during image acquisition, and by
the quality of the sensing elements themselves.
• For instance, in acquiring images with a CCD camera,
light levels and sensor temperature are major factors
affecting the amount of noise in the resulting image.
• Images are corrupted during transmission principally
by interference in the transmission channel. For example,
an image transmitted using a wireless network might be
corrupted by lightning or other atmospheric disturbance.
 Gaussian noise
• The PDF of a Gaussian random variable, z, is defined by the
following familiar expression:

where z represents intensity, 𝑧bar is the

mean (average) value of z, and σ is its
standard deviation.
 Rayleigh noise
• The PDF of a Rayleigh random variable, z, is defined
by the following familiar expression:

• The mean and variance of z when this

random variable is characterized by a
Rayleigh PDF are
 Gamma noise
• The PDF of a Erlang/Gamma noise is given by

where the parameters are such that a > b, b is

a positive integer, and “!”indicates factorial.
The mean and variance of z are
 Exponential noise
• The PDF of a Exponential noise is given as

where a > 0. The mean and variance

of z are
 Uniform noise
• The PDF of a uniform noise is:
 Salt and Pepper noise

• The PDF of a Salt and Pepper noise is given by

If k represents the number of bits used to

represent the intensity values in a digital
image, then the range of possible intensity
values for that image is [0, 2k-1], where V is
any integer value in the range 0 < V < 2k-1.
 Sources of Noise
• Gaussian noise arises in an image due to factors such as
electronic circuit noise and sensor noise due to poor
illumination and/or high temperature.

• Raleigh noise is helpful in characterizing noise phenomena

in range imaging.

• Exponential and Gamma noises are more common is Laser

Imaging.

• Impulse noise is found in situations where quick transients,

such as faulty switching, take place during imaging.
 Image Degradation with Additive Noise

g ( x, y )  f ( x, y )   ( x, y )

Degraded images

Original image

Histogram
 Image Degradation with Additive Noise

g ( x, y )  f ( x, y )   ( x, y )

Degraded images

Original image

Histogram
 Estimation of Noise Parameters

• The parameters of periodic noise typically are

estimated by inspection of the Fourier spectrum
of the image.

• The parameters of the noise PDFs may be known

partially from sensor specifications.

• But it is often necessary to estimate them for a

particular imaging environment.
 Restoration in presence of Noise only
• Clearly, g(x, y) = f(x, y) + η(x, y).

• In frequency domain, G(u, v) = F(u, v) + N(u, v).

• In general, neither η(x, y) nor N(u, v) is known in advance.

• We choose spatial domain filtering techniques when

additive random noise is expected to be present.

• We use frequency domain technique only if we feel that

periodic noise is present.
 Periodic Noise Reduction by Frequency
Domain Filtering
• Periodic noise appears as a concentrated bursts of energy
in the Fourier transform, at locations corresponding to
frequencies of the periodic interference.

• The approach for restoration is to use a selective filter to

isolate the noise.

• The type of selective filters we use is either band-reject or

band-pass filters.

• We also use notch-reject or notch-pass filters for the same

purpose.
Periodic Noise Reduction by Frequency Domain Filtering
Degraded image DFT
Periodic noise can
be reduced by
setting frequency
components
corresponding to
noise to zero.

Band reject filter Restored image

 Restoration in presence of Noise only:
Spatial Domain Filtering
• Spatial domain filtering are chosen for
restoration if the degradation present in the
image is additive noise only.

• There are two general types of filters used in

this category:
– Mean Filters.
– Order-statistics Filters.
 Mean Filters
Degradation model:

g ( x, y )  f ( x, y )  h( x, y )   ( x, y )

To remove this part

Arithmetic mean filter or moving average filter

ˆf ( x, y )  1

mn ( s ,t )S xy
g ( s, t )

Geometric mean filter mn = size of moving window

1
  mn

fˆ ( x, y )    g ( s, t ) 
 ( s ,t )S 
 xy 
 Geometric Mean Filter: Example

Image
Original
corrupted
image
by AWGN

Image Image
obtained obtained
using a 3x3 using a 3x3
arithmetic geometric
mean filter mean filter

AWGN: Additive White Gaussian Noise

 Harmonic and Contra-harmonic Filters
Harmonic mean filter
mn
fˆ ( x, y )  Works well for salt noise
1

( s ,t )S xy g ( s, t )
but fails for pepper noise

Contra-harmonic mean filter mn = size of moving window


( s ,t )S xy
g ( s, t ) Q 1
Positive Q is suitable for
eliminating pepper noise.
fˆ ( x, y )  Negative Q is suitable for

( s ,t )S xy
g ( s, t ) Q eliminating salt noise.

Q = the filter order

For Q = 0, the filter reduces to an arithmetic mean filter.
For Q = -1, the filter reduces to a harmonic mean filter.
 Contra-harmonic Filters: Example
Image Image
corrupted corrupted
by pepper by salt
noise with noise with
prob. = 0.1 prob. = 0.1

Image Image
obtained obtained
using a 3x3 using a 3x3
contra- contra-
harmonic harmonic
mean filter mean filter
With Q = 1.5 With Q=-1.5
 Contra-harmonic Filters: Incorrect Use Example

Image Image
corrupted corrupted
by pepper by salt
noise with noise with
prob. = 0.1 prob. = 0.1

Image Image
obtained obtained
using a 3x3 using a 3x3
contra- contra-
harmonic harmonic
mean filter mean filter
With Q=-1.5 With Q=1.5
 Order-Statistic Filters: Revisited
Original image

subimage

Statistic parameters
Mean, Median, Mode,
Min, Max, Etc.

Moving
window

Output image
 Order-Statistics Filters
Median filter

fˆ ( x, y )  median  g ( s, t )
( s ,t )S xy

Max filter

fˆ ( x, y )  max  g ( s, t ) Reduce “dark” noise

( s ,t )S xy (pepper noise)

Min filter

fˆ ( x, y )  min  g ( s, t )
Reduce “bright” noise
(salt noise)
( s ,t )S xy

Mid-point filter
ˆf ( x, y )  1  max  g ( s, t )  min  g ( s, t ) 

2  ( s ,t )Sxy 
( s , t )S xy 
 Median Filter : How it works
A median filter is good for removing impulse, isolated noise
Salt noise
Pepper noise
Median

Sorted
Degraded image Moving array
window
Salt noise Filter output
Pepper noise
Normally, impulse noise has high magnitude
and is isolated. When we sort pixels in the
moving window, noise pixels are usually at
the ends of the array.
Therefore, it’s rare that the noise pixel will be a median value.
 Median Filter : Example
1 2
Image
corrupted
by salt-and-
pepper
noise with
pa=pb= 0.1

3 4

Images obtained using a 3x3 median filter

 Max and Min Filters: Example
Image Image
corrupted corrupted
by pepper by salt
noise with noise with
prob. = 0.1 prob. = 0.1

Image Image
obtained obtained
using a 3x3 using a 3x3
max filter min filter
 Alpha-trimmed Mean Filter
Formula:

1
fˆ ( x, y ) 
mn  d

( s ,t )S xy
g r ( s, t )

where gr(s, t) represent the remaining mn - d pixels after

removing the d/2 highest and d/2 lowest values of g(s, t).

Here d can range from 0 to (mn – 1).

If d = 0, it reduces to Arithmetic mean filter.
If d = mn – 1, it reduces to median filter.

This filter is useful in situations involving multiple types

of noise such as a combination of salt-and-pepper and
Gaussian noise.
 Alpha-trimmed Mean Filter: Example
1 2
Image
additionally
Image corrupted
corrupted by additive
by additive salt-and-
uniform pepper
noise noise

Image 2 Image 2
obtained obtained
using a 5x5 using a 5x5
arithmetic geometric
mean filter mean filter
 Alpha-trimmed Mean Filter: Example (cont.)
1 2
Image
additionally
Image corrupted
corrupted by additive
by additive salt-and-
uniform pepper
noise noise

Image 2
obtained
Image 2 using a 5x5
obtained alpha-
using a 5x5 trimmed
median filter mean filter
with d = 5
 Alpha-trimmed Mean Filter: Example (cont.)
Image Image
obtained obtained
using a 5x5 using a 5x5
arithmetic geometric
mean filter mean filter

Image
obtained
Image using a 5x5
obtained alpha-
using a 5x5 trimmed
median filter mean filter
with d = 5
 Adaptive Median Filter
• Adaptive median filter has three main purposes:

– To remove salt-and-pepper noise.

– To provide smoothing of other noise that may not be

impulsive.

– To reduce distortion, such as excessive thinning or

thickening of object boundaries.

• The algorithm discussed previously consider zmin and

zmax to be “impulse-like” noise components.
 Estimation of Degradation Model
Degradation model:
g ( x, y )  f ( x, y )  h( x, y )   ( x, y )
or
G (u, v)  F (u, v) H (u, v)  N (u, v)

Purpose: to estimate h(x, y) or H(u, v)

Why? If we know exactly h(x, y), regardless of noise, we can do
deconvolution to get f(x, y) back from g(x, y).

Methods:
1. Estimation by Image Observation

2. Estimation by Experiment

3. Estimation by Modeling
 Restoration Using Degradation Model
• The degradations are modeled as being result of convolution, and
restoration seeks to find filters that apply process in reverse.

• Due to that reason, the term image deconvolution is used frequently to

signify linear image restoration.

• The filters used in the restoration process often are called deconvolution
function.

• The process of restoring an image by using a degradation function that has

been estimated in some way sometimes is called blind deconvolution.

• It is due to the fact that true degradation function is seldom known

completely.
 Estimation by Image Observation
• We are given a degraded image without any knowledge about the
degradation function H.

• One way to estimate the function H is to gather information from

the image itself.

• We take a small section of the image with simple structures, where

there are strong signal contents.

• Using the sample gray levels of the object and background, we can
construct an un-blurred image of the same size and characteristics
as the observed sub-image.

• Then we can extend the transfer function to the whole image.

 Estimation by Image Observation
Original image (unknown) Degraded image

f(x,y) f(x,y)*h(x,y) g(x,y)

Observation

DFT Subimage
Estimated Transfer
function Gs (u, v ) g s ( x, y )
Restoration
G ( u, v ) process by
H ( u, v )  H s ( u, v )  s estimation
Fˆs (u, v )
DFT Reconstructed
This case is used when we
Fˆs (u, v ) Subimage
know only g(x, y) and cannot
repeat the experiment! fˆs ( x, y )
 Estimation by Experimentation
• If the equipment similar to the equipment used to acquire the
degraded image is available, it is possible in principle to obtain an
accurate estimate of the degradation.

• Images similar to the degraded image can be acquired with various

system settings until they are as closely as possible to the image we
wish to restore.

• The idea is to obtain the impulse response of the degradation by

imaging an impulse using the same system settings.

• An impulse is simulated by a bright dot of light, as bright as possible

to reduce the effect of noise.
 Estimation by Experimentation
Used when we have the same equipment set up and can repeat the
experiment.
Response image from
Input impulse image the system

System
H( )

A ( x, y ) g ( x, y )
DFT DFT

DFT A ( x, y )  A G ( u, v )
G ( u, v )
H ( u, v ) 
A
 Estimation by Modeling
Used when we know physical mechanism underlying the image
formation process that can be expressed mathematically.
Original image Severe turbulence Example:
Atmospheric
Turbulence model
k (u2 v2 )5/6
H(u, v)  e
k = 0.0025

Mild turbulence Low turbulence K is a constant

that depends on
the nature of the
turbulence.

k = 0.001 k = 0.00025
Proposed by Hufnagel and Stanley in 1964
 Inverse Filter
From degradation model:

G (u , v)  F (u , v) H (u , v)  N (u , v)

After we obtain H(u, v), we can estimate F(u, v) by the inverse filter:
G (u , v) N (u, v)
Fˆ (u, v)   F (u, v) 
H (u , v) H (u , v)

Noise is enhanced
when H(u, v) is small.
To avoid the side effect of enhancing
noise, we can apply this formulation
to frequency component (u, v) with in
a radius D0 from the center of (u, v).

In practice, the inverse filtering is not a popular technique.

 Inverse Filter: Example

Result of applying Result of applying

Original image the full filter the filter with D0=40

Blurred image Result of applying Result of applying

Due to Turbulence the filter with D0=70 the filter with D0=85

 0.0025 ( u 2  v 2 ) 5/6
H (u , v )  e
Wiener Filter: Minimum Mean Square
Error Filter
• The inverse filtering approach discussed so far makes no
explicit provision for handling noise.
• The current approach incorporates both the degradation
function and statistical characteristics of noise into the
restoration process.
• The method is founded on considering images and noise as
random processes.
• Wiener filtering was one of the first methods developed to
reduce additive random noise in images. It works on the
assumption that additive noise is a stationary random
process, independent of pixel location; the algorithm
minimizes the square error between the original and
reconstructed images.
 Wiener Filter
• Wiener filtering is a statistical image processing technique
used to reduce noise and blur in images by estimating the
original image from a degraded version, assuming knowledge
of the image and noise spectral characteristics.
• It finds the optimal balance between noise removal and detail
preservation by minimizing the mean squared error between
the original and restored images.
 Wiener Filter
• Wiener filtering is a statistical image processing technique
used to reduce noise and blur in images by estimating the
original image from a degraded version, assuming knowledge
of the image and noise spectral characteristics.
• It finds the optimal balance between noise removal and detail
preservation by minimizing the mean squared error between
the original and restored images.
• The objective is to find an estimate of the uncorrupted image
function f such that the mean square error between the
original and the estimate is minimized.
• It is assumed that:
– the noise and the image are uncorrelated;
– one or the other has zero mean;
– the gray levels in the estimate are a linear function of the
levels in the degraded image.
 Wiener Filter: Minimum Mean Square Error Filter
Objective: optimize mean square error: e  E ( f  fˆ )
2 2
 
Wiener Filter Formula:
 H *
(u , v ) S ( u , v ) 
Fˆ (u, v)   f
2
 G (u , v)
 S f (u, v) H (u, v)  S (u, v) 
 H * (u , v) 
 2
 G (u , v )
 H (u, v)  S (u, v) / S f (u, v) 
 1 H (u , v)
2

 2
 G (u, v )
 H (u , v) H (u , v)  S (u, v) / S f (u, v) 
where
H(u, v) = Degradation function
Sh(u, v) = Power spectrum of noise
Sf(u, v) = Power spectrum of the un-degraded image
 Approximation of Wiener Filter
Wiener Filter Formula:
 1 H (u , v )
2

Fˆ (u, v)   2
 G (u, v)
 H (u, v) H (u , v)  S (u , v) / S f (u , v) 

Difficult to estimate
Approximated Formula:

 1 H (u, v)
2

ˆ
F (u, v)    G (u, v)
2
 H (u, v) H (u , v)  K 

Practically, K is chosen manually to obtained the best visual result!

 Wiener Filter: Example

Result of the Result of the inverse

Original image full inverse filter filter with D0=70

Blurred image Result of the

Due to Turbulence full Wiener filter
Wiener Filter: Example (cont.)

Result of the inverse

Original image filter with D0=70

Blurred image Result of the

Due to Turbulence Wiener filter
 Example: Wiener Filter and Motion Blurring
Image Result of the Result of the
degraded inverse filter Wiener filter
by motion
blur +
AWGN
s2=650

s2=325

Note: K is
chosen
manually

s2=130
 Different restoration approaches
Frequency domain Algebraic approaches
– Inverse filter – Unconstrained
– Wiener (minimum mean optimization
square error) filter – Constrained
optimization

52
Homomorphic filtering
 INTRODUCTION
• Homomorphic filtering is a generalized technique for signal and image
processing, involving a nonlinear mapping to a different domain in which
linear filter techniques are applied, followed by mapping back to the
original domain.
• It simultaneously normalizes the brightness across an image and increases
contrast.
Applications
• Homomorphic filter is sometimes used for image enhancement.
• Homomorphic filtering is one such technique for removing multiplicative
noise that has certain characteristics.
• Homomorphic filtering is most commonly used for correcting non-uniform
illumination in images.
• Homomorphic filtering can be used for improving the appearance of a
grayscale image by simultaneous intensity range compression (illumination)
and contrast enhancement (reflection).
 Illumination-Reflectance Model
• The illumination-reflectance model of image formation says that
the intensity at any pixel, which is the amount of light reflected
by a point on the object, is the product of the illumination of the
scene and the reflectance of the object(s) in the scene, i.e.,
I(x,y)=L(x,y) * R(x,y)
where I is the image, L is scene illumination, and R is scene
reflectance.
• Illumination Reflectance model can be used to address the
problem of improving the quality of an image that has been
acquired under poor illumination conditions.
• Illumination typically varies slowly across the image as
compared to reflectance which can change quite abruptly at
object edges.
 Continued…

• The idea of homomorphic filter is to separate out the

illumination component from the reflectance component
and apply two different transfer functions to have more
control.
• Image can be represented by product of illumination and
reflectance
• f(x,y)=i(x,y) r(x,y)
• Fourier transform of above is not simple multiplication. Its
convolution of the two functions.
F[f(x,y)] ≠ F[i(x,y)] F[r(x,y)]
• In homomorphic filtering we first transform the
multiplicative components to additive components by
moving to the log domain.
 Steps for applying Homomorphic filtering
1. Logarithmic (log function) is applied to the image so that
it can be expressed as a sum of its illumination and
reflectance components.
z(x,y) = ln{f(x,y)} = ln{i(x,y)} + ln{r(x,y)}
2. Then apply Fourier transform to the log of these
functions
F[z(x,y)] = F[ln{f(x,y)}] = F[ln{i(x,y)}] + F[ln{r(x,y)}]
Z(u,v) = F (u,v) + F (u,v)
i r

3. Process Z(u,v) through filters H(u,v), so design filters

separately for illumination and reflectance components.
S(u,v) = H(u,v) Z(u,v) = H(u,v) F (u,v) + H(u,v) F (u,v)
i r

4. Apply inverse Fourier transformation the filtered image

to convert it back to the spatial domain
 Steps for applying Homomorphic filtering
4. Apply inverse Fourier transformation the filtered image to
convert it back to the spatial domain

5. To offset the logarithmic applied in step-1, apply the

antilog function to recover the original image.
 Summarized Operation
we use a high-pass filter in the log domain to remove the low-
frequency illumination component while preserving the high-
frequency reflectance component. The basic steps in
homomorphic filtering are shown in the diagram below:

I 𝑥, 𝑦 In DFT H(u,v) (DFT) -1 exp g 𝑥, 𝑦

The illumination component of an image generally is

characterized by slow spatial variations, while the
reflectance component tends to vary abruptly, particularly at
junction of dissimilar objects.
Applying homomorphic filter to original image