DIP TOPIC 9: Introduction to Image

Restoration

Dr. V Masilamani
[email protected]
Professor
Department of Computer Science and Engineering
Indian Institute of Information Technology, DM, Kancheepuram
Chennai-127

V. Masilamani DIP TOPIC 9: Introduction to Image Restoration 1 / 33

Overview

1 Introduction to Image Restoration

2 Modeling Image degradation

3 Least Square and constraint Least Square Filters

4 Image Restoration from projections using Fourier Slice Theorem

5 Back Projection Algorithm

6 Filtered Back Projection Algorithm

7 Convolution Back Projection Algorithm

8 Acknowledgements

V. Masilamani DIP TOPIC 9: Introduction to Image Restoration 2 / 33

What is Image Restoration?

▶ Image Restoration: Given a degraded image g , and degradation

function h, Find the original image from g using inverse of h
▶ Examples
1. CT Scan

2. De blurring

V. Masilamani DIP TOPIC 9: Introduction to Image Restoration 3 / 33

What is Image Restoration? (cont.)

V. Masilamani DIP TOPIC 9: Introduction to Image Restoration 4 / 33

What is Image Restoration? (cont.)

V. Masilamani DIP TOPIC 9: Introduction to Image Restoration 5 / 33

What is Image Restoration? (cont.)

V. Masilamani DIP TOPIC 9: Introduction to Image Restoration 6 / 33

What is Image Restoration? (cont.)

V. Masilamani DIP TOPIC 9: Introduction to Image Restoration 7 / 33

What is Image Restoration? (cont.)

V. Masilamani DIP TOPIC 9: Introduction to Image Restoration 8 / 33

What is Image Restoration? (cont.)

V. Masilamani DIP TOPIC 9: Introduction to Image Restoration 9 / 33

What is Image Restoration? (cont.)

De-blurring

V. Masilamani DIP TOPIC 9: Introduction to Image Restoration 10 / 33

Image Restoration Vs Image Enhancement

▶ Image Enhancement produces the image with better image quality,

whereas Image restoration produces the original image
• In case the original image is of poor quality, Image enhancement will
produce better quality image, but Image restoration will produce the
original poor quality image only
• Image restoration tries to maintain image fidelity, whereas Image
enhancement attempts to improve visible quality
▶ Image restoration uses degradation function, whereas the image
enhancement does not use it

V. Masilamani DIP TOPIC 9: Introduction to Image Restoration 11 / 33

Modeling Image degradation

▶ Let f (x, y ) be the original image and g (x, y ) be the degraded image.
▶ Let the degradation function be H.
▶ To make the model more general, let us consider some noise is
added to the degraded image.
▶ Image Degradation Model:
g (x, y ) = H(f (x, y )) + η(x, y ), where η(x, y ) is the noise (1)
R∞ R∞
▶ We know that f (x, y ) = −∞ −∞ f (α, β)δ(x − α, y − β)dαdβ. (2)
▶
▶ If we assume that the degradation function H is Linear,
Position-invariant, then from equation (2),
R∞ R∞
▶ we get, H(f (x, y )) = −∞ −∞
f (α, β)H(δ(x − α, y − β))dαdβ
(since H is linear and f (α, β) is constan for H)

V. Masilamani DIP TOPIC 9: Introduction to Image Restoration 12 / 33

Modeling Image degradation (cont.)
▶ Since H is position-invariant, we get
R∞ R∞
H(f (x, y )) = −∞ −∞ f (α, β)h(x − α, y − β)dαdβ, where
h(x, y ) = H(δ(x, y ))
▶ By the definition of convolution, H(f (x, y )) = f (x, y ) ∗ h(x, y ),
where ∗ denotes convolution and h(x, y ) is impulse
response(ie.h(x, y ) = H(δ(x, y )) ).
▶ Therefore, equation (1) becomes,
g (x, y ) = f (x, y ) ∗ h(x, y ) + η(x, y ). Hence the degradation model
is given by g (x, y ) = f (x, y ) ∗ h(x, y ) + η(x, y ). (3).
▶ To reconstruct f (x, y ), we need to solve the dequation (3). One way
of solving the equation is to apply Fourier Transform for equation
(3), and obtain the value of f (x, y ) as given below.
FT (g (x, y )) = FT (f (x, y ) ∗ h(x, y ) + η(x, y ))
FT (g (x, y )) = FT (f (x, y ) ∗ h(x, y )) + FT (η(x, y ))
FT (g (x, y )) = FT (f (x, y )).FT (h(x, y )) + FT (η(x, y ))
G (u, v ) = F (u, v ).H(u, v ) + N(u, v )
G (u,v ) N(u,v )
F (u, v ) = H(u,v ) + H(u,v ) ,

V. Masilamani DIP TOPIC 9: Introduction to Image Restoration 13 / 33

Modeling Image degradation (cont.)

where H(u, v ) = ϵ when H(u, v ) ≤ ϵ;

H(u, v ) = H(u, v ) otherwise
G (u,v ) N(u,v )
f (x, y ) = IFT ( H(u,v ) + H(u,v ) )

▶ How to get degradation function H(u,v)?

Suppose the degradation system is available, then find the response
of the system for impulse function, say the response is g (x, y ). We
know that FT of impulse function is constant A. To find H(u, v ),
find G (u, v )/A as G (u, v ) = F (u, v ).H(u, v ) when there is no noise.
▶ Note: When noise is present, perfect reconstruction is not possible.
However, the reconstruction process reduces the noise component as
we see in the degradation model

V. Masilamani DIP TOPIC 9: Introduction to Image Restoration 14 / 33

Least Square and constraint Least Square Filters

▶ WKT the Image restoration can be done by finding

f (x, y ) = IFT (G (u, v )/H(u, v ) − N(u, v )/H(u, v )).
▶ To get f , both H and N are required. If we know degradation
process, we can find H(u, v ), but not N.
▶ Hence we can get f ′ (x, y ) = IFT (G (u, v )/H(u, v )).
▶ Reconstruction error can be defined as e = E (f − f ′ )2 .
▶ Can we get f ′ such that the reconstruction error is minimized?
▶ Yes. It has been proved that the wiener’s filter ensures the minimal
error.
▶ The wiener’s filter in frequency domain:
H ∗ (u,v )
|N(u,v )|2
|H(u,v )|2 +
|F (u,v )|2

V. Masilamani DIP TOPIC 9: Introduction to Image Restoration 15 / 33

Least Square and constraint Least Square Filters
(cont.)

▶ As we know the filtering in frequency domain is

IFT (F (u, v ).H(u, v )), where F (u, v ) is FT(f (x, y ))and H(u, v ) is
the filter in frequency domain.
▶ Here the signal that needs to be filtered is the degraded signal
g (x, y )
▶ Filtering g (x, y ) with wiener’s filter means
H ∗ (u,v )
IFT (G (u, v ). 2 |N(u,v )|
2 )
|H(u,v )| +
|F (u,v )|2

∗
H (u,v ) 2
▶ Constrained Least Square Filter: |H(u,v )|2 + γ|P(u, v )|
where P(u, v ) is the FT of laplacian operator.
▶ To find γ, iterative process can be used

V. Masilamani DIP TOPIC 9: Introduction to Image Restoration 16 / 33

Image Restoration from projections using Fourier Slice
Theorem

V. Masilamani DIP TOPIC 9: Introduction to Image Restoration 17 / 33

Image Restoration from projections using Fourier Slice
Theorem (cont.)
▶ Tomography: Study of reconstructing slice from projects
▶ Tomogram: The image of slice

V. Masilamani DIP TOPIC 9: Introduction to Image Restoration 18 / 33

Image Restoration from projections using Fourier Slice
Theorem (cont.)

R
▶ Projection of an image f (x, y ) is defined as (x,y )∈L
f (x, y )dxdy
▶ Imaging modalities that reconstruct from projections

V. Masilamani DIP TOPIC 9: Introduction to Image Restoration 19 / 33

Image Restoration from projections using Fourier Slice
Theorem (cont.)
• Computed Tomography (CT Scan) (uses X-ray)
• Magnetic Resonance Imaging(MRI) (uses Radio wave)
• Functional MRI(FMRI) (Radio wave)
• Positron Emision Tomography(PET)

▶ Although the X-ray imaging was in use since Rantgen took the first
X-ray image(his wife’s hand image), Only in 80s, Computed
tomography came into use
▶ In X -ray, 3D-information will not be available, where as CT image
will have 3D-information.
▶ To obtain 3D-volumetric image, it is sufficient to reconstruct slice
images.
▶ Let us see how slice image is reconstructed.

V. Masilamani DIP TOPIC 9: Introduction to Image Restoration 20 / 33

Image Restoration from projections using Fourier Slice
Theorem (cont.)
▶ Suppose I0 amount of X-ray is transmitted from X-ray source
through an object, and an X -ray detector is kept on the other side
of object along the path of X − ray ,Rthen the amount of X-ray
detected by the detector is I = I0 e − (x,y )∈L f (x,y )dxdy .
R
▶ From this equation, we can get −log (I /I0 ) = (x,y )∈L f (x, y )dxdy .
R
(x,y )∈L
f (x, y )dxdy = log (I0 /I ), which is the projection of f (x, y )
along the line L.
▶ Now the challenge is to find f (x, y ) from projection.
▶ The following is the formal definition of image restoration problem
from projection
R
▶ Given g (ρ, θ) = (x,y )∈L f (x, y )dxdy for all −∞ < ρ < ∞ and
ρ,θ
0 ≤ θ < 360, find f (x, y )
▶ Note that the line here is perameterized with ρ and θ, where ρ is the
length of the line segment which is perpendiculat to the line L, and
making angle θ to X -axis.

V. Masilamani DIP TOPIC 9: Introduction to Image Restoration 21 / 33

Image Restoration from projections using Fourier Slice
Theorem (cont.)
▶ The image restoration problem requires the system of integral
equations formed for all lines.
▶ The following theorem, called Fourier Slice theorem can be used to
solve this system of integral equations.
▶ Fourier Slice Theorem:
R∞
If G (ω, θ) = −∞ g (ρ, θ)e −j2ϕωρ dρ then
G (ω, θ) = F (ωcosθ, ωsinθ), where F (ωcosθ, ωsinθ) is the Fourier
transform, in polar coordinates, of f (x, y )
▶ Proof.
R∞
G (ω, θ) = ρ=−∞ g (ρ, θ)e −j2πωρ dρdxdy
R∞
= ρ=−∞ g (ρ, θ)e −j2πωρ dρdxdy
R∞ R∞ R∞
= x=−∞ y =−∞ f (x, y ) ρ=−∞ δ(ρ−(xcosθ +ysinθ))e −j2πωρ dρdxdy
R∞ R∞
= x=−∞ y =−∞ f (x, y )e −j2πω(xcosθ+ysinθ) dxdy
R∞ R∞
= x=−∞ y =−∞ f (x, y )e −j2π(xωcosθ+y ωsinθ) dxdy
= F (ωcosθ, ωsinθ) (By defn FT, in which ωcosθ, ωsinθ are present
in place of u and v respectively).
V. Masilamani DIP TOPIC 9: Introduction to Image Restoration 22 / 33
Image Restoration from projections using Fourier Slice
Theorem (cont.)

Hence the proof.

▶ An algorithm can be designed using the result above.
▶ Image-Restoration-Fourier-Slice-Theorem()
{
for(θ = 0; θ < 360; θ = θ + 0.5)
for(ρ = 0; ρ < ∞; ρ = ρ + 0.5)
{
F [θ][ρ]= G (ω, θ),(which is FT of samples of projection g (ρ, θ))
}
return(IFFT(F));
}

V. Masilamani DIP TOPIC 9: Introduction to Image Restoration 23 / 33

Back Projection Algorithm

▶ Given the projections g (ρ, θ), back projecton of g is defined as

Rπ
0
g (ρ, θ)dθ.
▶ BackProjection algorithm computes back projection of g (ρ, θ), and
returns as output.
▶ It can bee seen experimentally that back projection of projection
signal is the approximation of the original image f (x, y ).
▶ To understand the approximation, take the following original image

0 0 0 0
0 50 50 0
0 50 50 0
0 0 0 0

▶ The row projection: [0, 100, 100, 0]

▶ The column projection: [0, 100, 100, 0]

V. Masilamani DIP TOPIC 9: Introduction to Image Restoration 24 / 33

Back Projection Algorithm (cont.)

▶ Back projection from row projection:

0 0 0 0
50 50 50 50
50 50 50 50
0 0 0 0

Back projection from row and column projections:

0 50 50 0
50 100 100 50
50 100 100 50
0 50 50 0

V. Masilamani DIP TOPIC 9: Introduction to Image Restoration 25 / 33

Back Projection Algorithm (cont.)

V. Masilamani DIP TOPIC 9: Introduction to Image Restoration 26 / 33

Filtered Back Projection Algorithm

▶ Let us derive an expression for f (x, y ) using inverse Fourier

transform, and design an algorithm using the the expression
▶ We Know that the Inverse Fourier Transform is given by
R∞ R∞
f (x, y ) = −∞ −∞ F (u, v )e j2π(ux+vy ) dudv .
▶ Let us transform the integration to polar coordinates using change
of variables u = ωcosθ and v = ωsinθ
▶ The range of the new variables will be −∞ ≤ ω ≤ ∞ and
0 ≤ θ < π, and the jacobian will be
∂u ∂u
∂ω ∂θ
∂v ∂v
∂ω ∂θ

=
cosθ −ωsinθ
sinθ ωcosθ
=ω

V. Masilamani DIP TOPIC 9: Introduction to Image Restoration 27 / 33

Filtered Back Projection Algorithm (cont.)
R R∞
▶ Hence f (x, y ) = 0π −∞ F (ωcosθ, ωsinθ)e j2π(ωcosθx+ωsinθy ) ωdωdθ
RπR∞
= 0 0 F (ωcosθ, ωsinθ)e j2πω(cosθx+sinθy ) |ω|dωdθ
RπR∞
= 0 0 F (ωcosθ, ωsinθ)e j2πωρ |ω|dωdθ, where ρ = (xcosθ + ysinθ)
RπR∞
= 0 0 G (ω, θ)e j2πωρ |ω|dωdθ, where ρ = (xcosθ + ysinθ) (By
Fourier
R π R Slice Theorem)
∞
= 0 ( 0 G (ω, θ)|ω|e j2πωρ dω)dθ
Rπ
= 0 IFT (G (ω, θ)|ω|)dθ
= Back Projection(Filtering of g (ρ, θ) with frequencey domain filter
|ω| )
▶ Based on the above derivation, the following algorithm, called
filtered back projection algorithm is given
i/p: Digitized g (ρ, θ) Filtered Back Peojection()
{
Find h(ρ, θ)= Fiter the signal g (ρ, θ) using frequency domain filter
|ω|
f (x, y )= Back Projection ( h(ρ, θ))
Return(f (x, y ))

V. Masilamani DIP TOPIC 9: Introduction to Image Restoration 28 / 33

Filtered Back Projection Algorithm (cont.)

V. Masilamani DIP TOPIC 9: Introduction to Image Restoration 29 / 33

Convolution Back Projection Algorithm

▶ Since filtering in frequency domain is doing convolution in spatial

domain, we can design algorithm by filtering in frequency domain
with convolution.
▶ The following is such algorithm, and it is called as convolution back
projection algorithm//
i/p: Digitized g (ρ, θ) Convolution Back Projection()
{
Find h(ρ, θ)= g (ρ, θ) ∗ IFT (|ω|), where ∗ denotes convolution
f (x, y )= Back Projection ( h(ρ, θ))
Return(f (x, y ))
}
}

V. Masilamani DIP TOPIC 9: Introduction to Image Restoration 30 / 33

Convolution Back Projection Algorithm (cont.)

V. Masilamani DIP TOPIC 9: Introduction to Image Restoration 31 / 33

Acknowledgements

▶ Images are downloaded from internet sources

V. Masilamani DIP TOPIC 9: Introduction to Image Restoration 32 / 33

 Thank You! :)

V. Masilamani DIP TOPIC 9: Introduction to Image Restoration 33 / 33