Image Processing and Related Fields!

http://en.wikipedia.org/wiki/File:CVoverview2.svg!

Bernd Girod: EE368 Digital Image Processing! Introduction no. 22!

 What is an image?!

  Ideally, we think of an image as a 2-dimensional light

intensity function, f(x,y), where x and y are spatial
coordinates, and f at (x,y) is related to the brightness or
color of the image at that point.!
  In practice, most images are defined over a rectangle.!
  Continous in amplitude („continous-tone“)!
  Continous in space: no pixels!!

Bernd Girod: EE368 Digital Image Processing! Point Operations no. 1!

 Digital Images and Pixels!

  A digital image is the representation of a continuous image

f (x,y) by a 2-d array of discrete samples f [x,y]. The
amplitude of each sample is quantized to be represented by
a finite number of bits. !
  Each element of the 2-d array of samples is called a pixel
or pel (from „picture element“) !
  Think of pixels as point samples, without extent. !

Bernd Girod: EE368 Digital Image Processing! Point Operations no. 2!

 Image Resolution!

200x200 100x100 50x50 25x25!

•  These images were produced by simply picking every n-th sample

horizontally and vertically and replicating that value nxn times. !
•  We can do better!
–  prefiltering before subsampling to avoid aliasing
–  Smooth interpolation

Bernd Girod: EE368 Digital Image Processing! Point Operations no. 3!

 Color Components!
Monochrome image!

R[x,y] = G[x,y] = B[x,y]

Red R[x,y] Green G[x,y] Blue B[x,y]

Bernd Girod: EE368 Digital Image Processing! Point Operations no. 4!

 Different numbers of gray levels!
256
32
16

„Contouring“

8
4
2

Bernd Girod: EE368 Digital Image Processing! Point Operations no. 5!
 How many gray levels are required?!

  Contouring is most visible for a ramp!

32 levels!

64 levels!

128 levels!

256 levels!

  Digital images typically are quantized to 256 gray levels. !

Bernd Girod: EE368 Digital Image Processing! Point Operations no. 6!

 Brightness discrimination experiment!

  Can you see the circle?!

I + ΔI

Note: I is luminance,
125!
126!
128!
129!
130!
127!
I measured in cd m 2

  Visibility threshold!
„Weber fraction“!
ΔI I ≈ KWeber ≈ 1…2% „Weberʻs Law“!

Bernd Girod: EE368 Digital Image Processing! Point Operations no. 7!

 Contrast with 8 bits according to Weberʻs Law!

  Assume that the luminance difference between two


successive representative levels is just at visibility threshold!

I max 255
= (1 + KWeber )
I min

I max
  For! KWeber = 0.010.02 = 13156
  Typical display contrast!
I min
  Cathode ray tube 100:1!
  Print on paper 10:1!
  Suggests uniform perception in the log(I) domain
(„Fechnerʻs Law“)!

Bernd Girod: EE368 Digital Image Processing! Point Operations no. 8!

 Gamma characteristic!

  Cathode ray tubes (CRTs) are nonlinear!

Luminance!
I!
I ~ Uγ
γ = 2.0 . . . 2.3!

Voltage U!

  Cameras contain γ-predistortion circuit!

U ~ I1 γ

Bernd Girod: EE368 Digital Image Processing! Point Operations no. 9!

 log vs. γ-predistortion!

U

U ~ log(I )

U ~ I1 γ

Imax
= 100
Imin

  Similar enough for most practical applications!

Bernd Girod: EE368 Digital Image Processing! Point Operations no. 10!
 Photographic film!
Hurter & Driffield curve (H&D curve)!
for photographic negative!

slope -γ! Luminance!

2.0
shoulder!
I = I 0 ⋅10 − d
= I 0 ⋅10 −( − γ log E +d0 )
density d

„linear“ region!
1.0
= I 0 ⋅10 − d0 ⋅ E γ

toe! log E

0

d0
E is exposure

  γ measures film contrast!
  General purpose films: γ = -0.7 . . . -1.0!
  High-contrast films: γ = -1.5 . . . -10!
  Lower speed films tend to have higher absolute γ!

Bernd Girod: EE368 Digital Image Processing! Point Operations no. 11!
 Intensity scaling!

Original image! Scaled image!

f [ x, y ] a ⋅ f [ x, y ]

Scaling in the γ-domain is equivalent to scaling in the linear luminance

domain!
γ γ
I ~ ( a ⋅ f [ x, y ]) = a ⋅ ( f [ x, y ])
γ

. . . same effect as adjusting camera exposure time.!

Bernd Girod: EE368 Digital Image Processing! Point Operations no. 12!
 Adjusting γ!

Original image! γ increased by 50%!

f [ x, y ] γ
a ⋅ ( f [ x, y ]) with γ = 1.5

. . . same effect as using a different photographic film . . .!

Bernd Girod: EE368 Digital Image Processing! Point Operations no. 13!
 Image Processing Examples!

Face morphing!

Source: Yi-Wen Liu and Yu-Li Hsueh, EE368 class project, spring 2000.!

Bernd Girod: EE368 Digital Image Processing! Introduction no. 16!

 Gray level histograms!

  Distribution of gray-levels can be judged by measuring a


histogram:!
  For B-bit image, initialize 2B counters with 0 !
  Loop over all pixels x,y!
  When encountering gray level f [x,y]=i, increment counter #ι

  Normalized histogram may be thought of as an

empirical probability distribution.!
  You can also use fewer, larger bins to trade off amplitude

resolution against sample size.!

Bernd Girod: EE368 Digital Image Processing! Point Operations no. 15!
#pixels! Example histogram!

Cameraman!
image!
gray level!

Bernd Girod: EE368 Digital Image Processing! Point Operations no. 16!
#pixels!
Example histogram!

Pout!
image!
gray level!

Bernd Girod: EE368 Digital Image Processing! Point Operations no. 17!
 Histogram equalization!

  Idea: find a non-linear transformation!

g=T(f)
!to be applied to each pixel of the input image f [x,y], such

that a uniform distribution of gray levels in the entire range

results for the output image g[x,y].

  Analyse ideal, continuous case first, assuming!

  0
≤ f ≤ 1 0 ≤ g ≤1
  T(f) is strictly monotonically increasing, hence, there
exists!
f = T −1 ( g ) 0 ≤ g ≤1

  Goal: pdf pg(g) = 1 over the range !0 ≤ g ≤ 1

Bernd Girod: EE368 Digital Image Processing! Point Operations no. 18!
 Histogram equalization example!
Original image Pout! . . . after histogram equalization!
#pixels!

#pixels!
gray level! gray level!

Bernd Girod: EE368 Digital Image Processing! Point Operations no. 22!
 Histogram equalization example!

Original image Pout! Pout!

after histogram equalization!

Bernd Girod: EE368 Digital Image Processing! Point Operations no. 21!
 Histogram equalization example!
Original image Cameraman! . . . after histogram equalization!
#pixels!

#pixels!
gray level! gray level!

Bernd Girod: EE368 Digital Image Processing! Point Operations no. 24!
 Histogram equalization example!

Original image Cameraman!

Cameraman! after histogram equalization!

Bernd Girod: EE368 Digital Image Processing! Point Operations no. 23!
 Histogram equalization example!
Original image Moon! . . . after histogram equalization!

#pixels!
#pixels!

gray level! gray level!

Bernd Girod: EE368 Digital Image Processing! Point Operations no. 26!
 Histogram equalization example!

Original image Moon! Moon!

after histogram equalization!

Bernd Girod: EE368 Digital Image Processing! Point Operations no. 25!
 Adaptive histogram equalization!

  Apply histogram equalization based on a histogram obtained

from a portion of the image!

Sliding window approach:
 Tiling approach:


different histogram (and mapping)
 subdivide into overlapping regions,

for every pixel! mitigate blocking effect by smooth blending

between neighboring tiles !

  Must limit contrast expansion in flat regions of the image,


e.g. by clipping individual histogram values to a maximum!

Bernd Girod: EE368 Digital Image Processing! Point Operations no. 27!
 Adaptive histogram equalization!

Original! Global histogram! Tiling
 Tiling


8x8 histograms! 32x32 histograms!

Bernd Girod: EE368 Digital Image Processing! Point Operations no. 28!
 Adaptive histogram equalization!

Original image Tire! Tire after 
 Tire after 


equalization of
 adaptive histogram equalization

global histogram! 8x8 tiles!

Bernd Girod: EE368 Digital Image Processing! Point Operations no. 29!
 Point operations for combining images!

  Image averaging for noise reduction!

  Combination of different exposures for

high-dynamic range imaging!
  Image subtraction for change detection!
  Accurate alignment is always a requirement!

Bernd Girod: EE368 Digital Image Processing! Point Operations no. 30!
 Image averaging for noise reduction!
1 image 2 images 4 images!

http://www.cambridgeincolour.com/tutorials/image-averaging-noise.htm!

Bernd Girod: EE368 Digital Image Processing! Point Operations no. 31!
 High-dynamic range imaging!

16 exposures, one f-stop (2X) apart!

[Debevec, Malik, 1997] ! Combined image!

Bernd Girod: EE368 Digital Image Processing! Point Operations no. 33!
 Where is the defect?!

Image A (no defect)! Image B (w/ defect)!

Bernd Girod: EE368 Digital Image Processing! Point Operations no. 35!
Absolute difference between two images!

w/o alignment! w/ alignment!

Bernd Girod: EE368 Digital Image Processing! Point Operations no. 36!
 Digital subtraction angiography!

-! +!

Contrast!
enhancement!

http://www.isi.uu.nl/Research/Gallery/DSA/!
Bernd Girod: EE368 Digital Image Processing! Point Operations no. 37!
 Gray-level thresholding!
How can holes be filled?!

Original image! Thresholded ! f [ x, y ] ⋅ m [ x, y ]

Peter f [x,y]! Peter m [x,y]

Bernd Girod: EE368 Digital Image Processing! Thresholding no. 1!

 Error probability for thresholding!

pdf!
Background!

Foreground!

error

probability! Gray level!

Bernd Girod: EE368 Digital Image Processing! Thresholding no. 2!

 Optimal supervised thresholding!

“MAP (Maximum A Posteriori) detector”!

pdf!
Background!

Foreground!

error

probability! Gray level!
If different outcomes are associated with different costs: 

more general “Bayes minimimum risk detector”!

Bernd Girod: EE368 Digital Image Processing! Thresholding no. 3!

 Unsupervised thresholding (cont.)!

Bernd Girod: EE368 Digital Image Processing! Thresholding no. 6!

 Unsupervised thresholding (cont.)!

Bernd Girod: EE368 Digital Image Processing! Thresholding no. 7!

 Chroma keying!

  Color is more powerful for pixel-wise segmentation: 3-d vs. 1-d space!
  Take picture in front of a blue screen (or green, or orange)!

Bernd Girod: EE368 Digital Image Processing! Thresholding no. 8!

 MAP detector in RGB-space!

Original image! Skin color detector!

[S. Leahy, EE368 class project, 2003]!

Bernd Girod: EE368 Digital Image Processing! Thresholding no. 11!

 Region labeling and counting!

  How many bacteria in this picture?!

Original Bacteria image! after thresholding!

  Which pixels belong to the same object (region labeling)?!

  How large is each object (region counting)?!

Bernd Girod: EE368 Digital Image Processing! Thresholding no. 13!

 Example: region labeling!

Bernd Girod: EE368 Digital Image Processing! Thresholding no. 16!

Hole filling as dual to small region removal!

After NOT operation,!

Mask with holes! (background) region labeling,!
small region removal,!
and second NOT operation!

Bernd Girod: EE368 Digital Image Processing! Thresholding no. 19!

Sometimes, a global threshold does not work!

Original image! Thresholded with


Otsuʼs Method!
Bernd Girod: EE368 Digital Image Processing! Thresholding no. 20!
 Locally adaptive thresholding!
  Slide a window over the image!
  For each window position, decide whether to
do thresholding!
  Can use variance or other suitable criterion!
  Thresholding should not be performed in
uniform areas!
  Non-uniform areas: apply Otsuʼs method
(based on local histogram) !
  Uniform areas: classify the entire area as
foreground or background based on mean
value!

Bernd Girod: EE368 Digital Image Processing! Thresholding no. 21!

 Locally adaptive thresholding (example)!

Original image! Global threshold! Local threshold!

Bernd Girod: EE368 Digital Image Processing! Thresholding no. 22!

 Dilation!
  Binary dilation operator!

  Effects!
{ } (
g ⎡⎣ x, y ⎤⎦ = OR ⎡⎣W f ⎡⎣ x, y ⎤⎦ ⎤⎦ := dilate f ,W )
  Expands the size of 1-valued objects!
  Smoothes object boundaries!
  Closes holes and gaps!

Original (178x178)! dilation with ! dilation with !

3x3 structuring element! 7x7 structuring element!

Bernd Girod: EE368 Digital Image Processing! Morphological image processing no. 3!
Example: blob separation/detection by erosion!

Original! Erosion!
binary! by 11x11 !
image! structuring!
circles! element!

Erosion ! Erosion!
by 21x21 ! by 27x27 !
structuring! structuring!
element! element!

Bernd Girod: EE368 Digital Image Processing! Morphological image processing no. 5!
 Example: counting coins!

1 connected
Original thresholded component

22 connected
dilation thresholded after dilation
components
Courtesy: P. Salembier!

Bernd Girod: EE368 Digital Image Processing! Morphological image processing no. 26!
 Morphological edge detector!

original f
dilation g

Courtesy: P. Salembier!
g-f
(g-f) thresholded

Bernd Girod: EE368 Digital Image Processing! Morphological image processing no. 28!
 Example!

n  Image subsampling 2:1 horizontally and vertically!

n  Small input image of size 8x8, output image size 4x4!
! 1 0 0 0 $
# &
# 0 0 0 0 &
# 0 1 0 0 &
# 0 0 0 0 &
Hx = Hy = # &
# 0 0 1 0 &
# 0 0 0 0 &
# 0 0 0 1 &
# &
" 0 0 0 0 %

n  Nice for unified treatment, but NOT recommended for

implementation!

Bernd Girod: EE368 Digital Image Processing! Image Filtering and Deconvolution no. 9!
 Example (cont.)!

n  A somewhat better technique for 2:1 image size reduction!

! 0.5 0 0 0 $
# &
# 0.5 0 0 0 &
# 0 0.5 0 0 &
# 0 0.5 0 0 &
Hx = Hy = # &
# 0 0 0.5 0 &
# 0 0 0.5 0 &
# 0 0 0 0.5 &
# &
" 0 0 0 0.5 %

n  Can you figure out what this does to the image?!
n  Why is this a better technique than the previous one?!

Bernd Girod: EE368 Digital Image Processing! Image Filtering and Deconvolution no. 10!
 Side by side comparison!

! 1 0 0 0 $ ! 0.5 0 0 0 $
# & # &
# 0 0 0 0 & # 0.5 0 0 0 &
# 0 1 0 0 & # 0 0.5 0 0 &
# 0 0 0 0 & # 0 0.5 0 0 &
Hx = Hy = # & Hx = Hy = # &
# 0 0 1 0 & # 0 0 0.5 0 &
# 0 0 0 0 & # 0 0 0.5 0 &
# 0 0 0 1 & # 0 0 0 0.5 &
# & # &
" 0 0 0 0 % " 0 0 0 0.5 %

Bernd Girod: EE368 Digital Image Processing! Image Filtering and Deconvolution no. 11!
 Filtering Examples!

Original! Cameraman blurred by convolution!

Cameraman! Filter impulse response!
! 1 1 !1 1 1 $
# 1 1 !1 1 1 &
1 # &
# 1 1 [1] 1 1 &
25 #
# 1 1 1 1 1 &&
" 1 1 1 1 1 %

Bernd Girod: EE368 Digital Image Processing! Image Filtering and Deconvolution no. 20!
 Filtering Examples!

Original! Cameraman blurred horizontally!

Cameraman! Filter impulse response!
1 !
5
( 1 1 ![1] 1 1 )

Bernd Girod: EE368 Digital Image Processing! Image Filtering and Deconvolution no. 21!
 Filtering Examples!

Original! Cameraman blurred vertically!

Cameraman! Filter impulse response!
! !1 $
# 1 &
1#
! &
# [1] &
5# &
# 1 &
" 1 %
Bernd Girod: EE368 Digital Image Processing! Image Filtering and Deconvolution no. 22!
 Filtering Examples!

Original! Cameraman sharpened!

Cameraman! Filter impulse response!
!
" 0 !1 0 %
1$ !
!1 8 !1 '
[ ]
4 $$ '
# 0 !1 0 '&

Bernd Girod: EE368 Digital Image Processing! Image Filtering and Deconvolution no. 23!
 Filtering Examples!

Original! Cameraman sharpened!

Cameraman! Filter impulse response!
!
" 0 !1 0 %
$ !1 !5 !1 '
$ [ ] '
$# 0 !1 0 '&

Bernd Girod: EE368 Digital Image Processing! Image Filtering and Deconvolution no. 24!
 Why do we process images?!

  Acquire an image!
–  Correct aperture and color balance
–  Reconstruct image from projections
  Prepare for display or printing!
–  Adjust image size
–  Halftoning
  Facilitate picture storage and transmission!
–  Efficiently store an image in a digital camera
–  Send an image from Mars to Earth
  Enhance and restore images!
–  Remove scratches from an old movie
–  Improve visibility of tumor in a radiograph
  Extract information from images!
–  Read the ZIP code on a letter
–  Measure water pollution from aerial images

Bernd Girod: EE368 Digital Image Processing! Introduction no. 2!

 Image Processing Examples!

Restoration of image from Hubble Space Telescope!

!
Source: IVPL Northwestern University, Chicago

Bernd Girod: EE368 Digital Image Processing! Introduction no. 3!

 Color balancing example!

Original Lena! Scale-by-max


color balancing!
Bernd Girod: EE368 Digital Image Processing! Color no. 27!
 Image Processing Examples!

Noise reduction!

Degraded image Noise-reduced image!

Source: Jungwon Lee, EE 368 class project, Spring 2000!

Bernd Girod: EE368 Digital Image Processing! Introduction no. 5!

 Image Processing Examples!

Halftoning!

Bernd Girod: EE368 Digital Image Processing! Introduction no. 7!

 Wiener filtering example!

Image with motion blur! restored by Wiener filtering!

Source: http://www.cs.kun.nl/~ths/rt2/col/h5/5restoratieENG.html!

Bernd Girod: EE368 Digital Image Processing! Image Filtering and Deconvolution no. 55!
 Image Processing Examples!
Special Effects!

Photo! Simulated 
 Simulated 


color pencils! oil painting!

source: Feng Xiao, EE368 class project, spring 2000.

Bernd Girod: EE368 Digital Image Processing! Introduction no. 6!

 Image Processing Examples!
Pseudocolor enhancement for security screening!

Source: Gonzalez+Woods, Fig. 6.24!

Bernd Girod: EE368 Digital Image Processing! Introduction no. 8!

 Image Processing Examples!

Extraction of settlement area from an aerial image!

source: INRIA, Sophia-Antipolis, France!

Bernd Girod: EE368 Digital Image Processing! Introduction no. 9!

 Image Segmentation!

http://cs.stanford.edu/group/roadrunner/stanley.html!

Bernd Girod: EE368 Digital Image Processing! Introduction no. 14!

 Image Processing Examples!

Face Detection!

source: Henry Chang, Ulises Robles, EE368 class project, spring 2000.

Bernd Girod: EE368 Digital Image Processing! Introduction no. 11!

 Image Processing Examples!

Face Detection!

source: Michael Bax, Chunlei Liu, and Ping Li, EE368 class project, spring 2003.

Bernd Girod: EE368 Digital Image Processing! Introduction no. 12!

 Image Processing Examples!
FaceDetection!
Face Blurring for Privacy Protection!

Bernd Girod: EE368 Digital Image Processing! Introduction no. 13!

 Image Processing Examples!
Handwriting recognition!

Bernd Girod: EE368 Digital Image Processing! Introduction no. 17!

 Image Processing Examples!
License Plate Recognition!

http://www.platerecognition.info/1103.htm!

http://ezine.motorola.com/government?a=325!

Bernd Girod: EE368 Digital Image Processing! Introduction no. 18!

 Image Processing Examples!
Biometrics: Fingerprint recognition!

FBIʼs 

Integrated

Automated

Fingerprint

Identification

System!
IAFIS!

Bernd Girod: EE368 Digital Image Processing! Introduction no. 19!

 Image Processing Examples!
Biometrics: Iris recognition!

Source: J. Daugman, U. Cambridge!

Bernd Girod: EE368 Digital Image Processing! Introduction no. 20!

 Image Processing Examples!

Mugshot retrieval!

Source: MIT Media Lab!

Bernd Girod: EE368 Digital Image Processing! Introduction no. 21!

 Spring 2006 Project:

Visual Code Marker Recognition!

Bernd Girod: EE368 Digital Image Processing! Introduction no. 26!

 Spring 2007 Project:

Painting Recognition!

1! 2! 3! 4!

5! 6! 7! 8!

9! 10!
Bernd Girod: EE368 Digital Image Processing! Introduction no. 27!
 Spring 2008 Project:

CD Cover Recognition!

Bernd Girod: EE368 Digital Image Processing! Introduction no. 29!