Image Processing

Gabriel Brostow

GV12/3072 1
Image Processing.
 GV12/3072 2
Image Processing.
 Motivation and Goals
• Grounding in image processing techniques

• Concentrate on algorithms useful for

machine vision

• Best sensors ever!

GV12/3072 3
Image Processing.
 Motivation and Goals
• Grounding in image processing techniques

• Concentrate on algorithms useful for

machine vision

• Best sensors ever! With a few problems…

GV12/3072 4
Image Processing.
Transmission interference

5
Compression artefacts

6
Spilling

7
Scratches, Sensor noise

8
Bad contrast

9
Resolution  Super resolution?

10
Super resolution

11
 Removing motion blur

Cropped subwindow

Original image

12
After motion blur removal
[Images from Amit Agrawal]
Removing motion blur

13
14
Removing motion blur

15
 Syllabus
1. The digital image
2. Image segmentation
3. Image transformations (Simon Prince)
4. Morphological operations
5. Feature characterization

GV12/3072 16
Image Processing.
Features and Object
Recognition

GV12/3072 17
Image Processing.
 Syllabus (GV12/3072)
6. Image Filtering
7. Edge detection
8. Corner detection
9. Color images
10.Template matching

GV12/3072 18
Image Processing.
 Why Now?
•Medicine
•Automatic or assisted diagnosis
•Image-guided surgery
•Agriculture
•Film and television
•Surveillance and police work
•Military
•http://www.cs.ubc.ca/spider/lowe/vision.html

• Why are these sectors paying more attention?

GV12/3072 19
Image Processing.
 Course content
• Lots of material!

• Some mathematics
• Calculus
• Geometry and matrix algebra
• Probability and statistics

• Some programming – Matlab.

GV12/3072 20
Image Processing.
 Assessment
• Four Courseworks 20%.
• Implement and test algorithms in matlab

• Exam 80%

• Honor System

GV12/3072 21
Image Processing.
 Lectures and notes
• Monday 09:00-10:00 (1.20)
• Monday 11:00-13:00 (Rockefeller 337 David Sacks)

• Lab sessions
– Mon 10-11 (1.05)
– Tue 14-15 (4.06)
– Wed 11-12 (1.05)
• Web page coming soon!
• subject: “join” to [email protected]
GV12/3072 22
Image Processing.
 GV12/3072 23
Image Processing.
 Unassessed CW Assignment
• Matlab introduction
• Start matlab:
% matlab or
% /opt/matlab7/bin/matlab
• Download any simple image

• Load it into matlab:

>> I = imread(‘foo.jpg’);

GV12/3072 26
Image Processing.
 Unassessed CW Assignment
• Display the image in Matlab:
>> imshow(I);
• Print the image data array:
>> I
• Print the size of the image array and create a
subimage:
>> size( I )
>> Isubwindow = I(72:92, 62:82);
>> imshow(Isubwindow);
GV12/3072 27
Image Processing.
 Unassessed CW Assignment
• Start the Matlab help tool (Help menu).
• In the “Contents” pane to the left of the
window. Click on MATLAB.
• Go through the “Getting Started” section.
• Continue to the “Using MATLAB” section
when you have time.

GV12/3072 28
Image Processing.
 GV12/3072 29
Image Processing.
The Digital Image

GV12/3072 30
Image Processing.
 Outline
• What is an image?
• What is a pixel?
• How do we store them?

GV12/3072 31
Image Processing.
What is an image?

GV12/3072 32
Image Processing.
 Image as 2D signal

• Signal: function depending on some variable

with physical meaning
• Image: continuous function
2 variables: xy - coordinates
3 variables: xy + time (video)
• Brightness is usually the value of the function

• But can be other physical values too:

temperature, pressure,
33
depth …
 Example 2d images

ultrasound temperature

34
camera image CT
 Image?

>> t=rand(256,256);
>> imshow(t)

GV12/3072 39
Image Processing.
 What is a pix-el?
(0,0) x

f(x,y)

(x,y)

y
42
 Not a little square!
• A Pixel Is Not A Little
Square, A Pixel Is Not A
Little Square, A Pixel Is
Not A Little Square! (And
a Voxel is Not a Little
Cube),
– Alvy Ray Smith,
MS Tech Memo 6, Jul 17, 1995

GV12/3072 43
Image Processing.
 Not a little square!

Gaussian reconstruction filter

Illustrations: Smith, MS Tech Memo 6, Jul 17, 1995

GV12/3072 44
Image Processing.
 Not a little square!

Cubic reconstruction filter

Illustrations: Smith, MS Tech Memo 6, Jul 17, 1995
GV12/3072 45
Image Processing.
Not a little square!

Graphics: Dick Lyon, 2006

GV12/3072 46
Image Processing.
Where do images come from?
• Digital cameras
• MRI scanners
• Computer graphics packages
• Body scanners
• Laser range finders

• Many more…
GV12/3072 47
Image Processing.
Where do images come from?
• Digital cameras
• MRI scanners
• Computer graphics packages
• Body scanners
• Laser range finders

• Many more…
GV12/3072 48
Image Processing.
 The digital camera
• A Charge Coupled Device (CCD).
Sensor
array

Image array

ADC
Lens GV12/3072 49
Image Processing.
Capturing photons

From: Lecture Notes – EAAE

GV12/3072 and/or Science “Nuggets” 2000
50
Image Processing.
http://www.astro.virginia.edu/class/oconnell/astr121/im/CCD-fullframearc-FSU.jpg

GV12/3072 51
Image Processing.
 The sensor array
• Can be < 1cm2.
• An array of photosites.
• Each photosite is a
bucket of electrical
charge.
• They contain charge
proportional to the
incident light intensity
during exposure.

GV12/3072 53
Image Processing.
 Analog to Digital Conversion
• The ADC measures the
charge and digitizes the
result.
• Conversion happens line
by line.
• The charges in each
photosite move down
through the sensor array. ADC

RAM
GV12/3072 54
Image Processing.
ADC

RAM

GV12/3072 55
Image Processing.
 Blooming
• The buckets have finite capacity
• Photosite saturation causes blooming

GV12/3072 56
Image Processing.
 Dark Current
Yohkoh satellite, 9 years apart ..

• CCDs produce thermally-generated charge.

• They give non-zero output even in darkness.
• Partly, this is the dark current.
• Fluctuates randomly.

• How can we reduce dark current? From: Lecture Notes - EAAE

GV12/3072 57
Image Processing.
 Dark Current
• CCDs produce thermally-generated charge.
• They give non-zero output even in darkness.
• Partly, this is the dark current.
• Fluctuates randomly.

• How can we reduce dark current? From: Lecture Notes - EAAE

GV12/3072 58
Image Processing.
 GV12/3072 59
Image Processing.
 GV12/3072 62
Image Processing.
 Sampling 1D

Sampling in 1D takes a function, and returns a vector whose elements are

values of that function at the sample points.

63
 Sampling 2D

Sampling in 2D takes a function and

returns an array; we allow the
array to be infinite dimensional and to
have negative as well as positive indices.
64
Greyscale digital image

GV12/3072 65
Image Processing.
 Nyquist Frequency

• Half the sampling frequency of a discrete

signal processing system

GV12/3072 66
Image Processing.
Sampling grids

67
Retina-like sensors

68
 Quantization
• Real valued function will get digital values –
integer values
• Quantization is lossy!!
– After quantization, the original signal cannot be
reconstructed anymore
• This is in contrast to sampling, as a sampled but
not quantized signal can be reconstructed.
• Simple quantization uses equally spaced levels
with k intervals

k 2 b
69
 Quantization

11

10

01

00
70
 Quantization

11

10

01

00
71
 Usual quantization intervals

• Grayvalue image
8 bit = 2^8 = 256 grayvalues
• Color image RGB (3 channels)
8 bit/channel = 2^24 = 16.7Mio colors
• 12bit or 16bit from some sensors
• Nonlinear, for example log-scale

72
Photo: Paulo Barcellos Jr.
GV12/3072 73
Image Processing.
Winding Down

GV12/3072 74
Image Processing.
 Properties

• Image resolution

• Geometric resolution: How many pixel per area

• Radiometric resolution: How many bits per pixel

75
 Image resolution

512x512

1024x1024 512x1024

76
Geometric resolution

144x144 72x72 36x36

18x18 9x9 4x4

GV12/3072 77
Image Processing.
 Radiometric resolution

256 128 64 32

16 8 4 2
GV12/3072 78
Image Processing.
 Lossless vs. Lossy
• Name some formats?

GV12/3072 79
Image Processing.
 Aliasing and SNR
• What is the disadvantage of low sampling
resolution?

• What is the disadvantage of high sampling

resolution?

GV12/3072 80
Image Processing.
 Finish
Next week:
Image segmentation

GV12/3072 81
Image Processing.