Computer Vision

Image formation, pixel relationship,

preprocessing
Basic pixel relationships
• An image is denoted by: f(x,y)
• Lowercase letters (e.g. p, q) will denote individual pixels
• A subset of f(x,y) is denoted by S
• Neighbors of a pixel:
– A pixel p at (x,y) has 4 horizontal/vertical neighbors at (x+1,y), (x-1,y), (x,
y+1) and (x, y-1) called the 4-neighbors of p: N4(p)
– A pixel p at (x,y) has 4 diagonal neighbors at (x+1,y+1), (x+1,y-1), (x-1, y+1) and (x-
1, y-1) called the diagonal-neighbors of p: ND(p)
– The 4-neighbors and the diagonal-neighbors of p are called the
8-neighbors of p: N8(p)
Image Structure

• Pixels, 4-adjacency,
P P
8-adjacency, m-adjacency

• Enables summarized image description4-adjacency

too:
8-adjacency

Chain code directions

• Issues? for 4-connectivity (a)
and 8-connectivity (b).
Connectivity
• Connectivity is an important concept in establishing boundaries of object
and components of regions in an image
• When are two pixels connected?
– If they are adjacent in some sense (say they are 4-neighbors)
– and, if their gray levels satisfy a specified criterion of similarity (say they are
equal)
• Example: given a binary image (e.g. gray scale = [0,1]), two pixels may be
4-neighbors but are not considered connected unless they have the same
value
Connectivity
• Let V be the set of values used to determine connectivity
• –For example, in a binary image, V={1} for the connectivity of pixels with a value of
1
• –In a gray scale image, for the connectivity of pixels with a range of intensity
values of, say, 32 to 64, it follows that V={32,33,…,63,64}
• –Consider three types of connectivity
• •4-connectivity: Pixels p and q with values from V are 4-connected if q is in the set
N4(p)
• •8-connectivity: Pixels p and q with values from V are 8-connected if q is in the set
N8(p)
Pixel adjacencies and paths
• Pixel p is adjacent to q if they are connected
– We can define 4-, 8-, or m-adjacency depending on the specified type of
connectivity
• Two image subsets S1 and S2 are adjacent if some pixel in S1 is adjacent to
S2
• A path from p at (x,y) to q at (s,t) is a sequence of distinct pixels with
coordinates (x0,y0), (x1,y1),….., (xn,yn)
– Where (x0,y0)=(x,y) and (xn,yn)=(s,t) and
– (xi,yi) is adjacent to (xi-1,yi-1) for 1<= i <= n
– n is the length of the path
• If p and q are in S, then p is connected to q in S if there is a path from p to
q consisting entirely of pixels in S
Connected components
• For any pixel p in S, the set of pixels connected to p form a
connected component of S
• Distinct connected components in S are said to be disjoint

• Implications?
Distance measures
• Given pixels p, q, and z at (x,y), (s,t) and (u,v) respectively,
• D is a distance function (or metric) if:
– D(p,q) >= 0 (D(p,q)=0 iff p=q),
– D(p,q) = D(q,p), and
– D(p,z) <= D(p,q) + D(q,z).
• The Euclidean distance between p and q is given by:

• The pixels having distance less than or equal to some value r from (x,y) are
the points contained in a disk of radius r centered at (x,y)
Distance measures
• The D4 distance (also called the city block distance) between p and q is
given by:

• The pixels having a D4 distance less than some r from (x,y) form a
diamond centered at (x,y)
• Example: pixels where D4 <= 2
• The D8 distance (also called the chessboard distance) between p and q is
given by:

• The pixels having a D8 distance less than some r from (x,y) form a square
centered at (x,y)
• Example: pixels where D8 <= 2
Distance measures and connectivity

• The D4 distance between two points p and q is the shortest 4-path

between the two points
• The D8 distance between two points p and q is the shortest 8-path
between the two points
• D4 and D8 may be considered, regardless of whether a connected path
exists between them, because the definition of these distances involves
only the pixel coordinates
• For m-connectivity, the value of the distance (the length of the path)
between two points depends on the values of the pixels along the path
Image Structure
• Region = Connected set of pixels
• => connected-component: reachability
• Region boundary, inner and outer contour
Statistical Analysis

• Mean of greylevels:
M 1 N 1
1
m 
P
  I (i, j )
i 0 j 0

• Variance of greylevels:

M 1 N 1
1
   I (i, j)  m 
2 2
V  
P i 0 j 0
• Signal to noise ratio (SNR), reflects the dispersion of the grey levels in the
image :

SNR m
• 
Extremas (min and max levels) et scope (max-min)

15
Statistical Analysis

• Normalisation of the data

– Modification of raw data permits to compare images between
themselves
– By centering them (same means ~ 0) :

I ' (i, j )  I (i, j )  m

– By centering and reducing them :

I (i, j )  m
I ' (i, j ) 

• Profile of line segment …

16
Statistical Analysis

17
 Statistical Analysis

• Grey level histogram H(n)

– Distribution of the population of the pixels belonging to each greyscale component
– For 256 grey levels: number of pixels from 0 to 255
– Representation of the image in terms of intensity and contrast

n2k 1

• Probability of one grey level :

n0
H(n)P
– Normalize the histogram by the number of pixels P=MxN

n2k 1
H(n)
p(n)
P  p(n)1
n0
18
Statistical Analysis
• Quantity of information
– The amount of information provided by a level n is inversely
proportional to the probability of this greylevel

I(n)Log2(p(n))
• Entropy
– Measurement of the average amount of information provided by
each level of the image

2k 1

E(Image) p(n)Log2(p(n))
n0

19
 Statistical Analysis

• Cumulative Histogramm
– Constructed from the histogram of then gray levels according to the formula :
H C ( n)   H ( q )
q 0 
n  0;2 k  1 
– with Hc(0)=H(0) and Hc(2k-1)=P

• Cumulative probability of a grey level

– Normalize the cumulative histogram for the number of pixels P :

H c ( n)
pC (n) 
P
20
 4
x 10
7

0
1800 0 50 100 150 200 250 300

1600

1400

1200
Histogramme cumulation
1000
black
800
Histogramme
600

400
white
200

0
0 50 100 150 200 250 21
300
Statistical Analysis
• Histogramme in x and in y
– Consists in summing all the gray levels of the pixels present in the
same row or the same column and this for all the rows or columns of
the image.

1 M 1
– Respectively  N
i[0;M-1] and j [0;N-1]
H row (i )   I (i, j )
j 0
H column ( j )   I (i, j )
i 0

22
Statistical Analysis

Column histogram Row histogram

23
 Statistical Analysis

• Let: 2

M=12, N=12 or P=144 4

k=4 or K=16
6

10 11 10 12 12 12 11 12 11 10 12 7
10
11 11 11 12 10 9 8 8 7 7 7 7
13 12 11 10 9 8 7 7 7 6 6 8 12
13 11 11 10 8 7 6 6 5 6 5 7 2 4 6 8 10 12

14 12 11 12 7 7 4 4 4 6 5 5
13 14 13 14 8 7 5 6 5 4 3 5 Calculate and propose a code /
14 15 15 13 9 9 7 7 6 6 7 7 algorithm:? - Mean and standard
12 12 11 13 10 7 8 9 8 7 9 11
deviation? - Min and max value? -
3 0 2 1 3 10 11 12 11 10 11 12
2 3 2 1 2 10 12 13 13 14 13 13 Histogram row and column? -
2 0 0 0 11 12 13 14 13 14 14 15 Histogram and cumulative histogram
1 1 1 11 12 13 13 13 12 13 14 15
24
Brightness / Contrast
• Examples of « light » and « dark » images:

Large population of
pixels at low levels
5
x 10
2.5 6000

5000
2

Large population of
4000
1.5
pixels at high levels
3000

1
2000

0.5
1000

0
0 50 100 150 200 250 300
25 0
0 50 100 150 200 250 300
 Brightness / Contrast

• Exemples of image of « low » et « high» contrast :

4 4
x 10 x 10
2 18

1.8 16

1.6
14

1.4
12
1.2
10
1
Large population of pixels
8 high and low levels
0.8 Absence ofpixels high
0.6
and low levels 6

4
0.4

0.2 2

0
0 50 100 150 200 250 300
26 0
0 50 100 150 200 250 300
Image Enhancement and Correction
Distortion due to Camera Misfocus

Original image Distorted image

Distortion due to Camera Misfocus

Camera lens
Distortion due to motion

Camera lens
Distortion due to Random Noise

Original image Distorted image

Point Processing
• The smallest neighborhood of a pixel is 1x1 in size
• Here, g depends only on the value of f at (x,y)
• T becomes an intensity transformation function of the form

• where s and r represent the intensity of g and f at any point (x,y)

• Also called a gray-level or mapping function
Shading Correction
Image normalization
• Rescaling Intensity Values
Intensity transform example

a) Contrast stretching function, b) thresholding function

Some Basic Intensity Transformation Functions

• Here, T is a transformation that maps a pixel value r into a pixel value s

• Since we are concerned with digital data, the transformation can
generally be implemented with a simple lookup table
• Three basic types of transformations
– Linear (negative and identity transformations)
– Logarithmic (log and inverse-log transformations)
– Power-law (nth power and nth root transformations)
General Form for Basic Intensity
Transformations
Correcting for poor exposure
•The range of light intensities in a scene is often very much greater than
can be captured and presented by a photo or video image. In bright
light, you must either expose for the sunlight and lose detail in the
shadows, or vice versa.

•Video cameras operate 24 hours a day under widely varying lighting

conditions.

•Video cameras usually have electronic gain adjustment, but this can
only (attempt to) compensate for changes in overall brightness levels
Local Averaging - Smoothing
 Enhancing Contrast by Adjusting
Grey Values in the Image
A number of techniques, such as histogram equalisation and gamma correction,
involve remapping grey values in the image.

contrast suppressed
output grey values

mapping function

contrast
enhanced

0 dark bright original grey values

Input-Output Transformations – single gradient CS
CS – multiple piecewise gradients
Gamma correction
•Gamma: relationship between an input and the resulting output.

•Gamma correction involves a remapping curve given by the equation

1
 f  (Divide by 255 to get a value
f '  255  between 0 and 1, raise to a
 255  power, then rescale back to 0 to
255.)
Gamma correction: examples
Image normalisation revisited
• changes the range of pixel intensity values

• Go to slides 33-35
Histogram Equalisation

•The distribution of luminance intensities in an image is called its

histogram.
•The horizontal axis represents intensity levels and the vertical axis
represents the number of pixels at each intensity level.
•Most well-lit scenes have a wide dynamic range of intensities, with hills
and valleys.
•Histogram equalisation involves spreading out the image histogram to
one that is more uniform.
•useful in images with backgrounds and foregrounds that are both bright
or both dark
•Produces artifacts
Histogram Equalisation

• Let ni be the frequency of grey level i. The probability of an occurrence of a pixel of level i in
the image is

L = total number of image grey levels , n = total number of image pixels , and being in fact the
image's histogram for pixel value i, normalized to [0,1].

• Let us also define the cumulative distribution function corresponding to px as

which is also the image's accumulated normalized histogram.

Histogram Equalisation
• Objective: create a transformation of the form y = T(x) to produce a new image {y}, such that
its CDF will be linearised across the value range, i.e.

for some constant K. The properties of the CDF allow us to define it as

Now it is only a matter of applying the following simple transformation

Image Interpolation
Image Interpolation
Poor resolution…
 Poor resolution:
Zooming in does not help!
Create a ‘higher resolution’ image by
mathematically interpolating pixel
values
Image Interpolation
• Introduction
– What is image interpolation? Image interpolation refers to the “guess”
of intensity values at missing locations, i.e., x and y can be arbitrary
– Why do we need it? We want BIG images. We want GOOD images. We
want COOL images
• Interpolation Techniques
– 1D zero-order, first-order, third-order
– 2D = two sequential 1D (divide-and-conquer)
– Interpolation Applications
– Digital zooming (resolution enhancement)
– Image inpainting (error concealment)
– Geometric transformations (remember the lab anyone?)

58
1D Zero-order (Replication)

f(n)
n

f(x)

59
1D First-order Interpolation
(Linear)
f(n)
n

f(x)

60
Linear Interpolation Formula

Heuristic: the closer to a pixel, the higher weight is assigned

Principle: line fitting to polynomial fitting (analytical formula)
f(n)
f(n+a)
f(n+1)

a 1-a

f(n+a)=(1-a)f(n)+af(n+1), 0<a<1

Note: when a=0.5, we simply have the average of two

61
Numerical Examples

f(n)=[0,120,180,120,0]

Interpolate at 1/2-pixel

f(x)=[0,60,120,150,180,150,120,60,0], x=n/2

Interpolate at 1/3-pixel

f(x)=[0,20,40,60,80,100,120,130,140,150,160,170,180,…], x=n/6

62
1D Third-order Interpolation
(Cubic)*
f(n)

f(x)

x
Cubic spline fitting

63
 From 1D to 2D
Old wisdom: divide and conquer
 2D interpolation can be decomposed into two sequential
1D interpolations.
The ordering does not matter (row-column = column-row)
Such separable implementation is not optimal but enjoys low
computational complexity

64
 Graphical Interpretation
of Interpolation at Half-pixel

row column

f(m,n) g(m,n)

65
Numerical Examples

a b

c d
zero-order

first-order
a a b b

a a b b
a (a+b)/2 b
c c d d (a+c)/2 (a+b+c+d)/4 (b+d)/2
c (c+d)/2 d
c c d d

66
 Numerical Examples (Con’t)
Col n Col n+1

row m X(m,n) X(m,n+1)

b
a 1-a
Y

1-b

row m+1 X(m+1,n) X(m+1,n+1)

Q: what is the interpolated value at Y?

Ans.: (1-a)(1-b)X(m,n)+(1-a)bX(m+1,n)
+a(1-b)X(m,n+1)+abX(m+1,n+1)
67
Bicubic Interpolation*

68
 Thresholding

• Thresholding of an image
– Diminution in the number of grey levels of an image
• highlighting regions
• simplification of information  acceleration of processing
– Example of “binarisation” (Thresholding that produces a binary image)
• If we had an image I1, at grey level threshold S et I2 the image that results :
– If I1(i,j)  S, I2(i,j) = 0
– if I1(i,j) > S, I2(i,j) = 255

S=2

70
 N2 N2 N2
K-1 K-1 K-1
V2
V3
V2
V1 V1 V1

0 N1 0 N1 0 N1
0 S1 K-1 0 S1 S2 K-1 0 S1 K-D
Simple thresholding Double thresholding High-pass thresholding

N2 N2 N2
K-1 K-1 K-1

V2 V1
V1

V1

0 N1 0 N1 0 N1
0 S1 S2 K-1 0 S1 S2 K-1 S1 K-1
0
Band-pass thresholding Band-rejection Low-pass
71
 Thresholding

• The choice of threshold

– Should it be absolute?
• model implies a perfectly-defined image
• could serve a series of similar images

– Should it be adaptive?
• calculated image by image
• Optimal threshold value

– Should it be adapted depending on the image content?

• in the case of image that are heterogeneous, with local variation
• several threshold values ​in a single image

72
 Thresholding (Otsu’s algorithm)

• Example of automatic thresholding: maximizing the interclass variance

– Take an image ofmean m and population P
– Let s0 be the optimal threshold found
– Let s[0;K-1] be a roving threshold
– For each value of s, we associte two classes C0s et C1s of mean and population {m0s;P0s} et {m1s;P1s}
respectivement

H(n)

C0s
{m0s; P0s} C1s
{m1s; P1s}
n
0 s s0 K-1
73
Thresholding

– For each threshold s, we calculate the interclass variance V(s) defined by :

V ( s )  p (C0s )  (m0s  m) 2  p (C1s )  (m1s  m) 2

with:
p(C0s) P0
sand
p(C1s) P1
s

– We select s0 such that V(sP

)=max{V(s)} for s[0;K-1]
0
P
– Remarkes:
• Not very well adapted for classes having very différent standard-deviations
• Generalisable to N classes

74
 Thresholding

• Example of automatic thresholding: the ISODATA method

– Let[min, max] the interval of the ndg range
– Let’s have the threshold s such that s=(min+max)/2
– Take m0s et m1s the means of each class :

s max
 jh(j)  jh(j)
m0s  j min
s m1s  j max
s 1

h(j)
j min
h(j)
j s 1
– We calculate the new threshold s, so that s=(m + m )/2
0
s
1
s

– We iterate till determining the value of s0 which corresponds to the valeu of the theshold so that
there’s no further change (convergence of the algorithm)
– Generalisable to more than two classes
– Methode of the type method "dynamic clouds"

75