Image Segmentation

1
 Introduction to image segmentation
• The purpose of image segmentation is to
partition an image into meaningful regions
with respect to a particular application
• The segmentation is based on measurements
taken from the image and might be greylevel,
colour, texture, depth or motion

2
 Introduction to image segmentation
• Usually image segmentation is an initial and vital step in
a series of processes aimed at overall image
understanding
• Applications of image segmentation include
– Identifying objects in a scene for object-based
measurements such as size and shape
– Identifying objects in a moving scene for object-based
video compression (MPEG4)
– Identifying objects which are at different distances
from a sensor using depth measurements from a laser
range finder enabling path planning for a mobile
robots
3
 Introduction to image segmentation
• Example 1
– Segmentation based on greyscale
– Very simple ‘model’ of greyscale leads to
inaccuracies in object labelling

4
Introduction to image segmentation

• Example 2
– Segmentation based on texture
– Enables object surfaces with varying patterns
of grey to be segmented

5
Introduction to image segmentation

6
 Introduction to image segmentation
• Example 3
– Segmentation based on motion
– The main difficulty of motion segmentation is that
an intermediate step is required to (either
implicitly or explicitly) estimate an optical flow
field
– The segmentation must be based on this estimate
and not, in general, the true flow

7
Introduction to image segmentation

8
 Introduction to image segmentation
• Example 3
– Segmentation based on depth
– This example shows a range image, obtained with
a laser range finder
– A segmentation based on the range (the object
distance from the sensor) is useful in guiding
mobile robots

9
 Introduction to image segmentation

Range
Original
image
image

Segmente
d image

10
 Greylevel histogram-based segmentation

• We will look at two very simple image segmentation

techniques that are based on the greylevel histogram
of an image
– Thresholding
– Clustering
• We will use a very simple object-background test
image
– We will consider a zero, low and high noise image

11
Greylevel histogram-based segmentation

Noise free Low noise High noise

12
Greylevel histogram-based segmentation

• How do we characterise low noise and high noise?

• We can consider the histograms of our images
– For the noise free image, its simply two spikes at i=100,
i=150
– For the low noise image, there are two clear peaks
centred on i=100, i=150
– For the high noise image, there is a single peak – two
greylevel populations corresponding to object and
background have merged

13
Greylevel histogram-based segmentation

h(i)
2500.00

2000.00

1500.00
Noise free
Low noise
High noise
1000.00

500.00

0.00 i
0.00 50.00 100.00 150.00 200.00 250.00

14
Greylevel histogram-based segmentation

• We can define the input image signal-to-noise

ratio in terms of the mean greylevel value of
the object pixels and background pixels and
the additive noise standard deviation


S/N 


15
Greylevel histogram-based segmentation

• For our test images :

– S/N (noise free) = 
– S/N (low noise) = 5
– S/N (low noise) = 2

16
 Greylevel thresholding
• We can easily understand segmentation based
on thresholding by looking at the histogram of
the low noise object/background image
– There is a clear ‘valley’ between to two peaks

17
Greylevel thresholding

h(i)
2500.00

2000.00

Backgroun
1500.00
d

1000.00
Object

500.00

0.00 i
0.00 50.00 100.00 150.00 200.00 250.00

18
 Greylevel thresholding
• We can define the greylevel thresholding
algorithm as follows:
– If the greylevel of pixel p <=T then pixel p is an
object pixel
else
– Pixel p is a background pixel

19
 Greylevel thresholding
• This simple threshold test begs the obvious
question how do we determine the
threshold ?
• Many approaches possible
– Interactive threshold
– Adaptive threshold
– Minimisation method

20
 Greylevel thresholding
• We will consider in detail a minimisation
method for determining the threshold
– Minimisation of the within group variance
– Robot Vision, Haralick & Shapiro, volume 1, page
20

21
 Greylevel thresholding
• Idealized object/background image histogram

h(i)
2500.00

2000.00

1500.00

1000.00

500.00

0.00 i
0.00 50.00 100.00 150.00 200.00 250.00

22
T
 Greylevel thresholding
• Any threshold separates the histogram into 2 groups
with each group having its own statistics (mean,
variance)
• The homogeneity of each group is measured by the
within group variance
• The optimum threshold is that threshold which
minimizes the within group variance thus maximizing
the homogeneity of each group

23
 Greylevel thresholding
• Let group o (object) be those pixels with
greylevel <=T
• Let group b (background) be those pixels with
greylevel >T
• The prior probability of group o is po (T )
• The prior probability of group b is pb (T )

24
 Greylevel thresholding
• The following expressions can easily be derived for
prior probabilities of object and background
T
po (T )  P (i )
i 0
255
pb ( T )   P( i )
i T 1

P (i ) h(i ) / N
• where h(i )is the histogram of an N pixel image

25
 Greylevel thresholding
• The mean and variance of each group are as
follows :
T
 o ( T )   iP(i ) / po ( T )
i 0
255
 b ( T )   iP( i ) / pb ( T )
i T 1

T
 ( T )   i   o ( T ) P(i ) / po ( T )
2 2
o
i 0

255
 ( T )   i   b ( T ) P( i ) / pb ( T )
2 2
b
i T 1
26
 Greylevel thresholding

• The within group variance is defined as :

2 2 2
 ( T )   ( T ) po ( T )   ( T ) pb ( T )
W o b

• We determine the optimum T by minimizing

this expression with respect to T
– Only requires 256 comparisons for and 8-bit
greylevel image

27
Greylevel thresholding

h(i)
Topt
2500.00

2000.00

Histogram
1500.00
Within group variance

1000.00

500.00

0.00 i
0.00 50.00 100.00 150.00 200.00 250.00

28
 Greylevel thresholding
• We can examine the performance of this
algorithm on our low and high noise image
– For the low noise case, it gives an optimum
threshold of T=124
– Almost exactly halfway between the object and
background peaks
– We can apply this optimum threshold to both the
low and high noise images

29
 Greylevel thresholding

Low noise Thresholded at

image T=124

30
 Greylevel thresholding

Low noise Thresholded at

image T=124

31
 Greylevel thresholding
• High level of pixel miss-classification
noticeable
• This is typical performance for thresholding
– The extent of pixel miss-classification is
determined by the overlap between object and
background histograms.

32
Greylevel thresholding

p(x)

0.02

Backgroun
d

0.01
Object

x
0.00

o b

T
33
Greylevel thresholding

p(x)

0.02

Backgroun
0.01 d
Object

x
0.00
o b

T
34
 Greylevel thresholding
• Easy to see that, in both cases, for any value
of the threshold, object pixels will be miss-
classified as background and vice versa
• For greater histogram overlap, the pixel
miss-classification is obviously greater
– We could even quantify the probability of error
in terms of the mean and standard deviations
of the object and background histograms

35
 Greylevel clustering

• Consider an idealized object/background

histogram

Background

Object

36
c1 c2
 Greylevel clustering
• Clustering tries to separate the histogram into
2 groups
• Defined by two cluster centres c1 and c2
– Greylevels classified according to the nearest
cluster centre

37
 Greylevel clustering
• A nearest neighbour clustering algorithm
allows us perform a greylevel segmentation
using clustering
– A simple case of a more general and widely used
K-means clustering
– A simple iterative algorithm which has known
convergence properties

38
 Greylevel clustering
• Given a set of greylevels
g (1), g (2)......g ( N )
• We can partition this set into two groups
g1 (1), g1 (2)......g1 ( N1 )
g 2 (1), g 2 (2)......g 2 ( N 2 )

39
 Greylevel clustering
• Compute the local means of each group

1 N1
c1   g1 ( i )
N 1 i 1
N2
1
c2 
N2
g
i 1
2 (i )

40
 Greylevel clustering
• Re-define the new groupings

g1 (k )  c1  g1 (k )  c2 k 1 N1

g 2 (k )  c2  g 2 (k )  c1 k 1 N 2

• In other words all grey levels in set 1 are nearer to

cluster centre c1 and all grey levels in set 2 are
nearer to cluster centre c2

41
 Greylevel clustering
• But, we have a chicken and egg situation
– The problem with the above definition is that each
group mean is defined in terms of the partitions
and vice versa
– The solution is to define an iterative algorithm and
worry about the convergence of the algorithm
later

42
 Greylevel clustering
• The iterative algorithm is as follows
Initialize the label of each pixel randomly

Repeat
c1 = mean of pixels assigned to object label
c2= mean of pixels assigned to background label

Compute partition g1 (1), g1 (2)  g1 ( N1 )

Compute partition g 2 (1), g 2 (2)  g 2 ( N 2 )

Until none pixel labelling changes

43
 Greylevel clustering
• Two questions to answer
– Does this algorithm converge?
– If so, to what does it converge?
• We can show that the algorithm is guaranteed
to converge and also that it converges to a
sensible result

44
 Greylevel clustering
• Outline proof of algorithm convergence
– Define a ‘cost function’ at iteration r

1 N1 ( r ) 1 N2
E (r )

N1 i 1
 
( r  1) 2
  g1 (i )  c1 
N2
 g (r )
2 (i )  c 2 
( r  1) 2

i 1

E (r )  0

45
 Greylevel clustering
• Now update the cluster centres
1 N1 ( r )
c1( r )   g1 (i )
N1 i 1
N1
1
c (r )
2 
N2
 2 (i)
g (r )

i 1

• Finally, update the cost function

1 N1 ( r ) 1 N2
E(r )
1
N1 i 1
 (r ) 2
  g1 (i )  c1 
N2
  g (r )
2 (i )  c 2 
(r ) 2

i 1

46
 Greylevel clustering
• Easy to show that
( r 1) (r ) (r )
E E
1 E
• Since E ( r )  0, we conclude that the
algorithm must converge
– but
• What does the algorithm converge to?

47
 Greylevel clustering
• E1 is simply the sum of the variances within
each cluster which is minimised at
convergence
– Gives sensible results for well separated clusters
– Similar performance to thresholding

48
Greylevel clustering

g2

g1

c1 c2

49