Digital Image Processong

Region and Shape

Representation and Description

Christophoros Nikou
[email protected]

Images taken from: R. Gonzalez and R. Woods. Digital Image Processing, Prentice Hall, 2008

University of Ioannina - Department of Computer Science and Engineering

2
Region and Shape Representation
and Description

Well, but reflect; have we not several times

acknowledged that names rightly given are the
likeness and images of the things which they
name?

Socrates

C. Nikou – Digital Image Processing

3 Contents
• After performing image segmentation, a
region may be represented in terms of
– external characteristics (boundaries).
– internal characteristics (texture).
• A shape may be considered as a filled
region with a unique value, e.g. f (x,y)=1.

C. Nikou – Digital Image Processing

4 Boundary following
• Several algorithms require the points in an
ordered clockwise (or counterclockwise)
direction.
• We will describe an algorithm whose
output is an ordered sequence of points.
– Binary images (object and background
points).
– Images are padded with a border of zeros to
avoid object merging with the image borders.
– We limit the discussion to single regions. The
extension is straightforward.
C. Nikou – Digital Image Processing
5 Boundary following (cont.)
• Given a binary region R or its boundary,
the algorithm for following the border of R:
1. Let the starting point b0, be the uppermost,
leftmost point in the image labeled 1.
2. Denote by c0 the west neighbor of b0. c0 is
always a background point.

C. Nikou – Digital Image Processing

6 Boundary following (cont.)
3. Examine the 8-neighbors of b0, starting at c0
and proceeding in a clockwise direction.
4. Let b1 denote the first neighbor encountered
whose value is 1.
5. Let c1 denote the background point
immediately preceding b1 in the sequence.

C. Nikou – Digital Image Processing

7 Boundary following (cont.)
6. Store the locations of b0 and b1 for use in
Step 10.
7. Let b=b1 and c=c1.
8. Let the 8-neighbors of b, starting at c and
proceeding in a clockwise direction, be
denoted by n1, n2,…, n8. Find the first nk
labeled 1.

C. Nikou – Digital Image Processing

8 Boundary following (cont.)
9. Let b=nk and c=nk-1.
10. Repeat steps 8 and 9 until b=b0, that is, we
have reached the first point and the next
boundary point found is b1.
• The algorithm is due to G. Moore [1968]

C. Nikou – Digital Image Processing

9 Boundary following (cont.)
• The need for the stopping rule “… and
the next boundary point found is b1” is
shown below.
• We would only include the spur at the
right if we stop when we reach the initial
point without checking the next point.

C. Nikou – Digital Image Processing

10 Chain codes
• Freeman codes [1961] represent a
boundary by the sequence of straight line
segments of specified length and
direction.
• The direction is coded by a numbering
scheme (4 or 8-connectivity).

C. Nikou – Digital Image Processing

11 Chain codes (cont.)
• To avoid noise degradation and long
chains a resampling of the image grid is
commonly used to describe the boundary
at a coarser level.

C. Nikou – Digital Image Processing

12 Chain codes (cont.)
• The chain code depends on the starting
point.
• To normalize it, we treat the code as a
circular sequence of direction numbers
and redefine the starting point so that the
resulting sequence forms an integer of
minimum magnitude.
• To account for rotation, we use the first
differences of the chain code instead of
the code itself.

C. Nikou – Digital Image Processing

13 Chain codes (cont.)
• The first difference is obtained by
counting the number of direction changes
that separate two adjacent elements of
the code.
• For instance, the first difference of the
chain code 10103322 (e.g. in a
counterclockwise direction) is 3133030.

C. Nikou – Digital Image Processing

14 Minimum perimeter polygons
• A digital boundary may be approximated
by a polygon with arbitrary accuracy.
• The goal is to capture the essence of the
shape using the fewest possible
segments.
• Non trivial and time-consuming problem.
• The minimum perimeter polygon (MPP)
approximation is of modest complexity
and provides good representations for
image analysis applications.
C. Nikou – Digital Image Processing
15 Minimum perimeter polygons (cont.)
• The idea is to enclose a boundary (think of it as a
rubber band) by a set of concatenating cells.
• The boundary is allowed to shrink but it is
constrained by the inner and outer walls of the
bounding region defined by the cells.
• Ultimately, the shape shrinking provides the MPP.

C. Nikou – Digital Image Processing

16 Minimum perimeter polygons (cont.)
• The vertices of the MPP coincide with corners of
the inner or the outer wall.
• The size of the cells determines the accuracy of
the representation.
• The objective is to use the largest possible cell
size acceptable in a given application.

C. Nikou – Digital Image Processing

17 Minimum perimeter polygons (cont.)
• The shape of the object enclosed by the inner wall of
the light gray cells is shown in dark gray.
• Traversing the boundary (counterclockwise) encounters
convex (white dots) or concave (black dots) vertices.
• The vertices of the MPP coincide either with convex
vertices in the inner wall or with the “mirrors” of the
concave vertices in the outer wall.

C. Nikou – Digital Image Processing

18 Minimum perimeter polygons (cont.)
• Useful: the orientation of triplets of points:
a  ( x1 , y1 ), b  ( x2 , y2 ), c  ( x3 , y3 )

 x1 y1 1
A   x2 y2 1
 x3 y3 1

 0 (a, b, c) is a counterclockwise sequence


det( A)   0 a, b, c are colinear
 0 (a, b, c) is a clockwise sequence


• Convenient notation: sgn(a,b,c)=det(A)

C. Nikou – Digital Image Processing
19 Minimum perimeter polygons (cont.)
• sgn(a,b,c)>0 indicates that point c lies on the
positive side of the line passing from (a,b).
• sgn(a,b,c)<0 indicates that point c lies on the
negative side of the line passing from (a,b).
• Note: sgn(a,b,c)=sgn(c,a,b) because the
direction of traversal is the same.
– However, the geometrical interpretation is
different. For example, sgn(c,a,b)>0 indicates that
point b lies on the negative side of the line
passing from (c,a).

C. Nikou – Digital Image Processing

20 Minimum perimeter polygons (cont.)
• Data preparation
– List of coordinates of each vertex.
– Label each vertex as W (convex) or B (concave).
– List of the mirrors of B vertices.
– Vertices must be in sequential order.
– The first vertex V0 is the uppermost leftmost
vertex.
• It is always a W vertex (proof skipped).
– The algorithm uses a white crawler WC and a
black crawler BC crawling along the convex (W)
and mirrored concave (B) vertices respectively.
C. Nikou – Digital Image Processing
21 Minimum perimeter polygons (cont.)
• Initialization: WC = BC = V0.
• VL is the last vertex examined
• VK is the current vertex being examined.
• sgn(VL, WC, VK)>0 or VK lies to the positive side
of the line through (VL, WC).
– The next MPP vertex is WC,
– VL=WC
– Continue with the next vertex after VL.

C. Nikou – Digital Image Processing

22 Minimum perimeter polygons (cont.)
• sgn(VL, WC, VK) ≤0 and sgn(VL, BC, VK) ≥0
– VK becomes a candidate MPP vertex.
– If VK is convex then WC=VK
– Else BC=VK.
– Continue with the next vertex in the list.
• sgn(VL, WC, VK) ≤0 and sgn(VL, BC, VK)<0
– BC becomes a candidate MPP vertex.
– VL=BC
– Reinitialize the algorithm by setting WC = BC = VL
– Continue with the next vertex in the list.

C. Nikou – Digital Image Processing

23 Minimum perimeter polygons (cont.)

C. Nikou – Digital Image Processing

24 Signatures
• 1D functional representation of a boundary.
• A simple way s to plot the distance from the
centroid as a function of the angle.

C. Nikou – Digital Image Processing

25 Signatures (cont.)
• Translation invariant but not rotation invariant.
– We have to find a way to select the same
starting point.
• Select the point which is farthest from the centroid
(not unique for a family of shapes).
• Select the point on the largest eigen axis.
• Obtain the chain code and use the first differences.
• Scale changes
– Scale to [0, 1]. Depends on min and max of the
function. Noise affects this type of scaling.
– Divide each sample by the variance of the
signature.
C. Nikou – Digital Image Processing
26 Signatures (cont.)
• Distance vs angle is not the only way to obtain a
signature.
• Line tangent to the boundary vs a reference line.
– Horizontal segments in the signature would correspond
to straight lines as the tangent would be constant there.
• Slope density function
– Histogram of tangent-angle values at segments of the
boundary of a certain length.
– Peaks at sections of boundary with constant tangent and
valleys in sections with rapidly varying angles.

C. Nikou – Digital Image Processing

27 Boundary segments
• Decomposition into segments reduces the
boundary complexity.
• Attractive approach in presence of
concavities.
• We employ
– The convex hull H of the region S:
• The smallest convex set containing S.
– The convex deficiency H-S of S .

C. Nikou – Digital Image Processing

28 Boundary segments (cont.)
• The boundary is partitioned by following the
contour of S and marking the points at which
a transition is made into or out of the convex
deficiency.
• Smoothing may be necessary to remove
irregularities (noise).

C. Nikou – Digital Image Processing

29
Skeleton by medial axis
transformation
• It may be obtained by region thinning using
morphological operators
– The skeleton is not always connected.
• The medial axis transformation (MAT)
[Blum 1967] of a region R with border B
corrects this drawback:
– For each point p in R, we find each closest
neighbor in B.
– If p has more than one neighbor, it is said to
belong to the medial axis of R.
• The definition of a distance is crucial (e.g. Euclidean).
C. Nikou – Digital Image Processing
30
Skeleton by medial axis
transformation (cont.)
• MAT calculation is computationally
expensive.
• Numerous algorithms have been proposed
for improving the execution time.
• They are based on thinning algorithms
iteratively deleting boundary points subject to
the constraints that the deletion of points:
– does not remove end points.
– does not break connectivity.
– does not cause excessive erosion of the region.

C. Nikou – Digital Image Processing

31
Skeleton by medial axis
transformation (cont.)
• Such an iterative algorithm, • Example:
consists of two thinning N(p1) = 4, T(p1)=3.
steps and it is based on
– The number of nonzero
neighbors N(p1) of a pixel p.
– The number of 0-1
transitions T(p1) in the
ordered sequence p2, p3,…,
p8, p9, p2.

C. Nikou – Digital Image Processing

32
Skeleton by medial axis
transformation (cont.)
• Step 1 flags a boundary pixel p1 for deletion if
the following conditions are satisfied:
a) 2 ≤ N(p1) ≤ 6
b) T(p1) = 1
c) p2 *p4 *p6 = 0
d) p4 *p6 *p8 = 0

C. Nikou – Digital Image Processing

33
Skeleton by medial axis
transformation (cont.)
• In Step 2, conditions a) and b) remain the
same, but conditions c) and d) are changed:
a) 2 ≤ N(p1) ≤ 6
b) T(p1) = 1
c) p2 *p4 *p8 = 0
d) p2 *p6 *p8 = 0

C. Nikou – Digital Image Processing

34
Skeleton by medial axis
transformation (cont.)
• Step 1 is applied to every contour pixel and if all
conditions are satisfied the pixel is flagged for
deletion.
• However, it is not deleted until all contour pixels
have been processed in order not to change the
structure of the data during execution of Step 1.
• Then, Step 2 is applied to the remaining border
pixels in exactly the same manner.
• The procedure is repeated until no points are
deleted.

C. Nikou – Digital Image Processing

35
Skeleton by medial axis
transformation (cont.)
a) 2 ≤ N(p1) ≤ 6
– The condition is violated when contour pixel p1 has
only one or seven neighbors.
• One neighbor implies that p1 is the endpoint of a
skeleton and obviously it should not be deleted.
• Deleting p1 if it has seven neighbors would cause
erosion into the region.

C. Nikou – Digital Image Processing

36
Skeleton by medial axis
transformation (cont.)
b) T(p1) = 1
– The condition is violated when it is applied to
points on a stroke 1 pixel thick.
– It prevents breaking thin segments.

C. Nikou – Digital Image Processing

37
Skeleton by medial axis
transformation (cont.)
c) p2 *p4 *p6 = 0
d) p4 *p6 *p8 = 0
– The conditions are satisfied simultaneously by
the minimum set of values (p4=0 or p6=0) or
(p2=0 and p8=0).

C. Nikou – Digital Image Processing

38
Skeleton by medial axis
transformation (cont.)
– A point p1 that satisfies these conditions and
simultaneously the conditions a) and b) is either
• an east or south boundary point.
• a northwest corner point in the boundary.
– Therefore, p1 is not a point of the skeleton and it
should be removed.

C. Nikou – Digital Image Processing

39
Skeleton by medial axis
transformation (cont.)
– Similar assumptions hold for Step 2 concerning
north or west boundary points and southeast
corner points.

C. Nikou – Digital Image Processing

40 Boundary descriptors
• Geometric descriptors
• Shape numbers
• Fourier descriptors
• Statistical moments

C. Nikou – Digital Image Processing

41 Geometric boundary descriptors
• Length
• Diameter Diam( B)  max[Dist(
i, j
pi , p j )]
• Major axis (connecting the two extreme points of
the diameter)
• Minor axis (perpendicular to the major axis)
– The two axes define the basic rectangle completely
enclosing the boundary
• Eccentricity
– ratio of the lengths between the major and the minor
axis
• Curvature

C. Nikou – Digital Image Processing

42 Shape numbers
• Recall that the chain code depends on the
starting point.
– We select the smallest number in its
representation.
• The shape number of a boundary is the first
difference of smallest magnitude.
– First differences make it invariant to rotation.
• The order n of a shape number is defined as
the number of digits in its representation.

C. Nikou – Digital Image Processing

43 Shape numbers (cont.)
• Examples. All closed shapes of order n=4, 6 and 8.
• First differences are computed by treating the chain
as a circular sequence.

C. Nikou – Digital Image Processing

44 Shape numbers (cont.)
• Although first differences are invariant to rotation
the coded boundary depends on the orientation of
the grid.
• Solution: align the chain-code grid with the sides of
the basic rectangle.
– Compute the basic rectangle and the order n of the
shape.
– Approximate the basic rectangle with a rectangle of order
n (a rectangle with perimeter equal to n), e.g. if n=12, all
the rectangles with a perimeter 12 are in {2x4, 3x3,1x5}.
– Select the one that best matches the eccentricity of the
basic rectangle of the shape.
– Establish the grid on the new rectangle.
C. Nikou – Digital Image Processing
45 Shape numbers (cont.)
• Suppose that n=12 for the specified boundary.

• Basic rectangle

• The closest
rectangle is
of size 3x6
• The grid is
aligned to it.

C. Nikou – Digital Image Processing

46 Fourier descriptors
• The two coordinates are treated as a complex
number and the DFT is computed.
s(k )  x(k )  jy(k ), k  0,1, 2,..., K 1
K 1 2 u
j
a(u )   s(k ) e
k

, u  0,1, 2,..., K  1
k 0

K 1 2 k
s(k )   a(u ) e
j u

, k  0,1, 2,..., K  1
u 0

C. Nikou – Digital Image Processing

47 Fourier descriptors (cont.)
• Approximation by the first P<K first DFT
coefficients, a(u)=0 for u>P:
P 1 2 k
1
sˆ(k )   a(u ) e
j u
P
, k  0,1, 2,..., K  1
P u 0
• Notice that the number of points is always K,
that is the initial number of points is
represented by fewer frequencies.
• Recall from DFT that high frequency
components account for fine detail and low
frequencies capture the global shape.
C. Nikou – Digital Image Processing
48 Fourier descriptors (cont.)

• The properties of the DFT hold.

• A change in the starting point affects all

descriptors in a different but known way as
the term multiplying a(u) depends on u.
C. Nikou – Digital Image Processing
49 Fourier descriptors (cont.)

C. Nikou – Digital Image Processing

50 Statistical moments

• A boundary segment may be represented

as a 1D function g(r) by connecting the end
points and rotating the line segment to be
horizontal.
• Normalizing g(r) to unit area we can treat it
as a histogram.

C. Nikou – Digital Image Processing

51 Statistical moments (cont.)
K 1
n (r )   (ri  m) n g (ri )
i 0
K 1
m   ri g (ri )
i 0

• The second moment measures the spread

of the curve around the mean
• The third moment measures the symmetry
with respect to the mean.
• ….
C. Nikou – Digital Image Processing
52 Regional descriptors
• Simple descriptors
– Geometric descriptors similar with the
boundary case and descriptors including pixel
intensities
• Topological descriptors
• Texture descriptors
– Statistical approaches
– Structural approaches
– Spectral approaches
• Moment invariants
C. Nikou – Digital Image Processing
53 Simple region descriptors
• The area of a region S:
( S )   f ( x, y )
x y
f (x,y)=1 on the pixels belonging to the region
and zero otherwise.
• The perimeter of a region S:
P( S )   ( xi  xi 1 )2  ( yi  yi 1 ) 2
i

where (xi,yi) are the coordinates of the i-th pixel

of the boundary of the region.

C. Nikou – Digital Image Processing

54 Simple region descriptors (cont.)
• The compactness of a region S:
4 A( S ) A( S )
C (S )  2  2
P (S ) P ( S ) / 4

• The denominator is the area of a circle whose

perimeter is P(S).
• It measures the ratio of area of the shape and
the circle that can be traced with the same
perimeter.
• For a perfectly circular region A(S)=1.

C. Nikou – Digital Image Processing

55 Simple region descriptors (cont.)
• The dispersion of a region S:

I (S )  i

 max  xi  x    yi  y 
2 2

A( S )
• It measures the ratio of the major chord
length to the area.
• The numerator defines the area of the
maximum circle enclosing the region.

C. Nikou – Digital Image Processing

56 Simple region descriptors (cont.)
• An alternative definition of dispersion of a
region S measures the ratio of the
maximum to the minimum radius:

I (S )  i

max  xi  x    yi  y 
2
 2

min   x  x  y  y 
2 2
i i
i

• It is the ratio between the radius of the

maximum circle enclosing the region and
the minimum circle that may be contained
in the region.
C. Nikou – Digital Image Processing
57 Simple region descriptors (cont.)
• Other regional descriptors include statistics
on the intensity levels of the pixels in the
region
– mean
– median
– minimum
– Maximum
– Number of pixels with values above or below the
mean

C. Nikou – Digital Image Processing

58 Topological descriptors
• Topology is the study of properties that are
unaffected by any deformation.
– Rubber-sheet distortions.
• No tearings or joinings of the region.
• These properties do not depend on the notion
of distance or a distance measure.
– Number of connected components C.
– Number of holes H.
• Euler number:
E=C-H
C. Nikou – Digital Image Processing
59 Topological descriptors (cont.)

E = C - H = 1 - 2 = -1

C. Nikou – Digital Image Processing

60 Topological descriptors (cont.)
• Regions represented by straight line
segments (polygonal networks) have a
particular simple interpretation in terms of the
Euler number.
• Classification of interior regions into faces
and holes.
• The number of vertices (V) and the number of
edges (Q) and the number of faces (F) give
the Euler formula:
V–Q+F=C–H=E
C. Nikou – Digital Image Processing
61 Topological descriptors (cont.)

C–H=1–3=–2
V – Q + F = 7 – 11 + 2 = -2

C. Nikou – Digital Image Processing

62 Texture
• Basic texture representation
– Statistical approaches
• Histogram
• Co-occurrence matrices.
– Structural approaches
• relational descriptors based on rules (we will look
at them at the end of the section)
– Spectral methods.

C. Nikou – Digital Image Processing

63 Histogram approaches
• Statistical moments of the histogram of a
region.
– Let p(z) be the histogram of an image region
with L possible gray levels zi, i=1,…L.
– nth moment:
L 1 L 1
n ( z )   ( zi  m)n p( zi ), m   zi p( zi ),
i 0 i 0

• The third moment measures the skewness and

fourth moment measures flatness of the
histogram.
C. Nikou – Digital Image Processing
64 Histogram approaches (cont.)
– Contrast:
1
R( z )  1 
1   2 ( z)
– Uniformity (maximum for uniform images):
L 1
U ( z )   p 2 ( zi )
i 0

– Entropy:
L 1
E ( z )   p( zi ) log  p( zi )
i 0
C. Nikou – Digital Image Processing
65 Histogram approaches (cont.)

C. Nikou – Digital Image Processing

66 Co-occurrence Matrices
• Let Q be an operator that defines the
position of two pixels relative to each
other.
• Consider an image f, with L possible gray
levels.
• Let G be a matrix whose element gij is the
number of times that pixel pairs with
intensities zi and zj occur in f in the position
specified by Q.
• G is called co-occurrence matrix.
C. Nikou – Digital Image Processing
67 Co-occurrence Matrices (cont.)
• Quantization of intensities due to computational load.
Many matrices per region. Here L=8 gray levels.

C. Nikou – Digital Image Processing

68 Co-occurrence Matrices (cont.)

gij
pij 
g
i, j
ij

K
P(i )   pij
j 1

K
mr   iP(i )
i 1

K
   (i  mr )2 P(i)
2
r
i 1

C. Nikou – Digital Image Processing

69 Co-occurrence Matrices (cont.)

Random noise

Periodic texture (sine)

Mixed texture

256x256 co-occurence matrices

“one position immediately to the
right”.

C. Nikou – Digital Image Processing

70 Co-occurrence Matrices (cont.)

C. Nikou – Digital Image Processing

71 Co-occurrence Matrices (cont.)
• Is there an image portion containing a certain
texture (a repetitive pattern)?
• Sequences of co-occurrence matrices are
employed.
– the correlation descriptor for varying horizontal
offset of adjacent pixels may be calculated.
• In the next experiment, this is performed for
co-occurence matrices computed for
horizontal offsets from 1 to 50 pixels.

C. Nikou – Digital Image Processing

72 Co-occurrence Matrices (cont.)
Noise image Sine image Circuit image

Disadvantage: co-occurrence matrices are

not efficient for coarse texture description

C. Nikou – Digital Image Processing

73 Structural approaches
• Rule that indicates that the symbol S may be
rewritten as aS: S  aS
• Three repetitions of the rule yield the
string aaaS.
• If a represents a circle and the meaning
“circles to the right” is assigned to a string
of the form aaa…then the rule allows the
generation of the pattern:

a aaa…
C. Nikou – Digital Image Processing
74 Structural approaches (cont.)
• We add new rules, for example:
S  bA, A  cA, A  c, A  bS , S  a

• b means “circle down”.

• c means “circle to the left”.
• aaabccbaa corresponds to a 3x3 matrix of
circles.
• Larger texture patterns may be generated.
• We will treat the topic in detail later.

C. Nikou – Digital Image Processing

75 Spectral Approaches
• The Fourier transform (FT) is useful for
description of the directionality of periodic or
almost periodic structures.
– Peaks in the FT give the principal direction of
patterns.
– The location of peaks gives the fundamental
period of patterns.
– The FT is symmetric around the origin and only
half of the frequency plane needs to be
considered.

C. Nikou – Digital Image Processing

76 Spectral Approaches (cont.)
• The spectrum is expressed in polar
coordinates S(r,θ) for simplification
– We define Sr(θ) which is a 1D function for a given
r and Sθ(r) which is a 1D function for a given θ.
– Analyzing Sθ(r) for a fixed value of θ yields the
behavior of the spectrum along a radial direction
from the origin.
– Analyzing Sr(θ) for a fixed value of r yields the
behavior of the spectrum along a circle of radius
r which is centered at the origin.

C. Nikou – Digital Image Processing

77 Spectral Approaches (cont.)
• A more global description is obtained by
integrating these functions:
 R0

S (r )   S (r ) S ( )   Sr ( )
 0 r 1

– with R0 being a the radius of a circle centered at

the origin.
• The result constitute a par of values [S(r), S(θ)]
for each pair of coordinates (r,θ).

C. Nikou – Digital Image Processing

78 Spectral Approaches (cont.)
Images and FT The periodic bursts extending in 2D in
both spectra are due to the periodic
texture of coarse background material.

The other dominant components in the

left are due to the random orientation of
object edges.

In the right figure, the energy not

associated with the background is along
the horizontal axis.

C. Nikou – Digital Image Processing

79 Spectral Approaches (cont.)

S(θ)

S(r)
Random

S(θ)
Ordered S(r)

Periodic horizontal repetition: θ= 0, 90 and 180, indicate that there

Matches and background. is information at these orientations.
C. Nikou – Digital Image Processing
80 Moment invariants
• 2D moment of order (p+q) of a MxN image
is defined as:
M 1 N 1
m pq    x p y q f ( x, y ),
x 0 y 0

p  0,1, 2,... and q  0,1, 2,...

• It may be used on gray scale images as

well as on shapes. For a shape boundary
we may consider that f (x,y)=1 on the
boundary and zero otherwise.
C. Nikou – Digital Image Processing
81 Moment invariants (cont.)
• The corresponding central moment of
order (p+q) is defined as:
M 1 N 1
m10 m01
 pq    ( x  x ) ( y  y ) f ( x, y ), x 
p q
, y
x 0 y 0 m00 m00
p  0,1, 2,... and q  0,1, 2,...

• The normalized central moments are

defined as:
 pq pq
 pq   ,   1
00 2
C. Nikou – Digital Image Processing
82 Moment invariants (cont.)
• A set of seven invariant moments can be
derived from the second and third
moments. Invariance is obtained with
respect to:
– translation
– rotation
– scale change
– mirroring

C. Nikou – Digital Image Processing

83 Moment invariants (cont.)

1  20  02
2  (20  02 ) 2  4112
3  (30  312 ) 2  (321  03 ) 2
4  (30  12 ) 2  3(21  03 ) 2
5  (30  312 )(30  21 ) (30  12 )  3(21  03 ) 
2 2

 (321  03 )(21  03 ) 3(30  12 ) 2  ( 21  03 ) 2 

C. Nikou – Digital Image Processing

84 Moment invariants (cont.)

6  (20  302 ) (30  12 ) 2  (21  03 ) 2 

 411 (30  12 )(21  03 )
7  (321  03 )(30  12 ) (30  12 )  3( 21  03 ) 
 2 2

 (312  30 )(21  03 ) 3(30  12 )  ( 21  03 ) 

2 2

C. Nikou – Digital Image Processing

85 Moment invariants (cont.)

C. Nikou – Digital Image Processing

86 Moment invariants (cont.)

• Sgn(φi)log10(|φi|) is shown to reduce the dynamic

range.
• The values of the moments are very close.
• The sign of φ7 is different for the mirrored image.
C. Nikou – Digital Image Processing
87 Principal Component Analysis (PCA)

• We want to construct a low-dimensional linear

subspace that best explains the variation in the
components of a multidimensional image:
 x1 
x 
x   2

 
 xn 

• n=3 for RGB images or n=6 for LANSAT images.

C. Nikou – Digital Image Processing

88
Principal Component Analysis
(cont.)
• Multispectral images in the visible blue, visible green, visible red, near
infrared, middle infrared and thermal infrared bands.

C. Nikou – Digital Image Processing

89
Principal Component Analysis
(cont.)
• One n-dimensional vector for each pixel is created.

C. Nikou – Digital Image Processing

90
Principal Component Analysis
(cont.)
• Goal: find a low dimensional
representation for the vectors
that
• minimizes the projection
error (the distance
between the initial vectors
and their projections)
or equivalently
• maximizes the variance of
the projected data.

C. Nikou – Digital Image Processing

91
Principal Component Analysis
(cont.)
• Given: N data points x1, … ,xN in Rd

• We want to find a new set of features that

are linear combinations of the original
ones: N
1
w(xi) = uT(xi – mx), m x   xi
N i 1

• What unit vector u in Rd captures the most

variance of the data?

C. Nikou – Digital Image Processing

92
Principal Component Analysis
(cont.)
• The variance of the projected data:

var  w(xi )    w(xi ) w (xi )   u (xi  m x )  u (xi  m x ) 

1 N T 1 N T T T

N i 1 N i 1
Projection of data point

1 N T T 1  T
N
  u (xi  m x )(xi  m x ) u  u
T
  (xi  m x )(xi  m x )  u
N i 1 N  i 1 

 u Σu
T Covariance matrix of data

C. Nikou – Digital Image Processing

93
Principal Component Analysis
(cont.)
• We now estimate vector u maximizing the variance:
u Σu
T

subject to: u u || u ||  1

T 2

because any multiple of u maximizes the

objective function.
• The Lagrangian is J (u;  )  uT Σu   (1  uT u)
leading to the solution: Σu  u
which is an eigenvector of Σ. The one maximizing J
corresponds to the largest eigenvalue of Σ.
C. Nikou – Digital Image Processing
94
Principal Component Analysis
(cont.)
• The direction that captures the maximum
variance of the data is the eigenvector
corresponding to the largest eigenvalue of
the data covariance matrix.

• The top k orthogonal directions that

capture the most variance of the data are
the k eigenvectors corresponding to the k
largest eigenvalues.

C. Nikou – Digital Image Processing

95
Principal Component Analysis
(cont.)
• Because Σ is real, symmetric and positive
definite its eigenvalues are non negative and its
eigenvectors are orthogonal.
• Let 1  2  ...  d be the eigenvalues of Σ in
descending order and
 v1T 
 T
 v2 
A
 
 T
 v d 
be a matrix whose rows are the eigenvectors of Σ.
C. Nikou – Digital Image Processing
96
Principal Component Analysis
(cont.)
• The transformation that maps x to the new k-dimensional
(k<d) subspace (using the first k eigenvectors) is

 v1T 
 
y  A k (x - m x )     (x - m x )
 vTk 
 
• Properties of the new subspace:

my  0 1 0 0
  
0 The components of y
Σy  Α k ΣΑ T 
k 0
1



 are uncorrelated
 
 0 0 k

C. Nikou – Digital Image Processing

97
Principal Component Analysis
(cont.)
• Because the rows of A are orthonormal A1  AT

• Therefore, we can reconstruct the original data x

from their k-dimensional projections y:

xˆ  ATk y  mx

• The mean square reconstruction error is:

d k d
e  j  j   j
j 1 j 1 j  k 1

which is zero if we make use of all of the

eigenvectors.
C. Nikou – Digital Image Processing
98
Principal Component Analysis
(cont.)
• In our example, the eigenvalues of the covariance matrix of the 6-
dimensional pixels are:

λ1 = 10344
λ2 = 2966
λ3 = 1401
λ4 = 203
λ5 = 94
λ6 = 31

C. Nikou – Digital Image Processing

99
Principal Component Analysis
(cont.)
• The six principal component images y  A 6 (x - m x )
• The first two images account for 89% of the total variance.

C. Nikou – Digital Image Processing

100
Principal Component Analysis
(cont.)
• Reconstructed images using the first two principal components
xˆ  AT2 y  mx

C. Nikou – Digital Image Processing

101
Principal Component Analysis
(cont.)
• Reconstruction error (enhanced for better visualization).

C. Nikou – Digital Image Processing

102
Principal Component Analysis
(cont.)
• PCA is a convenient
way to normalize 2D
boundaries with respect
to rotation and
translation.
• Shifting by (y1min, y2min)
makes the coordinate
non negatives.
• Further division by
λ1and λ2 normalizes the
scale.

C. Nikou – Digital Image Processing

103
Principal Component Analysis
(cont.)
• Application:
eigenfaces for
face recognition

C. Nikou – Digital Image Processing

104
Principal Component Analysis
(cont.)
• Eigenfaces

Mean image

C. Nikou – Digital Image Processing

105 Relational descriptors
• Rewriting rules that capture the basic repetitive
pattern.
• It applies to both boundaries and regions.
• Example: the staircase structure has been
extracted from an image and we want to
describe it.
• We employ two primitive elements and a set of
rules.

C. Nikou – Digital Image Processing

106 Relational descriptors (cont.)
• S and A are variables and a and b are constants.
• Rule 1 indicates that the starting symbol S can be
replaced by a and a variable A.
• Rules 2 and 3 indicate that variable A in turn can be
replaced by b and S or b alone.
– Replacing A by bS leads to the first rule and the procedure may
be repeated.
– Replacing A by b terminates the process as there ar no more
variables to be processed.

C. Nikou – Digital Image Processing

107 Relational descriptors (cont.)
• Example derivations of the rules.
• The relationship between a and b is preserved
as the rules force an a to be followed by a b.

C. Nikou – Digital Image Processing

108 Relational descriptors (cont.)
• These strings are 1D structures.
• Applications of the rules to images requires an
establishment of methods for reducing 2D
positional relations to 1D relations.
• An approach is to follow the contour of an object
and code the result with head-to-tail segments of
specified direction and length.

C. Nikou – Digital Image Processing

109 Relational descriptors (cont.)
• Another approach is to employ directed line
segments with other ways, besides head-to-tail
connections.

Primitives and operations among them

C. Nikou – Digital Image Processing

110 Relational descriptors (cont.)
• Step by step generation of a specific shape.

C. Nikou – Digital Image Processing

111 Relational descriptors (cont.)
• Tree structures better represent non contiguous
textured regions.
• Important information in a tree
– A set of words describing the node (e.g. image
region).
– Relation between a node and its neighbors (e.g.
“inside of”).

C. Nikou – Digital Image Processing