Lecture 8

Object Descriptors

Azadeh Fakhrzadeh
Centre for Image Analysis
Swedish University of Agricultural Sciences
Uppsala University
 2

Reading instructions

Chapter 11.1 – 11.4 in G-W

Azadeh Fakhrzadeh 3

• PhD student
• Research topic:
study of morphological
changes in the testicular
tissue
• [email protected] http://www.udel.edu/biology
 4

Image analysis

• Our progress in the analysis process

 5

So far

Today
and next
lecture
 6

Representations and descriptors

Next step after segmentation is to represent the
object in a good way that makes it Possible to
describe it.

• Two ways to represent regions:

• Boundary (external characteristics)
• Shape, orientation
• Whole region (internal characteristics)
• Color, texture, histogram
 7

Scale, rotation and translation

• Most of the time we are interested to choose descriptors
that are invariant of variations of scale, rotation and
translation whenever possible

But not always in Optical character recognition (OCR) Rotation

and scale is important like P and d
Chain Coding
8

Boundary representation

• Walk around the object

boundary and describe
directional change in each
step by a number
 9

Chain Coding
Considerations

• Code become very long and noise sensitive

• Use larger grid spacing, 0710 = 00
• Scale dependent
• Choose appropriate grid spacing
• Start point determines result
• Treat code as circular (minimum magnitude integer)
754310 075431
• Depends on rotation
• Calculate difference code (counterclockwise)
075431 767767
 10

Polygonal Approximations
Boundary representation

• A digital boundary can be approximated

(simplified)
• For closed boundaries:
• Approximation becomes exact when no. of
segments of the polygons is equal to the no. of
points in the boundary
• Goal is to capture the essence of the object
shape
• Approximation can become a time consuming
iterative process
 11

Polygonal Approximations
• Minimum Perimeter Polygons (MPPs)
• Cover the boundary with cells of a chosen size and
force a rubber band like structure to fit inside the cells
Polygonal Approximations
12

• Merging techniques
1. Walk around the boundary and fit a least-square-error line to
the points until an error threshold is exceeded
2. Start a new line, go to 1
3. When start point is reached the intersections of adjacent lines
are the vertices of the polygon
 13

Polygonal Approximations

• Splitting techniques
1. Start with an initial guess, e.g., based on majority axes
2. Calculate the orthogonal distance from lines to all points
3. If maximum distance > threshold, create new vertex there
4. Repeat until no points exceed criterion
 14

Signatures
Boundary representation
 15

Signatures
Boundary representation

• A 1D representation of a boundary
• Could be implemented in different ways
• Distance from centre point to border as a function of angle
• Angle between the tangent in each point and a reference line
(slope density function)
• Independent of translation, but not rotation & scaling.
Possible solutions:
• Select unique starting point (e.g. based on major axis)
• Normalize amplitude of signature (divide by variance)
 16

Boundary segments
Boundary representation
• When a boundary contains major concavities that carry
shape information it can be worthwhile to decompose it
into segments

• A good way to achieve this is to calculate the convex

Hull of the region enclosed by the boundary

• Can be a bit noise sensitive

• Smooth prior to Convex hull calculation
• Calculate Convex Hull on polygon approximation
 17

Boundary segments
• Can be a bit noise sensitive
• Smooth prior to Convex Hull calculation
• Calculate Convex Hull on polygonal approximation
 18

Skeletons
Shape representation

• Skeletons could be used as curve representations of an

object
• Should in general be thin, centered, topologically
equivalent to original object and reversible
 19

Skeletons

• Example:
 20

Descriptors

• We have now represented our objects in different

ways (using boundary representation and
skeletons)
• The next step is to describe our regions so that
we later can classify them (next lecture)
 21

Simple boundary descriptors

Boundary descriptors

• Length (perimeter)
• Diameter = = major axis
• Minor axis (perpendicular to major axis)
• Basic rectangle = major × minor
• Eccentricity = major / minor
• Curvature= rate of change of slope
Fourier descriptors
22

Boundary descriptors

• Redefine the x- & y-coordinates of the boundary as the

real and imaginary parts respectively of a complex number
• Fourier transform of the new coordinates generates the
Fourier descriptors
• Inverse transformation will regenerate the original image
• Doing an inverse transform on a part of the descriptors will
result in an approximation of the shape
 23

Fourier descriptors

• Represent the boundary

as a sequence of
coordinates
• Treat each coordinate pair
as a complex number
(2D 1D )
 24

Fourier descriptors

• From the DFT of the complex number we get the Fourier

descriptors (the complex coefficients, a(u))

• The IDFT from these coefficients restores s(k)

• We can create an approximate reconstruction of s(k) if

we use only the first P Fourier coefficients
 25

Fourier descriptors

• Boundary reconstruction using 546, 110, 56, 28, 14 and 8 Fourier

descriptors out of a possible 1090.
 26

Fourier descriptors

• This boundary consists of 64 point, P is the number of

descriptors used in the reconstruction
Statistical moments
27

Boundary descriptors
• Useful for describing the shape of boundary segments
(or other curves)
• Suitable for describing the shape of convex deficiencies
• The histogram of the function (segment curve) can also
be used for calculating moments
• 2nd moment gives spread around mean (variance)
• 3rd moment gives symmetry around mean (skewness)
 28

Statistical Moments

• If v is a discrete random variable representing discrete

amplitude in the range [0,A-1] then the nth statistical
moment of v (about its mean) is calculated as:
 29

Simple Regional Descriptors

Regional descriptors
• Area = number of pixels in a region
• Compactness (P2A) = perimeter^2 / area
• Circularity ratio = 4×π×area / perimeter^2
• Graylevel measures
• Mean
• Median
• Max
• Etc.
 30

Topological descriptors
Regional descriptors
• Topology = The study of the properties of
a figure that are unaffected by any
deformation
• Topological descriptors
• Number of holes in a region, H
• Number of connected components, C
• Euler number, E = C – H

A B
 31

Topological descriptors
Regional descriptors

• Using connected components

IR image Thresholded
image

Largest Skeleton
connected
component
 32

Texture
Regional descriptors

• Textures can be very valuable when describing

objects
• Example below: Smooth, coarse and regular
textures
 33

Texture

• Statistical texture descriptors:

• Histogram based
• Co-occurence based
(Statstical moments, Uniformity, entropy,... )

• Spectral texture descriptor

• Use fourier transform
 34

Histogram based descriptors

Regional descriptors

• Properties of the graylevel histogram, of an image or

region, used when calculating statistical moments
• z : discrete random variable representing discrete graylevels in
the range [0 , L-1]
• P(zi) : normalized histogram component corresponding to the ith
value of z

2nd moment : Variance of z

3rd moment : Skewness
4th moment : Relative flatness
 35

Histogram based descriptors

Uniformity and average entropy also uses z & P(zi)

• Uniformity (maximum for image with just one grayvalue):

• Average entropy (measure of variability. Is 0 for constant

images)
 36

Co-occurrence matrix
Regional descriptors

• For an image with N graylevels, and P, a positional

operator, generate A, a N × N matrix, where ai,j is the
number of times a pixel with graylevel value zi is in
relative position P to graylevel value zj

• Divide all elements in A with the sum of all elements in

A. This gives a new matrix C where ci,j is the probability
that a pair of pixels fulfilling P has graylevel values zi and
zj which is called the co-
co-occurrence matrix
 37

Co-occurrence Matrix
 38

Co-occurrence Matrix Descriptors

• Maximum probability (strongest response

to P)

• Uniformity

• Entropy (randomness)
 39

Co-occurrence Matrix

• Example 1
• Maximum probability = 1/3
• Uniformity ≈ 0.264
• Entropy ≈ undefined
 40

Co-occurrence Matrix

• Example 2
• Maximum probability = 1/3
• Uniformity ≈ 0.167
• Entropy ≈ 2.918
 41

Co-occurrence Matrix

• Examples: match image to co-occurrence

matrix
 42

Spectral Texture
Regional descriptors

• Peaks in the Fourier spectrum give information

about direction and spatial period patterns
• The spectrum can be described using polar
coordinates S(r,θ)
• For each angle θ, S(r,θ) is a 1D function Sθ(r)
• Similarly, for each frequency r, Sr(θ) is a 1D
function
• A global description can be obtained by summing
Sθ(r) and Sr(θ)
 43

Spectral Analysis
Regional descriptors
 44

Spectral Analysis

S(r) S(θ)
 45

Moment Invariants
Regional descriptors

• For a 2D continuous function f(x,y), the moment of order

(p + q) is defined as

for p, q = 0,1,2,…

• The central moments are defined as

where
 46

Moment Invariants
• If f(x,y) is a digital image, the central moments become

• The normalized central moments, denoted ηpq, are

defined as

where for p+q = 2,3,…

 47

Moment Invariants

• A set of seven invariant moments can be derived from

the 2nd and 3rd moments
• These moments are invariant to changes in translation,
rotation and scale
48
 49

Principal components Analysis

 50

Matlab code
m = [0 0];
Sigma = [3 2;2 2]; R = chol(Sigma);
z = repmat(m,1000,1) + randn(1000,2)*R;
c=cov(z);
figure,scatter(z(:,1),z(:,2),'.');
c=cov(z);
[E,D]=eig(c)
figure,scatter(z(:,1),z(:,2),'.');
hold on
quiver(0,0,E(1,1),E(2,1),'r','LineWidth',4);
axis('equal');
hold on
quiver(0,0,E(1,2),E(2,2),'r','LineWidth',4);
axis('equal');

[E,D]=eig(c)
[pc score latent]=princomp(z);
figure,scatter(score(:,1),score(:,2),'.');
axis('equal');
 51

Principal components Analysis

• Calculate Cx, covariance matrix of data X

• Find eigenvectors and corresponding eigen values of covariance
matrix (Cxei = λiei)
• Find A which is a matrix with the eigenvectors as rows, ordered
corresponding to decreasing eigenvalue
• Use A to transform x to y: y = A(x
x – mx).
• Any vector x can be recovered from y by: x = ATy + mx and
approximated by only using some (say k) of the eigenvalues and an Ak
matrix constructed from the k eigenvectors
 52

Face recognition Using PCA

We need a training data set: bunch of
We need
sample a training
images data set
of people
we want to recognize

Using PCA analysis we find eigenfaces

(eigenvectors)

Every new image that we have, we

project the Image on eigenvectors
and based on the weights we obtained
we can classify it.

= w1 + w2 + w3

http://www.mathworks.co.uk/matlabcentral/fileexchange/
17032-pca-based-face-recognition-system
 53

Conclusion
• Boundary representation (chain code, polygonal,
signature, convex hull)
• Shape represent (skeleton)
• Boundary descriptor (Fourier descriptor , statistical
descriptor)
• Regional descriptor (Histogram, Texture )
• PCA

Exercise 11.19 and 11.25