Unit 3: Feature Extraction

Contents
● Point, Line and Edge Detection
● Color, Texture, Shape and Structure Features in Spatial and
Frequency Domains
● Corner Detection
● Hough Transform
Detection of Discontinuity

Types of Grey
Level
Discontinuities

Point Line Edge

Point Detection
● An isolated point is a point whose gray level is significantly different from its
background in an homogenous area.

W1 W2 W3 Z1 Z2 Z3

W4 W5 W6 Z4 Z5 Z6

W7 W8 W9 Z7 Z8 Z9

Mas Image
k
If |R| >= T, a point is detected where, T is a non negative integer.
-1 -1
Sample Mask for Point Detection -1

-1 8 -1

-1 -1 -1
Line Detection
● In this, four types of masks are used to get the responses i.e.,
R1,R2,R3,R4 for the directions vertical, horizontal, +45 degree and -
45 degree respectively.
-1 -1 -1 -1 2 -1 -1 -1 2 2 -1 -1

2 2 2 -1 2 -1 -1 2 -1 -1 2 -1

-1 -1 -1 -1 2 -1 2 -1 -1 -1 -1 2
Horizontal Vertical +45 -45
Edge Detection
● An edge is a set of connected pixels that lies on the boundary between
two regions which differ in grey value. Pixels on edge is known as edge
points.
● Edges provide an outline of the object.
● In physical plane edge corresponds to the discontinuities in depth,
surface orientation, change in material properties, light variations, etc.
● It locates sharp changes in the intensity function
● Edges are pixels where brightness changes abruptly
● An edge can be extracted by computing the derivative of the image
function
○ Magnitude of the derivative, indicates the strength or contrast of edge
○ Direction of derivative vector, indicates the edge orientation.
Edge Detection (Cont.)
● Example: A mug seen in the input image and once edges are
detected, one can find out the exact layout or boundary of the
object.
● The edges are detected based on the significant change in the
intensity between the objects in the image.
● To make it precise, the mug has a different intensity from the grey
background and it is the key idea for us to identify the edges. So, in
an image, if there are many objects, edges are the easiest way to
identify all of them.
Edge Detection (Cont.)
Edge Detection (Cont.)
Why to detect edges?
● One can understand the shape of the object in the image only when
the edges are detected. So, ideally to understand an object and its
shape, it becomes inevitable for someone to detect the edges.
● There are many technical issues and challenges mapped to the
segmentation, registration and object identification techniques.
Edges prove to be efficient with these above techniques at the
fundamental levels.
Edge Detection (Cont.)
Some of the commonly encountered edges in image processing

Ramp Spike
Step Edge Roof Edge
Edge Edge
Stages in Edge detection(Cont.)
● There are three steps followed in edge detection process. They are:
● Image Smoothening
● Edge Point Detection
● Edge Localization
Stages in Edge detection(Cont.)
● Image smoothening is all about removal of the noise from the
image. In addition, the removal or suppression of the noise should
be done in such a way that the quality of the image is not altered.
● Essentially, image smoothening is all about making the image noise
free.
Stages in Edge detection(Cont.)
● Edges are detected through identifying the sudden changes in the
intensity. Even noise is all about sudden change in the intensity.
Edge point detections is a process where the noise alone is carefully
removed or discarded retaining the edges appropriately
Stages in Edge detection(Cont.)
● The final step in the sequence is edge localization. Sometimes the
processes like thinning, linking etc. are to be carried out to locate
the edges appropriately. This process is called edge localization.
Edge Detection (Cont.)
Sobel Edge Detector
● The image is processed in the X and Y directions respectively.
● This would result in the formation of new image, which actually is
the sum of X and Y edges of the image.
● This approach works through calculation of the gradient of the image
intensity at every pixel within the image.
● Sobel filter has two kernels (3x3) matrix
● One of them corresponds to the X (horizontal) and Y (vertical)
direction.
● These two kernels shall be convoluted with the original image under
process and through which the edge points are calculated with ease.
Sobel Edge Detector (Cont.)
● The kernel values are fixed and cannot be altered.
● The Gaussian filter play a vital role in the entire process. The
fundamental idea is the center having more weight than the rest
● The general approach detecting the edges is with the first order or
second order derivatives as shown below:
Sobel Edge Detector (Cont.)
Sobel Edge Detector (Cont.)
● Convolute the Gx(Sobel X) and Gy(Sobel Y) over the input image
which shall enable to calculate the value for one pixel at a time.
● Then the shift happens and the move has to be made towards the
right i.e., the next column shift till the column end.
● Similar process is followed for the row shifting from top to bottom.

The left matrix is a 5x4 image and is being

convoluted with Gx(Sobel X) operator. In this
5x4 matrix, we take only the first 3x3 matrix,
in which the center value I is computed
Sobel Edge Detector (Cont.)
● The resulting gradient approximation can be calculated as:

● The G will be compared against the threshold and with which one
can understand the taken point is edge or not.
Sobel Edge Detector (Cont.)

Input Gradient

Edge
Sobel Edge Detector (Cont.)
Advantages
● Simple and time efficient computation
● Very easy at searching for smooth edges

Disadvantages
● Diagonal direction points are not preserved always.
● Highly sensitive to noise
● Not accurate in edge detection
● Detect with thick and rough edges does not give appropriate
results.
Prewitt Edge Detector
● It is similar to Sobel edge detection but with a minor change.The
prewitt operator shall give the values which are symmetric around
the centre. But sobel operator gives weight to the point which is
lying closer to (x,y)
● The Prewitt x and y operator are presented below:
Prewitt Edge Detector (Cont.)

Input Gradient

Edge
Prewitt Edge Detector (Cont.)
Advantages
● Good performance on detecting vertical and horizontal edges
● Best operator to detect the orientation of image.

Disadvantages
● The magnitude of coefficient is fixed and cannot be changed.
● Diagonal direction points are not preserved always.
Canny edge detector
● Canny edge detection technique, not just is a plain edge detection
technique. It has an additional feature. It also suppresses the noise
while detecting the edges flawlessly.

Step 1: Conversion to grayscale

Canny edge detector (Cont)
Step 2: Gaussian Blur: It is an operator, which helps in removing noise
in the input image. The noise removed shall enable further processing
to be smooth and flawless. The sigma value has to be set appropriately
for better results.
Canny edge detector (Cont)
Step 3: Intensity Gradient Calculation: Here Sobel filter is used. The
whole process remains same.
Canny edge detector (Cont)
● It is preferred to use Sobel operator and it is the general process. But it
is not mandatory rule to always go with sobel operator. It can be any
gradient operator and the result should be the gradient magnitude of
the image.
● The resulting gradient approximation can be calculated with:

● The G will be compared against the threshold and with which one can
understand the taken point is edge or not.
● The formula for finding the edge direction is Theta = inv tan (Gy/Gx)
Canny edge detector (Cont)
Step 4: Non Maximum Suppression: The gradient magnitude operators
discussed in the previous stages normally obtain thick edges. But final
image is expected to have thin edges.
Hence non maximum suppression shall enable to derive thin edges from
thicker ones through following steps:
● Edge direction is already known. The subsequent step is to relate the
identified edge direction to a DIRECTION that can be sketched in the
image. i.e., ideally it is a prediction of how the movement of edges
could happen.
● For reference 3x3 matrix is taken.
Canny edge detector (Cont)
Canny edge detector (Cont)
● The center cell is the region of interest for us.
● There can be only 4 possible directions for any pixel. They are
○ 0 degrees
○ 45 degrees
○ 90 degrees and
○ 135 degrees
● Hence it forces us to a situation where the edges has to be definitely
oriented to one of these four directions.
● This is a kind of approximation where if the orientation angle is
observed to be 5 degrees then it is taken as 0 degrees. Similarly if it is
43 degrees, it shall be made as 45 degrees.
● For easy understanding semicircle with color shading is drawn. It is
drawn 180 degree but actual scenario is for 360 degree.
Canny edge detector (Cont)
● Any edge come under yellow
range is set to 0 degree
● Any edge come under green
range is set to 45 degree
● Any edge coming under blue
range is set to 90 degree
● Any under red is set to 135
degree
Canny edge detector (Cont)
● After this process direction of the edges is mapped to any of the 4
directions mentioned above. The input image now shall be like the
one presented below where direction of the edges are appropriately
mapped.
Canny edge detector (Cont)
● The edge directions are all determined and now the non maximum
suppression is to be applied.
● As the name suggest, it is a process where suppression of the pixels
to zero which cannot be considered as an edge is carried out. This
shall enable the system to generate a thin line in the output image as
shown.
● This result are obtained before the thresholding and next stage is to
perform thresholding and smoothing.
Canny edge detector (Cont)
5. Thresholding
● Here two threshold value is set, one is high and another is low.
● Example high threshold value as 0.8. Any pixel with a value above 0.8
is to be seen as stronger edge.
● Another threshold lower can be 0.2. In that case, any pixel below this
value is not an edge at all and hence set them all to 0.
● The edge in between 0.2 to 0.8 are weak edge i.e., may or maynot be
edge. There needs to be a process which can determine which of the
weak edges are actual edges.
Canny edge detector (Cont)
6. Edge Tracking
● As discussed above it is important to know which of the weaker edges
are the actual edges.
● We can call the weak edges connected to the strong edges as
strong/actual edges and retain them and those which are not
connected to the stronger edges are to be removed.
7. The final cleansing
All the remaining weak edges can be removed and that is it. The process
is complete. Once this is done. The following output is obtained.
Canny edge detector (Cont)
Advantages
● It has good localization.
● It extract image features without altering the features.
● Less sensitive to noise

Disadvantages
● There is false zero crossing
● Complex computation and time consuming
Laplacian of Gaussian
● In edge detection we normally look for the peaks in the image,
sudden change in intensity and not about the color content. If there
is a sharper change in intensity, we declare it as an edge. But in this
there are challenges.
● Deciding some point as a peak or not is a tough call. We use
thresholding for this, above the threshold we call it as peak and
below ignore it.
Laplacian of Gaussian (Cont.)
● As shown in the figure left to the curve, where it is rising, the slope
is definitely positive. Coming to the right side, it is negative. Hence
there is a zero crossing. It reveals the edge location. The edge
detectors which take this as the fundamental concept behind are
called as Laplacian Edge detectors.
● The Laplacian edge detector uses only one kernel where it was two
with respect to Sobel edge detector. Laplacian edge detector is
capable and it calculates the second order derivative in a single go.
The following are the kernel used:
Laplacian of Gaussian (Cont.)
Laplacian of Gaussian (Cont.)
● Following is the 2-D Log function with Gaussian standard deviation
Laplacian of Gaussian (Cont.)
Advantages
● Easy to detect edges and their various orientations.
● There is fixed characteristics in all directions.

Disadvantages
● Very sensitive to noise
● The localization error may be severe at curved edges.
● It generates noisy responses that do not correspond to edges, so
called false edge.
Real world applications of Image Edge detection
● Medical imaging, study of anatomical structure
● Locate an object in satellite image
● Automatic traffic controlling system
● Face recognition and fingerprint recognition
Image Enhancement
● Image enhancement refers to the process of highlighting certain information of
an image, as well as weakening or removing any unnecessary information
according to specific needs. For example, eliminating noise, revealing blurred
details, and adjusting levels to highlight features of an image.

Image enhancement techniques can be divided into two broad categories:

● Spatial domain — enhancement of the image space that divides an image into
uniform pixels according to the spatial coordinates with a particular resolution.
The spatial domain methods perform operations on pixels directly.
● Frequency domain — enhancement obtained by applying the Fourier Transform
to the spatial domain. In the frequency domain, pixels are operated in groups as
well as indirectly.
Image Enhancement (Cont.)
Image Enhancement (Cont.)
Types of Spatial Domain Technique

Types of spatial domain operator:

● Point operation (intensity transformation) - Point operations refer to
running the same conversion operation for each pixel in a grayscale
image. The transformation is based on the original pixel and is
independent of its location or neighboring pixels.
● Spatial filter (or mask, kernel) - The output value depends on the
values of f(x,y) and its neighborhood.
Spatial Domain
● An image can be represented in the form of a 2D matrix where each
element of the matrix represents pixel intensity. This state of 2D
matrices that depict the intensity distribution of an image is called
Spatial Domain. It can be represented as shown below-
Spatial Domain (Cont.)
● For the RGB image, the spatial domain is represented as a 3D
vector of 2D matrices. Each 2D matrix contains the intensities for a
single color as shown below-
Spatial Domain (Cont.)
● Each pixel intensity is represented as I(x,y) where x,y is the co-
ordinate of the pixel in the 2D matrix. Different operations are
carried out in this value. For example- operation T(say, addition of 5
to all the pixel) is carried out in I(x,y) which means that each pixel
value is increased by 5. This can be written as-

I’(x,y) = T[I(x,y)]
● where, I’(x,y) is the new intensity after adding 5 to I(x,y).
Spatial Domain (Cont.)
● The spatial domain enhancement is based on pixels in a small range
(neighbor). This means the transformed intensity is determined by
the gray values of those points within the neighborhood, and thus
the spatial domain enhancement is also called neighborhood
operation or neighborhood processing.
● A digital image can be viewed as a two-dimensional function f (x, y),
and the x-y plane indicates spatial position information, called the
spatial domain. The filtering operation based on the x-y space
neighborhood is called spatial domain filtering.
Frequency domain
What is Frequency domain?
● It means “analysis of the signals or functions” with respect to the
frequency instead of “time”.

Difference between spatial and frequency domain

● In spatial domain the value of the pixels of an image normally
changes with respect to the scene. In short dealing the image as it is.
● In the spatial domain, it is going to work on the intensity values of the
image pixels. But when it comes to the frequency domain it is all
about transformation of the coefficient instead of the direct
manipulation of the pixels like the spatial domain.
Frequency domain (Cont.)
● For spatial domain it is all about
manipulation or processing of an
input image as it is to enhance the
image for further operations or for
the given application.
● All the techniques that we know
and we discuss are direct operation
or manipulation of the pixels in an
image in the spatial domain.
● The filters deployed normally are
the smoothing filters, edge
detection filters, sharpening filters,
etc.
Frequency domain (Cont.)
● In frequency domain the first step is all
about conversion of an image from
spatial to frequency domain.
● E.g., Fast Fourier Transform used to
convert spatial domain content to
frequency domain (direct transformation)
● Also, low pass filters for smoothing and
high pass filter for sharpening the image
(frequency filter)
● After the processing is carried out till this
stage, results available are not ideal to
display as output. Hence inverse
transformation is carried out to ensure
the output image is displayed
appropriately.
Frequency domain (Cont.)
Step 1: Use Fast Fourier Transform in order to get gray scaled image into frequency
transformation. (I/P image gets transformed into frequency) After end of this step, spectrum is
available for further processing.

Step 2: The spectrum available might not be an easier choice to operate with filters. Also it can
be visualize by humans. Hence, there needs further processing. Shifting to zero frequency
component to the center of the spectrum is the step to be carried out.

Step 3: Apply the corresponding filters like the low or high pass based on the requirement to
filter the frequencies.

Step 4: It is the time to decentralization. One should get the things back after the processing
through this decentralization process.

Step 5: One should apply the inverse Fast Fourier Transform which enables conversion of the
spectrum to grayscale image. This is output image.
Frequency domain (Cont.)
Frequency domain (Cont.)
Low pass filters: The purpose of low pass filter is all about getting the
image smoothened or blurred. The low frequency content will be
retained while attenuating the high frequency content.
Frequency domain (Cont.)
Ideal Low pass filter, which is commonly referred as ILPF is the most
used filter for image smoothening in the frequency domain. As
expected, the ILPF removes the high frequency content (noise) from the
image and retains the low frequency component.

Mathematically the low pass filter can be represented through the

function:
Frequency domain (Cont.)
● Do - Positive content. The filter retains all the frequency component
within the radius Do
● All the frequency components outside the radius of the circle shall be
eliminated. Most of the cases it would all be the high frequency
components getting eliminated. Within the circle, the retention
happens without attenuation.
● Do is the one referred as the cut-off frequency.
● Also D(u,v) is the Euclidean distance from any point (u,v) to the origin
of the frequency plane.
● If the value of Do is very minimal, there is a risk of losing the core
information and it would be over blurred or smoothened.
Frequency domain (Cont.)
Frequency domain (Cont.)
Frequency domain (Cont.)
● Butterworth Low pass filter is used to get as flat as frequency
response as possible. It is normally said to be used for image smoothing
when it comes to frequency domain.
● It is recognized as BLPF, which is very helpful in removal of high frequency
noise from the input image while preserving the low frequency contents.
● It has no sharp frequency response transition. When compared with ideal
low pass filter, this one is said to have very smooth transition.
● The transition between the stop-band and pass-band is totally determined
by the order of the filter.
● Here, represent by n. It determines the steepness of the transition
between the bands.
Frequency domain (Cont.)
● The transition could be soft and gentle or it can as well be abrupt
and sudden. So, based on the value of n, one could determine the
transition.
● Ideal low pass filter is said to have something called Ringing effect.
● It has to be avoided and BLPF certainly gets over the ringing effect.
Frequency domain (Cont.)
● Do is termed the cut off frequency and it is always regarded as the transition
point between 1 and 0 (H(u,v) = 1 and H(u,v) = 0). BLPF works based on the
Do value. It passes all the frequencies inferior to the Do, whereas it cuts off
all the frequencies above the Do.
● If ILPF is considered, the transition could have been sharper, but, when it is
BLPF it is smooth transition happening between 1 and 0. This is the reason
BLPF is said to fight the ringing effects.
● n- will define order. More the order, it will look and behave similar to ILPF.
● D(u,v) - is the euclidean distance from any point (u.v) to the origin of the
frequency plane. It is always important and in fact unavoidable to understand
the distance of every element of the transfer function to the origin (0,0)
Frequency domain (Cont.)
Frequency domain (Cont.)
● One of the disadvantages of BLPF is when the order increased, one
could still feel the ringing effect.
● So, it start behaving same as ILPF and hence ringing effect is
unavoidable.
● In order to overcome this, Gaussian filter can be used. Here order
of the filter is not considered.
● But there is another new component to be included. It is standard
deviation. It is also useful to measure the spreadness of Gaussian
curve
● The transfer function for the Gaussian Low pass filter is:
Frequency domain (Cont.)
● Do specifies the cut off frequency as prompted by the user.
● D(u,v) is the euclidean distance from any point (u,v) to the origin of
the frequency plane. It is always important and in fact unavoidable
to understand the distance of every element of the transfer function
to the origin (0,0)
Frequency domain (Cont.)
Frequency domain (Cont.)
Frequency domain (Cont.)
● A High Pass filter is something when enables passing the high
frequencies while managing to suppress the low frequencies.
● It is also referred as sharpening filter as it does sharpen in the
frequency domain. There are multiple variants of high pass filter
available for the usage.
● They are similar to the low pass filter as Ideal High Pass Filter,
Butterworth High Pass Filter, and Gaussian High Pass Filter
Frequency domain (Cont.)
● Ideal High Pass Filter used for image sharpening process.
● Image sharpening is carried out in order to enhance the fine sharp
details in the image and to clearly highlight the edges in a digital
image.
● As the name suggest the IHPF clearly removes and suppress the low
frequency components while retaining the high frequency
components.
● Since that we have learnt the functioning of ILPH, one can expect
the reverse of it with IHPF
Frequency domain (Cont.)
● Do - Positive content. The filter retains all the frequency component
outside the radius Do
● The filter retains all the frequency components inside the radius Do.
High frequency shall be retained and low shall be suppressed.
● Do is the transition point between H(u,v) = 1 and H(u,v) = 0, and
hence become the cut-off frequency.
● Also D(u,v) is the Euclidean distance from any point (u,v) to the
origin of the frequency plane.
Frequency domain (Cont.)
● Butterworth High Pass Filter often regarded as BHPF.
● It is used for the image sharpening like any other high pass filter.
Again, with BHPF, one could preserve the high frequency
components while surpassing the low frequency ones.
Frequency domain (Cont.)
● Do is termed the Cut off frequency and it is always regarded as the
transition point between 1 and 0 (H(u,v) =1 and H(u,v) = 0). BHPF
works based on the Do value, It passes all the frequencies above
Do, whereas suppresses all the frequencies below Do and discard
values inside.
● n- will define order
● D(u,v) - is the euclidean distance from any point (u,v) to the origin
of the frequency plane.
Frequency domain (Cont.)
● Gaussian High Pass Filter has the same ideology as the Ideal and
Butterworth High pass filter.
● Here the transition are much smoother than the previous two. The
transfer function is:
● Do- specifies the cut off frequency as prompted by the user.
● D(u,v) is the euclidean distance from any point (u,v) to the origin of
the frequency plane.
Frequency domain (Cont.)
Frequency domain (Cont.)
Corner Detection
● Corner detection is very useful for shape description and matching.
● Corner is very important feature of an image, it indicates some
important points of the image which is known as Landmark points.
● Corner is formed at the boundary when two bright regions meet
where boundary curvature is very high.
● Corner point exhibits the characteristics of having large variations in
all directions, while edges have variation in only one direction.
● Corner points are also known as interest points.
● Corner are useful in many applications like tacking, measurement
systems, aligning and stereo imaging.
Corner Detection (Cont.)
How it works?
● Corner detection works on the principle that if you place a small
window over an image, if that window is placed on a corner then if it
is moved in any direction there will be a large change in intensity.
● If the window is over a flat area of the image then there will be
obviously no intensity change when the window moves. If the
window is over an edge there will be an intensity change if the
window moves in one direction. If the window is over the corner
then there will be a change in all directions and therefore we know
there must be a corner.
Corner Detection (Cont.)
Moravec Corner
● It is one of the earliest corner detection algorithms.
● It calculates the edge strength in all four directions.
● Minimum of four directional responses is selected and threshold
process is applied.
● Then non-maximal suppression algorithm is applied to extract a set of
corner points.

Limitation

Moravec corner detector is not rotationally invariant and it is

sensitive to even small variations along the edge.
Corner Detection (Cont.)
Harris Corner Detector
● It is very popular corner detection algorithm.
● It uses auto autocorrelation function to find out the differential of
corner score unit w.r.t direction directly.
● Advantage is corner detection irrespective of different sampling
quantization, small changes in scales and affine transformations
● A small square window is placed around each pixel and window is
shifted in all eight direction => Autocorrelation is performed.
● If no variation: Region is patch
● If large variation: pixel is marked as an interest point.
Corner Detection (Cont.)
Corner Detection (Cont.)
Corner Detection (Cont.)
Corner Detection (Cont.)
Corner Detection (Cont.)
Corner Detection (Cont.)
Hough Transform
● Hough transform is a feature extraction method for detecting simple
shapes such as circles, lines, etc. in an image.
● Hough transform takes the images created be edge detection
operators but most of the time, edge map is disconnected.
● Therefore Hough Transform is used to connect the disjoint edge
points.
Hough Transform (Cont.)
Hough Transform (Cont.)
Hough Transform (Cont.)
Thank You