1906704 - DIGITAL IMAGE

PROCESSING
 OBJECTIVES:
The student should be made:
• To understand the digital image fundamentals.
• To get exposed to simple image enhancement techniques in Spatial
and Frequency domain.
• To learn concepts of degradation function and restoration techniques.
• To study the image segmentation and representation techniques.
• To familiarize with image compression and recognition methods.
 OBJECTIVES:

• UNIT – I: DIGITAL IMAGE FUNDAMENTALS

• UNIT – II: IMAGE ENHANCEMENT

• UNIT – III: IMAGE RESTORATION

• UNIT – IV: IMAGE SEGMENTATION

• UNIT – V: IMAGE COMPRESSION AND RECOGNITION

 OUTCOMES:
On completion of the course, the student should be able to:
• Explain the basics and fundamentals of digital image processing, such as digitization,
sampling, quantization, and 2D-transforms.

• Apply various techniques of smoothing, sharpening and enhancement on images.

• Analyse the restoration concepts and filtering techniques.

• Explore the basics of segmentation, features extraction.

• Enumerate the compression and recognition methods.

TEXT BOOKS:
• Rafael C. Gonzalez, Richard E. Woods, “Digital Image Processing”, Pearson, Third
Edition, 2010.
• Anil K. Jain, “Fundamentals of Digital Image Processing”, Pearson, 2002.

REFERENCE BOOKS:

• Kenneth R. Castleman, “Digital Image Processing”, Pearson, 2006.

• Rafael C. Gonzalez, Richard E. Woods, Steven Eddins, “Digital Image Processing
using MATLAB”, Pearson Education, Inc., 2011.
• D,E. Dudgeon and RM. Mersereau, “Multidimensional Digital Signal Processing”,
Prentice Hall Professional Technical Reference, 1990.
• William K. Pratt, “Digital Image Processing”, John Wiley, New York, 2002
• Milan Sonkaetal “Image processing, analysis and machine vision”, Brookes/Cole,
Vikas Publishing House, 2nd edition, 1999.
• What is an Image?
Picture, photograph
Visual data
Usually two or three dimensional

What is a digital image?

An image which is “discretized,”, i.e., defined on a
discrete grid
Two-dimensional collection of light values (or gray values)
What is Digital Image Processing (DIP) ?

Image Processing
→ Image Analysis (or Image Understanding)
→ Computer Vision (goal: emulate human vision, branch of Artificial Intelligence ...)

3 types of computerized processes:

1. Low-level processes: reduction of noise, enhancement, sharpening
2. Mid-level processes: segmentation, description of objects, classification (recognition)
3. High-level processes: "making sense" of recognized objects => perform cognitive
functions
 What is Digital Image Processing (DIP) ?
Image = two-dimensional function f(x,y) where:
• x and y are spatial (plane) coordinates
• the amplitude of f at any (x,y) is called the intensity or gray level of the
image at that point.
x, y and intensity values all finite, discrete quantities => digital image
DIP: processing digital images by means of a computer

• Elements of the image: picture elements, image elements, pels, pixels

Examples of Fields that Use Digital Image
Processing
Examples of Fields that Use Digital Image
Processing
Gamma-Ray Imaging
Major use: nuclear medicine and astronomical observations
Imaging in the Visible and Infrared Bands
Fundamental Steps in Digital Image Processing
 Fundamental Steps in Digital Image Processing
Image Acquisition-
This is the first step or process of the fundamental steps of digital image processing. Image
acquisition could be as simple as being given an image that is already in digital form.
Generally, the image acquisition stage involves pre-processing, such as scaling etc.
Image Enhancement –
The aim of image enhancement is to improve the interpretability or
perception of information in images for human viewers, or to provide `better' input for other
automated image processing techniques. Such as, changing brightness & contrast etc
Image Restoration-
Image restoration is an area that also deals with improving the appearance of an image.
However, unlike enhancement, which is subjective, image restoration is objective, in the sense
that restoration techniques tend to be based on mathematical or probabilistic models of image
degradation
Color Image Processing-Color image processing is an area that has been gaining its
importance because of the significant increase in the use of digital images over the Internet.
This may include color modeling and processing in a digital domain etc.
 Fundamental Steps in Digital Image Processing

Wavelets and Multi-Resolution Processing-

Wavelets are the foundation for representing images in various degrees of resolution. Images
subdivision successively into smaller regions for data compression and for pyramidal
representation
Compression-Compression deals with techniques for reducing the storage required to
save an image or the bandwidth to transmit it. Particularly in the uses of internet it is very much
necessary to compress data.
Morphological Processing-
Morphological processing deals with tools for extracting image components that are useful in
the representation and description of shape
Segmentation
Segmentation procedures partition an image into its constituent parts or objects. In general,
autonomous segmentation is one of the most difficult tasks in digital image processing. A rugged
segmentation procedure brings the process a long way toward successful solution of imaging
problems that require objects to be identified individually
 Fundamental Steps in Digital Image Processing
Representation and Description-
Representation and description almost always follow the output of a segmentation stage,
which usually is raw pixel data, constituting either the boundary of a region or all the points in
the region itself. Choosing a representation is only part of the solution for transforming raw
data into a form suitable for subsequent computer processing. Description deals with
extracting attributes that result in some quantitative information of interest or are basic for
differentiating one class of objects from another.
Object recognition-
Recognition is the process that assigns a label, such as, “vehicle” to an object based on its
descriptors
Knowledge Base
Knowledge may be as simple as detailing regions of an image where the information of interest
is known to be located, thus limiting the search that has to be conducted in seeking that
information. The knowledge base also can be quite complex, such as an interrelated list of all
major possible defects in a materials inspection problem or an image database containing high-
resolution satellite images of a region in connection with change detection applications
Components of an Image Processing System

• Short-term storage (during

processing).
E.g.: RAM, frame buffers
• On-line storage (fast recall).
E.g. disks
• Archival storage (infrequent access).
E.g. magnetic tapes, optical disks
 Components of an Image Processing System
Image Processing System is the combination of the different elements involved in the digital
image processing. Digital image processing is the processing of an image by means of a digital
computer. Digital image processing uses different computer algorithms to perform image
processing on the digital images.
•Image Sensors:
Image sensors senses the intensity, amplitude, co-ordinates and other features of the images and
passes the result to the image processing hardware. It includes the problem domain.
Two elements are required to acquire digital images.
The first is a physical device that is sensitive to the energy radiated by the object we wish to
image. The second, called a digitizer, is a device for converting the output of the physical sensing
device into digital form
Image Processing Hardware:
Specialized image processing hardware usually consists of the digitizer just mentioned, plus
hardware that performs other primitive operations, such as an arithmetic logic unit (ALU), that
performs arithmetic and logical operations in parallel on entire images It passes the result to
general purpose computer.
•Computer:
Computer used in the image processing system is the general purpose computer that is used by
us in our daily life.
•Image Processing Software:
Image processing software is the software that includes all the mechanisms and algorithms that
are used in image processing system.
Mass Storage:
Mass storage capability is a must in image processing applications. An image of size pixels, in
which the intensity of each pixel is an 8-bit quantity, requires one megabyte of storage space if
the image is not compressed.
When dealing with thousands, or even millions, of images, providing adequate storage in an
image processing system can be a challenge.
Digital storage for image processing applications falls into three principal categories:
(1) short-term storage for use during processing, (2) on-line storage for relatively
fast recall, and (3) archival storage, characterized by infrequent access.
Hard Copy Device:
Hardcopy devices for recording images include laser printers, film cameras, heat-sensitive
devices, inkjet units, and digital units, such as optical and CDROM disks. Film provides the
highest possible resolution, but paper is the obvious medium of choice for written material.
•Image Display:
It includes the monitor or display screen that displays the processed images.
Network:
Because of the large amount of data inherent in image processing applications, the key
consideration in image transmission is bandwidth. In dedicated networks, this typically is not a
problem, but communications with remote sites via the Internet are not always as efficient.
Fortunately, this situation is improving quickly as a result of optical fiber and other broadband
technologies
Elements of Visual Perception
• IP: foundation of mathematic and probabilistic formulations
• Human intuition and analysis play a central role
• Choices made based on subjective, visual judgments
• Physical limitations of human vision
• e.g: How human and electronic imaging devices compare in terms of
resolution and ability to adapt to changes in illumination ?
Structure
of the
Human
Eye
 The eye ball is approximately spherical. The vertical measure of an eye ball is
approximately 24 mm and is slightly less than the horizontal width. The eye
The eye ball offers a field of view covering 160o(width) × 135o height area. The anterior of
the eye has the outer coating cornea while the posterior has the outer layer of
sclera.
The cornea is a transparent, curved, refractive window through which the light
enters the eye. The corneal segment (typically 8 mm in radius) is linked to the
Cornea
larger unit, the sclera, which extends and covers the posterior portion of the
optic globe. The cornea and sclera are connected by a ring called the limbus.

The light entering the cornea is blocked by the visible colored and opaque
Iris
surface of the iris. The back of the iris is coated with a black pigment.

The pupil is the opening at the center of the iris. The pupil controls the amount
of light entering the eye ball. Its diameter varies from 1 to 8 mm in response to
Pupil
illumination changes. In low light conditions it dilates to increase the amount of
light reaching the retina. Behind the pupil is the lens of the eye.
 The lens is suspended to the ciliary body by the suspensory ligament
(Zonule of Zinn), made up of fine transparent fibers. The lens is
Lens transparent (has 70% water) and absorbs approximately 8% of the visible
light spectrum. The protein in the lens absorbs the harmful infrared and
ultraviolet light and prevents damage to the eye.

Beneath the sclera is a membrane called choroid. It contains blood vessels

Choroid to nourish the cells in the eye. Like the iris, it is pigmented to prevent light
from entering the eye from any other direction other than the pupil.

Beneath the choroid lies the retina, the innermost membrane of the eye
where the light entering the eye is sensed by the receptor cells. The retina
Retina
has 2 types of photoreceptor cells − rods and cones. These receptor cells
respond to light in the 330 to 730 nm wavelength range.

The central portion of the retina at the posterior part is the fovea. It is
Fovea
about 1.5 mm in diameter.
 There about 100 million rods in the eye. The rods help in the dim-light
(scotopic) vision. Their spatial distribution is radially symmetric about the
fovea, but varies across the retina. They are distributed over a larger area
Rods in the retina. The rods are extremely sensitive and can respond even to a
single photon. However they are not involved in color vision. They cannot
resolve fine spatial detail despite high number because many rods are
connected to a single nerve.
There are about 6 million cones in the eye. The cones help in the bright-
light (photopic) vision. These are highly sensitive to color. They are located
Cones primarily in the fovea where the image is focused by the lens. Each cone
cell is connected to its separate nerve ending. Hence they have the ability
to resolve fine details.
Though the photo-receptors are distributed in radially symmetric manner
about the fovea, there is a region near the fovea where there are no
Blind spot receptors. This region is called as the blind spot. This is the region where
the optic nerve emerges from the eye. Light falling on this region cannot be
sensed.
Distribution of discrete
light receptors over the
surface of the retina
2 classes of receptors:
cones and rods :
• Cones: 6-7 million in each
eye, mainly located in the
fovea. Highly sensitive to
colour, fine details.
“Photopic” or bright-light
vision
• Rods: 75-150 million,
distributed. Sensitive
to low level of illumination,
not involved in
colour vision. “Scotopic” or
dim-light vision
Photo camera: lens has fixed focal length. Focusing at various distances by varying distance
between lens and imaging plane (location of film or chip)
Human eye: converse. Distance lens-imaging region (retina) is fixed. Focal length for proper
focus obtained by varying the shape of the lens.
 Brightness Adaptation and Discrimination
• Eye's ability to discriminate between different
intensity levels
• Range of light intensity levels to which the
human visual system can adapt: on the order of
1010
 Mach bands
• The visual system tends to
undershoot or overshoot
around the boundary of
regions of different
intensities
• Simultaneous contrast phenomenon: a region’s perceived brightness
does not depend simply on its intensity.
Optical illusions:
The eye fills in non-
existing info or
wrongly perceives
geometrical
properties of objects
 A Simple Image Formation Model
• Images denoted by two-dimensional functions f(x,y)
• Value of amplitude of f at (x,y): positive scalar quantity
• Image generated by physical process: intensity values are proportional to
the energy radiated by a physical source

f(x,y) may be characterized by 2 components:

(1) The amount of source illumination incident on the scene: illumination i(x,y)
(2) The amount of illumination reflected by the objects of the scene: reflectance
r(x,y)
A Simple Image Formation Model
A Simple Image Formation Model
 Image Sensing and Acquisition
• Transform of illumination energy into digital images:
• The incoming energy is transformed into a voltage by the combination
of input electrical power and sensor material.
• Output voltage waveform = response of the sensor(s)
• A digital quantity is obtained from each sensor by digitizing its
response.
Image Sensing and Acquisition
 Image Sensing and Acquisition
Image acquisition using a single sensor
 Image Sensing and Acquisition
Image acquisition using sensor strips
 Image Sensing and Acquisition
Image acquisition using sensor arrays
 Image Sensing and Acquisition
Image acquisition using sensor arrays

CCD cameras: widely used in

modern applications: private
consumers, industry,
astronomy…
CCD: Charge Couple Device

Rectangular grid of electron-

collection sites laid over a thin
silicon wafer

Image readout of the CCD one

row at a time, each row
transferred in parallel to a
serial output register
 Image Sensing and Acquisition
Image acquisition using sensor arrays

Alternative to CCD cameras:

CMOS technology
CMOS: Complementary Metal-
Oxyde-Semiconductor

CMOS can potentially be

implemented with fewer
components, use less power
and provide data faster than
CCDs
CCD: more mature technology
NB: a CMOS-based camera can
be significantly smaller than a
comparable CCD
camera
 Image Sampling and Quantization
• Digitizing the coordinate values = Sampling
• Digitizing the amplitude values = Quantization
 Image Sampling and Quantization
Basic Concepts in Sampling and Quantization
Method of sampling determined by the sensorarrangement:
• Single sensing element combined with motion:
spatial sampling based on number of individual mechanical increments
• Sensing strip:
the number of sensors in the strip establishes the sampling limitations in one
image direction; in the other: same value taken in practice
• Sensing array:
the number of sensors in the array establishes the limits of sampling in both
directions
• The quality of a digital image is determined to a large degree by the number of
samples and discrete intensity levels used in sampling and quantization.
• However image content is also an important consideration in choosing these
parameters
 Image Sampling and Quantization

(a) Continuous image projected onto a sensor array. (b) Result of image sampling and quantization.
 Image Sampling and Quantization
Representing Digital Images
Continuous image: function of 2 continuous variables f(s,t)
→ digital image by sampling and quantization
→ 2D array f(x,y), M rows and N columns, (x,y) = discrete coordinates
x = 0, 1, 2,…, M-1 and y = 0, 1, 2…, N-1
Section of the real plane spanned by the coordinates of an image = spatial domain x and y
are called spatial variables or spatial coordinates

• The digitization process requires decisions on the values of M, N and L (number of

discrete intensity levels)
• No (theoretical) restrictions on M and N other than: M > 0 and N > 0
• Due to storage and hardware, typically: L = 2k
• Assume that discrete levels are equally spaces and integers in [0,L-1]
 Image Sampling and Quantization
Representing Digital Images

(a) Image plotted as a surface.

(b) Image displayed as a
visual intensity array.
(c) Image shown as a 2-D
numerical array (0, .5, and 1
represent black, gray, and white,
respectively).
 Image Sampling and Quantization
Representing Digital Images
Image Sampling and Quantization
 Image Sampling and Quantization
Spatial and Intensity Resolution
• Dynamic range = ratio of maximum measurable intensity to minimum detectable intensity
level in the system
• Rule: upper limit determined by saturation, lower limit determined by noise
• Contrast = difference in intensity between the highest and the lowest intensity levels in an
image
• High dynamic range => high contrast expected
• Low dynamic range => dull, washed-out gray look

spatial resolution is a measure of the smallest

discernible detail in an image.
Quantitatively, spatial resolution can be stated in a
number of ways, with line pairs per unit distance, and
dots (pixels) per unit distance being among the most
common measures.
 Image Sampling and Quantization
• Image Interpolation
• Interpolation is a basic tool used extensively in tasks such as zooming,
shrinking, rotating, and geometric corrections. which are basically image
resampling methods.
• interpolation is the process of using known data to estimate values at
unknown locations
• nearest neighbor interpolation -Intensity level assignment for one pixel look for its
closest pixel in the original image and assign its intensity
• bilinear interpolation-use the 4 nearest neighbours to estimate the intensity at a given
location (x,y):

• bicubic interpolation-use the 16 nearest neighbours of a point:

Image Sampling and Quantization
Image Sampling and Quantization
Image Sampling and Quantization
Image Sampling and Quantization
 Image Sampling and Quantization

Reconstruction of the Image from Its Samples

 Some Basic Relationships between Pixels
Given an image f(x,y) and pixels p or q
Some Basic Relationships between Pixels
Some Basic Relationships between Pixels
 Some Basic Relationships between Pixels
Distance Measures
 Color Image Fundamentals

Motivation to use colour:

•Powerful descriptor that often simplifies object identification and
extraction from a scene
•Humans can discern thousands of colour shades and intensities,
compared to about only two dozen shades of gray Two major areas:
•Full-colour processing: e.g. images acquired by colour TV camera or
colour scanner
•Pseudo-colour processing: assigning a colour to a particular monochrome
intensity or range of intensities Some of the gray-scale methods are
directly applicable to colour images
Color Image Fundamentals

Color
spectrum seen by
passing white
light through a
prism.
 Color Image Fundamentals
• Perception of colours by the human eye
• Cones can be divided into 3 principal sensing categories: (roughly) red, green and blue ~65% are
sensitive to red light, ~33% to green light and ~2% to blue (but most sensitive)
• Colours are seen as variable combinations of the primary colours: Red, Green, Blue
• From CIE* (1931), wavelengths: blue = 435.8nm, green = 546.1nm, red = 700nm

Absorption of
light by the red,
green, and blue
cones in the
human eye as a
function of
wavelength.
Color Image Fundamentals
 Color Image Fundamentals
Three basic quantities are used to describe the quality of a chromatic light source: radiance, luminance, and
brightness.
Radiance:
Radiance is the total amount of energy that flows from the light source, and it is usually measured in watts
(W).
Luminance:
Luminance, measured in lumens (lm), gives a measure of the amount of energy an observer perceives from
a light source.
Brightness:
Brightness is a subjective descriptor that is practically impossible to measure. It embodies the achromatic
notion of intensity and is one of the key factors in describing color sensation

Characteristics of a colour:
• Brightness: embodies the achromatic notion of intensity
• Hue: attribute associated with the dominant wavelength in a mixture of light waves
• Saturation: refers to the relative purity or the amount of white light mixed with a hue
Hue and Saturation together = chromaticity
Colour may be characterized by its brightness and chromaticity
Color Image Fundamentals
Color Image Fundamentals
Color Image Fundamentals
 Color Image Fundamentals
Colour Models
• Also called: colour spaces or colour systems
• Purpose: facilitate the specification of colours in some “standard” way
• Colour model = specification of a coordinate system and a subspace within it where each
colour is represented by a single point.

Most commonly used hardware-oriented models:

• RGB (Red, Green, Blue), for colour monitors and video cameras
• CMY (Cyan, Magenta, Yellow) and CMYK (CMY+Black) for colour printing
• HSI (Hue, Saturation, Intensity)
 Color Image Fundamentals
The RGB Colour Model
• Each colour appears in its primary
spectral components of Red, Green and
Blue
• Model based on a Cartesian coordinate
System
• Colour subspace = cube
• RGB primary values: at 3 opposite
corners (+ secondary values at 3 others)
• Black at the origin, White at the opposite
corner

Schematic of the RGB color cube. Points along the

main diagonal have gray values, from black at the
origin to white at point (1, 1, 1).
 RGB Colour Model
Number of bits used to represent each pixel in
the RGB space = pixel depth
Example: RGB image in which each of the red,
green and blue images is a 8-bit image
Each RGB colour pixel (i.e. triplet of values
(R,G,B)) is said to have a depth of 24 bits (full-
colour image)

Total number of colours in a 24-bit RGB image

is: (28)3 = 16,777,276
RGB Colour Model
While high-end display cards and monitors provide a
reasonable rendition of the colors in a 24-bit RGB image,
many systems in use today are limited to 256 colors.
Given the variety of systems in current use, it is of
considerable interest to have a subset of colors that are likely
to be reproduced faithfully, reasonably independently of
viewer hardware capabilities.
This subset of colors is called the set of safe RGB colors, or the
set of all-systems-safe colors. In Internet applications, they are
called safe Web colors or safe browser colors.
On the assumption that 256 colors is the minimum number of
colors that can be reproduced faithfully by any system in which
a desired result is likely to be displayed
Forty of these 256 colors are known to be processed differently
by various operating systems, leaving only 216 colors that are
common to most systems. These 216 colors have become the
de facto standard for safe colors, especially in Internet
applications.
 HSI Colour Model
RGB and CMY models: straightforward + ideally suited for hardware implementations + RGB
system matches nicely the human eye perceptive abilities
But, RGB and CMY not well suited for describing colours in terms practical for human
interpretation.

Human view of a colour object described by Hue, Saturation and Brightness (or Intensity)
• Hue: describes a pure colour (pure yellow, orange or red)
• Saturation: gives a measure of the degree to which a pure colour is diluted by white light
• Brightness: subjective descriptor practically impossible to measure. Embodies the
achromatic notion of intensity => intensity (gray level), measurable

=> HSI (Hue, Saturation, Intensity) colour model

 HSI Colour Model
Intensity is along the line joining
white and black in the RGB cube
• To determine the intensity
component of any colour point:
pass a plane perpendicular to the
intensity axis and containing the
colour point. Intersection of the
plane with the axis is the
normalized intensity value
• Saturation (purity) of a colour
increases as a function of
distance from the intensity axis
Conceptual relationships between the RGB and HSI
(on the axis, saturation = 0, gray color models.
points)
 HSI Colour Model

Hue and saturation in the HSI color model. The dot is an arbitrary color point. The angle from the red axis gives
the hue, and the length of the vector is the saturation. The intensity of all colors in any of these planes is given
by the position of the plane on the vertical intensity axis.
 HSI Colour Model

HSI color model based on

(a) triangular and (b) circular color planes.
The triangles and circles are perpendicular to the vertical intensity axis.
Conversion from RGB to HSI
Conversion from HSI to RGB
 Conversion from HSI to RGB

a b

HSI components of the RGB 24-bit colour cube image (a) Hue, (b) saturation, and (c) intensity images