Digital Image Processing

Handouts

Dr. Umair Ali Khan

Associate Professor,
Computer Systems Engineering Department,
Quaid-e-Awam University of Engineering, Science & Technology,
NawabShah, Sindh, Pakistan.
Ph. # +92-244-9370381(ext. 2640), +92-333-7024006
Email: [email protected]
Website: http://umair-han.quest.edu.pk/
 1 Introduction

1. What Is Digital Image Processing?

An image may be defined as a two-dimensional function, f(x, y), where x and y are spatial (plane)
coordinates, and the amplitude of f at any pair of coordinates (x, y) is called the intensity or gray level of
the image at that point. When x, y, and the amplitude values of f are all finite, discrete quantities, we call
the image a digital image. The field of digital image processing refers to processing digital images by
means of a digital computer. A digital image is composed of a finite number of elements, each of which
has a particular location and value. These elements are referred to as picture elements, image elements,
pels, and pixels. Pixel is the term most widely used to denote the elements of a digital image.

Figure 1: Representation of an image

2. Examples of Digital Images

1. Natural landscape
2. Synthetically generated scene
3. Poster graphic
 4. Computer screenshot
5. Black and white illustration
6. Barcode
7. Fingerprint
8. X‐ray
9. Microscope slide
10. Satellite Image
11. Radar image
12. Astronomical object

3. A Simple Image Formation Model

When an image is generated from a physical process, its values are proportional to energy radiated by a
physical source (e.g., electromagnetic waves). As a result, f(x, y) must be nonzero and finite; that is,

0 < f(x, y) < q

The function f(x, y) may be characterized by two components: (1) the amount of source illumination
incident on the scene being viewed, and (2) the amount of illumination reflected by the objects in the
scene. These are called the illumination and reflectance components and are denoted by i(x, y) and r(x, y),
respectively. The two functions combine as a product to form f(x, y):

f(x, y) = i(x, y)r(x, y)

where

0 < i(x, y) < q

and

0 < r(x, y) < 1

The nature of i(x, y) is determined by the illumination source, and r(x, y) is determined by the
characteristics of the imaged objects. Figure 2 shows a simple image formation model.

Figure 2: A simple image formation model

In Figure 1, the light incident on a scene element from a light source is captured by the imaging system (a
camera). The imaging system focuses the image on an internal image plane. The digitization process in
the imaging system converts the image into a digital form.

4. Example Image Processing Operations

There are many image processing operations performed for the analysis and manipulation of digital
images. Some of them are described as follows.
4.1. Noise removal
Images taken with both digital cameras and conventional film cameras will pick up noise from a variety
of sources. Further use of these images will often require that the noise be (partially) removed -
for aesthetic purposes as in artistic work or marketing, or for practical purposes such as computer vision.

Figure 3: Image noise removal

4.2. Contrast Enhancement
Contrast is an important factor in any subjective evaluation of image quality. Image enhancement
techniques have been widely used in many applications of image processing where the subjective quality
of images is important for human interpretation.

Figure 4: Image contrast enhancement

4.3. Edge Detection
Edge detection is the name for a set of mathematical methods which aim at identifying points in a digital
image at which the image brightness changes sharply or, more formally, has discontinuities. Edge
detection is usually performed for feature detection and feature extraction for object detection and
recognition.
 Figure 5: Example of edge detection
4.4. Image Segmentation
Image segmentation is the process of partitioning a digital image into multiple segments and extracting
those segments for further analysis. An example of image segmentation includes extracting a certain part
in a Medical Resonance Image (MRI) for the detection of a disease.

Figure 6: Example of image segmentation

4.5. Image Compression
The objective of image compression is to reduce irrelevance and redundancy of the image data in order to
reduce the size of images and be able to store or transmit them in an efficient form.

Figure 7: Example of image compression

4.6. Image Inpainting
Inpainting is the process of reconstructing lost or deteriorated parts of images and videos. It is used
for film restoration; to reverse the deterioration (e.g., cracks in photographs or scratches and dust spots in
film; see infrared cleaning). It is also used for removing red-eye, the stamped date from photographs and
removing objects to creative effect.
 Figure 8: Example of image inpainting
4.7. Face Recognition
Face recognition is used for identifying or verifying a person from a digital image or a video framefrom
a video source. One of the ways to do this is by comparing selected facial features from the image and a
facial database.

Figure 9: Face recognition based on facial features

4.8. Gesture Recognition
Gesture recognition is used for interpreting human gestures via mathematical algorithms. Gestures can
originate from any bodily motion or state but commonly originate from the face or hand.

Figure 10: Example of gesture recognition

4.9. Object Detection
Object detection deals with detecting instances of semantic objects of a certain class (such as humans,
buildings, or cars) in digital images and videos. Well-researched domains of object detection include face
detection and pedestrian detection. Object detection has applications in many areas of computer vision,
including image retrieval and video surveillance.

Figure 11: Example of object detection

4.10. Object Tracking
Object tracking is the process of locating a moving object (or multiple objects) over time using a camera.
It has a variety of uses, some of which are: human-computer interaction, security and surveillance, video
communication and compression, augmented reality, traffic control, medical imaging and video editing.

Figure 12: Example of object tracking

5. Key Stages in Digital Image Processing
Digital image processing comprises following key stages.
5.1. Image Acquisition
Image acquisition refers to providing an image to the image processing system for analysis. It could be as
simple as being given an image that is already in digital form. Generally, the image acquisition stage
involves preprocessing, such as scaling.
5.2. Image Enhancement
The idea behind enhancement techniques is to bring out detail that is hidden, or simply to highlight
certain features of interest in an image. A familiar example of enhancement is when we increase the
contrast of an image because it looks better. Image enhancement is a very subjective area of image
processing.
 Figure 13: Key stages of digital image processing
5.3. Image Restoration
Image restoration is an area that also deals with improving the appearance of an image. However, unlike
enhancement, which is subjective, image restoration is objective, in the sense that restoration techniques
tend to be based on mathematical or probabilistic models of image degradation. Enhancement, on the
other hand, is based on human subjective preferences regarding what constitutes a “good” enhancement
result.
5.4. Morphological Processing
Morphological processing deals with tools for extracting image components that are useful in the
representation and description of shape.
5.5. Segmentation
Segmentation procedures partition an image into its constituent parts or objects. In general, autonomous
segmentation is one of the most difficult tasks in digital image processing. A rugged segmentation
procedure brings the process a long way toward successful solution of imaging problems that require
objects to be identified individually. On the other hand, weak or erratic segmentation algorithms almost
always guarantee eventual failure. In general, the more ac-curate the segmentation, the more likely
recognition is to succeed.
5.6. Representation & Description
After segmentation, converting the data to a form suitable for computer processing is necessary. The first
decision that must be made is whether the data should be represented as a boundary or as a complete
region. Boundary representation is appropriate when the focus is on external shape characteristics, such as
corners. Regional representation is appropriate when the focus is on internal properties, such as texture or
skeletal shape. A method must also be specified for describing the data so that features of interest are
highlighted. Description, also called feature selection, deals with extracting attributes that result in some
quantitative information of interest or are basic for differentiating one class of objects from another.
5.7. Object Recognition
Recognition is the process that assigns a label (e.g., “vehicle”) to an object based on its descriptors.
5.8. Compression
Compression deals with techniques for reducing the storage required to save an image, or the bandwidth
required to transmit it. Although storage technology has improved significantly over the past decade, the
same cannot be said for transmission capacity. This is true particularly in uses of the Internet, which are
characterized by significant pictorial content.
5.9. Color Image Processing
Color image processing is an area that has been gaining in importance because of the significant increase
in the use of digital images over the Internet. Color is also used as the basis for extracting features of
interest in an image.
6. Components of Image Processing System
Digital image processing comprises the following basic components.

Figure 14: Basic components of a digital image processing system

Image sensors basically comprise two elements to acquire digital images. The first is a physical device
that is sensitive to the energy radiated by the object we wish to image. The second, called a digitizer, is a
device for converting the output of the physical sensing device into digital form. For instance, in a digital
video camera, the sensors produce an electrical output proportional to light intensity. The digitizer
converts these outputs to digital data.
Specialized image processing hardware usually consists of the digitizer, plus hardware that performs
other operations, such as an arithmetic logic unit (ALU), which performs arithmetic and logical
operations in parallel on entire images. One example of how an ALU is used is in averaging images as
quickly as they are digitized, for the purpose of noise reduction.
The computer in an image processing system is a general-purpose computer and can range from a PC to
a supercomputer. In a general-purpose image processing system, almost any well-equipped PC-type
machine is suitable for off-line image processing tasks.
Image processing software consists of specialized modules that perform specific tasks. A well-designed
package also includes the capability for the user to write code that utilizes the specialized modules.
Mass storage capability is a must in image processing applications. An image of size 1024 x1024 pixels,
in which the intensity of each pixel is an 8-bit, requires one megabyte of storage space if the image is not
compressed. Hence, providing adequate mass storage in a digital image processing system is necessary.
Image displays in use today are mainly color TV monitors. Monitors are driven by the outputs of image
and graphics display cards that are an integral part of the computer system. In some cases, it is necessary
to have stereo displays, and these are implemented in the form of headgear containing two small displays
embedded in goggles worn by the user.
Hardcopy devices for recording images include laser printers, film cam-eras, heat-sensitive devices,
inkjet units, and digital units, such as optical and CD-ROM disks.
Networking component is required for transmission of data to remote sites. Most of the general purpose
image processing systems nowadays are equipped with networking components.

7. Elements of Visual Perception

Although the digital image processing field is built on a foundation of mathematical and probabilistic
formulations, human intuition and analysis play a central role in the choice of one technique versus
another, and this choice often is made based on subjective, visual judgments. Hence, developing a basic
understanding of human visual perception is appropriate.
7.1. Structure of the Human Eye
Figure 15 shows a simplified horizontal cross section of the human eye. The eye is nearly a sphere, with
an average diameter of approximately 20 mm. Three membranes enclose the eye: the cornea and sclera
outer cover; the choroid; and the retina. The cornea is a tough, transparent tissue that covers the anterior
surface of the eye. Continuous with the cornea, the sclera is an opaque membrane that encloses the
remainder of the optic globe.
The lens is made up of concentric layers of fibrous cells and is suspended by fibers that attach to the
ciliary body. The innermost membrane of the eye is the retina. When the eye is properly focused, light
from an object outside the eye is imaged on the retina. Pattern vision is afforded by the distribution of
discrete light receptors over the surface of the retina. There are two classes of receptors: cones and rods.
The cones in each eye number between 6 and 7 million. They are located primarily in the central portion
of the retina, called the fovea, and are highly sensitive to color. Humans can resolve fine de-tails with
these cones largely because each one is connected to its own nerve end. Muscles controlling the eye rotate
the eyeball until the image of an object of interest falls on the fovea.
The number of rods is much larger: Some 75 to 150 million are distributed over the retinal surface. Rods
serve to give a general, overall picture of the field of view. They are not involved in color vision and are
sensitive to low levels of illumination. For example, objects that appear brightly colored in day-light
when seen by moonlight appear as colorless forms because only the rods are stimulated.
 Figure 15: Simplified diagram of a cross section of the human eye

7.2. Image Formation on the Eye

The distance between the center of the lens and the retina (called the focal length) varies from
approximately 17 mm to about 14 mm, as the refractive power of the lens increases from its minimum to
its maximum. When the eye focuses on an object farther away than about 3 m, the lens exhibits its lowest
refractive power. When the eye focuses on a nearby object, the lens is most strongly refractive. This
information makes it easy to calculate the size of the retinal image of any object. In Fig. 16, for example,
the observer is looking at a tree 15 m high at a distance of 100 m. If h is the height in mm of that object in
the retinal image, the geometry of Fig. 16 yields 15/100=h/17 or h=2.55 mm. The retinal image is
reflected primarily in the area of the fovea. Perception then takes place by the relative excitation of light
receptors, which transform radiant energy into electrical impulses that are ultimately decoded by the
brain.

Figure 16: Graphical representation of the eye looking at a palm tree.

8. Light and the Electromagnetic Spectrum
Electromagnetic waves can be conceptualized as propagating sinusoidal waves of varying wavelengths, or
they can be thought of as a stream of massless particles, each traveling in a wavelike pattern and moving
at the speed of light. Each massless particle contains a certain amount (or bundle) of energy. Each bundle
of energy is called a photon. If spectral bands are grouped according to energy per photon, we obtain the
spectrum shown in Figure 17, ranging from gamma rays (highest energy) at one end to radio waves
(lowest energy) at the other.

Figure 17: The electromagnetic spectrum arranged according to energy per photon
8.1. Gamma-Ray Imaging
Major uses of imaging based on gamma rays include nuclear medicine and astronomical observations. In
nuclear medicine, the approach is to inject a patient with a radioactive isotope that emits gamma rays as it
decays. Images are produced from the emissions collected by gamma ray detectors.
8.2. X-Ray Imaging
X-rays are among the oldest sources of EM radiation used for imaging. The best known use of X-rays is
medical diagnostics, but they also are used extensively in industry and other areas, like astronomy. X-rays
for medical and industrial imaging are generated using an X-ray tube, which is a vacuum tube with a
cathode and anode.
8.3. Ultraviolet Imaging
Ultraviolet imaging includes lithography, industrial inspection, microscopy, lasers, biological imaging,
and astronomical observations. Ultraviolet light is used in fluorescence microscopy, one of the fastest
growing areas of microscopy.
8.4. Visible and Infrared Imaging
The infrared band often is used in conjunction with visual imaging. The examples range from
pharmaceuticals and micro inspection to materials characterization. Another major area of visual
processing is remote sensing, which usually includes several bands in the visual and infrared regions of
the spectrum.
8.5. Microwave Imaging
The dominant application of imaging in the microwave band is radar. The unique feature of imaging radar
is its ability to collect data over virtually any region at any time, regardless of weather or ambient lighting
conditions. Some radar waves can penetrate clouds, and under certain conditions can also see through
vegetation, ice, and extremely dry sand. In many cases, radar is the only way to explore inaccessible
regions of the Earth’s surface.
8.6. Radio Imaging
The major applications of imaging in the radio band are in medicine and astronomy. In medicine radio
waves are used in magnetic resonance imaging (MRI). This technique places a patient in a powerful
magnet and passes radio waves through his or her body in short pulses. Each pulse causes a responding
pulse of radio waves to be emitted by the patient’s tissues. The location from which these signals
originate and their strength are determined by a computer, which produces a two-dimensional picture of a
section of the patient.
9. Image Sensing and Acquisition
An image sensor or imaging sensor is a sensor that detects and conveys the information that constitutes
an image. It does so by converting light waves (as they pass through or reflect off objects) into signals,
the small bursts of current that convey the information. The waves can be light or other electromagnetic
radiation. Image sensors are used in electronic imaging devices of both analog and digital types, which
include digital cameras, camera modules, medical imaging equipment, night vision equipment such
as thermal imaging devices, radar, sonar, and others.
Figure 18 shows the single imaging sensor used to transform illumination energy into digital images.
Incoming energy is transformed into a voltage by the combination of input electrical power and sensor
material that is responsive to the particular type of energy being detected. The output voltage waveform is
the response of the sensor(s), and a digital quantity is obtained from each sensor by digitizing its
response.

Figure 18: Single Imaging Sensor

9.1. Types of Image Sensors
Image sensors have two types.
9.1.1. CCD (Charged Coupled Devices) Sensor
CCD sensors are silicon-based integrated circuits consisting of a dense matrix of photodiodes that operate
by converting light energy in the form of photons into an electronic charge. Electrons generated by the
interaction of photons with silicon atoms are stored in a potential well and can subsequently be transferred
across the chip through registers and output to an amplifier.
In a CCD for capturing images, there is a photoactive region, and a transmission region made out of a
shift register. An image is projected by a lens on the capacitor array (the photoactive region), causing
each capacitor to accumulate an electric charge proportional to the light intensity at that location. Once
the array has been exposed to the image, a control circuit causes each capacitor to transfer its contents to
its neighbor. The last capacitor in the array dumps its charge into a charge amplifier, which converts the
charge into a voltage. By repeating this process, the controlling circuit converts the entire semiconductor
contents of the array to a sequence of voltages, which it samples, digitizes and stores in some form of
memory. Figure 19 shows the working principle of a CCD sensor.
9.1.2. CMOS (Complementary Metal Oxide Semiconductor) Sensor
In CMOS sensors, there are several transistors at each pixel that amplify and move the charge using
wires. The CMOS approach is more flexible because each pixel can be read individually.
 Figure 19: Working principle of CCD sensor
Besides, each pixel has its own charge-to-voltage conversion, and the sensor often also includes
amplifiers, noise-correction, and digitization circuits, so that the chip outputs digital bits. Figure 20 shows
the working principle of a CMOS sensor.

Figure 20: Working principle of a CMOS sensor

9.1.3. CMOS vs. CCD sensor
CMOS and CCD cameras have following major differences.
- CCD sensors create high-quality, low-noise images. CMOS sensors generate more noise.
- Because each pixel on a CMOS sensor has several transistors located next to it, the light
sensitivity of a CMOS chip is lower. Many of the photons hitting the chip hit the transistors
instead of the photodiodes.
- CMOS sensors consume little power.
- CMOS chips can be fabricated on just about any standard silicon production line, so they tend to
be inexpensive compared to CCD sensors.
- CCD sensors are produced for a longer period of time, so they are more mature. They generally
have higher quality and more pixels.
Table 1 summarizes the main differences between CMOS and CCD sensors.

Table 1: CMOS vs. CCD sensor

10. Image Sampling and Quantization
The output of most sensors is a continuous voltage waveform whose amplitude and spatial behavior are
related to the physical phenomenon being sensed. To create a digital image, we need to convert the
continuous sensed data into digital form. This involves two processes: sampling and quantization.
Figure 21(a) shows a continuous image, f(x, y), that we want to convert to digital form. An image may be
continuous with respect to the x- and y-coordinates, and also in amplitude. To convert it to digital form,
we have to sample the function in both coordinates and in amplitude. Digitizing the coordinate values is
called sampling. Digitizing the amplitude values is called quantization.
The one-dimensional function shown in Fig. 21(b) is a plot of amplitude (gray level) values of the
continuous image along the line segment AB in Fig. 21(a). The random variations are due to image noise.
To sample this function, we take equally spaced samples along line AB, as shown in Fig. 21(c). The
location of each sample is given by a vertical tick mark in the bottom part of the figure. The samples are
shown as small white squares superimposed on the function. The set of these discrete locations gives the
sampled function. However, the values of the samples still span (vertically) a continuous range of gray-
level values. In order to form a digital function, the gray-level values also must be converted (quantized)
into discrete quantities. The right side of Fig. 21(c) shows the gray-level scale divided into eight discrete
levels, ranging from black to white. The vertical tick marks indicate the specific value assigned to each of
the eight gray levels. The continuous gray levels are quantized simply by assigning one of the eight
discrete gray levels to each sample. The assignment is made depending on the vertical proximity of a
sample to a vertical tick mark. The digital samples resulting from both sampling and quantization are
shown in Fig. 21(d). Starting at the top of the image and carrying out this procedure line by line produces
a two-dimensional digital image.

Figure 21: Generating a digital image. (a) Continuous image. (b) A scan line from A to B in the continuous image,
used to illustrate the concepts of sampling and quantization. (c) Sampling and quantization. (d) Digital scan line.
11. Spatial Resolution
Spatial resolution is a term that refers to the number of pixels utilized in construction of a digital image.
Images having higher spatial resolution are composed with a greater number of pixels than those of lower
spatial resolution. The detail discernible in an image is dependent on the spatial resolution of the sensor
and refers to the size of the smallest possible feature that can be detected. Figure 22 shows an image with
different spatial resolutions.
12. Gray Level Resolution
In short gray level resolution is equal to the number of bits per pixel. Gray level resolution refers to the
predictable or deterministic change in the shades or levels of gray in an image. The mathematical relation
that can be established between gray level resolution and bits per pixel can be given as L  2 , where L
k

refers to the number of gray levels and k refers to bpp or bits per pixel. Figure 23 shows an image with
different gray level resolutions.
 Figure 22: An image with different spatial resolutions

Figure 23: A 452 x 376 image with gray levels of 256, 128, 64, 32, 16, 8, 4 and 2.