Chapter One

1.1 What is Computer Vision?

Computer vision is a field of Artificial Intelligence (AI) that enables computers and
systems to derive meaningful information from digital images, videos and other visual
inputs — and take actions or make recommendations based on that information. If AI
enables computers to think, computer vision enables them to see, observe and understand.

1.1.1 How does Computer Vision Work?

Computer vision mainly performs three steps(figure 1), which are:
1. Capturing an Image
A computer vision software or application always includes a digital camera or
CCTV to capture the image. So, firstly it captures the image and puts it as a digital
file that consists of Zero and one's.
2. Processing the image
In the next step, different CV algorithms are used to process the digital data stored
in a file. These algorithms determine the basic geometric elements and generate
the image using the stored digital data.
3. Analyzing and taking required action
Finally, the CV analyses the data, and according to this analysis, the system takes
the required action for which it is designed.

Figure 1:Steps of Computer Vision

1.2 What is Image?
An image is defined as a two-dimensional function(x,y), where x and y are spatial(plane)
coordinates, and the amplitude of F at any pair of coordinates (x,y) is called the intensity
of that image at that point. When x,y, and amplitude values of F are finite, we call it a
digital image.

1.3 Related Fields in Computer Vision

Computer vision is a multidisciplinary field that intersects with various other areas of
study. Here are some related fields in computer vision:
1. Machine Learning and Deep Learning: Machine learning and deep learning are
essential components of computer vision. Techniques such as convolutional neural
networks (CNNs) and recurrent neural networks (RNNs) are widely used for
image and video analysis.
2. Image Processing: Image processing deals with manipulating and enhancing
images to improve their quality or extract useful information. Many computer
vision applications involve image processing techniques.
3. Pattern Recognition: Pattern recognition is the study of recognizing patterns
within data. In computer vision, this involves recognizing objects, shapes, or
gestures in images or videos.
4. Robotics: Computer vision plays a significant role in robotics, enabling robots to
perceive and navigate their environment. It's used in tasks like object detection,
localization, and mapping (SLAM).
5. Natural Language Processing (NLP): Combining computer vision with NLP
allows for multimodal analysis, where both text and visual data are used together,
enabling applications like image captioning and sentiment analysis of images.
6. Biometrics: Biometrics involves recognizing individuals based on their unique
physical or behavioral characteristics. Face recognition and fingerprint analysis
are examples of computer vision applications in biometrics.
7. Augmented Reality (AR) and Virtual Reality (VR): Computer vision is
essential in AR and VR for tracking the user's movements and the environment to
provide an immersive experience.
8. Medical Imaging: In the field of medical imaging, computer vision is used for
tasks like image segmentation, tumor detection, and disease diagnosis through the
analysis of medical images like X-rays, MRIs, and CT scans.
9. Autonomous Vehicles: Computer vision is a key technology in the development
of self-driving cars. It's used for tasks like lane detection, object detection, and
path planning.
10. Remote Sensing: Remote sensing involves collecting data about the Earth's
surface from a distance, often using satellite imagery. Computer vision is used for
land cover classification, monitoring natural disasters, and more.
11. Agriculture: In precision agriculture, computer vision is used for crop
monitoring, yield prediction, and disease detection in plants.
12. Security and Surveillance: Computer vision is used for video surveillance, facial
recognition, and anomaly detection in security applications.
 13. Entertainment and Gaming: The gaming industry often uses computer vision for
motion capture, gesture recognition, and creating immersive experiences.
14. Geographic Information Systems (GIS): Computer vision is used to extract
geographical information from images and satellite data, assisting in map creation
and geospatial analysis.
15. Artificial Intelligence Ethics and Bias: As computer vision systems become
more pervasive, ethical considerations related to privacy, bias, and fairness in AI
are gaining importance.
16. Hardware Development: The development of specialized hardware, such as
GPUs (Graphics Processing Unit) and TPUs (Tensor Processing Unit), plays a
crucial role in accelerating the computation required for computer vision tasks.
These fields often overlap, and interdisciplinary research and collaboration are
common in computer vision, as solutions to real-world problems frequently require expertise
from multiple domains.

1.4 Computer Vision vs Image Processing

Computer vision and image processing are closely related but distinct fields within the
domain of visual information processing. Here's a breakdown of their differences:
1.4.1 Computer Vision
Goal: Computer vision focuses on enabling computers to interpret and understand the visual
world, much like human vision. It's about replicating human vision and cognition in
machines.
Tasks: Computer vision involves higher-level tasks such as object recognition, scene
understanding, image understanding, and even making decisions based on visual input. This
can include tasks like object detection, image segmentation, facial recognition, and 3D
reconstruction.
Complexity: Computer vision tasks often involve understanding the context and semantics of
images. It requires higher-level reasoning and can involve machine learning and AI
techniques, including deep learning.
Applications: Computer vision is used in various applications, including autonomous
vehicles, robotics, medical image analysis, and augmented reality. It often deals with real-
world, dynamic environments.
Example: An autonomous car using computer vision might identify pedestrians, other
vehicles, traffic signs, and road conditions to make driving decisions.

1.4.2 Image Processing

Goal: Image processing is primarily focused on manipulating and enhancing images to
extract information or improve image quality. It deals with the modification of the visual
content itself.
Tasks: Image processing tasks include tasks like image filtering, noise reduction, image
sharpening, contrast enhancement, and color correction. It's often concerned with low-level
operations on images.
Complexity: Image processing typically involves simpler operations that are applied to
individual pixels or small neighborhoods of pixels within an image. It's not concerned with
high-level interpretation or recognition.
Applications: Image processing is used in applications like image compression, medical
image enhancement, and improving the quality of photos. It's commonly applied in scenarios
where the goal is to improve the visual quality or extract specific information from images.
Example: A medical image processing application might enhance the contrast in an X-ray
image to make it easier for a radiologist to identify abnormalities.
In summary, computer vision is broader and more complex, focusing on
understanding and interpreting visual data to make higher-level decisions, often utilizing
artificial intelligence and machine learning techniques. Image processing, on the other hand,
is more about enhancing and manipulating the visual content to improve its quality or extract
specific information, usually using low-level operations. While they have distinct goals and
applications, they can also complement each other, with image processing techniques often
being a part of the preprocessing step in computer vision tasks to prepare images for higher-
level analysis.

1.5 Applications of Computer vision and Image processing

Computer vision and image processing have a wide range of applications across
various industries. Here are some of the key applications in both fields:

1.5.1 Applications of Computer Vision:

Object Recognition and Detection: Computer vision is used to recognize and locate objects
in images or video streams, such as pedestrian detection in autonomous vehicles, facial
recognition for security, and barcode scanning in retail.
Image Segmentation: This involves dividing an image into meaningful parts, which is used
in medical image analysis, industrial quality control, and scene understanding in robotics.
Gesture Recognition: Computer vision enables the interpretation of hand and body gestures,
which is useful in applications like sign language translation, gaming, and human-computer
interaction.
Face Recognition: Facial recognition is widely used in security, access control, and
authentication systems, as well as in social media for tagging photos.
Optical Character Recognition (OCR): OCR technology is used to convert printed or
handwritten text in images into machine-readable text. Applications include digitizing printed
documents, license plate recognition, and reading text from images for the visually impaired.
Robotics: Computer vision plays a crucial role in enabling robots to perceive their
environment and navigate, allowing them to perform tasks like object manipulation and
autonomous navigation.
Augmented Reality (AR) and Virtual Reality (VR): AR and VR applications use computer
vision for tracking users' movements and the environment, enabling immersive experiences
and overlaying digital information on the real world.
Medical Image Analysis: Computer vision is used in medical imaging to analyze X-rays,
MRIs, CT scans, and other medical images for tasks like tumor detection, organ
segmentation, and disease diagnosis.
Autonomous Vehicles: Self-driving cars rely on computer vision for tasks like lane
detection, obstacle detection, and road sign recognition.
Agriculture: Computer vision is used for crop monitoring, yield prediction, and disease
detection in precision agriculture.
Industrial Quality Control: Computer vision systems are used to inspect and ensure the
quality of manufactured products, such as checking for defects in electronics, textiles, and
food items.

1.5.2 Applications of Computer Vision:

Image Enhancement: Image processing techniques are used to enhance the quality of
images, improve contrast, reduce noise, and adjust brightness and color levels.
Image Compression: Compression algorithms are used to reduce the size of images for
efficient storage and transmission, as in JPEG and PNG formats.
Image Restoration: Restoring old or damaged images by removing scratches, dust, or other
imperfections is a common application.
Image Filtering: Techniques like edge detection and blurring are used for various
applications, including feature extraction and special effects in photography.
Color Correction: Adjusting the color balance, saturation, and contrast of images to improve
their visual quality in photography and video editing.
Morphological Operations: These are used in applications like document processing and
shape analysis to extract or manipulate image features.
Image Registration: Aligning multiple images of the same scene, often used in medical
imaging for comparing images taken at different times or with different modalities.
Panorama Stitching: Combining multiple images to create a single, wide-angle or
panoramic view, used in photography and virtual tours.
Biometric Identification: Image processing is used in biometric systems for fingerprint
recognition, iris scanning, and hand geometry.
Remote Sensing: Analyzing satellite and aerial imagery for applications in agriculture, urban
planning, and environmental monitoring.
Both computer vision and image processing have a significant impact on numerous
industries, making tasks more efficient, accurate, and accessible, and enabling a wide range
of innovative applications.

1.6 Image processing Example

Image Denoising:
Description: Image denoising is the process of removing unwanted noise or artifacts
from an image, resulting in a cleaner and more visually appealing representation of the
original content. Noise can be caused by factors such as low lighting conditions, sensor
limitations in cameras, or transmission errors in digital images.
Image Processing Techniques:
Filtering: Filtering techniques such as median filtering or Gaussian smoothing are commonly
used to reduce noise. Median filtering replaces each pixel value with the median value of its
local neighborhood, which can effectively remove impulsive noise. Gaussian smoothing
applies a weighted average to each pixel's neighborhood, reducing high-frequency noise.
Wavelet Transform: The wavelet transform is used for multiresolution analysis, and it helps
to separate image content from noise in different scales and orientations. This makes it useful
for denoising applications.
Non-local Means Denoising: This technique compares similar patches of the image to
estimate the noise level and then reduces noise while preserving image details. It is effective
in reducing random noise.
Example:
Imagine you have a digital photograph taken in low light conditions. As a result, the image
contains noticeable speckles, which are a type of noise. To improve the image, you can apply
image denoising techniques:
Gaussian Smoothing: You apply a Gaussian filter to the image, which blurs it slightly,
reducing high-frequency noise. The resulting image appears smoother and cleaner.
Wavelet Transform: Using a wavelet denoising algorithm, you decompose the image into
different scales and remove noise in the high-frequency components while preserving the
main features.
Non-local Means Denoising: This technique identifies similar patches in the image and
averages them, reducing noise while maintaining the essential image features.
After applying these techniques, you obtain a cleaner, less noisy version of the original
image, making it more visually appealing and suitable for various applications, such as
photography, medical imaging, or video processing.
Image denoising is just one of the many image processing applications, demonstrating how
this field plays a critical role in improving image quality and making images more suitable
for analysis, visualization, or further processing.
1.7 Fundamentals steps in image processing
The fundamental steps in image processing typically include the following stages,
which are essential for various image manipulation and analysis tasks:
1. Image Acquisition: Image acquisition is the process of capturing an image using
devices like cameras, scanners, or sensors. It’s the first step where the raw data
(pixels) is obtained from the real world or a digital source.
2. Preprocessing: Image preprocessing involves operations to prepare the image for
subsequent analysis or enhancement.
Common preprocessing steps include:
Image Scaling: Resizing the image to a specific dimension.
Image Cropping: Removing unwanted parts of the image.
Color Conversion: Converting between color spaces (e.g., RGB to grayscale).
Noise Reduction: Reducing noise using filters or techniques like averaging.
Image Enhancement: Adjusting contrast, brightness, and sharpness.
3. Image Segmentation:
Image segmentation divides the image into meaningful regions or objects.
It helps identify and isolate areas of interest within the image.
Techniques like thresholding, edge detection, or clustering are used for segmentation.
4. Feature Extraction:
Feature extraction involves selecting relevant information or attributes from the
segmented regions.
These features can be characteristics like color, texture, shape, or edges.
Feature extraction is crucial for subsequent analysis and classification tasks.
5. Image Analysis:
Image analysis encompasses tasks like object recognition, classification, or
measurement.
This step uses the extracted features to derive information or make decisions.
Machine learning, pattern recognition, and computer vision techniques are often
employed here.

6. Post-processing:
Post-processing may involve further adjustments or operations to refine the results.
 Techniques like filtering, morphological operations, and spatial transformations can
be applied.
7. Visualization and Display:
The final processed image can be displayed, saved, or transmitted for interpretation
and decision-making.
Visualization can include generating visual representations, reports, or interactive
displays.
8. Storage and Management:
Processed images are often stored in databases or file systems for future reference and
retrieval.
Effective management ensures that images are organized and accessible.
9. Interpretation and Decision-Making:
The ultimate goal of image processing is often to extract information, make decisions,
or gain insights from the processed images.
Interpretation can be performed by humans or automated systems.
These steps represent a typical image processing workflow. However, the specific
techniques and tools used in each step can vary based on the application and the desired
outcome. Moreover, advancements in deep learning and neural networks have led to more
automated and integrated pipelines for image analysis, where the feature extraction and
analysis stages may overlap or be performed jointly.