Digital Image ProcessingAbout the Authors S Jayaraman obtained his BE and ME from PSG College of Technology in the years 1974 and 1976 respectively, and his PhD from Bharathiar University in 1993. He has more than thirty years of experience in teaching and research and has been cg associated with many sponsored projects funded by AICTE, DST, DMIT and SDC. yw He has guided three PhD research works and is currently supervising three scholars in their PhD studies, He has published more than 80 papers in various national and international journals and conferences, and has delivered many special lectures in short-term training programmes and faculty development programmes sponsored by the AICTE and ISTE, His specific areas of interest include Bio-Medical Signal Processing, Multidimensional System Analysis, Image Compression and Non-Linear Signal Processing. Esakkirajan completed his BSc in Physics from Sadakathullah Appa College, Palayamkottai, Blech from Cochin University of Science and Technology, Cochin, and ME from PSG College of Technology, Coimbatore, also being an ME rank holder. He has received the Alumni Award in his ME degree programme. He has published many papers in international and national journals and his areas of interest include Database Management Systems, and Digital Image Compression. T Veerakumar received his BE in Electronics and Communication Engineering from RVS College of Engineering, and did his ME in Applied Electronics from PSG College of Technology. He has presented papers in international and national journals and his areas of interest include Artificial Neural Networks, Image ‘Compression and Video CompressionDigital Image Processing S Jayaraman Retired Professor Department of Electronics and Communication Engineering PSG College of Technology Coimbatore, Tamil Nadu $ Esakkirajan Lecturer Department of Instrumentation and Control Engineering PSG College of Technology Coimbatore, Tamil Nadu T Veerakumar Lecturer Department of Electronics and Communication Engineering PSG College of Technology Coimbatore, Tamil Nadu NY SI Tata McGraw Hill Education Private Limited NEW DELHI McGraw-Hill Offices New Delhi New York St.Louis San Francisco Auckland Bogota Caracas Kuala Lumpur Lisbon London Madrid Mexico City Milan Montreal San Juan Santiago Singapore Sydney Tokyo TorontoINA Tata McGraw Hill Published by the Tata McGraw Hill Education Private Limited, 7 West Patel Nagar, New Delhi 110 008. Copyright © 2009 by Tata MeGraw Hill Education Private Limited. No part of this publication may be reproduced or distributed in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise or stored in a database or retrieval system without the prior written permission of the publishers. The program listings (if any) may be entered, stored and executed in a computer system, but they may not be reproduced for publication, This edition can be exported from India only by the publishers, ‘Tata MoGraw Hill Education Private Limited ISBN (13): 978-0-07-014479-8 ISBN (10): 0-07-014479-6 ‘Managing Director: Ajay Shukla General Manager: Publishing SEM & Tech Ed: Vibha Mahajan Manager—Sponsoring: Shalini Jia Asst. Sponsoring Editor: Siwman Sen Executive—Euiforial Services: Sohini Mukherjee ‘Senior Production Manager: P-L Pandita General Manager: Marketing—Higher Education & School: Michael J Cruz Sr. Product Manager: SEM & Tech Ed: Biju Ganesan General Manager Production: Rajender P Ghansela Asst. General Manager—Produetion: B L Dogra {Information contained in this work las been obtained by Tata McGraw Hill, from sources believed to be reliable. However, neither Tata McGraw Hill nor its authors guarantee the accuracy or completeness of any information published herein, ‘and neither Tata MeGraw Hill nor its authors shall be responsible for any errors, omissions, or damages arising out of use of this information, This work is published with the understanding that Tata McGraw Hill and its authors are supplying information but are not attempting to render engineering or other professional services. If such services are required, the assistance of an appropriate professional should be sought. ‘Typeset at Text-o-Graphies, B-1/S6, Aravali Apartment, Sector-34, Noida 201 301 and printed at ‘Avon Printets, Plot No. 16, Main Loni Road, Jawahar Nagar Industrial Area, Shadara, Delhi 110 094 Cover: SDR RAZLCRAFDCQDL EaContents Preface Acknowledgements List of Acronvins Image-Processing Taxonomy Visual Walkthrough 1. Introduction to Image-processing System Introduction 1 Image Sampling 4 Quantisation 12 Resolution £3 Human Visual System 4 Classification of Digital Images 19 Image Types 20 Elemenis of an Image-processing System Image File Formats 38 Applications of Digital Image Processing 42 Summary 43 Review Questions 43 Problems 45 References 46 SeRURRE0RE 2. 2D Signals and Systems 2.1 Introduction 47 2.2 2D Signals 48 2.3 Separable Sequence 50 24 Periodic Sequence 50 2.5 2D Systems 51 2.6 Classification of 2D Systems 57 2.7 2D Comolution 55 2.8 2D Z-Transform 55 2.9 2D Digital Filter 77 Summary 77 Review Questions 77 Problems 81 References 82 3. Convolution and Correlation 3.1 Introduction 84 2D Convolution Through Graphical Method 85 Convolution Through Z-Transform 706 2D Convolution Through Matrix Analysis 110 Circular Convolution Through Matrix Method Application of Circular Convolution 124 2D Correlation 127 Summary 148 Review Questions 149 Problems 150 References 151 SRaRoS 47 84vi Contents 4. Image Transforms 4.1 Introduction 152 4.2. Need for Transform 43 Image Transforms 4.4 Fourier Transform 4.5. 2D Discrete Fourier Transform 4.6 Properties of 2D-DFT 159 4.7 Importance of Phase _174 4.8 Walsh Transform 175 4.9 Hadamard Transform 4.10 Haar Transform 182 Slant Transform 793 Discrete Cosine Transform 194 Karhunen-Loeve Transform (KL Transform) gular Value Decomposition 209 Radon Transform 222 4.16 Comparison of Different Ima, Solved Problems 230 Summary 238 Review Questions Problems 240 References 242 153 153 158 1st Transforms 239 5. Image Enhancement Introduction 243 Image Enhancement in Spatial Domain Enhancement Through Point Operation ‘Types of Point Operation 245 Histogram Manipulation 24 evel Transformation 252 Nonlinear Gray-level Transformation Local or Neighbourhood Operation Median Filter 270 Spatial Domain Hi Bit-plane Slici Image Enhancement Homomorphic Filter 292 Zooming Operation 293 Image Arithmetic 297 Solved Problems 304 Summary 316 Review Questions 317 Problems 320 Matlab Exercises 322 References 323 260 Se Ign eage sass Filtering or Ima 5 aEoS 6. Image Restoration and Denoising 6.1. Introduction 6.2 Image Degrad 3 6.3 Types of Image Blur 64 Classification of Image-restoration Techniques 6.5 Image-restoration Model 328 6.6 Linear Image-restoration Techniques 329 6.7 Non-linear Image-restoration Techniques .¢ Sharper the Frequency Domain 340 202 209 243 ing 273, 324 3276.8 Blind Deconvolution 344 6.9 Classification of Blind-deconvolution Techniques 344 6.10 Image Denoising 348 6.11 Classification of Noise in tmage 348 6.12 Median Filtering 349 6.13 Trimmed Average Filter 354 6.14 Performance Metrics in Image Restoration 357 6.15 Applications of Digital Image Restoration Solved Problems 358 unmary 362 Review Questions 363 Problems 365 References 367 7. Image Segmentation 7.1 Introduction 368 7.2 Classification of Image-segmentation Techniques 369 7.3 Region Approach to Image Segmentation 369 74 Clustering Techniques 376 7.5 Image Segmentation Based on Thresholding 379 7.6 Edge-based Segmentation 380 7.7 Classification of Edges 380 7.8 Edge Detection 387 7.9 Edge Linking 397 0 Hough Transform 392 1 Active Contour 393 2 Watershed Transformation 394 3 Shape Representation 397 '4 Classification of Shape-representation Techniques 397 Summary 400 Review Questions 401 Problems 405 References 407 8 Object Recognition Introduction 409 £2 Necd foram Okfectaccognition System 409 8.3 Automated Object-recognition Systems 409 84 Patterns and Pattern Class 411 8.5 Selection of Measurement Parameters 474 8.6 Relationship between Image Processing and Object Recognition 8.7 Approaches to Object Recognition 415 8.8 Bayes’ Parametric Classification 476 8.9 Template-Matching-based Object Recognition 478 8.10 Non-parametric Density Estimation 478 8.11 Neural-network Approach to Object Recognition 419 8.12 Classification of Artificial Neural Networks 423 8.13 Learning Rules in Artificial Neural Networks 425 8.14 Perceptron 426 8.15 Multi-layer Perceptron 427 8.16 Radial-basis Function Network 431 8.17 Advantages of Neural Networks 431 8.18 Structural Pattern Recognition 432 8.19 Applications of Object Recognition 434 4s Contents vi 368 408viii Contents Summary 437 Review Questions 438 References 443 9. Image Compression 444 9.1 Introduction 444 9.2 Need for Image Compression 445 93 Redundancy in Images 445 9.4 Classification of Redundancy in Images 445 9.5 Image-compression Scheme 446 Classification of Image-compression Schemes 447 Fundamentals of Information Theory 447 Run-length Coding 449 Shannon-Fano Coding 450 Huffman Coding 452 Arithmetic Coding 457 Dictionary-based Compression 469 Predictive Coding 469 Transform-based Compression 487 Image-compression Standard 488 Scalar Quantisation 494 Vector Quantisation 497 Types of Vector Quantisation 505 Wavelet-based Image Compression 507 Fractal Image Compression 507 Block Truncation Coding 51 Solved Problems 315 Summary 536 Review Questions 537 Problems $38 References 541 10. Binary Image Processing 543 10.1 102 103 lod los 10.6 10.7 10s. 10.9) 10.10 10.11 10.12 Di 10.13 Introduction 543 Binarisation 544 Mathematical Morphology 544 Structuring Elements 544 Morphological Image Processing 544 Basic Set Theory 546 Logical Operations 547 Standard Binary Morphological Operations 548 Dilation- and Erosion-based Operations 554 Properties of Morphological Operation 555 Morphological Algorithms 564 tance Transform 576 alient Features of Morphological Approach 578 Summary 578 Review Questions 579 Problems 580 References 583 11. Colour-Image Processing 584 It 12 13 ia Introduction 583 Light and Colour 585 Colour Formation 585 Human Perception of Colour 586Contents ix Colour Model 586 The Chromaticity Diagram 590 Colour-image Quantisation 593 Histogram of a Colour Image 594 Colour-image Filtering 597 11,10 Gatnma Correction of a Colour Image 600 IL-IL Pseudo-colour 604 11.12 Colour-image Segmentation 605 Summary 606 Review Questions 607 Problems 608 References 610 12. Wavelet-based Image Processing oll 12.1 Introduction 6/2 12.2 Evolution of Wavelet Transform 612 123 Wavelet 613 12.4 Wavelet Transform 614 12.5 Continuous Wavelet Transform 615 12.6 2D Continuous Wavelet Transform 615 12.7 Multi-resolution Analysis 6/6 12.8 Examples of Wavelets 6/7 12.9 Wavelet-based Image Compression 6/9 12.10 Embedded Image Coding 626 12.11 Embedded Zero Tree Wavelet 626 12.12 Sct Partitioning in Hierarchical Trees (SPIHT) 634 12.13 JPEG2000 Compression Standard 639 12.14 Desirable Properties of Wavelets 644 12.15 Wavelet Packet Transform 645 12.16 Multi-wavelets 646 12.17 Contourlet Transform 648 12.18 Image Pyramid 650 12.19 Wavelet-based Denoising 655 12.20 Wavelet-thresholding Methods 656 12.21 Comparison of Different Thresholding Methods 659 12.22 Digital Image Watermarking 659 12.23 Classification of Watermarking Methods 660 12.24 Watermarking in the Spatial Domain 667 12.25 Watermarking in Frequency Domain 664 12.26 Applications of Digital Watermarking 667 Summary 667 Review Questions 668 Problems 670 References. 671 13. An Introduction to Video Processing Available on the website ‘Appendices Appendix |: Image Processing Related MATLAB Commands 673 Appendix II: Overview of Vector Space Concepts 680) Appendix Ill: Fundamentals of Matrices 684 Appendix IV: Objective Type Questions 695 Glossary 710 Index 79Preface Vision is the most powerful of the five human senses. Visual information, conveyed in the form of images gives better impact than textual information. The fields of digital image processing have grown tremendously over the past 30 years. The growth of digital image processing has been fueled by technological advances in digital imaging, computer processors and mass storage devices. Research and development of image pro- cessing technologies have advanced very rapidly in the past decade, Digital image processing is concerned primarily with the extraction of useful information from images. In this process, it also deals with 1) image representation, 2) enhancing the quality of an image, 3) restoration of the original image from its degraded version, and 4) compression of the large amount of data in the images for efficient archiving and retrieval. Image-processing algorithms can be classified into three different categories. At the lowest level, the algo- rithm deals directly with the raw pixel values (image denoising and edge detection are good examples). In the middle level, the algorithm utilises low-level results for processes such as segmentation and edge linking. At the highest level, the algorithm attempts to extract semantic information from those provided by the lower levels. Examples of high-level algorithms are handwriting recognition, face recognition and machine vision algorithms. The objective of this book is to not only introduce different concepts of digital image processing to undergraduate and postgraduate students but also to serve as a handbook for practicing engineers Simulation is an essential tool in any ficld related to engineering techniques. In this book, the image- processing algorithms are simulated using MATLAB. It has been the endeavour of the authors to present a large number of detailed worked examples to illustrate the various digital image-processing concepts. ‘Organisation of the Book This book contains twelv, chapters. Chapter 1 e-processing system. The -processing systems like image acquisition, image sampling, quantisation, imag. sensors, image scanners and image storage devices are covered in this chapter. The highlight of this chapter -ussion of the different types of image file formats in practice. Chapter 2 deals with two-dimensional signals and systems. Different types of two-dimensional signals, and properties of two-dimensional signals like separability and periodicity are discussed in this chapter. This chapter also gives an overview of the two-dimensional system. The focus is mainly on the linear shift-invariant system, As Z-transform is widely used to analyse two dimensional signals and systems, the understanding of the 2D signals and systems is enriched through numerous examples of forward and inverse two-dimensional Z transforms in this chapter. Chapter 3 is devoted to 2D convolution and correlation. Convolution is one of the most powerfal mathe- ‘matical operators which is widely used in the field of digital image processing. In this chapter, the computa- tion of 2D convolution and correlation using graphical, matrix and Z-transform methods are discussed in a step-by-step approach. The examples related to 2D convolution and correlations are illustrated through MATLAB examples. The focus in Chapter 4 is on image transforms. The need for image transforms, different types of image transforms and their properties are explained in this chapter. The image transforms discussed in this chapter include two-dimensional Fourier transform, Walsh transform, Hadamard transform, Slant transform, KL transform, Discrete Cosine Transform, Radon transform and Singular Value Decomposition. different elements of ima; is thePreface xi enhancement of Chapter § discusses different techniques employed for th mages. This chapter includes techniques in spatial as well as frequency domains. While the frequency domain methods discuss the design of low-pass, high-pass and band-pass filters, the discussion of techniques related to the spatial domain include y-level transformations together with spatial fit ss, high-pass and high boost filtering which are illustrated with MATLAB examples. Further, in this chapter, certain test images like flowers, animals and popular monuments are considered rather than the usual images, like Cameraman and Lena, to bring out variety and emphasis on generality in approach. Chapter 6 provides an overview of image restoration and image denoising techniques. The different causes for image degradation, and deterministic and stochastic methods of image restoration are the crucial topics that are discussed in this chapter. Also, this chapter covers the different types of image denoising techniques like average filtering, median filtering, alpha-trimmed filtering and min-max filtering. Chapter 7 deals with different techniques in image segmentation. The different techniques include region, boundary and edge-based segmentation methods. Advanced image segmentation algorithms like the Snake algorithm and Watershed algorithm are covered in this chapter. Chapter 8 gives an overview of different types of object-recognition techniques. Different types of representation of patterns are discussed in this chapter, namely Statistical approach, Structural approach and Neural-Network approach. Different applications of object recognition are also discussed in this chapter. Chapter 9's devoted to image compression. The need for image compression, spatial domain and frequency domain techniques of image compression and the applications of image compression form the heart of this chapter. One of the powerful tools to achieve compression is vector quantisation. This chapter gives the basic idea of vector quantisation (VQ) and different approaches to VQ through illustrative examples Chapter 10 deals with binary image processing. In this chapter, different morphological operations like dilation, erosion, opening and closing are given with suitable MATLAB examples along with their proper- ties, Other topics discussed in this chapter include thinning, thickening, distance transform and hit-or-miss transform, The concepts related to colour image processing are given in Chapter 11. The concepts discussed in this, chapter include human perception of colour, colour models, colour image quantisation, colour image filtering, pseudo-colouring, colour image histogram and colour image segmentation. Finally, in Chapter 12, the application of wavelets in image processing are dealt with. The different wavelet- based applications considered in this chapter are image compression, image denoising and watermarking. The embedded quantisation techniques widely used in the field of image compression like EZW and SPLHT are explained through numerical examples. An introduction to advanced concepts like multi-wavelet trans- forms, contourlet transforms and directional filter banks are also given in this chapter to provide some sense of completeness and stimulate the researchers” interest. Each chapter is equipped with a set of solved problems with solutions, and review ques The problem sets help readers to understand the basie concepts in digital image processing. In addition to the twelve chapters, four appendices are also given in this book. Since MATLAB has been used extensively to highlight the various concepts throughout this book, a list of commonly used MATLAB commands in image processing are provided in Appendix 1 From our past teaching experience, it is the considered opinion of the authors that a holistic picture of Image Transforms can be brought about by providing a unified treatment from the standpoint of vector spaces. An introductory knowledge related to Vector Space, thus, becomes essential to the understanding of image transforms. An introduction to Vector Spaces is provided in Appendix IL. ing methods such as low-xii Preface A digital image can be represented in terms of a matrix. Hence, a basic knowledge of matrices will be seful for effective manipulation of images. Fundamental concepts related to different types of matrices are given in Appendix I. Objective-type questions measure one’s ability to remember facts and figures and comprehend the concepts related to any subject. In this book, a rich set of objective-type questions along with answers are separately provided in Appendix 1V Apart from this, the book also has additional web supplements and an accompanying CD. The online con- tents can be accessed at http:/ivww.mhhe.com/jayaraman/dip and contain the following material: ‘* An Introduction to Video Processing Introduction to video processing = Spatio-temporal sampling = Interfiame and Intraftame coding Motion Estimation Techniques Video Compression Standards © Interactive Quiz ‘* Downloadable images from the text ‘© Solution Manual for Instructors ‘+ The accompanying CD contains = Powerpoint tutorials = Images from the text = MATLAB codes y hope that this book will initiate many persons to the exciting world of digital image processing, book free from mistakes, some errors may still have survived ons, suggestions and constructive criticism We earnest Though we have spared no pains to make tl our scrutiny, We gladly welcome all corrections, recommenda from our reader S Jayaraman S Esakkirajan ‘T-VeerakumarAcknowledgements The authors are always thankful to the Almighty for guiding them in their perseverance and blessing them with achievements. The authors wish to thank Mr Rangaswamy, Managing Trustee, PSG Institutions; Mr C R Swaminathan, Chief Executive; and Dr R Rudramoorthy, Principal, PSG College of Technology, Coimbatore, for their wholchearted coo ration and constant encouragement given in this successful endes Dr S Jayaraman would like to thank his parents, Sri P'S Subramanyam and Mrs J Chellam, for inculcating -s and providing inspiring guidance: wife, Mrs V Ganga; and daughter, J Pavitra, for their constant encour- ent and moral support along with patience and understanding, Sincere thanks is also due to his teacher, friend, philosopher and guide, Dr SN Sivanandan, Professor and Head, Department of CSE, PSG College of Technology, for being a source of inspiration in all his endeavours. His enriching wisdom, experience and timely advice have been extremely influential in shaping the author's outlook on Mr § Esakkirajan would like to thank his father, Mr G Sankaralingam, and wi dered a lot of extra responsibilities during the months teachers Dr N Malmurt Mrs Akila, who shoul- book was being written. He also likes to thank his mand Dr R Sudhakar, who taught him the basics of digital image processing, MrT Veerakumar would like to thank his parents, Mr N Thangaraj and Mrs T Muniammal, for their suppor The authors wholeheartedly thank, appreciate and sincerely acknowledge Mr V Senthil Murugan, aalraj, Mr Afsar Ahmed, Ms Kavitha and Ms Padmapriya for their excellent, unforgettable help and fance towards the documentation related to this book. The authors wish to thank Mr D $ Joseph Mathew and Mr A R Ramakrishnan for all their support B Lokeswara Rao Geethanjali College of Engineering and Technology Hyderabad, Andhra Pradesh GN Srinivasan RV College of Engineering Bangalore, Karnataka E Bengeorge OIS College of Engineering Ongole, Andhra Pradesh K Krishna Prasad Narional Institure of Technology Warangal, Andhra Pradesh MY Raghunadh National Institute of Technology Warangal, Andhra Pradesh ssamma Thomas Cochin University of Science and Technology Cochin, Kerala Y Wiselin Jiji Dr Sivanthi Adithanar College of Engineering Tiruchendwr, Tamil Nadu D Laxmi Bannari Amman Institute of Technology Sathyamangalam, Tamil Naduxiv Acknowledgements AN Rajagopalan S Allin Christy AK Sachan ‘Manish Kumar Verma Udai Shanker S Bhattacharya ‘SM Patil Tanish Zaveri Jignasa P Mehta AN Sarvaiya US Reviews Dapeng Wu Frank Shih, Amar Raheja Bahadir K Gunturk Xin Li Indian Institute of Technology Madras Chennai, Tamil Nadu PSG College of Technology Coimbatore, Tamil Nadu Truba College of Research and Technology Bhopal, Madhya Pradesh Bharat Institute of Technology Meerut, Uttar Pradesh MMMEC University Gorakhpur, Uttar Pradesh Indian School of Mines Dhanbad, Jharkhand Bharati Vidvapecth College of Engineering Navi Mumbai, Maharashtra Nirma Institute of Technology Ahmedabad, Gujarat PVP Institute of Technology Rajkot, Gujarat SYNIT Surat, Gujarat University of Florida Gainesville, Florida Department of Computer Science New Jersey Institue of Technology Newark, New Jersey Department of Computer Engineering California State Polytechnic University Pomona, California Department of Electrical and Computer Engineering, Louisiana State University Baton Rouge, Los Angels Department of Electrical and Computer Engineering, The University of Texas at San Antonio, San Antonio, Texas 34 Antietam Drive, Morgantown West VirginiaMarketing Reviews G Sudha Vani Kilari Veeraswamy S Selvan ary Joans Sudhanshu Tripathi Narendra Kohli RP Arora Satish Chavan SD Joshi Acknowledgements x RVR and JC College of Engineering Guntur, Andhra Pradesh QIS College of Engineering and Technology Bapatia, Andhra Pradesh St Peter's Engineering College Chennai, Tamil Nadu Vellamal Engineering College Chennai, Tamil Nadu Amity School of Engineering and Technology New Delhi, Dethi HBTI, Kanp Dehradun Institute of Technology Dehradun, Uttarakhand iar Pradesh Don Bosco Institute of Technology Mumbai, Maharashora Vishwakarma Government Engineering College Ahmedabad, GujaratWoden aed D ADALINE CODEC CPN cR CRI CTE per DFB DFT DICOM DM DPCM DPI DWT EBCOT EC EPS: EZW FDCT FFT FIR FLC GIF HVS Iwcr IEEE iid MR IFS JFIF JPEG KLT Two-Dimensional Adaptive Linear Element Artificial Neural Network Adaptive Resonance Theory Additive White Gaussian Noise Bidirectional Associative Memory Bounded Input Bounded Output Back Propagation Network Block Truncation Coding Charge Coupled Device Commission International d’Eclairage Constrained Least Square Color Look-up Table Complementary Metal Oxide Semiconductor Coder Decoder Counter Propagation Network Compression Ratio Cathode Ray Tube Discrete Cosine Transform Directional Filter Bank Discrete Fourier Transform Digital Imagin Delta Modulation Differential Pulse Code Modulation Dots Per Inch Discrete Wavelet Transform Embedded Block Coding with Optimized Truncation diogram lated Post Seript Embedded Zero Tree Wavelet Forward Discrete Cc Fast Fourier Transform Finite Impulse Response Fixed Length Code Graphic Interchan; Human Visual Sys Inverse Discrete Cosine Transform Institute of Electrical and Electronic Engineers Independent and identically distributed Infinite Impulse Response Iterated Function System JPEG File Interchange Format Joint Photographic Expert Group Karhunen-Loeve Transform ng and Communications in Medicine e Transform FormatLMs LP LsI Lzw MADALINE MAE MPEG MR MSE MsvQ NIST NISC PACS PCA PDFB PEL PIES PNG PSF PSNR RBF SQ SIFT SURE svD SVG TL TSVQ vic VOXEL va WHT we wr www List of Acronyms xvii Least Mean Square Laplacian Pyramid Linear Shift Invariant System Lempel-Ziv-Welch Multiple Adaptive Linear Element Mean Absolute Error Moving Pictures Expert Group Multiresolution ‘Mean Square Error Multi-Stage Vector Quantization National Institute of Standards and Technology National Television System Committee Picture Archiving and Communication Systems Principal Component Analysis, Pyramidal Directional Filter Bank Picture Element Partitioned Iterated Function System Portable Network Graphics Point Spread Function Peak Signal-to-Noise Ratio Radial Basis Function Rate-Distortion Run Length Coding Root Mean Square Error Sequential Chrominance Signal-to-Noise Ratio Self Organizing Map Sound Navigation and Ranging Set Partitioning in Hierarchical Trees Scalar Quantization Short Time Fourier Transform Stein's Unbiased Risk Estimator Singular Value Decomposition Scalable Vector Graphics Tagged Image File Format Tree Structure Vector Quantization Variable Length Coding Volume Pixel Vector Quantization Walsh Hadamard Transform Wavelet Packet Wavelet Transform World Wide Web Signal and MemoryRan RL Natural Scene | q @iraoe Aeon ‘Samping and Quanticaton BD Signal image) Image Representation Digital image Processing Frequency Domain -—_ or Convolution Grayscale Catour A. [wavoletiaseg |—+_var Image Image Imago Processing | | Processing” | Provossing Correlation +lwaeh |] Fadamara Compression | $+ Siant Denoisng Lae t Watemrereng| 4 Erranconant(@) Roauaton (@) ———! — SpatiaiDomain] [Frequenoy | [ Deterministe ] [Stochastic Region] [Boundary] [Edge Domain based based | |_based Paint Operation Inverse Fiterng 7 ‘Gbjeet Recognition Local Operation Peoudo-Iverse Fill eval esren Wiener Fiterng Staisvoal | [Symtactio | [ Neurai Netware Approach | | Approach ‘Approach (Compression ea = { { { t { { q 4 Repave Tiatisical Lossless Biplane Lossy] [Transform | [Fractal | [ato Sequence Encosing Preseive coang | | Presicwve | | cosing win | | coaing Encoding Codng Goring | | Quantization L+[ Hitman v ¥ 7 Ror-engih PCM Encoding | |e Antimetic Pow Law ‘ADPCMVISUAL WALKTHROUGH Each chapter begins with a set of learning objectives. Learning Objectives describe what the reader should be able to do after participating in the learning activity. Learning Objectives give learners a clear picture of what to expect and what is expected of them. 2D Signals and Systems Each chapter begins with an Introduction that gives a brief summary of the back- ground and contents of the chapter. Introduction to Image-processing SystemWorked Examples are provided in sufficient number in each chapter and at appropriate locations, to aid the understanding of the text material 1 Z ; op % > , 4

ter. Answers to all the objective questions are providedA list of commonly used terms in digital im- age processing and their meanings are given in the form of a glossary. The glossary will serve as a dictionary for the students.This chapter provides an overview of the image-processing system which includes various elements like image camptng, cunrteaton" proces Introduction to storage and display. On completion of this chapter, the readeris expected t©be familiar wth the flowing con I ma g e- p rocessi n g = System wore image display isu Digital images play an important role, both in daily-life applications such as satellite television, magnetic resonance imaging, computer tomography as well as in areas of research and technology such as geographical information systems and astronomy. An ima representation of a three-dimensional scene. A digital image is basically a numerical representation of an object. The term digital image processing refers to the ‘manipulation of an image by means of a processor. The different elements of an image-processing system include image acquisition, image storage, image processing and display. This chapter begins with the basic definition of a digital image and is followed by a detailed discussion on two-dimensional sampling. This is, followed by different elements of imag ei rocessing systems.2. Digital mage Processing 1.1.1 Image ‘An image is a two-dimensional function that represents a measure of some characteristic such as brightness or colour of a viewed scene. An image is a projection of a 3D scene into a 2D projection plane. It can be defined as a two-variable function f(x,y) where for each position (x, y) in the projection plane, f(x, y) defines the light intensity at this point. 1.1.2 Analog Image An analog image can be mathematically represented as a continuous range of values representing position and inter An analog image is characterised by a physical magnitude varying continuously in space. For ‘example, the image produced on the screen of a CRT monitor is analog in nature. 1.13 Digital mage A digital image is composed of picture elements called pixels. Pixels are the smallest sample of an image. A pixel represents the brightness at one point. Conversion of an analog image into a digital image involves two important operations, namely, sampling and quantisation, which are illustrated in Fig. Ll acaes Lt samsing bef avntzaton fo] Dt Fig. 1.1 Digital image from analog image Advantages of Digital Images ‘The advantages of digital images are summarised below: The processi cs is faster and cost-effective ently transmitted from one pl to another ga digital image, one can immediately see if the image is good or not. age is easy. The quality of the digital image will not be degraded even if itis copied I format, the reproduction of the image is both faster and cheaper. nty of scope for versatile image manipulation. Drawbacks of Digital Images Sore of the drawhacks of di ial image are given below ‘+ Misuse of copyright has become easier because images can be copied from the Internet just by clicking the mouse a couple of times. + A digital file cannot be enlarged beyond a certain size without compromising on quality. + The memory required to store and process good-quality digital images is very high. + For real-time implementation of digital-image-processing algorithms, the processor has to be very fast because the volume of data is very high. 1.1.4 Digital Image Processing The processing of an image by means of a computer is, tages of using computers for the processing of image: ally termed digital image processing. The advan- marised below:Introduction to Image-Processing System 3 (1) Flexibility and Adaptability The main advantage of digital computers when compared to analog elee- tronic and optical information processing devices is that no hardware modifications are necessary in order to reprogram digital computers to solve different tasks. This feature makes digital computers an ideal device for processing image signals adaptively. (2) Data Storage and Transmission With the development of different image-compression algorithms, the digital data can be effectively stored. The digital data within the computer can be casily transmitted from one place to another. The only limitation of digital imaging and digital image processing are memory and processing speed capabilities of computers. Different image-processing techniques include image enhancement, image restora- tion, image fusion and image watermarking 1.1.5 Digital Image Representation image is a two-dimensional diserete signal al image is also an NV x N array of elements. Each element in the array is a number which repr. sents the sampled intensity: For example, the repre sentation of a4 x 4 image in matrix format andits | y=; = 4 CO three-dimensional view is shown in Fig 1.2 LD Converting an image into a digital format can be ° done either with a digital camera, or by a scanner. (0 0 Digital images can be created directly on a com- < ¢ © Fig.1.2 Digitahimage representation puter screen. However, it is restricted both in spatial Be NIP coordinates (sampling) and in its allowed intensities (quantisation) 1.1.6 Neighbours of a Pixel A pixel will have four neighbours if the neighbours exist in the EAST, WEST, NORTH and SOUTH direc tion. The four neighbours of the pixel “P” pixel are represented in Fig. 1.3. A pixel ‘P” will have eight neighbours if the neighbours are in eight directions such as EAST, WEST. NORTH, SOUTH, NORTH-EAST (NE), NORTH-WEST (NW), SOUTH-EAST (SE) and SOUTH-WEST (SW). The eight neighbours of the pixel *P* are represented in Fig. 1.4 NORTH NORTH x nw] x | x | x [Ne west| x | p | x |east west| x | p | x Jeast x sw| x | x | x |se ‘SOUTH ‘SOUTH Fig.1.3. Four neighbours of the pixel Fig.1.4 Eight neighbours of the pixel P4 Digital mage Processing Sampling is the process of measuring the brightness information only at a discrete spatial location. A continuous image function / (x, y) can be sampled using a discrete grid of sampling points in the plane. 1.2.1 Theory of 2D Sam, Let f(x, ») represent the analog image. It means that function /(x, ») is defined at each and every val xand y. The discrete version of f(x,1) is obtained by defining f(x,y) at specific instants. This is mathemati- of cally represented as F(m,n) = f(mAx, nAy) ay where Av and Ay are positive real constants. It is to be noted that f(m, n) is not defined for all values of ‘mand n. Here Ar and Ay are known as sampling intervals: The 2D sampling starts with the analog signal f(x, y). On taking the Fourier transform of this analog the spectrum of f(x, y) which is denoted by Fy FO, ») J J Fosse eM de dy a2) In Eq. (1.2) e-*** and e~/"* represent the Fourier basis, On taking inverse Fourier transform, we get fouy) Ff PO. 20 eM aN AM a3) When we perform sampling of the analog signal f(x,y), the values are specified only at specific instants which is represented mathematically as S (m,n) = f(mAx, nAy) a4) femme ff FO. 0, ee dd, as) For a discrete signal, we define w= 0)Ae a6) and wy = Ay (7) where wand wy ane expressed in radians From Ea, (1.6), we have + =. Differentiating ®), we gotIntroduction to Image-Processing System 5 de, Sa, a8) ar ' Similarly from Eq. (1.7), we get dea yor, a) Substituting Eq. (1.8) and (1.9) in Eq. (1.5), we get # (m,n) ve (1.10) f(m,ny eee ef day dy (uy ‘The signal f(m, 2) is discretised; hence the integral can be replaced by summation. The double integral over the entire (.2,, v2) plane can be broken into an infinite series of integrals, cach of which is over a square of area 4°, The range of «, and w, is given by —7 + + 2k SD bh Ax, y—hydy) (18) ax Fig. 1.5 2D view ofa comb function Fig. 16. Three-dimensional view of a comb function After multiplying th ct the discrete version of the analog ima F(, y) with the 2D comb function, we F (m,n) = Fs») x comb(x, y, Ax, Avy (1.19) fimm= >> DE FRAY, kyAv) 8 r= KAN, y= bv) (1.20) 4 We know that convolution in spatial domain is equal to multiplication in the frequency domain and vice versa. For the purpose of analysis in the frequency dor and the 2D comb fun. he Fourier transform of the signal f(x,y) is f(y, 2). ‘The Fourier transform of 2D comb function is another comb function which is represented by n, fet us take the Fourier transform of the input analog comb(), 22) = F7(comb(x, y, Ax, Avy) (21 11s > q comb(®,%) = 33> dX 4% Ree, (1.22)Introduction to Image-Processing System 7 comb(02,, 2) (1.23) Now the spectrum of the 2D comb function is convolved with the spectrum of the analog image which is given by £1 > comb (9, %) F (4,2) = FM), %) & comb (My, 2) (2a) Substituting Eq. (1.22) in Eq, (1.24), we get q 5: 4 1.25 x (1.28) >) 7 8 P F(a, 0g) F = (Q.2, 2, ==. (ped P= QMO YY | Ie pam Upon convolving the spectrum of the signal with the spectrum of the comb function, we get F(ej.623) nw Yo Qk) VY Se 1 pomo0 gs As summation is a Linear operator, interchanging the order of summation, we get (127) cot G0 11 See ee F034.) =e @, —k. Op —DdLk F(x.) ao YY x FO, ~k, 2 — De {« 4 (1.28) Equation (1.28) resembles Eq. (1.17). 1.2.2 Retrieving the Image from its Samples We know that discreteness in one domain leads to periodicity in another domain. Hence sampli tial domain leads to periodic spectrum in the frequency domain, which is illustrated in Fig. 1.7. In onder to retrieve the original sampled spectrum, the following conditions have to be Way > 220 where us,, = and 20,9 is the bandwidth of the spectrum in the wy direction, a Similarly, yy > 2g (130) where ,, s and 24,9 is the bandwidth of the spectrum in the 2, direction. The condition given in ) Eqs. (1.29) and (1.30) implies that the sampling frequency should be greater than twice the maximum signal frequency, which is generally termed the sampling theorem. Here, 2st, and 2,9 are called Nyguist rates.8B Digital Image Processing A low-pass filter is normally employed in order to extract the desired spectrum. The transfer function of the low-pass filter is given as follows: shes (uise2) € region of support H( yy) = |S OP? (3b 0 otherwise ‘The region of support is indicated as # in Fig. 1.7. The continuous image can be obtained from the sampled spectrum by multiplying the sampled spectrum with the low pass filter which is given as Pj, 2p) = H (oy, 2) F(w1, 4) (1.32) By taking inverse Fourier transform, we get the continuous image as Foxy) = FF.) (1.33) Fig.1.7 Periodic spectrum of the sampled image 1.2.3 Violation of Sampling Criterion In this section, let us discuss the consequences of violation of the sampling criterion. Violation of sampling criterion given in Eq. (1.29) and (1.30) leads to aliasing. Aliasing basically occurs due to under-sampling Violation of sampling criterion given in Eq, (1.29) leads to overlapping of the spectrum in the w direction, which is illustrated in Fig. 1.8, Here, ,, <2,q. Whereas W,, > 20Introduction to Image-Processing System 9 Fig. 1.8. Under-sampling along the w direction we10 Digital Image Processing Fig. 1.10 Under-sampling along the, and, directions 4c0s 22(2x + y) is to be sampled such that one can reconstruct the ignal from its samples without errors. Suggest a sampling scheme. Solution The image is given by f(x, ») = 4eos 2x(2x + y). The given image can be written as PrtQety) 4 9 /Aaxty) Fs y)= 4x —$ $Y aff rtn 4 Peer] (34) Taking Fourier transform of the image, we get (wp 9) = 2[B(i0y + 2, 29 + 1) + Bory — 2, wp — 1} (35) This implies 2 = 2 and wg = 1. To avoid aliasing, w,, > 2.049 and w,, > 2.019. In this ease, w1, > 4 1 and w,, > 2. Also, Av and Ay should be smaller than the Nyquist sampling rate. That is, Ax < 1 x0 and Ay <57— . Arshould betess than 0.25 and Ay shouldbe less than 0.5. Choosing Av=0.2and Ay 7 0.4, aliasing can be avoided. This corresponds to,, =! =! 5 and w,, === 25 The sampled spectrum is given by Ar 0.2 Ay 04 Fo 2)= eye DS YD X (ej hogs 2 he) (1.36) 1 ote Substituting the values of ty, w3, We get F(aw)=125 SY Xl 25) (1.37) 101 Substituting the expression of F(., 3) from Eq, (1.37) in Eq, (1.35), we get F(oy,09)=25 SS YD 6, +2 Sk, wy 41-251) +6, -2 B00 5k, 1-251)Introduction to Image-Processing System 11 Example 1.2 _An image is given by the 2D function f (x, y) = 2cos [2s(4x + 6] which is sampled on an infinite grid of points with sampling intervals Ax = 0.1 and Ay = 0.2, in the x and y directions respec- tively. Determine the () Fourier transform of the sampling function (i) Fourier transtorm of the sampled image before low-pass filtering (ii) Fourier transform of the image after it has been low-pass filtered. (iv) reconstructed image In order to reconstruct the original image from sampled data without distortion, what are the ‘maximum intervals Ax and Ay that can be used, and the minimum and maximum bandwidths of the low-pass reconstruction fiers that can be used? Solution The sampling frequency along the x-direction is given by w,, a 10 The sampling frequency along the y-direction is given by ws, (i) The Fourier transform of the given 2D function f(x. F (ys 2g) = Ol, —4, ty — 6) + (0, +4, wry +6) (38) Also, Flo}, 09) is zero for fu] 4. o9|> 6 . This implies 9 —4, yo 6. The spectrum F(.!,, w2) will be periodic due to sampling and it is diagrammatically represented in Fig. 1.11, The periodicity along the wo, axis is 10, and along the w, axis is 5. In Fig. 1.11, #8 represents the sampling interval along the x and y direction. Here the symbol @ represents the occurrences of Fiuiy, w2). The letter indicates the region of support of the low pass filter. The points which lie within the region of support of the low-pass filter alone will be considered for reconstruction Fig. 1.11 Representation of Fy si)12. Digital mage Processing The Fourier transform of the sampling function is given by 50.5 Y3 au, 104, 51) (139) Ate toto (ii) The Fourier transform of the sampled image before low-pass filtering is given by 22) =50 S22 Huy — 410k, we 6-51) +64 +4 10k, wy +6-S1) (1.40) Pe (iii) The low-pass filter has a frequency response 1 H(y.82)= | 22 aap 0, otherwise The Fourier transform of the image after it has been low-pass filtered is given by the equation F (ty, sig) = 8 —4, ey D+ 8(ry +4, 029 +) (1.42) (iv) The reconstructed image is obtained by taking the inverse Fourier transform of the equation (4.422) = Hy —4, 0 I) + 6(4, +4, FD to Fy) =2cos[27(4x+ y) (1.43) We ean expect aliasing effect as the sampling rate is lower than the Nyquist frequency in the y axis According to the Nyquist riterion, 1. 9.5, . Hence, Ax < 0.125 and similarly, Ay < 0,0833 = Ax “ 2 Using low-pass filters, the maximum values for Ax and Ay are 0.125 ~ € and 0.0833 ~ € where £ > 0 positive real number can be taken as an arbitrarily smal Example 1.3 Suppose an image of dimension 4 x 6 inches has details to the frequency of 400 dots per inch in each direction. How many samples are required to preserve the information in the image? Solution The bandwidth is 400 in both the directions; therefore samples must be taken at 800 dots per inch in both the dimensions. A total of 4 x 800 x 3 x 800 = 7680000 samples are needed. Quantisation involves representing the sampled data by a finite number of levels based on some eriteria such as minimisation of quantiser distortion. Quantiser design includes input decision level, output representation level and the number of levels. Quantisers can be broadly classified into two types, namely, (i) sealar quamtis- crs and (ii) vector quantisers. The classification of quantisers is shown in Fig. 1.12 Examples of uniform scalar quantisers are midtread and midrise quantisers. An example of a non-uniform scalar quantiser is the Lloyd-Max quantiser. A detailed description of scalar and veetor quantisers is given in the chapter on image compression.Introduction to Image-Processing System 13 ‘Quantiser classification Scalar quantiser Vector quantiser tS Uniform quantiser —_Non-unitorm quantiser Fig. 1.12 Classification of quantiser 4 RESOLUTION Resolution gives the degree of distinguishable details, Resolution can be broadly classified into (i) spatial resolution, and (ii) gray-level resolution, i) Spatial Resolution Spatial resolution is the smallest discernible detail in an image. Spatial resolution depends on the number of pixels. The principal factor determining spatial resolution is sampling. (i) Gray-level Resolution _Gray-level resolution refers to the smallest discernible change in the gray level Gray The use of insufficient number of gray levels in smooth areas of the digital image is termed false contouring. The MATLAB code that illustrates the concept of false contouring is shown in Fig. 1.13 and the corresponding results are shown in Fig. 1.14. aay levels evel resolution depends on the number of sthis program illust; te clear all lose all a-imread('tigerpub. jpg") (a),titie(*original image’) 128 gray levels ouring (28), title(*"Image with 12 level') sing 64 levels figure, imshow (grayslice (a, 64) ,gray title(*'Image with 64 gray lev using 32 gray levels figure, imshow (gzayslice (a, gx: 2), gray (32)), level figure, imshow ( (a, 16) gray (16)), title(*image with 16 gray level’) levels ice (a,8),gray(8)), ray level") Fig. 1.13 False contouring MATLAB code14. Digital Image Processing Image with 128 gray level Original image Image with 64 gray level Image with 32 gray level Image with 8 gray level Fig. 1.14 illustration of false contouring Snell ‘The Human Visual System (HVS) is one of the most complex systems in existence. Our vistal system allows, us to organise and understand the many complex elements in our environment. The visual system consists of, an eye that transforms light into neural signals, and the related parts of the brain that process the neural sig nals and extract necessary information. The human eye serves to project and convert light into neural activity Light enters the cornea, passes through the aqueous humor, then through the lens into the vitreous humor, and finally onto the photoreceptors located at the back of the retina. The ciliary muscles are responsible for accommodating the lens so as to focus the light rays onto the fovea, the region of the retina containing the greatest density of cones, and thus the high acuity for spatial and colour vision, 1.5.1 Anatomy of the Human Visual System. The human eye is a slightly asymmetrical sphere with an approximate sagittal diameter or length of 24 to 25 mm and a transverse diameter of 24 mm. It has a volume of about 6.5 ce. The view of a human eye is, shown in Fig. 1.15Introduction to Image-Processing System 15 A black-looking aperture, the pupil allows light to center the eye (it appears dark because of the absorbing pigments in the retina). The pupil of the eye contracts when exposed to bright light. The effect is to reduce the amount of light that falls on the retina. However, as time passes, the retina adapts to the new level and the Pupil retums to its original size. The pupil can control the amount of light entering by about a factor of 30, A coloured circular musele, the iris, which is beautifully pigmented, gives us our eye colour (the central aperture of the iris is the pupil). This circular muscle controls the size of the pupil so that more or less light, depending on conditions, is allowed to enter the eye. Eye colour, or more correctly, iris colour is due to variable amounts of eumelanin (brownvblack melanins) and pheomelanin (red/yellow melanins) produced by melanocytes. More of the former is present in brown-eyed people and of the later in blue-and green-eyed people. The horizontal section of the human eye is shown in Fig. 1.16. A transparent external surface, the cornea, covers both the pupil and the iris. This is the first and most powerful lens of the optical system of the eye and allows, together with the erystalline lens the production of a sharp image at the retinal photoreceptor level. The cornea and lens act together like a camera lens to focus aan image on the retina at the back of the eye, which acts like the film. ‘The “white of the eye’, the sclera, forms part of the supporting wall of the eyeball. ‘The sclera is continuous with the comea. The sclera is a nearly spherical shell with a radius of 11 mm and is I-mm thick. At the front of the eye, the sclera merges into the transparent cornea. Fig.1.15 View ofthe human eye Wns, Conjunctive Ciliary body Pigment’ epithelium Sdlera Choroid Lemina eribose Macula lutea Sheath Fig.1.16 Horizontal section of the human eye Optic nerve’16 Digital Image Processing The cross-sectional view of the eye shows three different layers. They are (i) the external layer formed by the sclera and comea, (i) the intermediate layer, divided into two parts—anterior (iris and ciliary body) and posterior choroids, (ii) the internal layer or the sensory part of the eye, the retina. There are three chambers of fluid—anterior chamber (between comea and itis), posterior chamber (between iris, zonule fibers and lens) and the vitreous chamber (between the lens and the retina), The first ‘wo chambers are filled with aqueous humor whereas the vitreous chamber is filled with a more viscous fluid, the vitreous humor, ‘The sagittal section of the eye also reveals the sclera Choroid Jens which is a transparent body located behind the iris. The lens is suspended by ligaments (called zonule fibers) attached to the anterior Cornea pottion of the ciliary body. The contraction or relaxation of these ligaments as a consequence of ciliary muscle actions, changes the shape of the lens, a process called accommodation, that allows us to form a sharp image on the retina, ‘The vertical section of the human eye is shown in Fig. 1.17. Light rays are focussed through the transpar- cent comea and lens upon the retina. The central point for image focus (the visual axis) in the human retina is the fovea. Here, a maximally focussed image initiates resolution of the finest detail and direct transmission of that detail to the brain for the higher operations needed for perception. Slightly more nasally situated than the visual axis is the optic axis projecting closer to the optie nerve head. The optic axis is the longest sagittal distance between the front or vertex of the cornea and the furthest posterior part of the eyeball. It is about the optic axis that the eye is rotated by the eye muscles. The neural signals from the photoreceptors are processed in the retina by other cell types, and also passed directly over the optic nerve, together with the outputs from, the other retinal cell types, to the brain. 1.5.2 Photopic and Scotopic Vision The rods are s« dete the Ciliary body Fig.1.17 Vertical section of the human eye uation and are responsible for scotopic vision. The order of lie in fine with itive to very low illun imum table luminance is about I nL. The cones, which are very tightly packed in the fove ral axis and are responsible for the most acute vision, photopic vision. The mi cones is of the order of a microlambert. Rods are used to see at night or under very low illumination. Colour ion, also known as photopic vision, is provided by the cones, of which there are three distinct classes, cach containing a different photosensitive pigment. The three pigments have maximum absorptions at 430, ‘530 and 560 nm and the cones are often called blue, green and red. The cones provide colour vision that can distinguish fine wavelength changes 1.5.3 Brightness and Contrast Brightness | concept or sensation associated with the amount of light stimulus. le from which the psycholo; Light source intensity depends upon the total light emitted and the size of the solid aIntroduction to Image-Processing System 17 Fig.1.18 Ilustration of simultaneous contrast it is emitted. Two sources of equal intensity do not appear equally bright. Luminance, the intensity per unit area, is a psychophysical property that can be measured. hasise the difference in luminance of objects. The perceived brightness background which is illustrated in Fig, 1.18. In Fig. 1.18, the small The term contrast is used to em of a surface depends upon the loc conirast Colours tend to look darker and smaller against white, and lighter and larger against black. This fact is illustrated in Fig. 1.19 (Plate 1). The apparent changes in size arise from the flare in the eye's opties, which causes to the neighbouring dark areas. It is to be noted that simultaneous light from the bright areas to be scattered the same colours look different. Also, it can make different colours look the same contrast ean m 1.5.4 Laws of Physics Associated with Vision The important laws associated with vision are given below. Weber's Law First, Ict us define the term Just Noticeable Difference (IND). Just noticeable difference is the magnitude of the smallest stimulus that is perceivable. For luminance *L’, the just noticeable difference AL when making the experiment of having a uniform background with intensity L and a spot intensity L + AL can be written as AL=KL (aay where K is a constant and Eq, (1.44) is known as Weber's law. Weber's law holds over a large rang sity magnitudes. The visual system is not linear but logarithmic in amplitude sensitivity. The amount of light coming from a surface is the product of the illuminance and the surface reflectance. of inten- Steven’sLaw Steven's law models how different types of stimuli are perceived by humans. If is the physi- cal stimulus and / the perceived sensation then the mathematical form of Steven's law is given by Tso" (as) For a point with intensity L against a black background, the value of n is 0.5 Steven's law is taken into consideration when digitising image samples by introducing a non-linearity prior to quantisation which is given by Joa (1.46)18 Digital mage Processing where + is typically slightly over 2. Eq. (1.46) dese bes transformation of luminance into intensity 1. The process of using Eq. (1.46) at the input of the image-processing system and the inverse at the output side is known as gamma correction. The energy of the spot is important for detection. This is given by Bloch’s law and Ricco’s law. Ifthe of the spot is 4 and the duration of the stimulus is T then the energy deviation AZ is given by AE=Ax Tx AL (1.47) Bloch’s Law _Bloch’s law is given by Tx AL=C (1.48) where Cis a constant which is valid fora duration time shorter than about 0.1 second. Ricco’s Law _Ricco’s law is given by {x AL=C’ (1.49) where C’is a constant, wiich holds good for stimulus sizes with a diameter of less than about 10 minutes of an are in the visual angle. Piper's Law The mathematical form of Piper's law is given by VaxAL=c" (1.80) where C”’is a constant which is in accordance with experiments for stimulus sizes between 10 minutes of an are and 24 degrees of an are. 1.5.5 Machband Effect The Machband de: surfaces with different luminance. The Machband effe over the width of the bar. However, the visual appea n subconsciously increases the contrast between two is illustrated in Fig. 1.20. The intensity is uniform ce is that each strip is darker at its left side than its nd its surrounding creates the Machband effect shows that brightness is not monotonic funetion of luminance. right. The spatial interaction of luminance from an object Fig. 1.20 Machband effect Machbanding is caused by lateral inhibition of the receptors in the eye. As the receptors receive light, they raw light-sensitive chemical compounds from adjacent regions, thus inhibiting the response of receptors in those regions. Receptors directly on the lighter side of the boundary can pull in unused chemicals from the darker side, and thus produce a stronger response, Receptors on the darker side of the boundary, however, produce a weaker effect because of that same migration,Introduction to Image-Processing System 19 The human eye senses the visible spectrum using the combination of rods and cone sensors in the retina, Rod cells are better for low light vision, but can only sense the intensity of light, whereas the cone cells are sensitive to colour vision. Three types of cone cells exist in the human eye, each being more sensitive to either short or medium or long wavelengths of light. The set of signals at all the three cone cells describes the range of colours one sees with the eyes rene easy Digital images can be broadly classified into two types and they are (i) raster image, and (ii) vector image. 1.6.1 Raster Image or A raster image file is generally defined as a rectangular array of regularly sampled values known as pixels Scanned graphics and web graphics are the most common forms of raster images, Raster images are mapped to grids which are not easily scalable, A raster image is resolution dependent because it contains a fixed number of pixels that are used to ereate the image. Since there are a fixed and limited number of pixels, a raster image will lose its quality if it is enlarged beyond that number of pixels as the computer will have to “make up’ for the missing information, This fact is illustrated in Fig, 1.21 which shows the portion of the raster image being zoomed by a factor of 3 and 24, When the image is zoomed by a factor of 24, the clarity is lost. Bitmaps are used for photorealistic images, and therefore, involve complex colour variations, Raster images can show well the gradations of colour and detailed images such as photographs. Also, they can be acquired by optical scanners, digital CCD cameras and other raster-imaging devices. The spatial resolution of a raster image is determined by the resolution of the acquisition device and the quality of the original data source. map Image Fig. 1.21 Zooming of araster image ‘Common raster image formats include BMP (Windows Bitmap), PCX (Paintbrush), TIFF (Tag Interleave Format), JPEG (Joint Photographics Expert Group), GIF (Graphics Interchange Format), PNG (Portable Network Graphics), PSD (Adobe Photoshop) and CPT( Corel PhotoPaint) 1.6.2 Vector Image Aw is defined by objects which the computer. A vector ly defined and hene ctor im: fe made of lines and curves that are mathematically defined in aan have various attributes such as line thickness, length and colour. Vector images they scalable. This implies that vectors can be printed at are mathemati20 Digital Image Processing any size, on any output device, at any resolution, without losing the detail and without altering the resolu- tion of the image. Fig, 1.22 shows the zooming of a vector image. The vector image is zoomed by a factor of three and twenty-four. From the figure, itis evident that vector images can be scaled by several factors without altering the resolution of the image. Vector images are thus suitable for typography, line art and illustrations. Za CS s yn dy Fig. 1.22 Zooming ofa vector image Wtctatasd s, (ii) gray’ Images can be broadly classified under four categories: (i) Black and white or binary imag images, (iti) colour images, and (iv) multispectral images Binary images take only two values, ic., either ‘0’ or I’. The brightness graduation cannot be differe in the binary image. The binary image representation is illustrated in Fig. 1.23. Mother Teresa's image in Fig. 1.24 is an example of a black-and- white image. A grayscale image can be converted to a black- and-white or binary image by the thresholding operation. The detailed description of the thresholding operation is given in Chapter 5. Geometric properties of an object, like the location of the centroid of the object or the orientation of the object, can be easily extracted from a binary image. 1 Binary Images white 255 black 0 ig. 1.24 Mother Teresa in black and whit Fig. 1.23 Binary image representation Fig. 1.24 Mother Teresa in black and whiteIntroduction to Image-Processing System 21 1.7.2 Grayscale Images Grayscale images contain only brightness information. Each pixel val ‘an amount or quantity of light. The white 255 black 0 Fig. 1.25 Grayscale image representation Fig. 1.26 Mother Teresa in grayscale 1.7.3 Colour Image A colour image has three values per pixel and they measure the intensity and chrominance of light. Each pixel is a vector of colour components. Colour images can be modeled as three-band monochrome image data, where each band of data corresponds to a different colour. The actual information stored in the di image data is the brightness information in each spectral band. Common colour spaces are RGB (Red, Green and Blue), HSV (Hue, Saturation, Value), and CMYK (Cyan, Magenta, Yellow, Black). Colours ean be rep- resented with three colour components as shown in Fig. 1.27 (Plate 1), so the typical uncompressed data rate of a colour image is three times the data rate of a grayscale image. Kalpana Chawla’s image in Fig. 1.28 (Plate 1) is an example of an RGB image. Colour images usually consist of three separate image representations called colour planes, with pixel values in each plane corre- sponding to the intensity of a certain colour at a specific point in the image. Each colour plane consists of an array of pixel values similar to that of the grayscale representation. The most popular system is RGB, where three colour planes are used to represent the intensity of red, green and blue in the scene. 1.7.4 Volume Image A three-dimensional image is an example of volume imas cal imaging equipment in which the individual data points are called “voxels”. Voxels stands for volume pixels. A CAT scan is an example ofa volume imag. The volume image can be obtained from some medi- 1.7.5 Range Image Range im: a spe na known refer sof digital im ice frame and a visible point in the sere are also referred as depth im expresses the distance the 3D struc- reproduc ture of a scene. Rang22. Digital Image Processing 1.7.6 Multispectral Image mages of the same object taken in different bands of visil ctrum. Images acquired for remote-sensing applications are ge in nature. Multisp 's typically contain information outside the normal human perceptual range. This includes infrared, ultraviolet and other bands ctral images are le or infrared ual form by mappin ctral bands to RGB components. A hyper-spectral image is a set of 224 images, which are referred as bands, at a specific location on the cearth’s surface. These hyper-spectral images are taken froma Moderate Resolution Imaging Spectroradiometer through Terra and Aqua satellites. They are used by scientists on the earth to study and analyse dynamics and processes occurring in the earth's surface. All the 224 bands are pictures at the same location ET Trtig. a unas et The different elements in an image-processing system are (i) image-acquisition element which involves image sensors like CCD sensors, CMOS sensors, and image scanners, (ii) image-storage devices, (iii) image- processing elements, and (iv) image-display devices, 1.8.1 Image Sensor and Acquisition The term image acquisition refers to the process of capturing real-world images and storing them into a computer. Conventional silver-based photographs in the form of negatives, transparencies or prints ean be scanned using a variety of scanning devices. Digital cameras which capture im tly in digital form are more popular nowadays. Films are not used in digital cameras. Instead, they use a charge-coupled device or CMOS device as the im: tri An image sensor is a 2D array of e clements that convert photons to el 1e sensor. A typical solid ents, some form of charge-transport mechanism, and an output circuit The photosensitive sites convert the incoming photons into electrical charges and integrate these charges into a charge packet. The charge packet is then transferred through the transport mechanism to the output converted into a measurable voltage. The types of photo-sensing elements used in solid- trons. Most of the or consists of ate i iclude photodiodes, MOS capacitors, Schottky-barrier diodes and photoconductive layers. The output circuit typically consists of a floating diffusion and source-follower amplifier. In practical applications, image sensors are configured in a one-dimensional (Linear devices) or a two-dimensional (area devi es) manner. Image-Sensor Terminology Some of the definitions of the most commonly used terms in solid-state image sensors are given below: () Charge-coupled Device (CCD) CCD is a charge-transfer device that collects light in pixels and then uses clock pulses to shift the charge along a chain of pixels. (ii) Dark Current ‘The charge of the signal collected by the pixel in the absence of light is termed dark current. (ii) Photo Site Photo site is the portion of the silicon that functions as a light-sensitive area. (iv) Pixel Pixel is a discrete photosensitive cell that collects and holds a photo chargeIntroduction to Image-Processing System 23 (v) FillFactor A pixel is divided into a sensing portion and a read-out portion, Fill factor is the ratio of the sensing area to the total area (vi) Quantum Efficiency Quantum efficiency is the ratio of the photon-generated electrons that the pixel captures to the photons incident on the pixel area. 1.8.2 CCD Sensor Charge-coupled devices (CCDs) find wide applications in almost all digital image-acquisition devices. Invented in the late 1960s by researchers at Bell Labs, these were initially conceived as a new type of com- puter memory circuit, and were demonstrated in 1970 for that facility. It soon became apparent that CCDs have many other potential applications, including that of signal processing and imaging, the latter because of silicon’s light sensitivity that responds to wavelengths less than 1.1 ym. (The visible spectrum falls between 0.4 jm and 0.7 pam.) The CCDs early promise as a memory clement has since disappeared, but its superb abil- light has turned the CCD into a premier image sensor. Like the integrated circuits (ICs), CCDs , which are processed in a series of elaborate steps that define various functions it. Each wafer contains several identical devices (chips), cach capable of yielding a functional few chips are selected based on applications. The selected chips, based on a variety of preliminary screening tests, are then cut from the wafer and packaged into a carrier for use in a system Operation of CCD Sensor -coupled device (CCD) is basically a series of closely spaced MOS ;pacitors. CCD imaging is Step 1:Exposure In this step, the sensor is exposed to the incident light, Upon exposure, light is converted into an electronic charge at discrete sites called pixels. Step 2: Charge transfer Charge transfer moves the packets of charge within the silicon substrate. ‘Step 3: Charge-to-voltage conversion and output amplification Converting Light (Photons) into Electronic Charge An image is acquired when incident light, in the form of photons falls on the array of pixels. The energy associated with each photon is absorbed by the silicon and causes a reaction to occur. This reac- tion yields an electron-hole charge pair. Fig. 1.29 Me ee ee illustrates the photon interaction with silicon. The overveg number of electrons collected at each pixel is lin- ‘im early dependent on the light level and exposure . time and non-linearly dependent on the wavelength = [SF 1 of the incident light. cs Conoco Many factors can affect the ability to detect a photon. Thin films of intentionally grown x raion and deposited on the surface of the silicon durin e toabsorb or reflect d light, Photons are absorbed at different depths in the ‘scon silicon depending on their wavel substrate instances during which photon cannot be detected because of the location within the silicon where they were created. Fig. 1.28 Photon interaction with siicon24 Digital Image Processing Potential Wells and Barrier CCDs follows the basic physics of basic metal-oxide semiconductor (MOS) devices. A CCD MOS structure simply consists of a vertically stacked conductive material (doped polysilicon) ‘overlying a semiconductor (silicon), which is separated by a highly insulating material (silicon dioxide). By applying a voltage potential to the polysilicon or “gate” electrode, the electrostatic potentials within the silicon can be changed. With an appropriate voltage, a potential “well” can be formed which is capable of collecting the localised electrons created by the incident light, Fig, 1.30 shows the potential well and barrier. The electrons can be confined under this gate by forming zones of higher potentials called barriers surround ing the well, Incident light LELLETEELTT si v WV v icon Suede | | | Palyslicon Potential barrier Potential barrier Potential well Phologenrated eee Silicon substrate Fig.1.30 Potential well and barrier Depending charge on the voltage, each gate can be biased to form a potential well or a barrier to the integrated Charge Transfer Once charge has been integrated and held locally by the bounds of the pixel architecture, one should have a means of getting that charge to the sense amplifier that is physically separated from the pixels. 1.8.3 Four-Phase CCD phase CCD structure is illustrated in Fig. 1.31. Application of bias voltage to the gate of the MOS or results in the creation of a localised potential well in the semiconductor. The photo-generated the poter By varying the gate ss in a systematic and sequential manner, the char from under one electrode to the adja ‘rode. This provides a simple m through the CCD shift register. A CCD array consists of a series of column registers. The charge packets are confined within each row or column by channel stops. At the end of each column there is a hi packet can be transported nism for movil eIntroduction to Image-Processing System 25 Fig.1.31 Four-phase CCD register. The charged packets are shifted, one line at a time, into the horizontal shift register. The entire line then clocked out from the horizontal shift register to the output amplifier before the next line the enters horizontal register. ‘One important requirement fora good CCD operation is to maintain high charge transfer efficiency (CTE) in the CCD shift register. CTE is a measure of the percentage of electrons that are successfully transferred from under one electrode to under the adjacent electrode. Typically, CTE needs to be no less than 0.99999. Limited time available for charge transfer during high-speed operation, and potential obstacles (barriers and wells) that arise from the device design and processing, can cause an incomplete charge transport and reduction of CTE, 1.8.4 CCD Formats Image sensing can be performed using three basic teh Point scanner nhigues—point scanning, line scanning and area scanning Point Scanning Point scanning uses a single-cell detec- tor or pixel (picture element). An image can be scanned by sequentially detecting scene information at discrete X, ¥ coordinates as illustrated in Fig, 1.32, Some of the advan- tages of this approach include high resolution, uniformity ‘of measurement from one site to another and the simplicity of the detector Disadvantages of the point-scanning mechanism include registration errors due to the X—Y movement of the scene or detector, frame-scanning rates because of the repeated number of exposures and system complexity due to the X-Y movement. Fig. 1.32 Point scanning26 Digital Image Processing Line Scanning In line scanning, an array of single-cell detectors Line scanner can be placed along a single axis such that scanning thereupon takes place in only one direction, In this case, a line of information from the scene is captured and subsequently read out of the device before stepping to the next line index which is shown in Fig. 1.33. The physical length of a linear CCD scanner is limited only by the size of the starting silicon wafer used to fabricate the device. This limitation can be overcome at times by mounting several linear CCDs end to end to inerease the overall length. Line scanning greatly improves the scan time over point scan- ning. Other benefits include reasonably high resolution and less complicated scanning mechanies. However, the pixel spacing and size in one direction limit the resolution. The measurement a racy at each pixel has finite non-uniformities that must be occs sionally factored out with the system. Scan times, of the order of several seconds or minutes, are still unsuitable for many applics tions and the costs of linear CCDs are considerably higher than I detectors. The finite number of CCD chips on each silicon wafer and the resulting yield loss dictate the costs from processing variations. Fig.1.33 Linescanning ‘Area scanner Area Scanning _ two-dimensional array of detectors can be ere- ated such that the entire image can be captured with a single expo- sure, thus eliminating the need for any movement by the detector or scene. The area-scanning mechanism is illustrated in Fig, 1.34. Area scanners are capable of producing the highest frame rates with the greatest amount of registration accuracy berween pixels, Also, the system complexities are kept to a mininum, However, resolution is now limited in two directions, Other disad- ‘vantages include generally lower signal-to-noise and higher cost aris- ing due to the fact that fewer deviees can be placed on a wafer with the yield being inherently lower fora number of reasons. Fig.1.34 Area scanning 1.8.5 The Architecture of CCD There are two different types of CCD architecture and they are (i) full-frame CCD, and (ii) frame-transfer CCD. Full-frame CCD Full-trame CCDs have the simplest architecture and are the easiest to fabricate and operate. ‘These consist of a parallel shift regis serial shift register and a signal-sensing output amplifier as shown in Fig. 1.35, Images are optically projected onto the parallel array that acts as the image plane. The device takes the scene information and partitions the image into discrete elements that are defined by the number of pixels, thus ‘quantising” the scene. The resulting rows of scene information are then shifted in a parallel fashion to the serial register, which subsequently shifts the row of information to the output as a serial stream of data The process repeats until all rows are transferred off the chip. The image is then reconstructed as dictated by the system,Introduction to Image-Processing System 27 Since the parallel register is used for both scene detection and read-out, a mechanical shutter or synchronised strobe illumination mist be used to preserve the scene integrity The simplicity of the FF design yields CCD images with qualities of highest resolution and highest density Direction of Parallel shi I] Parallel clocks Frame-transfer CCD. Frame-transfer CCDs are much like full-frame CCDs, except that they have an additional identical parallel reg- Sorial ister called a stonige array which is not light- Clocks sensitive, The idea is to shift a captured scene Parallel register Serial register Output ‘from the photo-sensitive image array quickly to ‘amplifier the storage array. Read-out off the chip from the Fig. 1.35 Fullframe CCD storage register is then performed as deseribed in the full-frame device, while the storage array is integrating with the next frame. The advantage of this architecture is that it allows for faster frame updates due to a continuous or shut- terless operation. The resulting performance is compromised, however, because integration is still occurring during the image dump on the storage array, thus resulting in an image ‘smear’. Since twice the silicon area is required to implement this architecture, frame transfer CCDs have lower resolution and cost much higher than full frame CCDs. Advantage of a CCD Image Sensor The quality of the image obtained using a CCD image sensor is high due to optimised photo detectors. Also, CCD image sensors do not introduce noise or non-uniformity. Disadvantage of CCD image Sensor CCD image sensor power requirement is high-due to high speed shifting clocks. Also, the frame rate is limited due to analog serial read-out. Yet another drawback of a CCD image sensor is its inability to integrate with other camera functions such as clock drivers and timing logic. Signal processing on the same chip is another drawback of a CCD image sensor. 1.8.6 CMOS Image Sensor CMOS image sensors have been in existence since the late 1960s. The basic structure of a pixel consists of a light-sensitive element and a MOS transistor acting as a switch. The early design had a photodiode as the photo-sensing element. However, due to the issues of large fixed-pattern noise (PPN), scalability, and difficulty to obtain fast read-out, MOS sensors have lost out to CCDs as the sensor of choice. In the past few years, CMOS sensors have attracted a lot of attention from the imaging industry. The demand for low-cost, low-power, and compact imaging systems and the significant improvement in device scaling are the major contributing factors to the renewed interest in CMOS sensors. Advantages of CMOS Image Sensors The advantages of CMOS, image sensors are oltage operation and reduced power consumption «© the ability to integrate timing, control, and image processing circuitry onto the sensor chip © random access of the image data that allows electronic windowing, pan and zoom