ICDCCom-2014

Microscopic Image
Analysis for Identification
of Indian Herbal Plants
Date: 13th September, 2014

By
Prof. Bhupendra Fataniya

Electrical Engineering Department

Electronics & Communication Engineering Program
Institute of Technology,
Nirma University
Ahmedabad-382481
 Outline
• Introduction
• Problem Statement
• Introduction to Plant
• Literature Review
• Proposed Method
• Results
• Future work

13-09-2014 2
Introduction to Microscopic Image Processing
• The optical microscope has seen steady improvement and increasing use in
biomedical research and clinical medicine as well as in many other fields.
• Allowing us to peer into spaces much too small to be seen with the
unaided eye
• Digital image processing greatly enhances the process of extracting
information about the specimen from a microscope image.
• Digital processing can be used to extract quantitative information about
the specimen from a microscope image
• For that reason, digital imaging is steadily becoming an integral part of
microscopy.

13-09-2014 3
Digital Microscopy

Microscope Image Processing, by Qiang Wu, Fatima A Merchant and Kenneth R. Castleman,
Elsevier,2008
13-09-2014 4
 Motivation

• This is the era of low-cost computer hardware

• Allows diagnosis/analysis of images quantitatively
• Compensate for defects in imaging process (restoration)
• Speed & reduced specimen irradiation
• Avoidance of human error
• Consistency and repeatability

13-09-2014 5
 Manual Optical Microscopy
Advantages
• Relatively inexpensive
• Each particle individually examined - detect aggregates, 2D shape, colour,
melting point etc.
• Permanent record - photograph
• Small sample sizes required
Disadvantages
• Time consuming - high operator fatigue - few particles examined
• Very low throughput
• No information on 3D shape
• Certain amount of subjectivity associated with sizing - operator bias

13-09-2014 6
 Automatic Image Analysis of Microscopic
Image
Advantages
• Faster and less operator fatigue than manual
• No operator bias

Disadvantages
• Can be very expensive
• No human judgement retained e.g. to separate out aggregates, select or reject
particles etc. (unlike semi-automatic)

13-09-2014 7
 Applications
1. More Sugar is consumed by tumor cells. They
Illuminate in MRI. Detect change in Brightness
in the image!

13-09-2014 8
 Medical Applications

• Assist a physician to reach a

diagnosis.
• Construct 2D, 3D anatomy models of
the human body.
• Analyze the image to extract useful
features.

13-09-2014 9
 Research Fields Examples of Applications

Medicine Tumor recognition, quantification of change and/or

deformation of anatomical structures (e.g., endocardial
contour of left ventricle of heart, corpus callosum),
morphometric analysis for diagnosis (e.g., multiple sclerosis
and Alzheimer's disease), numerical analysis of
chromosomes, identification of genetic pathologies,
laparoscopy, genetic studies of dentofacial morphology, ...

Biology Morphometric-based evolution comparison, taxonomy,

interplay between form and function, comparative anatomy,
cytology, identification and counting of cells (e.g., white blood
cells), characterization of cells and nuclear shapes, growth
and shape modifications, analysis of human gait, analysis of
electrophoretic gels, microscopy, ...

Shape Analysis and Classification Theory and Practice, by Luciano da

Fontoura Costa and Roberto Marcondes Cesar Jr., 2001 CRC press[4]
13-09-2014 10
 Research Fields Examples of Applications

Physics Analysis of particle trajectories, crystal growth, polymers,

characterization of star clusters in astronomy, several types of
microscopy, ...

Engineering Semiconductors, quality control, danger detection, machine

interpretation of line drawings, computer aided design of
mechanical parts and buildings, automation, robotics, remote
sensing, image and video format standards, spatial
exploration, ...
Security Fingerprint/face/iris detection, biometrics, human gait,
signature verification, ...

Agriculture Harvest control, seed counting and quality control, species

identification, fruit maturation analysis, ...

Shape Analysis and Classification Theory and Practice, by Luciano da

Fontoura Costa and Roberto Marcondes Cesar Jr., 2001 CRC press[4]
13-09-2014 11
 Problem Statement
• Microscopic image analysis for identification of Indian Herbal Plants
based on its unique characteristic features present in respective plant
images

13-09-2014 12
 Data base of Microscope Image
• Sample slide prepared from the powdered form of the herbal plant
using Microscope ( Pharmacy Department of Nirma University)
• Herbal plant
• Vasaka, Kurchi, Rhubarb, Licorice, Datura, Ephedra.
• Design an algorithm for automatic detection of plant
• Benefit to Pharmaceutical industries

13-09-2014 13
 Introduction to Plants

13-09-2014 14
 Datura

The main feature used to identify datura is the shape of

trichome. It is broad at the top and tapers towards its tip.
Also they are longer in length compared to trichomes of
other plants.
13-09-2014 15
 Vasaka

The trichome is bent at an angle as shown with yellow circles. Also,

the trichome is broader at the top and tapers towards its end.
13-09-2014 16
 Licorice

Large stonecells with larger area. The stonecells lie hidden under the
cell array and are clearly visible when the image is converted to a
binary image.
13-09-2014 17
 Rhubarb

Rhubarb crystal is small in size and has a hole in between. The

crystal is circular in shape

13-09-2014 18
 Kurchi

The crystals are smaller in size and have rhombic shape

13-09-2014 19
 Ephedra

The main feature used for detection is the crystal shape and intensity. The
crystals are orange in color.

13-09-2014 20
 Literature Review

Research Area References

Object Detection in Microscopic Images [14-16]
Features Selection for object classification [7,11,17,19,24,26]
Image Segmentation [ 5,6,12,21]
Microscopic Image Analysis [11,18,20,22,24,25]

13-09-2014 21
 Block Diagram for Image Analysis
Image
Image Acquisition Thresholding
Enhancement

Measurement
Labelling and Morphological
of various
Cropping operations
properties

Decision
making

13-09-2014 22
 Image Acquisition
• Microscope camera for Image Acquisition
• Sample slide of herbal plant images which is in powdered form

13-09-2014 23
 Image Enhancement
• Image enhancement is the process of enhancing the appearance of
an image or a subset of the image for better contrast or visualization
of certain features and to facilitate subsequently more accurate image
analysis.
Licorice Image Enhanced Image

After thresholding Object Detected

13-09-2014 24
 Segmentation using Thresholding
• Image segmentation techniques that is based on the grey level
histogram of an image

13-09-2014 25
 original image original image
original image

Threshold image
Threshold image Threshold image

Threshold value is 150

Threshold value is 100
Threshold value is 121

Segmentation on licorice image using thresholding

13-09-2014 26
 original image
original image original image

Threshold image
Threshold image Threshold image

Threshold value is 150

Threshold value is 118
Threshold value is 50

Segmentation on kurchi image using thresholding

13-09-2014 27
 Basic Global Thresholding
• Threshold midway between maximum and minimum gray levels
• Automatic algorithm ( iterative algorithm)
• Segment with initial T into regions G1 and G2
• Compute the average gray level m1 and m2
• Compute new T=0.5(m1+m2)
• Repeat until reach an acceptably small change of T in successive iterations is
smaller than a predefined parameter ΔT
• ΔT is used to control the number of iterations

13-09-2014 28
 Licorice Image
Original Grayscale

After Iterative Thresholding

Result of Iterative threshold and threshold value is 121.

13-09-2014 29
 Kurchi Image
Original Grayscale

After Iterative Thresholding

Result of Iterative threshold and threshold value is 118.

13-09-2014 30
 Morphological Image Processing
• Morphology operations involve a binary image, f, to
be processed and a structuring element, g
• Morphology operation
• Erosion
• Dilation
• Opening ( erosion followed by dilation)
• Closing ( dilation followed by erosion)

13-09-2014 31
 Applications of MP operators
• To extract the boundary
• Remove noise
• Opening followed by a closing operation
• Thinning
• Thickening
• Identify components
• Image Segmentation

13-09-2014 32
 after morphological

13-09-2014 33
 Licorice Image Enhanced Image

After thresholding Object Detected

13-09-2014 34
 Feature Representation and Description
• What is Feature?
• To describe the object in a meaningful manner
• Features are required for image recognition
• Object can be characterized by
• Natural features: brightness and texture
• Artificial features: derived features that are manipulated

13-09-2014 35
Characteristics of Good features
• Robustness
• Shift Invariance
• Rotation Invariance
• Size Invariance
• Discrimination
• Reliability
• Independence

13-09-2014 36
 Feature extraction
Task: to extract features which are good for classification.
Good features: • Objects from the same class have similar feature
values.
• Objects from different classes have different values.

“Good” features “Bad” features

13-09-2014 37
 Area
• The simplest approach to estimate the area of an object is
to count the number of pixels representing that shape.
• Supposing that g is a binary image where g(p, q) = 1 for
shape pixels and g(p, q) = 0 for background pixels,

Shape Analysis and Classification Theory and Practice, by Luciano da Fontoura

Costa and Roberto Marcondes Cesar Jr., 2001 CRC press

13-09-2014 38
 Perimeter
• It is obtained by counting the number of boundary
pixels that belong to an object.

Shape Analysis and Classification Theory and Practice, by Luciano da

Fontoura Costa and Roberto Marcondes Cesar Jr., 2001 CRC press

13-09-2014 39
 Centroid

• The easiest way to estimate the shape centroid is as the average

values of the shape points’ coordinates.
• Suppose that g is a binary image containing only the shape of
interest,
where g(p,q) = 1 for shape pixels and
g(p,q) = 0 for background pixels.

13-09-2014 40
 Shape Analysis and Classification Theory and Practice, by Luciano da
Fontoura Costa and Roberto Marcondes Cesar Jr., 2001 CRC press
13-09-2014 41
 Parameters of Licorice Image After object
detection Calculated
Using Regionprops function Using Pixel based coding
• Area: 23022 • Area = 23022
• Centroid: [344.2502 324.2971] • Perimeter = 772
• BoundingBox: [287.5000 155.5000 • centroid = 344.2502 324.2971
116 347]
• MajorAxisLength: 385.7725
• MinorAxisLength: 83.1018
• EulerNumber: 1
• EquivDiameter: 171.2090
• Perimeter: 857.3280
13-09-2014 42
 Database of Licorice and Kurchi

13-09-2014 43
 Features of licorice and kurchi
Area Major Minor Ratio
Axis Axis
Length Length
Licorice 22854 384.40 82.8 4.64
16119 458.24 67.82 6.75
36454 407.03 125.28 3.24
14661 417 108.12 4.78
29862 473.79 98.99 6.75

Kurchi 2427 71.98 48.83 1.47

2184 64.43 55.03 1.17
4272 120.20 54.85 2.19
7561 143.65 105.47 1.36
2130 70.53 53.01 1.33

13-09-2014 Ratio = Major Axis Length / Minor Axis Length 44

 Object Classification
• Supervised
• One or more examples, known as training set, of each
previously known class of objects are provided as
prototypes for classifying additional objects.
• Unsupervised
• No specific prototypes or suggested features and
criteria are available
• Unsupervised classification is normally called clustering,
and each obtained group of objects a cluster
• Unsupervised classification is usually much more
difficult than supervised classification

13-09-2014 45
 Object Classification using KNN
Unknown
image or
Object

Test Features

Known Object Learning Classification

Features Algorithm Model
Label

Classification scheme

• TrainingPhase
• Testing Phase
13-09-2014 46
 Result Statistics
• Data base of 50 Images of licorice and 50
images of kurchi.
• Any 10 images of licorice and 10 images of
kurchi given for training.
• Then 50 test images of licorice and 50 test
images of kurchi given to the algorithm

13-09-2014 47
 Result

• TARGET is LICORICE
• 50 images of licorice, 50 of kurchi are given
• 34 images of licorice are detected correctly-TRUE POSITIVE
• 4 images of kurchi are detected as licorice- FALSE POSITIVE
• 46 images of kurchi are detected negatively- TRUE NEGATIVE
• 16 images of licorice are falsely misdetected- FALSE NEGATIVE

13-09-2014 48
 Confusion Matrix for LICORICE (Area)
Expert vs Classifier Predicted class as Total number of
per classifier instances
+ -
Actual class as + TP(34) FN(16) P(50)
per the expert
- FP(4) TN(46) N(50)

• True positive rate: TP/P=68%, where P=TP+FN

• False positive rate: FP/N=8%, where N=FP+TN
• False negative rate: FN/P=32%
• True negative rate: TN/N=92%
• Accuracy: TP+TN/TP+TN+FP+FN= 80%
• Error rate: FP+FN/TP+TN+FP+FN=20%
13-09-2014 49
 Confusion Matrix for LICORICE (Ratio)
Expert vs Classifier Predicted class as Total number of
per classifier instances
+ -
Actual class as + TP(43) FN(7) P(50)
per the expert
- FP(2) TN(48) N(50)

• True positive rate: TP/P=86%, where P=TP+FN

• False positive rate: FP/N=4%, where N=FP+TN
• False negative rate: FN/P=14%
• True negative rate: TN/N=96%
• Accuracy: TP+TN/TP+TN+FP+FN= 91%
• Error rate: FP+FN/TP+TN+FP+FN=9%
13-09-2014 50
 Result

• TARGET is Kurchi
• 50 images of kurchi, and 50 images of licorice given.
• 46 kurchi correctly detected-TP
• 4 kurchi not detected-FN
• 16 images falsely detected -FP
• 34 images correctly ignored-TN

13-09-2014 51
 Confusion Matrix for kurchi (Area)
Expert vs Classifier Predicted class as Total number of
per classifier instances
+ -
Actual class as + TP(46) FN(4) P(50)
per the expert
- FP(16) TN(34) N(50)

• True positive rate: TP/P=92%, where P=TP+FN

• False positive rate: FP/N=32%, where N=FP+TN
• False negative rate: FN/P=8%
• True negative rate: TN/N=68%
• Accuracy: TP+TN/TP+TN+FP+FN= 80%
• Error rate: FP+FN/TP+TN+FP+FN= 80%
13-09-2014 52
 Confusion Matrix for kurchi (Ratio)
Expert vs Classifier Predicted class as Total number of
per classifier instances
+ -
Actual class as + TP(48) FN(2) P(50)
per the expert
- FP(7) TN(43) N(50)

• True positive rate: TP/P=96%, where P=TP+FN

• False positive rate: FP/N=14%, where N=FP+TN
• False negative rate: FN/P=4%
• True negative rate: TN/N=86%
• Accuracy: TP+TN/TP+TN+FP+FN= 91%
• Error rate: FP+FN/TP+TN+FP+FN= 9%
13-09-2014 53
 Future work
• Increase the data base of the microscopic images
• images of more plants are planned to be added to increase the
database
• Implementation of SVM algorithm
• Selection of unique features to see the performance of classifiers
• Find the good features for the object representation
• Design an algorithm more robust, reduce the computation time and
increase the efficiency.

13-09-2014 54
 References
1. Microscope Image Processing, by Qiang Wu, Fatima A Merchant and Kenneth R.
Castleman, Elsevier,2008
2. Digital Image Processing, by S. Shridhar, OXFORD University press
3. Digital Image Processing, By Rafael C. Gonzalez and Richard E. Woods, 3rd Edition,
Pearson Prentice Hall,2008
4. Shape Analysis and Classification Theory and Practice, by Luciano da Fontoura Costa
and Roberto Marcondes Cesar Jr., 2001 CRC press
5. Shiping Zhu, Xi Xia, Qingrong Zhang, Kamel Belloulata,” An Image Segmentation
Algorithm in Image Processing Based on Threshold Segmentation,” Third
International IEEE Conference on Signal-Image Technologies and Internet-Based
System,2008,pp.673-678
6. Lu ChengEn, Bai Xiang, Zhu Guangxi, Liu Wenyu,”An Efficient Image Segmentation
Method with Application to Cell Images,” ICSP2008 Proceedings, IEEE, pp.1067-1070
7. VINAYAKA MISSON'S,” A Survey of Shape Descriptors for Digital Image Processing”,
IRACST - International Journal of Computer Science and Information Technology &
Security (IJCSITS), ISSN: 2249-9555 Vol. 3, No.1, February 2013

13-09-2014 55
8. Dijana Tralic, Jelena Bozek, Sonja Grgic,”Shape Analysis and Classification of Masses in
Mammographic Images using Neural Networks “
9. A.Phinyomark, S. Jitaree, P. Phukpattaranont, P. Boonyapiphat,” Texture Analysis of Breast
Cancer Cells in Microscopic Images Using Critical Exponent Analysis Method” Published by
Elsevier Ltd. ,Procedia Engineering 32, 2012,pp. 232 – 238
10. L.I. Kuncheva, J.J.Charles, N. Miles, A Collins, B. Wells and I.S. Lim,” Automated Kerogen
Classification in Micriscope Images of Dispered Kerogen Preparation”
11. Naveen Pandey, Satyanarayan Krishna, Shanu Sharma,”Automatic Seed Classification by Shape
and Color Features Using Machine Vision Technology”, International Journal of Computer
Application Applications Technology and Research Volume 2- Issue 2, 208-213,2013
12. Shen Pan, Mineichi Kudo,”Segementation of pores in wood microscopic images based on
mathematical morphology with a variable structuring element”, Computers and Electronics in
Agriculture 75 (2011) 250-260
13. Ms. Shivani Gupta, Mr. Satyajit Sen Purkayashta, “ Image Enhancement and Analysis of
Microscopic Images using Various Image Processing Techniques”, International Journal of
Engineering Research and Applicatios (IJERA), ISSN:2248-9622, vol. 2, Issue 3, may-June 2012,
pp. 044-048
13-09-2014 56
14. Priyanka M.S., Faheena A.G., Subrata Rakshit, “ Object Detection Using Signature Library”,
ICCCNT’12, 26th-28th July 2012, IEEE-20180
15. Anuj Mohan, Constantine Papagerious, and Tomaso Poggio, “ Example Based Object Detection
in Images by Components”, IEEE Transaction of Pattern Analysis and Machine Intelligence, Vol.
23, No. 4, April 2001.
16. Li, He, Hui Wang and Hong Zhang, “ Object Detection by Parts Using Appearance, Srtuctural
and Shape Features”, Proceedings of the IEEE, International Conference on Mechatronics and
Automation , August 7-10,
17. D. Karakus , A.H. Onur, A.H. Deliormanh, G. Konak, “ Size and Shape Analysis of Mineral
Particles Using Image processing Technique”, The Journal of ORE DRESSING, 2010, Vol. 12-
Issue 23
18. Chesner Desir, Caroline Petitijean, Laurent Heatte, Luc Thiberville, Mathieu Salaun,” An SVM-
based distal lung image classification using texture descriptors”, Computerized Medical
Imaging and Graphics 36 ( 2012) 264-270
19. Paul L. Rosin , “Measuring Shape: Ellipticity, Rectangularity, and Triangularity”
20. Richard iL Peterson and Christopher L Whliams, “Determining paper-coating thickness with de
on microscopy and image analysis “, October 1992 Tappi Journal
13-09-2014 57
21. Zhao Wencang, Wei Hongli, “Study of Medical Image Segmentation Algorithm Based Multi-
wavelet Analysis”, Proceedings of the IEEE International Conference on Automation and
Logistics Qingdao, China September 2008
22. Franz Pernkopf, “ Bayesian network classifiers versus selective k-NN classifier”, Pattern
Recognition 38 (2005) 1 – 10, Published by Elsevier
23. Nicolas Lomenie, Daniel Racoceanu, “Point set morphological filtering and semantic spatial
configuration modeling: Application to microscopic image and bio-structure analysis” ,
Pattern Recognition 45 (2012) 2894–2911
24. Snehamoy Chatterjee, Ashis Bhattacherjee, “Genetic algorithms for feature selection of image
analysis-based quality monitoring model: An application to an iron mine “, Engineering
Applications of Artificial Intelligence 24 (2011) 786–795
25. Rubiyah Yusof , Marzuki Khalid , Anis Salwa M. Khairuddin , “Application of kernel-genetic
algorithm as nonlinear feature selection in tropical wood species recognition system”,
Computers and Electronics in Agriculture 93 (2013) 68–77
26. Marina E. Plissiti , Christophoros Nikou , Antonia Charchanti , “Combining shape, texture and
intensity features for cell nuclei extraction in Pap smear images”, Pattern Recognition Letters
32 (2011) 838–853

13-09-2014 58
 Questions?

13-09-2014 59
 Thank you.

13-09-2014 60