IEEE REVIEWS IN BIOMEDICAL ENGINEERING, VOL.

7, 2014

97

Methods for Nuclei Detection, Segmentation,

and Classification in Digital Histopathology:
A ReviewCurrent Status and Future Potential
Humayun Irshad, Student Member, IEEE, Antoine Veillard, Ludovic Roux, and Daniel Racoceanu, Member, IEEE
Methodological Review

AbstractDigital pathology represents one of the major evolutions in modern medicine. Pathological examinations constitute the
gold standard in many medical protocols, and also play a critical
and legal role in the diagnosis process. In the conventional cancer diagnosis, pathologists analyze biopsies to make diagnostic and
prognostic assessments, mainly based on the cell morphology and
architecture distribution. Recently, computerized methods have
been rapidly evolving in the area of digital pathology, with growing
applications related to nuclei detection, segmentation, and classification. In cancer research, these approaches have played, and
will continue to play a key (often bottleneck) role in minimizing
human intervention, consolidating pertinent second opinions, and
providing traceable clinical information. Pathological studies have
been conducted for numerous cancer detection and grading applications, including brain, breast, cervix, lung, and prostate cancer
grading. Our study presents, discusses, and extracts the major
trends from an exhaustive overview of various nuclei detection,
segmentation, feature computation, and classification techniques
used in histopathology imagery, specifically in hematoxylineosin
and immunohistochemical staining protocols. This study also enables us to measure the challenges that remain, in order to reach
robust analysis of whole slide images, essential high content imaging with diagnostic biomarkers and prognosis support in digital
pathology.
Index TermsDigital pathology, histopathology, microscopic
analysis, nuclei classification, nuclei detection, nuclei segmentation, nuclei separation.

I. INTRODUCTION AND MOTIVATION

ATHOLOGY is the microscopic study of the cell morphology supplemented with in situ molecular information. The
tissue sample is removed from the body and then prepared for
viewing under the microscope by placing it in a fixative, which
stabilizes the tissue to prevent decay. For the sake of visualiz-

Manuscript received October 1, 2013; revised December 7, 2013; accepted

December 16, 2013. Date of publication December 18, 2013; date of current
version April 28, 2014. This work was supported in part by the French National
Research Agency ANR, Project MICO, reference ANR-10-TECS-015.
H. Irshad and L. Roux are with the University Joseph Fourier, 38041 Grenoble, France, and Image Pervasive Access Lab, Centre national de la recherche
scientifique Unit Mixed International 2955, Singapore (e-mail: humayun.
[email protected]; [email protected]).
A. Veillard and D. Racoceanu are with the Sorbonne Universites, University
of Pierre and Marie Curie, Unit Mixed International 2955, 75005 Paris, France,
and also with the Image and Pervasive Access Laboratory, Singapore (e-mail:
[email protected]; [email protected]).
Digital Object Identifier 10.1109/RBME.2013.2295804

ing under the microscope, different components of the tissue

are dyed with different stains. Then, different staining techniques are applied to reveal specific tissue components under the
microscope.
The pathologist plays a central role in therapeutic decision
making [1], [2]. Accordingly, diagnosis from pathology images
remains the gold standard in diagnosing a number of diseases
including most cancers [3]. Diagnosing a disease after manually analyzing numerous biopsy slides represents a labor intensive work for pathologists. Thanks to recent advances in digital
pathology, the automated recognition of pathology patterns in a
high-content whole slide image (WSI) has the potential to provide valuable assistance to the pathologist in his daily practice.
Researchers in pathology have been familiar with the importance of quantitative analysis of pathological images. Quantitative analysis can be used to support pathologists decision about
the presence or the absence of a disease, and also to help in
disease progression evaluation. In addition, quantitative characterization is important, not only for clinical usage (e.g., to
increase the diagnostic reliability), but also for research applications (e.g., drug discovery [4] and biological mechanisms
of disease [5]). As a consequence, the use of computer-aided
diagnosis (CAD) in pathology can substantially enhance the
efficiency and accuracy of pathologists decisions, and overall
benefit the patient.
Earliest works in the field date back to the early 90s [6][8]
but are in relatively small number, presumably due to the limited penetration of digital equipment in pathology. Thanks to
recent advances in digital pathology, numerous cancer detection
and grading applications have been proposed, including brain
[9][11], breast [12][21], cervix [22], [23], liver [24], lung [25],
and prostate [26][28] cancer grading.
Among the various studies, automated nuclei segmentation
and classification is a recurring task, particularly difficult on
pathology images. Indeed, the detection and segmentation of
nuclei in cytopathology images are generally facilitated due
to the well-separated nuclei and the absence of complicated
tissue structures. In contrast, the segmentation of nuclei on
histopathological images (tissue preserving its original structure) is more difficult since most of the nuclei are often part of
histological structures presenting complex and irregular visual
aspects.
A review on automated cancer diagnosis based on histopathological images [29], [30] and a review on histopathological

1937-3333 2014 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution
requires IEEE permission. See http://www.ieee.org/publications standards/publications/rights/index.html for more information.

98

IEEE REVIEWS IN BIOMEDICAL ENGINEERING, VOL. 7, 2014

TABLE I
DESCRIPTION OF NOTATION

image analysis [5] already exist in the literature, addressing

different types of problems associated with different image
modalities. This paper is intended as a comprehensive state-ofthe-art survey on the particular issues of nuclei detection, segmentation, and classification methods restricted to two widely
available types of image modalities: hematoxylin-eosin (H&E)
and immunohistochemical (IHC). A list of symbols and notation
commonly used in this paper is shown in Table I.
This paper is organized in six sections. Section II introduces
the different image modalities in histopathology. In Section III,
we highlight the challenges in nuclei detection, segmentation,
and classification. Section IV illustrates the recent advances in
nuclei detection, segmentation, and classification methods used
in histopathology. Section V addresses the challenges in nuclei
detection, segmentation, and classification, and suggests ways
to overcome them. We conclude with a discussion, pointing to
future research directions and open problems related to nuclei
detection, segmentation, and classification.

II. STAINING AND IMAGE MODALITIES IN DIGITAL PATHOLOGY

Digital pathology is the microscopic investigation of a biopsy
or surgical specimen that is chemically processed and sectioned
onto glass slides to study cancer expression, genetic progression,
and cellular morphology for cancer diagnosis and prognosis.
For tissue components visualization under a microscope, the
sections are dyed with one or more stains.
H&E staining is a widespread staining protocol in pathology. H&E staining has been used by pathologists for over a
hundred years [31] and is still widely used for observing morphological features of the tissue under white light microscopes.
Hematoxylin stains nuclei in dark blue color, while eosin stains
other structures (cytoplasm, stroma, etc.) with a pink color [see
Fig. 1(a)]. Nuclei are susceptible to exhibit a wide variety of
patterns (related to the distribution of chromatin, prominent nucleolus) that are diagnostically significant.

Fig. 1.

Examples of H&E and IHC images. (a) H&E. (b) IHC.

IHC is a technique used for diagnosing whether a cancer is

benign or malignant and for determining the stage of a tumor.
By revealing the presence or absence of specific proteins in the
observed tissue, IHC helps in determining which cell type is at
the origin of a tumor. According to the proteins revealed by IHC,
specific therapeutic treatments adapted for this type of cancer
are selected. Fig. 1(b) shows an example of IHC under light
microscopy.
After staining, fast slide scanners are used to generate digital images that contain relevant information about the specimen
at a microscopic level. They embark one or multiple lenses to
magnify the sample and capture digital images with a camera.
They are capable of digitizing complete slides usually at 20 or

IRSHAD et al.: METHODS FOR NUCLEI DETECTION, SEGMENTATION, AND CLASSIFICATION IN DIGITAL HISTOPATHOLOGY

99

Fig. 2. Different types of nuclei. (a) LN. (b) EN. (c) EN (Cancer). (d) EN
(Mitosis).

40 magnifications. The output of the digital scanners is multilayered images, stored in a format that enables fast zooming
and panning.
For illumination, uniform light spectrum is used to highlight the tissue slide. The microscope setup, sample thickness,
appearance, and staining may cause uneven illumination. In addition, most camera technologies have low response to short
wavelength (blue) illumination and have a high sensitivity at
long wavelength (red to infrared) regions. To reduce these differences in illumination, most slide scanners provide standard
packages to normalize and correct spectral and spatial illumination variations. To address the problem of color nonstandardness, Monaco et al. [32] presented a robust Bayesian color segmentation algorithm that dynamically estimates the probability
density functions describing the color and spatial properties of
salient objects.
III. CHALLENGES IN NUCLEI SEGMENTATION
AND CLASSIFICATION
Among the different types of nuclei, two types are usually the
object of particular interest: lymphocyte and epithelial nuclei.
Nuclei may look very different according to a number of factors such as nuclei type, malignancy of the disease, and nuclei
life cycle. Lymphocyte is a type of white blood cell that has a
major role in the immune system. Lymphocyte nuclei (LN) are
inflammatory nuclei having regular shape and smaller size than
epithelial nuclei (EN) [see Fig. 2(a)]. Nonpathological EN have
nearly uniform chromatin distribution with smooth boundary
[see Fig. 2(b)]. In high-grade cancer tissue, EN are larger in
size, may have heterogeneous chromatin distribution, irregular
boundaries, referred to as nuclear pleomorphism, and clearly
visible nucleoli as compared to normal EN [see Fig. 2(c)]. The
variation in nuclei shape, size, and texture during nuclei life
cycle, mitotic nuclei (MN), is another factor of complexity [see
Fig. 2(d)].
Automated nuclei segmentation is now a well-studied topic
for which a large number of methods have been described in
the literature and new methodologies continue to be investigated. Detection, segmentation, and classification of nuclei in
routinely stained histopathological images pose a difficult computer vision problem due to high variability in images caused by
a number of factors including differences in slide preparation
(dyes concentration, evenness of the cut, presence of foreign
artifacts or damage to the tissue sample, etc.) and image acquisition (artifacts introduced by the compression of the image,
presence of digital noise, specific features of the slide scanner,
etc.). Furthermore, nuclei are often organized in overlapping

Fig. 3. Examples of challenging nuclei to detect and segment. (a) Blur.

(b) Overlaps. (c) Heterogeneity.

clusters and have heterogeneous aspects. All these problems

(highlighted in Fig. 3) make the nuclei detection, segmentation,
and classification a challenging problem. A successful image
processing approach will have to overcome these issues in a
robust way in order to maintain a high level in the quality and
accuracy in all situations.
IV. NUCLEI DETECTION, SEGMENTATION,
AND CLASSIFICATION
Nuclei detection and segmentation are important steps in cancer diagnosis and grading. The aspect of nuclei is critical for
evaluating the existence of disease and its severity. For example,
infiltration of LN in breast cancer is related to patient survival
and outcome [33]. Similarly, nuclei pleomorphism has diagnostic value for cancer grading [34][36]. Furthermore, mitotic
count is also an important prognostic parameter in breast cancer
grading [34]. In Section IV-A, we introduce the most commonly
used image processing methods. Numerous works, described in
Sections IV-B, IV-C, IV-D, and IV-E, use a single or a combination of these image processing methods for preprocessing,
detection, segmentation, and separation, respectively.
A. Image Processing Methods
We begin with basic definitions. An image I is a function
I : U [0, 1]c

(1)

where U = [[0; m 1]] [[0; n 1]] are the pixels, m and n

are the number of rows and columns, and c is the number of
channels (also called colors), usually c {1, 3}. I(i) is the ith
pixel value in the image I, where i U. A part of image I
denoted Ij is a restriction of I to a connected subset of pixels.
1) Thresholding: Thresholding is a method used for converting intensity image I into a binary image I  by assigning
all pixels to the value one or zero if their intensity is above or
below some threshold T . Threshold T can be global or local. If
T is a global threshold, then I  is a binary image of I as

1, if I(i) T

(2)
I (i) =
0, otherwise.
A threshold value can be estimated using computational
methods like the Otsu method which determines an optimal
threshold by minimizing the intraclass variance [37]. Another
thresholding technique is local (adaptive) thresholding that handles nonuniform illumination. It can be determined by either
splitting an image into subimages and calculating thresholds for

100

IEEE REVIEWS IN BIOMEDICAL ENGINEERING, VOL. 7, 2014

each subimage or examining the image intensity in the pixels

neighborhood [38].
2) Morphology: Morphology is a set-theoretic approach that
considers an image as the elements of a set [39] and process
images as geometrical shapes [40]. The basic idea is to probe an
image I with a simple, predefined shape, drawing conclusions
on how this shape fits or misses the shapes in the image. This
simple probe is called the structuring element and is a subset of
the image. The typically used binary structuring elements are
crosses, squares, and open disks.
The two basic morphological operators are the erosion  and
the dilation . Let I : U {0, 1} be a binary image and Uf =
I 1 ({1}) be the foreground pixels. The erosion and dilation of
the binary image I by the structuring element S Z Z are
defined as
Erosion: Uf  S = {x|s S, x + s Uf }
Dilation: Uf S = {x + s|x I s S}.

(3)

The basic effect of erosion (dilation) operator on an image is to shrink (enlarge) the boundaries of foreground pixels.
Two other major operations in morphology are opening and
closing . Opening is an erosion of an image followed by dilation; it eliminates small objects and sharpens peaks in the object.
Opening is mathematically defined as
Uf S = [Uf  S] S.

(4)

Closing is a dilation of an image followed by an erosion; it

fuses narrow breaks and fills small holes and gaps in the image.
Closing is mathematically defined as
Uf S = [Uf S]  S.

The P r that are often used are gray level (average intensity
and variance), color, texture, and shape related.
4) Watershed: Watershed is a segmentation method that usually starts from specific pixels called markers and gradually
floods the surrounding regions of markers, called catchment
basin, by treating pixel values as a local topography. Catchment
basins are separated topographically from adjacent catchment
basins by maximum altitude lines called watershed lines. It allows to classify every point of a topographic surface as either
belonging to the catchment basin associated with one of the
local minima or to the watershed line. Details about watershed
can be found in [42]. The basic mathematical definition contains
lower slope LS(i) that is the maximum slope connecting pixel i
in the image I to its neighbors of lower altitude as


I(i) I(j)
(9)
LS(i) = max
d(i, j)
j N (i)
where N (i) is neighbors of pixel i and d(i, j) is the Euclidean
distance between pixels i and j. In case of i = j, the lower
slope is forced to be zero. The cost of moving from pixel i to j
is defined as

if I(i) > I(j)

LS(i) d(i, j),
cost(i, j) = LS(j) d(i, j),
if I(i) < I(j)
1
(LS(i)
+
LS(j))

d(i,
j),
if
I(i) = I(j).
2
(10)
The topographical distance between the two pixels i and j is
expressed as
min

(i 0 ,...,i t )

(5)

t1


d(ik , ik +1 ) cost (ik , ik +1 )

(11)

k =0

(8)

where is the set of all paths from i to j. The watershed transformation is usually computed on the gradient image instead of
the intensity image.
5) Active Contour Models and Level sets: Active contour
models (ACMs) or deformable models, widely used in image
segmentation, are deformable splines that can be used to depict
the contour of objects in an image using gradient information by
seeking to minimize an energy function [43]. In case of nuclei
segmentation, the contour points that yield the minimum energy
level form the boundary of nuclei. The energy function is often
defined to penalize discontinuity in the curve shape and graylevel discontinuity along the contour [12]. The general ACM is
defined using the energy function E over the contour points c as

(12)
E = (EInt (c) + EIm g (c) + EExt (c)) dc

3) Region Growing: Region growing [41] is an image segmentation method consisting of two steps. The first step is the
selection of seed points and the second step is a classification
of neighboring pixels to determine whether those pixels should
be added to the region or not by minimizing a cost function.
Let P r(Ii ) is a logical predicate which measures the similarity
of a region Ii . The segmentation results in a partition of I into
regions (I1 , I2 , . . . , In ), so that the following conditions hold:
1) P r(Ii ) = TRUE for all i = 1, 2, . . . , n;
2) P r(Ii Ij ) = FALSE, Ii , Ij (i = j) adjacent regions.

where EInt controls the shape and length of the contour (often
called internal energy), EIm g influences adjustment of local
parts of the contour to the image values regardless of the contour
geometry (referring as image energy), and EExt is the userdefined force or prior knowledge of object to control the contour
(referring as external energy). , , and are empirically derived
constants.
There are two main forms of ACMs. An explicit parametric representation of the contour, called snakes, is robust to
image noise and boundary gaps as it constrains the extracted

White and black top-hat transforms are two other operations

derived from morphology. They allow to extract small elements
and details from given images. The white top-hat transform is
defined as the difference between image I and its opening as
Tw (I) = Uf [Uf S].

(6)

The black top-hat transform is defined as the difference between image I and its closing as
Tb (I) = Uf [Uf S].

(7)

In addition, morphological gradient, which is the difference

between the dilation and the erosion of a given image, is useful
for edge detection. It is defined as
G(I) = [Uf S] [Uf S].

IRSHAD et al.: METHODS FOR NUCLEI DETECTION, SEGMENTATION, AND CLASSIFICATION IN DIGITAL HISTOPATHOLOGY

boundaries to be smooth. However, in case of splitting or merging of contours, snakes are restricted for topological adaptability
of the model. Alternatively, the implicit ACM, called level sets,
is specifically designed to handle topological changes, but they
are not robust to boundary gaps and have other deficiencies as
well [44]. The basic idea is to determine level curves from a
potential function.
6) K-means Clustering: The K-means clustering [45] is an
iterative method used to partition an image into K clusters. The
basic algorithm is as follows.
1) Pick K cluster centers, either randomly or based on some
heuristic.
2) Assign cluster label to each pixel in the image that minimizes the distance between the pixel and the cluster center.
3) Recompute the cluster centers by averaging all the pixels
in the cluster.
4) Repeat steps 2) and 3) until convergence is attained or no
pixel changes its cluster.
The difference is typically based on the pixel value, texture,
and location, or a weighted combination of these factors. Its
robustness depends mainly on the initialization of clusters.
7) Probabilistic Models: Probabilistic models can be
viewed as an extension of K-means clustering. Gaussian mixture
models (GMMs) are a popular parametric probabilistic model
represented as weighted sum of Gaussian cluster densities. The
image is modeled according to the probability distribution
P (I(i)) =

K


wk N (I(i)|k , k2 )

(13)

k =1

where K is the number of clusters (objects in the image), k , k2 ,

k, respectively.
and wk are mean, variance, and weight of cluster

w
The wk are positive real values such that K
k =1 k = 1.
The parameters of GMM are estimated from training data
using the computation method like expectation maximization
(EM) [46] that iteratively finds maximum likelihood. The EM
is based on the following four steps.
(0)
(0)
(0)
1) Initialization: The parameters k , k2 , and wk are
randomly initialized for each cluster Ck .
2) Expectation: For each pixel I(i) and cluster Ck , conditional probability P (Ck |I(i)) is computed as
(t)

(t)

(t )

w N (I(i)|k k2 )
P (Ck |I(i))(t) =
K k (t)
.
(t) 2 ( t )
)
j =1 wj N (I(i)|j j
(t)

(t )

(14)
(t)

3) Maximization: The parameters k , k2 , and wk of

each cluster Ck are now maximized using all pixels and
the computed probabilities P (Ck |I)(t) from expectation
step as

U
P (Ck |I(i))(t) I(i)
(t+1)
k
= i
U
(15)
(t)
i P (Ck |I(i))

U
(t+1) 2
P (Ck |I(i))(t) (I(i) k
)
(t+1)
k
= i
(16)

U
(t)
i P (Ck |I(i))

U
P (Ck |I(i))(t)
(t+1)
.
(17)
= i
wk
U

101

with U, the total number of pixels in I.

4) Termination: Steps 2) and 3) are repeated until parameters
converge.
Instead of pixel values, other features can be used like texture.
Carson et al. [47] described the use of a new set of texture
features polarity, anisotropy, and contrast. Polarity is a measure
of a gradient vector for all neighborhood pixels, anisotropy is
a ratio of the eigenvalues of the second moment matrix, and
contrast is a measure of homogeneity of pixels.
8) Graph Cuts: Graph cuts (Gcuts) refers to a wide family
of algorithms, in which an image is conceptualized as weighted
undirected graph G(V, E) by representing nodes V with pixels,
weighted edges E with similarity (affinity) measure between
nodes W : V 2 R+ . A similarity measure is computed from
intensity, spatial distribution, or any features between two pixels.
The Gcuts method partitions the graph into disjoint subgraphs
so that similarity is high within the subgraphs and low across
different subgraphs. The degree of dissimilarity between two
subgraphs A and B can be computed as the sum of weights
of the edges that must be removed to separate A(VA , EA ) and
B(VB , EB ). This total weight is called a cut

cut(A, B) =
w(u, v).
(18)
u V A ,v V B

An intuitive way is to look for the minimum cut in the graph.

However, the minimum cut criterion favors small isolated regions, which are not useful in finding large uniform regions.
The normalized cut (Ncut) solves this problem by computing
the cut cost as a fraction of total edge connections to all the
nodes in the graph. It is mathematically defined as
cut(A, B)
cut(A, B)
+

.
u V A ,tV w(u, t)
v V B ,tV w(v, t)
(19)
Ncut value would not be small for the cut that partitions
isolating points, because the cut value will be a large percentage
of the total connection from that set to the others. The basic
procedure used to find the minimum Ncut is explained here [48].
These image processing methods are extensively used in recently proposed frameworks for preprocessing, nuclei detection,
segmentation, separation, and classification. Based on these image processing methods, we compiled a list of existing frameworks for nuclei detection, segmentation, separation, and classification in histopathology as shown in Table II. In the following
sections, we discuss how different image processing methods
have been used.
Ncut(A, B) =

B. Preprocessing
Preprocessing can be performed to compensate for adverse
conditions such as the presence of batch effects. Batch effect
refers to unevenness in illumination, color, or other image parameters recurring across multiple images. Noise reduction and
artifacts elimination can also be performed prior to detection and
segmentation. Additionally, region of interest (ROI) detection
can also be performed in order to reduce the processing time.
1) Illumination Normalization: The illumination can be corrected either by using white shading correction or by estimating

102

IEEE REVIEWS IN BIOMEDICAL ENGINEERING, VOL. 7, 2014

TABLE II
SUMMARY OF STATE-OF-THE-ART NUCLEI DETECTION AND SEGMENTATION FRAMEWORKS IN HISTOPATHOLOGY

IRSHAD et al.: METHODS FOR NUCLEI DETECTION, SEGMENTATION, AND CLASSIFICATION IN DIGITAL HISTOPATHOLOGY

the illumination pattern from a series of images. In white shading correction, a blank (empty) image is captured and used to
correct images pixel by pixel [73]. A common equation is
Specimen value Background value
.
White Reference value Background value
(20)
A downside of this method is that a blank image must be
acquired for each lens magnification whenever the microscope
illumination settings are altered.
An alternative normalization method is based upon the intrinsic properties of the image which are revealed through Gaussian
smoothing [74]. Another possible way is to estimate background
by exploiting the images of the specimen directly, even in the
presence of the object [75], [76]. Can et al. [77] introduced a
method to correct nonuniform illumination variation by modeling the observed image I(i) as product of the excitation pattern,
E(i), and the emission pattern, M (i) as
Transmittance =

I(i) = E(i) M (i).

(21)

While the emission pattern captures the tissue-dependent

staining, the excitation pattern captures the illumination. From
a set of J images, Ij (i) denotes an ordered set of pixels. Assuming that a certain percentage, g, of the image is formed from
stained tissue (nonzero background), then a trimmed average of
the brightest pixels can be used to estimate the excitation pattern

(i) =
EAVE

J

1
Ij (i)
J K +1

(22)

j =K

where K is set to an integer closest to J(1 g) + 1.

2) Color Normalization: Many color normalization techniques have been proposed [78][81], including histogram or
quantile normalization in which the distributions of the three
color channels are normalized separately. Kothari et al. [81]
used histogram-based normalization in histopathological images. They proposed a rank function which maps the intensity
ranges across all pixels. Alternatively, Reinhard et al. [82] proposed a method for matching the color distribution of an image
to that of the reference image by use of a linear transform in
a perceptual color model (Lab color space). Magee et al. [83]
extended Reinhards normalization approach to multiple pixel
classes by using a probabilistic (GMM) color segmentation
method. It applies a separate linear normalization for each pixel
where class membership is defined by a pixel being colored by
a particular chemical stain or being uncolored, i.e., background.
In order to deal with stains colocalization, a very common
phenomenon in histopathological images, color deconvolution
is effective in separation of stains [84]. Ruifrok et al. [84] explains how virtually every set of three colors can be separated by
color deconvolution and reconstructed for each stain separately.
It requires prior knowledge of color vectors (RGB) of each specific stain. Later, Macenko et al. [80] proposed the automatic
derivation of these color vectors, a method further refined by
Niethammer et al. [85] and Magee et al. [83]. Several nuclei detection and segmentation methods [25], [49], [59], [67], [86]
are using color deconvolution-based separation of stains in
histopathological images.

103

Different color models can be used. Most detection and segmentation methods [9], [10], [17], [24], [25], [50], [64] use the
RGB color model, although the RGB model is not a perceptually uniform color model. Other more perceptual color models
such as HSV, Lab, and Luv are sometimes used [11], [18], [19],
[27], [51], [70], [72], [86][89].
3) Noise Reduction and Image Smoothing: Thresholding is
used for noise reduction that usually follows filtering and background correction in order to minimize random noise and artifacts [22], [90]. The pixels that lie outside threshold values are
often determined using intensity histogram are considered to be
noisy. Alternatively, applying the threshold function on a group
of pixels instead of an individual pixel eliminates a noisy region.
While such techniques are successful to eliminate small spots
of noise, they fail at eliminating large artifacts [91].
Alternatively, morphological operations can also be used for
noise reduction. Noise and artifacts are eliminated using morphological operations like closings and openings [59]. Morphological gray-scale reconstruction methods are used to eliminate
noise while preserving the nuclei shape [24], [54], [55], [70].
While thresholding and filtering reduce noise according to pixel
intensities, morphology reduces noise based on the shape characteristics of the input image, as characterized by a structuring
element. Morphology cannot distinguish the nuclei areas and artifacts having a nuclear-like shape but different intensity values.
Thresholding (prior or subsequent to applying the morphological operations) removes such artifacts.
Adaptive filters [92], Gamma correction [17], and histogram
equalization [52] have been used to increase the contrast between foreground (nuclei) and background regions. Anisotropic
diffusion is used to smooth nuclei information without degrading nuclei edges [52], [86]. Gaussian filtering is also used to
smooth nuclei regions [18], [26], [61].
4) ROI Detection: In some frameworks, noise reduction and
ROI detection are performed simultaneously. For example, for
tissue level feature computation, the preprocessing step selects
the ROI by excluding regions with little content and noise [91].
For nuclei level feature computation, noise reduction is succeeded by ROI detection to determine the nuclei region [70],
[86].
Thresholding is popular for ROI detection. Sertel et al. [52]
introduced the nuclei and cytological components as ROI for
grading of follicular lymphoma (FL). Red blood cells (RBCs)
and background regions show uniform patterns as compared to
other nuclei in FL tissue; thus, thresholding is performed in RGB
color model for elimination of RBCs and background. Similarly,
Dalle et al. [17] selected neoplasm ROI for nuclei pleomorphism
in breast cancer images by using Otsu thresholding along with
morphological operations.
Clustering is another method that is commonly used for ROI
detection. Cataldo et al. [25] performed automated separation of
cancer from noncancerous regions (stroma, blood vessels) using unsupervised clustering. Then, cancerous and noncancerous
regions are refined using morphological operations. Dundar
et al. [19] proposed a framework for classification of intraductal breast lesions as benign or malignant using the cellular component. The intraductal breast lesions contain four

104

IEEE REVIEWS IN BIOMEDICAL ENGINEERING, VOL. 7, 2014

components: cellular, extra cellular, regions with hues of red,

and illumina. The H&E-stained image data are modeled into
four components using GMM. Parameters of the GMM model
are estimated using EM [46]. The resulting mixture distribution
is used to classify pixels into four categories. Those classified as the cellular component are further clustered by dynamic
thresholding to eliminate bluepurple pixels with relatively less
luminance. The remaining pixels are considered cellular region
and are used in lesion classification.
Using textural information, Khan et al. [70] proposed a
novel and unsupervised approach to segment breast cancer
histopathology images into two regions; hypo-cellular stroma
(HypoCS) and hyper-cellular stroma (HyperCS). This approach
employs magnitude and phase spectrum in the Gabor frequency
domain to segment HypoCS and HyperCS regions, respectively.
For MN detection in breast cancer histopathology images, the
false positive rate (FPR) is reduced by four times by using this
technique [86].
C. Nuclei Detection
The identification of initial markers or seed points, usually
one per nucleus and close to its center, is a prerequisite for most
nuclei segmentation methods. The accuracy of segmentation
methods depends critically on the reliability of the seed points.
Initial works in this field rely upon the peaks of the Euclidean
distance map [17]. The H-maxima transform detects local maxima as seed points [26], [53][55], being highly sensitive to
texture and often resulting in overseeding. The Hough transform detects seed points for circular-shaped nuclei but requires
heavy computation [49]. The Centroid transform also detects
seeds but limitations make it useful only for binarized images,
being unable to exploit additional cues.
The Euclidean distance map is commonly used for seed detection and Laplacian of Gaussian (LoG) is a generic blob detection
method. Using multiscale LoG filter with a Euclidean distance
map offers important advantages, including computational efficiency and ability to exploit shape and size information. Alkofahi et al. [58] proposed a distance-constrained multiscale
LoG filtering method to identify the center of nuclei by exploiting shape and size cues available in the Euclidean distance map
of the binarized image. The main steps of this methodology are
as follows.
1) Initially, compute the response of the scale-normalized
LoG filter (LoGnorm (i; ) = 2 LoG(i; )) at multiple
scales = [m in , . . . , m ax ].
2) Use the Euclidean distance map DN (i) to constrain the
maximum scale values when combining the LoG filtering
results across scales to compute a single response surface
RN (i) as
RN (i) =

arg max {LoGnorm (i; ) IN (i)} (23)

[ m i n , M A X ]

where M AX = max{m in , min{m ax , 2 DN (i)}}, and

IN (i) is the nuclear channel image extracted by separating the foreground pixel from background pixel using
automatic binarization.

3) Identify the local maxima of RN (i) and impose a minimum region size to filter out irrelevant minima.
This methodology improves the accuracy of seed locations.
The main disadvantage of this methodology is its sensitivity
to even minor peaks in the distance map that results in over
segmentation and detection of tiny regions as nuclei.
The radial symmetry transform (RST) is also used for seed
detection. Loy and Zelinsky [93] proposed fast gradient-based
interest operator for detection of seed points having high radial
symmetry. Although this approach is inspired by the results of
the generalized symmetry transform, it determines the symmetrical contribution of each pixel around it, rather than considering
the contribution of a local neighborhood to a central pixel. Veta
et al. [59] also employed RST for seed detection.
Recently, several other approaches have been proposed to
detect the seed points. Qi et al. [64] proposed a novel and
fast algorithm for seed detection by utilizing single-path voting with the shifted Gaussian kernel. The shifted Gaussian
kernel is specifically designed by amplifying the voting at
the center of the targeted object and resulted in low occurrence of false seeds in overlapping regions. First, a cone shape
(rm in , rm ax , ) with its vertex at (x, y) is used to define the
voting area A(x, y; rm in , rm ax , ), where rm in is a minimum
radius, rm ax is a maximum radius, and is the aperture angle
of the cone. The voting direction (x, y) is computed using the
negative gradient direction (cos((x, y)), sin((x, y)), where
is the angle of the gradient direction with respect to x-axis.
The voting image V (x, y; rm in , rm ax , ) is generated using the
shifted Gaussian kernel with its means x , y and standard deviation located at the center (x, y) of the voting area A and
oriented in the voting direction using single path approach as
V (x, y; rm in , rm ax , ) =

I(x, y) N (x, y, x , y , )

(u ,v )A

(24)
where I(x, y) is the magnitude of gradient image and
N (x, y, x , y , ) is a 2-D shifted Gaussian kernel defined as


1
(x x )2 + (y y )2
exp

,
2 2
2 2
(25)
where x = x + cos2 (rm ax + rm in ) and y = y sin2 (rm ax +
rm in ). Later, the seed points are determined by executing mean
shift on the sum of voting images. They have compared their
results with iterative voting method in [94].
Counting nuclei by type is highly important for grading purpose. However, manual counting of nuclei is tedious and subject to considerable inter- and intrareader variations. Fuchs and
Buhmann [95] reported 42% disagreement between five pathologists on classification of nuclei as normal or atypical. They
also reported intrapathologist error of 21.2%. This shows the
high potential added value of automatic counting tools.
MN count provides clues to estimate the proliferation and the
aggressiveness of the tumor [62]. Anari et al. [88] proposed the
fuzzy c-means (FCM) clustering method along with the ultraerosion operation in the Lab color model for detection of MN in
IHC images of meningioma. They reported detection accuracy

N (x, y, x , y , ) =

IRSHAD et al.: METHODS FOR NUCLEI DETECTION, SEGMENTATION, AND CLASSIFICATION IN DIGITAL HISTOPATHOLOGY

nearly equal as manual annotation. The FCM clustering method

is based on the minimization of the following objective function:
Jm (V, C) =

c 
U


vkmi I(i) Ck 2

. In order to estimate unknown parameter (), the EM method

is employed for the maximum likelihood estimation. The loglikelihood function  of parameter vector is defined as

(26)
() =

k =1 i=1

with m > 1 (m R), U is the total number of pixels in I,

C = {C1 , C2 , . . . , Cc } are the cluster centers, and V = [vk i ] is
a c U matrix in

which vk i is the kth membership value of ith

pixel, such that Ui=1 vk i = 1. The membership function vk i is
vk i =

j =1

with the cluster center

Ck =

1
I (i)C k
I (i)C j

 m 21

c () =

2
U 


(32)

wik logk +

U


U


{wi1 log[(Ii ; , )]}

i=1

{wi2 log[N (Ii ; , )]}

(33)

i=1

Recently, Roullier et al. [62] proposed a graph-based multiresolution framework for MN detection in breast cancer IHC
images. This approach consists in unsupervised clustering at
low resolution followed by refinements at a higher resolution.
At multiresolution level, mitotic regions are initially segmented
by using the following discrete label regularization function:




0 2

f f
min R(f ) +
(29)
2
f H(V )
where the first term R(f ) is the regularizer defined as the
discrete Dirichlet form of the function f H(V ) : Rw (f ) =

1
2 12
and H(V ) is the
u V [
v u w(u, v)(f (v) f (u)) ]
2
Hilbert space of real valued functions defined on the vertices V
of a graph. The second term is a fitting term. 0 is a fidelity
parameter called the Lagrange multiplier which specifies the
tradeoff between the two competing terms. The GaussJacobi
method is used to approximate the solution of minimization in
(29) by the following iterative algorithm:
(0)
0
f (u) = f (u)

f 0 (u) + v u w(u, v)f (t) (v)

f (t+1) (u) =

, u V
+ v u w(u, v)
(30)
where f (t) is function at the iteration step t. More details on these
definitions can be found in [62]. This discrete regularization is
adapted for labeling the mitotic regions at higher resolution. The
authors reported more than 70% TPR and 80% TNR.
The use of EM for GMM was recently proposed by Khan
et al. [86] for the detection of MN in breast cancer histopathological images. In this framework, pixel intensity of mitotic and
nonmitotic region is modeled by a GammaGaussian mixture
model as
f (Ii ; ) = 1 (I(i); , ) + 2 N (I(i); , )

logf (I(i); )

where f (I(i); ) is the mixture density function in (31). The

EM method finds the maximum likelihood estimation of the
marginal likelihood by iteratively applying expectation and
maximization steps iteratively as

(27)

(28)

U

i=1

i=1 k =1
m
i=1 vk i I(i)
.

U
m
i=1 vk i

105

(31)

where 1 and 2 represent the mixing proportions (prior) of

the intensities belonging to mitotic and nonmitotic regions, respectively. (I(i); , ) represents the Gamma density function
for mitotic regions; it is parameterized by shape () and scale
() parameters. N (I(i); , ) represents the Gaussian density
function for nonmitotic regions; it is parameterized by and

= argmax ()

(34)

where wik , k = 1, 2 are indicator variables showing the component membership of each pixel I(i) in the mixture model
(31). This method reported 51% F-score during ICPR 2012
Contest [96].
Ciresan et al. [97] used deep max-pooling convolutional neural networks (CNNs) to detect MN and achieved highest F-score
(78%) during ICPR 2012 contest [96]. A training dataset consisting of patch images centered on ground truth mitosis is used
to train a CNN. The trained CNN is then used to compute a map
of probabilities of mitosis over the whole image. Their approach
proved to be very efficient and to have a much lower number of
false positives (FPs) as compared to the other contestants.
Grading of lymphocytic infiltration based on detection of
large number of LN in IHC HER2+ breast cancer histopathology
was reported by Basavanhally et al. [18]. In this framework, LN
are automatically detected by a region growing method which
uses contrast measures to find optimal boundary. High detection
sensitivity has been reported for this framework, resulting in a
large number of nuclei other than lymphocytes being detected.
In order to reduce the number of FP, size and luminance information based maximum a posteriori (MAP) estimation is applied
to temporarily labeled candidates as either LN or CN. Later,
Markov random field (MRF) theory with spatial proximity is
used in order to finalize the labels. This framework has been
evaluated on 41 HER2+ WSI and reported 90.41% detection
accuracy as compared to 94.59% manual detection accuracy.
D. Nuclei Segmentation
Nuclei features such as size, texture, shape, and other morphological appearance are important indicators for grading and
prognosis of cancer. Consequently, classification and grading
of cancer is highly dependent on the quality of segmentation
of nuclei. The choice of the nuclei segmentation method is
correlated with the feature computation method. For instance,
some feature computation method requires the exact boundary points of nuclei to compute the nuclei morphology. In this
case, high magnification images are required to utilize the exact

106

details of nuclei. Other feature computation methods require

their course location to compute topology features. A large
number of publications on nuclei segmentation in histopathology use state-of-the-art image segmentation methods based on
thresholding, morphology, region growing, watershed, ACMs,
clustering, and Gcuts, separately or in combination.
The simplest way to detect and segment nuclei in histopathological images is based on thresholding and morphological operations, a simple methodology to segment nuclei [9], [10],
[15], [89], [98]. This methodology reports higher performance
on well-defined, preferably uniform background. The main parameters to tune are the threshold level and the size and shape
of the structuring elements. The difference between nuclei and
background regions may be diffuse, making it harder to find a
reliable threshold level. Even though this methodology is usually defined only on gray-scale images, it can be extended to
color images or stacks of images, using multidimensional kernels. This methodology actually suffers from its simplicity by
including little object knowledge. In addition, it lacks robustness on size and shape variations, as well as on texture variations, which are very frequent in histopathological images. This
methodology is not meant to segment clustered or overlapping
nuclei.
Several authors have been using the watershed transform for
nuclei segmentation [26], [54], [99]. The main advantage of watershed is that there is no tuning to do before using it. However,
it requires the prior detection of seed points. The edge map and
distance transform are used for seed detection [26], [54]. The reported results are suboptimal for ring-shaped nuclei having clear
homogeneous regions. Furthermore, the watershed transform
does not include any prior knowledge to improve its robustness.
ACMs can combine both shape characteristics (smoothness
and shape model) with image features (image gradient and
intensity distribution). However, the resulting segmentation is
strongly dependent upon the initial seed points. Cosatto et al.
[49] described an automated method for accurately and robustly
measuring the size of neoplastic nuclei and providing an objective basis for pleomorphism grading. First, a difference of
Gaussian (DoG) filter is used to detect nuclei. Then, the Hough
transform is used to pick up radially symmetric shapes. Finally,
an ACM with shape, texture, and fitness parameters is used to
extract nuclei boundaries. The authors claimed 90% TPR.
Huang and Lai [24] proposed watershed and ACM-based
framework for nuclei segmentation in hepatocellular carcinoma
biopsy images. Initially, a dual morphological gray-scale reconstruction method is employed to remove noise and accentuate
the shapes of nuclei. Then, a marker-controlled watershed transform is performed to find the edges of nuclei. Finally, ACM is
applied to generate smooth and accurate contours for nuclei.
This framework achieves poor segmentation in case of low contrast, noisy background, and damaged/irregular nuclei.
Dalle et al. proposed gradient in polar space (GiPS), a novel
nuclei segmentation method [17]. Initially, nuclei are detected
using thresholding and morphological operations. Then, transformation into polar coordinate system is performed for every
patch with the center of mass of the nucleus as the origin. Finally,
a biquadratic filtering is used to produce a gradient image from

IEEE REVIEWS IN BIOMEDICAL ENGINEERING, VOL. 7, 2014

which nuclei boundaries are delineated. GiPS reports overall

7.84% accuracy error.
Ta et al. [53] proposed a method based on graph-based regularization. The specificity of this framework is to use graphs as a
discrete modeling of images at different levels (pixels or regions)
and different component relationships (grid graph, proximity
graph, etc.). Based on Voronoi diagrams, a novel image partition (graph reduction) algorithm is proposed for segmentation of
nuclei in serous cytological and breast cancer histopathological
images. A pseudometric : V V R is defined as
(u, v) =

min

P G (u ,v )

m
1


w(ui , ui+1 ) (f (ui+1 ) f (ui ))

i=1

(35)
where w(ui , ui+1 ) is a weight function between two pixels and
PG (u, v) is a set of paths connecting two vertices. Given a set
of K seeds S = (si V), where i = 1, 2, . . . , K, the energy
: V R induced by the metric for all the seeds of S can be
expressed as
S (u) = min (si , u),
s i S

u V.

(36)

The influence zone z (also called Voronoi cell) of a given

seed si S is the set of vertices which are closer to si than to
any other seeds with respect to the metric . It can be defined,
j = 1, 2, . . . , K and j = i, as
z(si ) = {u V : (si , u) (sj , u)} .

(37)

Then, the energy partition of graph, for a given set of seeds

S and a metric , is the set of influence zones Z(S, ) =
{Z(si ), si S}. The authors compared this method with kmeans clustering and Bayesian classification methods in [100].
This method reported 95.73% segmentation accuracy as compared to k-means clustering and Bayesian classification methods
which reported 93.67% and 96.47% accuracy, respectively.
Kofahi et al. [58] proposed another Gcuts-based method for
segmentation of breast CN. Initially, the foreground is extracted
using Gcut-based binarization. The pixel labeling I  (i) is done
by minimizing the following energy function:
 
(I  (i), I  (j))
E(I  (i)) = ln P(I(i)) +
i

j N (i)



I(i) I(j)
exp
2I2

(38)

where P(I(i)|k), k = 0, 1 is a Poisson distribution, N (i) is a

spatial neighborhood of pixel i, and

1, if I  (i) = I  (j)
(39)
(I  (i), I  (j)) =
0, otherwise.
In (38), the first term is a data term that represents the cost of assigning a label to a pixel and the second term is a pixel continuity
term that penalizes different labels for neighboring pixels when
|I(i) I(j)| < I  . After binarization, nuclear seed points are
detected by combining multiscale LoG filtering constrained by
a distance map-based adaptive scale selection (23). These detected seed points are used to perform initial segmentation which

IRSHAD et al.: METHODS FOR NUCLEI DETECTION, SEGMENTATION, AND CLASSIFICATION IN DIGITAL HISTOPATHOLOGY

is refined later using a second Gcuts-based method with combination of alpha expansion and graph coloring to reduce computational complexity. The authors reported 86% accuracy on
25 histopathological images containing 7400 nuclei. The framework often causes oversegmentation when chromatin is highly
textured and the shape of nuclei is extremely elongated. In case
of highly clustered nuclei with weak borders between nuclei,
undersegmentation may occur.
For nuclei segmentation in glioblastoma histopathology images, Chang et al. [66] proposed a multireference Gcuts framework for solving the problem of technical and biological variations by incorporating geodesic constraints. During labeling,
a unique label L(i) is assigned to each vertex v V and the
image cutout is performed by minimizing the energy
E=

(Eg f L(v) + Elf L(v))

v V

Esm o othness (L(v), L(u))

(40)

(v ,u )E

where Eg f and Elf are the global and local data fitness
terms applying the fitness cost for assigning L(v) to v, and
Esm o othness (L(v), L(u)) is the prior energy, denoting the cost
when the labels of adjacent vertices, v and u are L(v) and L(u),
respectively. The authors reported 85% TPR and 75% PPV on
TCGA dataset [101] of 440 WSI.
Vink et al. introduced a deterministic approach using machine
learning technique to segment EN, LN, and fibroblast nuclei in
IHC breast cancer images [69]. Initially, the authors report that
one detector cannot cover the whole range of nuclei as diversity
in appearance is too large to be covered by a single detector.
They formulate two detectors (pixel-based and line-based) using modified AdaBoost. The first detector focuses on the inner
structure of nuclei and second detector covers the line structure
at the border of nuclei. The outputs of these two detectors are
merged using an ACM to refine the border of the detected nuclei. The authors report 95% accuracy with computational cost
of one second per field of view image.
These nuclei segmentation frameworks have reported good
segmentation accuracy on LN, MC, and EN having regular
shape, homogeneous chromatin distribution, smooth boundaries, and individual existence. However, these frameworks have
poor segmentation accuracy for CN especially when CN are
clustered and overlapping. Furthermore, they are intolerant to
chromatin variations, which are very common in CN.
E. Nuclei Separation
A second generation of nuclei segmentation frameworks tackles the challenges of heterogeneity, overlapping, and clustered
nuclei by using machine learning algorithms together with classical segmentation methods. In addition, statistical and shape
models are used to separate overlapping and clustered nuclei.
As compared with nuclei segmentation methods, these methods are more tolerant to variations in shape of nuclei, partial
occlusion, and differences of the staining.

107

The watershed transform is employed to address the problem of overlapping nuclei by defining a group of basins in the
image domain, where ridges in-between basins are borders that
isolate nuclei from each other [9], [19], [25], [54], [60]. Wahlby
et al. [26] addressed the problem of clustered nuclei and proposed a methodology that combined the intensity and gradient
information along with shape parameters for improved segmentation. Morphological filtering is used for finding nuclei seeds.
Then, seeded watershed segmentation is applied on the gradient
magnitude image to create the region borders. Later, the result
of the initial segmentation is refined with gradient magnitude
along the boundary separating neighboring objects, resulting
in the removal of poorly contrasted objects. In final step, distance transform and shape-based cluster separation methodologies are applied keeping only the separation lines, which went
through deep valleys in the distance map. The authors reported
90% accuracy for overlapping nuclei. Cloppet and Boucher [99]
presented a scheme for segmentation of overlapping nuclei in
immunofluorescence images by providing a specific set of markers to the watershed algorithm. They defined markers as split
between overlapping structures and resulted in 77.59% accuracy in case of overlapping nuclei and 95.83% overall accuracy.
In [102], a similar approach is used for segmentation of clustered
and overlapping nuclei in tissue micro array (TMA) and WSI
colorectal cancers. First, combined global and local thresholding are used to select foreground regions. Then, morphological
filtering is applied to detect seed points. Region growing from
seed points produces initial segmented nuclei. At last, clustered
nuclei are separated using watershed and ellipse approximation.
The authors claimed 80.3% accuracy.
The main problem with most ACMs is their sensitivity to
initialization. To solve this initialization problem, Fatakdawala
et al. [57] proposed EM-driven Geodesic ACM with overlap
resolution for segmentation of LN in breast cancer histopathology and reported 86% TPR and 64% PPV. EM-based ACM
initialization allows the model to focus on relevant objects of
interest. The magnetostatic active contour [103] model is used as
a force F guiding contour toward boundary. Based on contours
enclosing multiple objects, high concavity points are detected
on the contours and used in the construction of an edge-path
graph. Then, a scheme based on high concavity points and size
heuristic is used to resolve overlapping nuclei. The degree of
concavity/convexity is proportional to the angle (cw ) between
contour points. It is computed as follows:

(cw ) = arccos

(cw cw 1 ) (cw +1 cw )
|cw cw 1 ||cw +1 cw |


(41)

where cw is a point on the contour.

Yang et al. [51] proposed a nuclei separation methodology in
which concave vertex graph and Ncut algorithm are used. Initially, the outer boundary is delineated via robust estimation and
color active model, and a concave vertex graph is constructed
from automatically detected concave points on boundaries (41)
and inner edges. By minimizing a morphological-based cost
function, the optimal path in graph is recursively calculated to
separate the touching nuclei.

108

IEEE REVIEWS IN BIOMEDICAL ENGINEERING, VOL. 7, 2014

Mouelhi et al. proposed an automatic separation method for

clustered nuclei in breast cancer histopathology [61]. First, a
modified GAC with the ChanVese energy model is used to
detect the nuclei region [104]. Second, high concavity points
along touching nuclei regions are detected (41). Third, the inner
edges are extracted by applying the watershed transform on a
hybrid distance transform image, which combines the geometric
distance and color gradient information. Fourth, the concave
vertex graph using high concavity points and inner edges is
constructed. Last, the optimal separating curve is selected by
computing the shortest path in the graph.
Moreover, for the recognition of single nuclei in nuclei cluster, Kong et al. [60] integrated a framework consisting of a
novel supervised nuclei segmentation and touching nuclei splitting method. For initial segmentation of nuclei, each pixel is
classified into nuclei or background regions by utilizing colortexture in the most discriminant color model. The differentiation
between clustered and separated nuclei is computed using the
distance between the radial symmetry center and the geometrical center of the connected component. For splitting of clustered
nuclei, the boundaries of touching clumps are smoothed out by
Fourier shape descriptor and then concave point detection is
carried out. The authors evaluated this framework on FL images
and achieved average 77% TPR and 5.55% splitting ER.
Another adaptive AC scheme that combines shape, boundary,
region homogeneity, and mutual occlusion terms in a multilevel
set formulation was proposed by Ali et al. [28], [63]. The segmentation of K overlapping nuclei with respect to shape prior
is solved by minimizing the following level set function:

E(, , IF , IB ) =

K
=2

k =1


(k (I) (I))2 |k |(k )dI



Shape + boundary energy


(F H 1 2 )dI +
(B H 1 2 )dI
r
+ 
 

Region energy

K
=2 


H 1 2 dI +
(k k )2 dI

+
k =1 



Mutual occlusion energy

(42)

where = (1 , 2 ), = (1 , 2 ), IF and IB are foreground

and background regions, s , r , > 0 are constants that balance contributions of the shape and boundary, region and mutual
occlusion term, respectively, () is the Dirac delta function, and
(k ) is the contour measure on { = 0}, H() is the Heaviside function, H 1 2 = (H 1 + H 2 H i H 2 ), H 1 2 =
H 1 H 2 , and j = |I Ij |2 + |Ij |2 and j {F, B}. The
watershed transform is used for model initialization. The authors
evaluated this framework on overlapping nuclei in prostate and
breast cancer images and reported 86% TPR and 91% OR on
breast images and 87% TPR and 90% OR on prostate images.
Qi et al. [64] proposed a two-step method for the segmentation of overlapping nuclei in hematoxylin-stained breast TMA
specimens that require very little prior knowledge. First, seed

Fig. 4. Results of segmentation and separation using different methods on

same area of an image. (a) Original (b) GIPS [17]. (c) Level set [105]. (d) MPP
[65].

points are computed by executing mean shift on the sum of the

voting images (24). Second, the following level set representation of the contours is used:
E = N

K 

k =1

K  1

k =1

|I k | di + B
2

K 

k =1

|I b |2 di

g(|I(k (z))|)|k (z)|dz

K
K



k j

(43)

k =1 j =1,j = k

where N , B , > 0 are constants that balance contributions of

each term, k (k = 1, . . . , K) is the nuclei contours that evolve
toward boundaries, K is the number of nuclei, k is the region
inside each contour k , B is the background which represents
the regions outside all the nuclei, k and b are mean intensities
of nuclei and background regions, and g is a sigmoid function
x
g(x) = (1 + e( ) ), where controls the slope of the output
curve and controls the window size. The last term in (43) is
the repulsion term used to represent the repulsion energy between each touching nuclei and is a regulation parameter. The
repulsion term separates the touching nuclei to create smooth
and complete contour of each nuclei. The authors claimed 78%
TPR and 90% PPV in case of touching nuclei.
To overcome ACMs initialization sensitivity, Kulikova et al.
[65] proposed a method based on marked point processes
(MPPs). This methodology, a type of high-order ACM, is able to
segment overlapping nuclei as several individual objects. There
is no need to initialize the process with seed points giving the
location of the nuclei to be segmented. A shape prior term is
used for handling overlapping nuclei. Fig. 4 shows a comparison of nuclei segmentation results using MPP, GiPS [17], and
levelset [105].
Recently, Veillard et al. [67] proposed a method based on
the creation of a new image modality consisting in a gray-scale
map where the value of each pixel indicates its probability to
belong to a nucleus. This probability map is calculated from
texture, scale information, and simple pixel color intensities.
The resulting modality has a strong object-background contrast and smoothing out the irregularities within the nuclei and
background. Later, segmentation is performed using an ACM
with a nuclei shape prior [65] to solve the problem of overlapping nuclei. Fig. 5(a) shows the result of ACM segmentation on

IRSHAD et al.: METHODS FOR NUCLEI DETECTION, SEGMENTATION, AND CLASSIFICATION IN DIGITAL HISTOPATHOLOGY

Fig. 5. Segmentation results using ACM methods on probability and

hematoxylin-stained image [67]. (a) Probability map image. (b) ACM on probability map image. (c) Hematoxylin stained image. (d) ACM on Hematoxylin
stained image.

probability map image and hematoxylin-stained image, produced after color deconvolution [84].
In general, model-based approaches segment nuclei using a
prior shape information, which may introduce a bias favoring the
segmentation of nuclei with certain characteristics. To address
this problem, Wienert et al. [68] proposed a novel contourbased minimum model for nuclei segmentation using minimal
a prior information. This minimum model-based segmentation
framework consists of six internal processing steps. First, all
possible closed contours are computed regardless of shape and
size. Second, all initially generated contours are ranked using
gradient fit. Third, nonoverlapping segmentation is performed
with ranked labeling in a 2-D map. Fourth, segmentation is
improved using contour optimization. Fifth, cluster nuclei are
separated using concavity point detection (41). Last, segmented
regions are classified as nuclei or background using stained
related information. This framework avoids a segmentation bias
with respect to shape features. The authors managed to achieved
86% TPR and 91% PPV on a dataset of 7931 nuclei.
RST is an iterative algorithm attributing votes to pixels inside
a region [93]. After the final iteration, maxima are used as
marker of a nuclei segmentation algorithm such as watershed.
Each boundary point contributes to votes for a region defined
by oriented cone-shape kernels as

A(x, y; rm in , rm ax , ) = (x + r cos , y + r sin )
|rm in r rm ax ,

(x, y) +
(x, y)
2
2


(44)

where the radial range is parameterized by rm in , rm ax and the

angular range . (x, y) is the angle between the positive xaxis and the voting direction. These parameters are updated
using votes from the previous iterations.
Schmitt and Hasse [106] separated the clustered nuclei using
RST based on the idea that the center of mass in a nucleus is
considered as a basic perceptual event that supports separation
of clustered nuclei. They initialized iterative voting along the
gradient direction where, at each iteration, the voting direction
and shape of the kernel are refined iteratively. The voting area
can be regulated by selecting the number of steps in the evolution of the kernel shape. Few number of steps resulted in the
fragmentation of the center of mass, while a large number of
steps increases computational cost. They also proposed a way

109

to deal with holes and sub holes in the region by processing

boundaries iteratively.
One limitation of RST is the prior knowledge of scale, which
cannot be generalized. To overcome this limitation, multiscale
extension of the RST seems to be reasonable. A similar method
[106] is used in [50] to decompose regions of clustered nuclei
in H&E-stained prostate cancer biopsy images. They initially
obtained regions of clustered nuclei by clustering and levelset segmentation. Recently, Veta et al. [59] proposed a method
similar to [24] that met the objective of nuclei segmentation
in H&E-stained breast cancer biopsy images by applying the
fast RST [93] to produce markers for the watershed segmentation. Sertel et al. [56] proposed adaptive likelihood-based nuclei
segmentation for FL centroblasts. Initially, nuclear components
are clustered using GMM with EM. Using fast RST, the spatial
voting matrix is computed along the gradient direction. Finally,
local maxima locations associated with individual nuclei are
determined.
Alternatively, EM- and GMM-based unsupervised Bayesian
classification scheme was used for segmentation of overlapping
nuclei in IHC images [55]. The separation of overlapping nuclei is formulated as cluster analysis problem. This approach
primarily involves applying the distance transform to generate
topographic surface, which is viewed as a mixture of Gaussian. Then, a parametric EM algorithm is employed to learn
the distribution of topographic surface (GMM). On the basis
of extracted regional maxima, cluster validation is performed
to evaluate the optimal number of nuclei. The cluster validity
index consists of a compactness measure (the smaller value
means more compact) and a separation measure between the
clusters. The main idea is to have nuclei as compact and as
well separated as possible. Thus, cluster parameters are chosen
to maximize . A prior knowledge for the overlapping nuclei
is incorporated to obtain separation line without jaggedness,
as well as to reconstruct occluded contours in overlapping region. They achieved improvements of up to 6.80%, 5.70%, and
3.43% with respect to classical watershed, conditional erosion,
and adaptive H-minima transform schemes in terms of separation accuracy. Overall, they achieved 93.48% segmentation
accuracy for overlapping nuclei on specimens of cervical nuclei
and breast invasive ductal carcinomas.
The novelty of these approaches corresponds to the use of machine learning and statistical methods to eliminate malformed
nuclear outlines and thus, to allow robust nuclei segmentation.
These methods are mainly dependent on the availability of expert annotations. Furthermore, these models may not be generalizable and have limited application due to the manual training
step, sensitivity to initialization, and limited ability to segment
multiple overlapping objects.
F. Nuclei Features and Classification
Features computed from segmented nuclei are usually a prerequisite to nuclei classification that generate higher level information regarding the state of the disease. The classifiers use
nuclei features, which capture the deviations in the nuclei structures, to learn how to classify nuclei into different classes. In

110

IEEE REVIEWS IN BIOMEDICAL ENGINEERING, VOL. 7, 2014

TABLE III
SUMMARY OF NUCLEI FEATURES USED IN HISTOPATHOLOGY

order to extract features, there are two types of information

available in the image: 1) the intensity values of pixels; and 2)
their spatial interdependency [29].
We found a compilation of features for cytopathology imagery [107], but found relatively little such work for histopathology imagery. In histopathology, these features can be categorized into the following four categories: 1) cytological; 2) intensity; 3) morphological; and 4) texture features. A summary
of nuclei features is listed in Table III. Definitions for all listed
features can be found in [29], [72], and [108].
In some frameworks, the computed features, like intensity and
texture features, are explicitly used for segmentation of nuclei
with K-means clustering [56], [57]. To address the problem of
heterogeneity in CN, Veillard et al. [67] used intensity and textural features with support vector machine (SVM) classifier for
the creation of a new image modality to segment CN. Recently,
Vink et al. [69] constructed a large set of features and modified
AdaBoost to create two detectors that solved the problem of
variations in nuclei segmentation. The first detector is formulated with intensity features; the second detector is constructed
using Haar like features.
In addition to the morphological features computed from cytological regions, Huang and Lai [24] extracted intensity and
cooccurrence (CO) features. They extracted a total of 14 features (intensity, morphological, and texture features) from segmented nuclei in biopsy images, which comprise both local and
global characteristics so that benignancy and different degrees
of malignancy can be distinguished effectively. An SVM-based
decision graph classifier with feature subset selection on each
decision node of classifier is used in comparison with k-nearest
neighbor and simple SVM; the accuracy rate of classification
promoted from 92.88% to 94.54% with an SVM-based decision
graph classifier.
Intensity and morphological features are extensively used for
nuclei classification as epithelial and CN in [17], [19], and [49].
An exhaustive set of features including morphological and texture features are explored to determine the optimal features
for nuclei classification [109]. Their results of feature selection demonstrated that Zernike moment, Daubechies wavelets,
and Gabor wavelets are the most important features for nuclei
classification in microscopy images. Recently, Irshad et al. [89],
[98], [110] used intensity, morphology, CO, and run-length (RL)
features in selective color channels from different color models
with a decision tree and SVM classifiers for mitosis detection
in MITOS dataset of breast cancer histopathology images and

ranked second with 72% F-score in ICPR 2012 contest [96].

Similarly, Malon et al. [72] computed intensity, texture, and
morphological features and used these features with SVM for
the classification of segmented candidate regions into mitotic
and nonmitotic regions. This method reported 66% F-score during ICPR 2012 contest [96].
According to Al-Kadi [10], the combination of several texture
measures instead of using just one might improve the overall
accuracy. Different texture measures tend to extract different
features each capturing alternative characteristics of the examined structure. They computed four different texture features,
two of them are model-based: Gaussian Markov random field
(GMRF) and fractal dimension (FD); the other two are statistically based: CO and RL features. Using selected features
after excluding highly correlated features, Bayesian classifier
was trained for meningioma subtype classification. They studied the variation of texture measure as the number of nuclei
increased; the GMRF was nearly uniform, while the RL and FD
performed better in the high frequencies. They also studied the
texture measures response to additive texture distortion noise
while varying nuclei shape densities. The GMRF was the least
affected, yet the RL and FD performed better in high and low
shape frequency, respectively. The combination of GMRF and
RL improved the overall accuracy up to 92.50% with none of
the classified meningioma subtypes achieving below 90%.
By observing the cancer detection procedure adopted by
pathologists, Nguyen et al. [27] developed a novel idea for
cancer detection in prostate using cytological (nuclear) and textural features. Prominent nucleoli (cytological feature) inside
nuclei region is used to classify nuclei as cancerous or not. In
addition, prostate cancer is detected using cytological, intensity, morphological, and textural features having 78% TPR on a
dataset including six WSI for training and 11 for testing.
V. DISCUSSION AND FUTURE DIRECTIONS
Since the last decade, a significant number of articles have
been published in the field of histopathology, focusing on nuclei segmentation and classification in different image modalities. Still, there are some open research areas where little study
has been done. These open research areas have unique challenges, which should be covered in future research. One of the
aforementioned challenges is the lack of unified benchmarks.
Studies cited in this review have been performed using their
own private datasets. Moreover, it is not straightforward to evaluate and numerically compare different studies solely based on
their reported results since they use different datasets, various
evaluation methods, and multiple performance metrics. For numerical comparison of the studies, it is definitely necessary to
build benchmark datasets. These datasets should be medically
validated, comprise samples coming from a large number of patients, and annotated by different pathologists to accommodate
subjective variations in annotation. Such an effort would make
possible the numerical comparison of the results obtained by different studies and the identification of distinguishing features.
To the best of our knowledge, we know of only a few benchmark
datasets: UCSB Bio-Segmentation [111], the MITOS mitosis

IRSHAD et al.: METHODS FOR NUCLEI DETECTION, SEGMENTATION, AND CLASSIFICATION IN DIGITAL HISTOPATHOLOGY

detection [71] benchmark, as well as a recent similar initiative

AMIDA [112].
The UCSB Bio-Segmentation Benchmark dataset consists of
2-D/3-D images and time-lapse sequences that can be used for
evaluating the performance of novel state-of-the-art computer
vision methods. The data cover subcellular, cellular, and tissue
level. Tasks include segmentation, classification, and tracking.
The MITOS benchmark has been set up to provide a database
of mitosis freely available to the research community. Mitotic
count is an important parameter in breast cancer grading as it
gives an evaluation of the aggressiveness of the tumor. Detection of mitosis is a very challenging task, since mitosis are small
objects with a large variety of shape configurations; however, it
has not been addressed well in the literature, mainly because of
the lack of available data. The MITOS benchmark has been set
up as an international contest of mitosis detection in the framework of conference ICPR 2012. AMIDA benchmark reedited
in 2013 the same type of mitosis detection challenge as MITOS
did in 2012.
Most of these benchmarks highlighted the fact that despite
the promising results, there are still progresses to be made to
reach clinically acceptable results. For instance, the overall best
results on mitosis detection presented during the recent MITOS
and AMIDA contests achieved an F-score of 78.21% for MITOS [71] and 61.1% for AMIDA [112], which would not be
considered accurate detection under medical terms.
The issue of inter- and intrapathologist disagreements is also
to be taken into account. Fuchs and Buhmann [95] reported 42%
disagreement between five pathologists on nuclei classification
as normal or atypical. They also reported intrapathologist error
of 21.2%. A conclusion of this study is that that self-assessment
is not a reliable validation method. A similar study by Malon
et al. [113] reported a moderate agreement between three pathologists for identifying MN on H&E-stained breast cancer slides.
Although the seemingly large figures are to be interpreted into
the specific context of the study, it shows that validation by
medical expert is not a straightforward issue.
It is also important to address the issue of robustness to varying clinical/technical conditions including: 1) different scanners
used for image acquisition, 2) different staining characteristics,
3) different lightening conditions, and 4) magnification.
Segmentation methods like thresholding, region growing,
and watershed can locate the nuclei region but problems arise
when they try to segment the touching and overlapping nuclei.
They employ only local intensity information without any prior
knowledge about the object to be segmented and produce inaccurate nuclei boundaries.
Dealing with overlapping and clustered nuclei is still a major
challenge in the field of nuclei segmentation. While different
methods have been developed with various levels of success in
the literature for the problem of overlapping and clustered nuclei, the problem has not yet been completely solved. A variety
of schemes taking into account concavity point detection [28],
[51], [57], [60], [61], [68], distance transform [11], [26], [54],
marker-controlled watershed [9], [19], [24], [25], [59], adaptive
ACM with shape and curvature information [63][65], [67],
GMM and EM [55], and graphs [51], [58], [61] have been

111

investigated to separate overlapping and clustered/touching nuclei. These methods have good results for nuclei that are slightly
touching or overlapping each other, but they are not suitable for
specimens containing larger numbers of nuclei with extensive
overlapping and touching. These methods suffer from dependencies inducing instability. For instance, the computation of
curvature is highly dependent on concavity point detection algorithm, region growing tends to rely on shape and size of
nuclei, marker-controlled watershed needs true nuclei markers,
and ellipse-fitting techniques are unable to accommodate the
shape of most nuclei. Most of these methods also require prior
knowledge. In spite of the availability of few methods like clustering, GMM and EM, and new image modality [67] able to
deal with heterogeneity, accurate segmentation of touching or
overlapping nuclei is still an open research area.
To the best of our knowledge, only few supervised machinelearning techniques like Bayesian [18], [55], SVM [67], and
AdaBoost [69] are used for nuclei segmentation. The basic philosophy of the machine learning approach is that human provides examples of the desired segmentation, and leaves the optimization and parameter tuning tasks to the learning algorithm.
The two main avenues to be explored in terms of supervised
machine-learning algorithms are the use of more domain specific features and limitation of overfitting issues.
REFERENCES
[1] W. W. Ma and A. A. Adjei, Novel agents on the horizon for cancer
therapy, CA: A Cancer J. Clinic., vol. 59, no. 2, pp. 111137, 2009.
[2] M. May, A better lens on disease: Computerized pathology slides may
help doctors make faster and more accurate diagnoses, Sci. Amer.,
vol. 302, pp. 7477, 2010.
[3] R. Rubin and D. S. Strayer, in Rubins Pathology: Clinicopathologic
Foundations of Medicine, E. M. Rubin, F. Gorstein, R. Schwarting, and
D. S. Strayer, Eds., 4th ed. Philadelphia, PA, USA: Lippincott Williams
& Wilkins, Apr. 2004.
[4] X. Zhou and S. Wong, Informatics challenges of high-throughput microscopy, IEEE Signal Process. Mag., vol. 23, no. 3, pp. 6372, May
2006.
[5] M. N. Gurcan, L. E. Boucheron, A. Can, A. Madabhushi, N. M. Rajpoot,
and B. Yener, Histopathological image analysis: A review, IEEE Rev.
Biomed. Eng., vol. 2, pp. 147171, Dec. 2009.
[6] P. H. Bartels, D. Thompson, M. Bibbo, and J. E. Weber, Bayesian belief
networks in quantitative histopathology, Anal. Quant. Cytol. Histol.,
vol. 14, no. 6, pp. 459473, 1992.
[7] J. P. Thiran and B. Macq, Morphological feature extraction for the classification of digital images of cancerous tissues, IEEE Trans. Biomed.
Eng., vol. 43, no. 10, pp. 10111020, Oct. 1996.
[8] T. Mouroutis, S. J. Roberts, and A. A. Bharath, Robust cell nuclei segmentation using statistical modeling, Bioimaging, vol. 6, pp. 7991,
1998.
[9] M. N. Gurcan, T. Pan, H. Shimada, and J. Saltz, Image analysis for
neuroblastoma classification: Segmentation of cell nuclei, in Proc. IEEE
28th Annu. Int. Conf. Eng. Med. Biol. Soc., New York, NY, USA, 31
Aug.3 Sep. 2006, pp. 48444847.
[10] O. S. Al-Kadi, Texture measures combination for improved meningioma classification of histopathological images, Pattern Recog.,
vol. 43, no. 6, pp. 20432053, 2010.
[11] J. Kong, L. Cooper, T. Kurc, D. Brat, and J. Saltz, Towards building
computerized image analysis framework for nucleus discrimination in
microscopy images of diffuse glioma, in Proc. IEEE 33rd Annu. Int.
Conf. Eng. Med. Biol. Soc., Boston, MA, USA, Aug. 30Sep. 3, 2011,
pp. 66056608.
[12] W. N. Street, W. H. Wolberg, and O. L. Mangasarian, Nuclear feature
extraction for breast tumor diagnosis, in Proc. Int. Symp. Electron.
Imag.: Sci. Technol., San Jose, CA, USA, 14 Feb. 1993, vol. 1905,
pp. 861870.

112

[13] J. Gil, H. Wu, and B. Y. Wang, Image analysis and morphometry in the
diagnosis of breast cancer, Microsc. Res. Tech., vol. 59, no. 2, pp. 109
118, 2002.
[14] C. Gunduz, B. Yener, and S. H. Gultekin, The cell graphs of cancer,
Bioinformatics, vol. 20, pp. 145151, 2004.
[15] S. Petushi, F. U. Garcia, M. M. Haber, C. Katsinis, and A. Tozeren, Large-scale computations on histology images reveal gradedifferentiating parameters for breast cancer, BMC Med. Imag., vol. 6,
pp. 1424, 2006.
[16] A. E. Tutac, D. Racoceanu, T. Putti, W. Xiong, W.-K. Leow, and V. Cretu,
Knowledge-guided semantic indexing of breast cancer histopathology
images, in Proc. Int. Conf. Biomed. Eng. Informat., Sanya, Hainan,
China, 2008, vol. 2, pp. 107112.
[17] J.-R. Dalle, H. Li, C.-H. Huang, W. K. Leow, D. Racoceanu, and
T. C. Putti, Nuclear pleomorphism scoring by selective cell nuclei detection, in Proc. IEEE Workshop Appl. Comput. Vis., 2009, 6 pp.
[18] A. N. Basavanhally, S. Ganesan, S. Agner, J. P. Monaco, M. D. Feldman,
J. E. Tomaszewski, G. Bhanot, and A. Madabhushi, Computerized
image-based detection and grading of lymphocytic infiltration in HER2+
breast cancer histopathology, IEEE Trans. Biomed. Eng., vol. 57, no. 3,
pp. 642653, Mar. 2010.
[19] M. Dundar, S. Badve, G. Bilgin, V. C. Raykar, R. K. Jain, O. Sertel, and
M. N. Gurcan, Computerized classification of intraductal breast lesions
using histopathological images, IEEE Trans. Biomed. Eng., vol. 58,
no. 7, pp. 19771984, Jul. 2011.
[20] C.-H. Huang, A. Veillard, L. Roux, N. Lomenie, and D. Racoceanu,
Time-efficient sparse analysis of histopathological whole slide images,
Computer. Med. Imag. Graph., vol. 35, pp. 579591, Nov. 2011.
[21] N. Lomenie and D. Racoceanu, Point set morphological filtering and semantic spatial configuration modeling: Applications to microscopic image and bio-structure analysis, Pattern Recog., vol. 45, no. 8, pp. 2894
2911, 2012.
[22] S. J. Keenan, J. Diamond, W. Glenn McCluggage, H. Bharucha,
D. Thompson, P. H. Bartels, and P. W. Hamilton, An automated machine vision system for the histological grading of cervical intraepithelial neoplasia (CIN), J. Pathol., vol. 192, no. 3, pp. 351362,
2000.
[23] X. He and Q. Liao, A novel shape prior based segmentation of touching
or overlapping ellipse-like nuclei, in Proc. SPIE, San Diego, CA, USA,
vol. 6914, 2008, 8 pp.
[24] P. W. Huang and Y. H. Lai, Effective segmentation and classification
for HCC biopsy images, Pattern Recog., vol. 43, no. 4, pp. 15501563,
2010.
[25] S. D. Cataldo, E. Ficarra, A. Acquaviva, and E. Macii, Automated segmentation of tissue images for computerized IHC analysis, Comput.
Methods Progr. Biomed., vol. 100, no. 1, pp. 115, 2010.
[26] C. Wahlby, I. M. Sintorn, F. Erlandsson, G. Borgefors, and E. Bengtsson,
Combining intensity, edge and shape information for 2D and 3D segmentation of cell nuclei in tissue sections, J. Microsc., vol. 215, no. 1,
pp. 6776, 2004.
[27] K. Nguyen, A. K. Jain, and B. Sabata, Prostate cancer detection: Fusion
of cytological and textural features, J. Pathol. Informat., vol. 2, no. 2,
pp. 727, Dec. 2011.
[28] S. Ali, R. Veltri, J. Epstein, C. Christudass, and A. Madabhushi, Adaptive
energy selective active contour with shape priors for nuclear segmentation and gleason grading of prostate cancer, in Proc. 14th Int. Conf.
Med. Image Comput. Comput.-Assist. Interv., Toronto, ON, Canada, 18
22 Sep. 2011, pp. 661669.
[29] C. Demir and B. Yener, Automated cancer diagnosis based on
histopathological images: A systematic survey, Dept. Comput. Sci.,
Rensselaer Polytechnic Inst., Troy, NY, USA, Tech. Rep. TR-05-09,
2005.
[30] L. He, L. R. Long, S. Antani, and G. Thoma, Computer assisted diagnosis in histopathology, Sequ. Genome Anal.: Methods Appl., 2010, pp.
271287.
[31] H. Fox, Is H&E morphology coming to an end?, J. Clin. Pathol.,
vol. 53, pp. 3840, 2000.
[32] J. Monaco, J. Hipp, D. Lucas, S. Smith, U. Balis, and A. Madabhushi,
Image segmentation with implicit color standardization using spatially
constrained expectation maximization: Detection of nuclei, in Proc.
15th Int. Conf. Med. Image Comput. Comput.-Assist. Interv., 2012,
pp. 365372.
[33] G. Alexe, G. S. Dalgin, D. Scanfeld, P. Tamayo, J. P. Mesirov, C.
DeLisi, L. Harris, N. Barnard, M. Martel, A. J. Levine, S. Ganesan,
and G. Bhanot, High expression of lymphocyte-associated genes in
node-negative HER2+ breast cancers correlates with lower recurrence
rates, Cancer Res., vol. 67, no. 22, pp. 10 66910 676, 2007.

IEEE REVIEWS IN BIOMEDICAL ENGINEERING, VOL. 7, 2014

[34] C. Elston and I. Ellis, Pathological prognostic factors in breast cancer.

the value of histological grade in breast cancer experience from a large
study with long-term follow-up, Histopathology, vol. 41, pp. 151153,
2002.
[35] M. Stierer, H. Rosen, and R. Weber, Nuclear pleomorphism, a strong
prognostic factor in axillary node-negative small invasive breast cancer,
Breast Cancer Res. Treatment, vol. 20, no. 2, pp. 109116, 1991.
[36] B. Dunne and J. Going, Scoring nuclear pleomorphism in breast cancer,
Histopathology, vol. 39, no. 3, pp. 259265, 2001.
[37] N. Otsu, An automatic threshold selection method based on discriminate
and least squares criteria, Denshi Tsushin Gakkai Ronbunshi, vol. 63,
pp. 349356, 1979.
[38] P. D. Wellner, Adaptive thresholding for the digital desk, in Xerox,
EPC1993-110, 1993, 14 pp.
[39] J. Serra, Image Analysis and Mathematical Morphology. Orlando, FL,
USA: Academic, 1983.
[40] H. Heijmans, Mathematical morphology: Basic principles, in Proc.
Summer School Morphological Image Signal Process., Zakopane,
Poland, Sep. 1995, pp. 118.
[41] S. W. Zucker, Region growing: Childhood and adolescence, Comput.
Graph. Image Process., vol. 5, no. 3, pp. 382399, 1976.
[42] J. B. Roerdink and A. Meijster, The watershed transform: Definitions,
algorithms and parallelization strategies, Fundamenta Informaticae,
vol. 41, no. 1, pp. 187228, 2000.
[43] M. Kass, A. Witkin, and D. Terzopoulos, Snakes: Active contour models, Int. J. Comput. Vis., vol. 1, no. 4, pp. 321331, 1988.
[44] J. S. Suri, K. Liu, S. Singh, S. N. Laxminarayan, X. Zeng, and L. Reden,
Shape recovery algorithms using level sets in 2-D/3-D medical imagery:
A state-of-the-art review, IEEE Trans. Inf. Technol. Biomed., vol. 6,
no. 1, pp. 828, Mar. 2002.
[45] J. MacQueen, Some methods for classification and analysis of multivariate observations, in Proc. 5th Berkeley Symp. Math. Statist. Probabil.,
Berkeley, CA, USA, 1967, vol. 1, pp. 281297.
[46] A. P. Dempster, N. M. Laird, and D. B. Rubin, Maximum likelihood
from incomplete data via the EM algorithm, J. Roy. Statist. Soc. Series
B (Methodological), vol. 39, pp. 138, 1977.
[47] C. Carson, S. Belongie, H. Greenspan, and J. Malik, Blobworld: Image
segmentation using expectation-maximization and its application to image querying, IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 8,
pp. 10261038, Aug. 2002.
[48] J. Shi and J. Malik, Normalized cuts and image segmentation, in
Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recog., San Juan,
Puerto Rico, 1997, pp. 731737.
[49] E. Cosatto, M. Miller, H. P. Graf, and J. S. Meyer, Grading nuclear
pleomorphism on histological micrographs, in Proc. 19th Int. Conf.
Pattern Recog., Tampa, FL, USA, Dec. 2008, pp. 14.
[50] A. Hafiane, F. Bunyak, and K. Palaniappan, Clustering initiated multiphase active contours and robust separation of nuclei groups for
tissue segmentation, in Proc. 19th Int. Conf. Pattern Recog., Tampa,
FL, USA, Dec. 2008, pp. 14.
[51] L. Yang, O. Tuzel, P. Meer, and D. Foran, Automatic image analysis
of histopathology specimens using concave vertex graph, in Proc. Med.
Image Comput. Comput.-Assist. Interv., New York, NY, USA, 2008,
pp. 833841.
[52] O. Sertel, U. V. Catalyurek, H. Shimada, and M. N. Gurcan, Computeraided prognosis of neuroblastoma: Detection of mitosis and karyorrhexis
cells in digitized histological images, in Proc. IEEE 31st Annu. Int. Conf.
Eng. Med. Biol. Soc., Minnesota, USA, Sep. 2009, pp. 14331436.
[53] V.-T. Ta, O. Lezoray, A. Elmoataz, and S. Schupp, Graph-based tools
for microscopic cellular image segmentation, Pattern Recog., vol. 42,
no. 6, pp. 11131125, 2009.
[54] C. Jung and C. Kim, Segmenting clustered nuclei using h-minima
transform-based marker extraction and contour parameterization, IEEE
Trans. Biomed. Eng., vol. 57, no. 10, pp. 26002604, Oct. 2010.
[55] C. Jung, C. Kim, S. W. Chae, and S. Oh, Unsupervised segmentation of
overlapped nuclei using Bayesian classification, IEEE Trans. Biomed.
Eng., vol. 57, no. 12, pp. 28252832, Dec. 2010.
[56] O. Sertel, G. Lozanski, A. Shanaah, and M. N. Gurcan, Computer-aided
detection of centroblasts for follicular lymphoma grading using adaptive
likelihood-based cell segmentation, IEEE Trans. Biomed. Eng., vol. 57,
no. 10, pp. 26132616, Oct. 2010.
[57] H. Fatakdawala, J. Xu, A. Basavanhally, G. Bhanot, S. Ganesan,
M. Feldman, J. E. Tomaszewski, and A. Madabhushi, Expectation
maximization-driven geodesic active contour with overlap resolution
(EMaGACOR): Application to lymphocyte segmentation on breast cancer histopathology, IEEE Trans. Biomed. Eng., vol. 57, no. 7, pp. 1676
1689, Jul. 2010.

IRSHAD et al.: METHODS FOR NUCLEI DETECTION, SEGMENTATION, AND CLASSIFICATION IN DIGITAL HISTOPATHOLOGY

[58] Y. Al-Kofahi, W. Lassoued, W. Lee, and B. Roysam, Improved automatic detection and segmentation of cell nuclei in histopathology images, IEEE Trans. Biomed. Eng., vol. 57, no. 4, pp. 841852, Apr. 2010.
[59] M. Veta, A. Huisman, M. A. Viergever, P. J. van Diest, and J. P. W. Pluim,
Marker-controlled watershed segmentation of nuclei in H & E stained
breast cancer biopsy images, in Proc. 8th IEEE Int. Symp. Biomed.
Imag.: Nano Macro, Chicago, IL, USA, Apr. 2011, pp. 618621.
[60] H. Kong, M. Gurcan, and K. Belkacem-Boussaid, Partitioning
histopathological images: An integrated framework for supervised colortexture segmentation and cell splitting, IEEE Trans. Med. Imag., vol. 30,
no. 9, pp. 16611677, Sep. 2011.
[61] A. Mouelhi, M. Sayadi, and F. Fnaiech, Automatic segmentation of clustered breast cancer cells using watershed and concave vertex graph, in
Proc. Int. Conf. Commun., Comput. Control Appl., Hammamet, Tunisia,
Mar. 2011, pp. 16.
[62] V. Roullier, O. Lezoray, V. T. Ta, and A. Elmoataz, Multi-resolution
graph based analysis of histopathological whole slide images: Application to mitotic cell extraction and visualization, Comput. Med. Imag.
Graph., vol. 35, pp. 603615, 2011.
[63] S. Ali and A. Madabhushi, An integrated region-, boundary-, shapebased active contour for multiple object overlap resolution in histological
imagery, IEEE Trans. Med. Imag., vol. 31, no. 7, pp. 14481460, Jul.
2012.
[64] X. Qi, F. Xing, D. J. Foran, and L. Yang, Robust segmentation of
overlapping cells in histopathology specimens using parallel seed detection and repulsive level set, IEEE Trans. Biomed. Eng., vol. 59,
no. 3, pp. 754765, Mar. 2012.
[65] M. Kulikova, A. Veillard, L. Roux, and D. Racoceanu, Nuclei extraction from histopathological images using a marked point process approach, presented at the Proc. SPIE Med. Imag., San Diego, CA, USA,
2012.
[66] H. Chang, L. A. Loss, and B. Parvin, Nuclear segmentation in H&E
sections via multi-reference graph cut (MRGC), in Proc. 9th IEEE Int.
Symp. Biomed. Imag.: Nano Macro, Barcelona, Spain, 2012, pp. 614
617.
[67] A. Veillard, M. Kulikova, and D. Racoceanu, Cell nuclei extraction from
breast cancer histopathology images using color, texture, scale and shape
information, in Proc. 11th Eur. Congr. Telepathol. 5th Int. Congr. Virt.
Microsc., Venice, Italy, 69, Jun. 2012, 3 pp.
[68] S. Wienert, D. Heim, K. Saeger, A. Stenzinger, M. Beil, P. Hufnagl,
M. Dietel, C. Denkert, and F. Klauschen, Detection and segmentation of
cell nuclei in virtual microscopy images: A minimum-model approach,
Sci. Rep., vol. 2, 7 pp., 2012.
[69] J. Vink, M. V. Leeuwen, C. V. Deurzen, and G. Haan, Efficient nucleus
detector in histopathology images, J. Microsc., vol. 249, no. 2, pp. 124
135, 2013.
[70] A. M. Khan, H. El-Daly, E. Simmons, and N. M. Rajpoot, A hybrid
magnitude-phase approach to unsupervised segmentation of tumor areas
in breast cancer histology images, J. Pathol. Inform., vol. 4, 7 pp., Mar.
2013.
[71] (2012). MITOS, ICPR 2012 Contest, IPAL UMI CNRS Lab Std., [Online]. Available: http://ipal.cnrs.fr/ICPR2012/?q=node/5
[72] C. D. Malon and E. Cosatto, Classification of mitotic figures with convolutional neural networks and seeded blob features, J. Pathol. Inform.,
vol. 4, pp. 913, May 2013.
[73] G. D. Marty, Blank-field correction for achieving a uniform white background in bright field digital photomicrographs, BioTechniques, vol. 42,
no. 6, pp. 714720, 2007.
[74] F. W. Leong, M. Brady, and J. O. McGee, Correction of uneven illumination (vignetting) in digital microscopy images, J. Clin. Pathol.,
vol. 56, no. 8, pp. 619621, 2003.
[75] A. Gherardi, A. Bevilacqua, and F. Piccinini, Illumination field estimation through background detection in optical microscopy, in Proc.
IEEE Symp. Comput. Intell. Bioinform. Comput. Biol., Paris, France,
2011, pp. 16.
[76] F. Piccinini, E. Lucarelli, A. Gherardi, and A. Bevilacqua, Multi-image
based method to correct vignetting effect in light microscopy images,
J. Microsc., vol. 248, no. 1, pp. 622, 2012.
[77] A. Can, M. Bello, H. E. Cline, T. Xiaodong, F. Ginty, A. Sood, M. Gerdes,
and M. Montalto, Multi-modal imaging of histological tissue sections, in Proc. IEEE 5th Int. Symp. Biomed. Imag.: Nano Macro,
Sun Valley, Idaho, USA, 2008, pp. 288291.
[78] L. E. Grenier, B. V. Funt, P. H. Orth, and D. M. McIntosh, Transillumination method apparatus for the diagnosis of breast tumors and other
breast lesions by normalization of an electronic image of the breast,
U.S. Patent 5 079 698, Jan. 7, 1992.

113

[79] D. Magee, D. Treanor, D. Crellin, M. Shires, K. Smith, K. Mohee, and

P. Quirke, Colour normalisation in digital histopathology images, in
Proc. Opt. Tissue Image Anal. Microsc., Histopathol. Endosc., London,
U.K., 2024 Sep. 2009, pp. 100111.
[80] M. Macenko, M. Niethammer, J. Marron, D. Borland, J. T. Woosley,
X. Guan, C. Schmitt, and N. E. Thomas, A method for normalizing histology slides for quantitative analysis, in Proc. IEEE Int. Symp. Biomed.
Imag.: Nano Macro, Boston, MA, USA, 30 Aug3 Sep. 2011, pp. 1107
1110.
[81] S. Kothari, J. H. Phan, R. A. Moffitt, T. H. Stokes, S. E. Hassberger,
Q. Chaudry, A. N. Young, and M. D. Wang, Automatic batch-invariant
color segmentation of histological cancer images, in Proc. IEEE Int.
Symp. Biomed. Imag.: Nano Macro, 2011, pp. 657660.
[82] E. Reinhard, M. Adhikhmin, B. Gooch, and P. Shirley, Color transfer
between images, IEEE Comput. Graph. Appl., vol. 21, no. 5, pp. 3441,
Sep./Oct. 2001.
[83] D. Magee, D. Treanor, P. Chomphuwiset, and P. Quirke, Context aware
colour classification in digital microscopy, in Proc. Med. Image Understand. Anal., Coventry, U.K., 67 Jul. 2010, pp. 15.
[84] A. C. Ruifrok and D. A. Johnston, Quantification of histochemical staining by color deconvolution., Anal. Quant. Cytol. Histol./Int. Acad. Cytol.
Amer. Soc. Cytol., vol. 23, no. 4, pp. 291299, 2001.
[85] M. Niethammer, D. Borland, J. Marron, J. Woosley, and N. E. Thomas,
Appearance normalization of histology slides, in Machine Learning in
Medical Imaging. New York, NY, USA: Springer, 2010, pp. 5866.
[86] A. M. Khan, H. El-Daly, and N. M. Rajpoot, A GammaGaussian mixture model for detection on mitotic cells in breast histology images, in
Proc. 21th Int. Conf. Pattern Recog., Tsukuba, Japan, 1115 Nov. 2012,
pp. 149152.
[87] L. Yang, P. Meer, and D. J. Foran, Unsupervised segmentation based
on robust estimation and color active contour models, IEEE Trans. Inf.
Technol. Biomed., vol. 9, no. 3, pp. 475486, Sep. 2005.
[88] V. Anari, P. Mahzouni, and R. Amirfattahi, Computer-aided detection of
proliferative cells and mitosis index in immunohistichemically images
of meningioma, in Proc. 6th Iran. Conf. Mach. Vis. Image Process.,
Isfahan, Iran, 2010, pp. 15.
[89] H. Irshad, Automated mitosis detection in histopathology using morphological and multi-channel statistics features, J. Pathol. Inform., vol. 4,
pp. 1015, May 2013.
[90] S. Baheerathan, F. Albregtsen, and H. E. Danielsen, New texture features based on the complexity curve, Pattern Recog., vol. 32, no. 4,
pp. 605618, 1999.
[91] P. W. Hamilton, P. H. Bartels, D. Thompson, N. H. Anderson,
R. Montironi, and J. M. Sloan, Automated location of dysplastic fields
in colorectal histology using image texture analysis, J. Pathol., vol. 182,
pp. 6875, 1997.
[92] R. C. Gonzalez and R. E. Woods, Digital Image Processing. Upper
Saddle River, NJ, USA: Pearson, 2008.
[93] G. Loy and A. Zelinsky, Fast radial symmetry for detecting points of
interest, IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 8, pp. 959
973, Aug. 2003.
[94] B. Parvin, Q. Yang, J. Han, H. Chang, B. Rydberg, and M. H. BarcellosHoff, Iterative voting for inference of structural saliency and characterization of subcellular events, IEEE Trans. Image Process., vol. 16, no. 3,
pp. 615623, Mar. 2007.
[95] T. J. Fuchs and J. M. Buhmann, Computational pathology: Challenges
and promises for tissue analysis, Comput. Med. Imaging Graph., vol. 35,
no. 7, pp. 515530, 2011.
[96] L. Roux, D. Racoceanu, N. Lomenie, M. Kulikova, H. Irshad, J. Klossa,
F. Capron, C. Genestie, G. L. Naour, and M. N. Gurcan, Mitosis detection in breast cancer histological images an ICPR 2012 contest,
J. Pathol. Inform., vol. 4, pp. 814, May 2013.
[97] D. C. Ciresan, A. Giusti, L. M. Gambardella, and J. Schmidhuber, Mitosis detection in breast cancer histology images with deep neural networks, in Proc. Med. Image Comput. Comput.-Assist. Interv., Nagoya,
Japan, 2226 Sep. 2013, pp. 411418.
[98] H. Irshad, L. Roux, and D. Racoceanu, Multi-channels statistical
and morphological features based mitosis detection in breast cancer
histopathology, in Proc. IEEE 35th Annu. Int. Conf. Eng. Med. Biol.
Soc., Osaka, Japan, 37 Jul. 2013, pp. 60916094.
[99] F. Cloppet and A. Boucher, Segmentation of overlapping/aggregating
nuclei cells in biological images, in Proc. 19th Int. Conf. Pattern Recog.,
Tampa, FL, USA, 811 Dec. 2008, pp. 14.
[100] O. Lezoray and H. Cardot, Cooperation of color pixel classification
schemes and color watershed: A study for microscopic images, IEEE
Trans. Image Process., vol. 11, no. 7, pp. 783789, Jul. 2002.

114

[101] Glioblastoma TCGA Dataset, National Cancer Institute Std.

(2008).
[Online].
Available:
https://tcga-data.nci.nih.gov/tcga/
tcgaAnalyticalTools.jsp
[102] J. Shu, H. Fu, G. Qiu, P. Kaye, and M. Ilyas, Segmenting overalapping
cell nuclei in digital histopathology images, in Proc. IEEE 35th Annu.
Int. Conf. Eng. Med. Biol. Soc., Osaka, Japan, 37 Jul. 2013, pp. 5445
5448.
[103] X. Xie and M. Mirmehdi, Correction to MAC: Magnetostatic active
contour model [apr 08 632-646], IEEE Trans. Pattern Anal. Mach.
Intell., vol. 30, no. 5, pp. 632646, May 2008.
[104] T. F. Chan and L. A. Vese, Active contours without edges, IEEE Trans.
Image Process., vol. 10, no. 2, pp. 266277, Feb. 2001.
[105] K. Mosaliganti, L. Cooper, R. Sharp, R. Machiraju, G. Leone, K. Huang,
and J. Saltz, Reconstruction of cellular biological structures from optical
microscopy data, IEEE Trans. Vis. Comput. Graph., vol. 14, no. 4,
pp. 863876, Jul./Aug. 2008.
[106] O. Schmitt and M. Hasse, Radial symmetries based decomposition of
cell clusters in binary and gray level images, Pattern Recog., vol. 41,
no. 6, pp. 19051923, 2008.
[107] K. Rodenacker and E. Bengtsson, A feature set for cytometry on digitized microscopic images, J. Cellul. Pathol., vol. 25, pp. 136, 2003.
[108] L. E. Boucheron, Object and spatial-level quantitative analysis of multispectral histopathology images for detection and characterization of
cancer, Ph.D. dissertation, Dept. Electr. Comput. Eng., Univ. California, Santa Barbara, CA, USA, 2008.
[109] S. Liu, P. A. Mundra, and J. C. Rajapakse, Features for cells and nuclei
classification, in Proc. IEEE 33rd Annu. Int. Conf. Eng. Med. Biol. Soc.,
Boston, MA, USA, Aug. 30Sep. 3, 2011, pp. 66016604.
[110] H. Irshad, S. Jalali, L. Roux, D. Racoceanu, L. Joo-Hwee, G. L. Naour,
and F. Capron, Automated mitosis detection using texture, SIFT features
and HMAX biologically inspired approach, J. Pathol. Inform., vol. 4,
pp. 1218, Mar. 2013.
[111] E. D. Gelasca, B. Obara, D. Fedorov, K. Kvilekval, and B. Manjunath, A
biosegmentation benchmark for evaluation of bioimage analysis methods, BMC Bioinform., vol. 10, pp. 112, 2009.
[112] (2013). AMIDA, MICCAI 2013 Challenge Std., [Online]. Available:
http://amida13.isi.uu.nl/
[113] C. Malon, H. Brachtel, E. Cosatto, H. P. Graf, A. Kurata, M. Kuroda,
J. S. Meyer, A. Saito, S. Wu, and Y. Yagi, Mitotic figure recognition:
Agreement among pathologists and computerized detector, Anal. Cellul.
Pathol., vol. 35, pp. 97100, 2012.

Humayun Irshad (S13) received the B.S. degree

in computer science from the University of Central
Punjab, Lahore, Pakistan, in 2005 and the M.S. degree in computer science from the National University of Computer and Emerging Sciences, Islamabad,
Pakistan, in 2009. He is currently working toward the
Ph.D. degree at the University Joseph Fourier, Grenoble, France.
He is with the joint FrenchSingaporean Research
Lab Image and Pervasive Access Lab UMI, French
National Center for Scientific Research, Singapore.
His research interests include pattern recognition, machine learning, medical
image analysis, and computer vision.

IEEE REVIEWS IN BIOMEDICAL ENGINEERING, VOL. 7, 2014

Antoine Veillard received the Diplome dingenieur

(M.Sc.) degree from Ecole

Polytechnique, Paris,
France, in 2009, with a double major in computer
science, and the Ph.D. degree from the National University of Singapore, Singapore, in 2013.
He is a Research Fellow at the University Pierre
and Marie Curie, Paris, France, and a member of Image and Pervasive Access Lab UMI, French National
Center for Scientific Research Lab, Singapore. His
research interests include machine learning and the
integration of prior-knowledge into kernel machines,
in particular.

Ludovic Roux received the Ph.D. degree in computer

science from the University Paul Sabatier, Toulouse,
France., in 1997
He is currently with the University Joseph Fourier,
Grenoble, France, and is a Research Fellow at the
joint FrenchSingaporean Research Lab Image and
Pervasive Access Lab UMI, French National Center
for Scientific Research, Singapore. His research interests include machine learning, image processing,
segmentation, classification, and data fusion.

Daniel Racoceanu (M08) received the M.Eng. degree from the Politehnica University, Timisoara, Romania, in 1992, and the M.Sc., Ph.D., and Dr. habil
degrees from the University of Besancon, Besancon,
France, in 1993, 1997, and 2006, respectively,
He is currently with the University Pierre and
Marie Curie, Paris, France. He is a Senior Research
Fellow at the French National Center for Scientific
Research, Singapore, and a Professor (adj.) at the National University of Singapore, Singapore. He is the
Director of the FrenchSingaporean-Joint Lab Image and Pervasive Access Lab UMI, French National Center for Scientific
Research. His research interests include symbolic and connectionist cognitive
exploration methods for high-content biomedical images. He is involved in
the ANR-A*STAR program and French Ministry of Industry founded grants
(http://www.comp.nus.edu.sg/danielr/see webpages).